An AI chatbot shouldn't scroll faster than you can read
A person trying to check their rental-car insurance started reading the chatbot's answer aloud. The text kept moving up the window faster than they could read it. They stopped: ‘[I’ll] wait for it to finish.’ That's an observation from Nielsen Norman Group's existing website-chatbot study, not a test I've run. Showing words early didn't let that person use the answer early. The scrolling made them wait anyway, then find their place again. NN/g also observed Mississippi's government chatbot pushing readers to the bottom while a long answer appeared. I'd keep the beginning of a new answer still when it won't fit in the window. Once someone scrolls back to reread an insurance condition, leave them there even if more text arrives below. A ‘Jump to latest’ button can wait for their choice. For a small chat window, I'd test whether someone can read the first paragraph and return to a clause without chasing either one. A quick-looking reply isn't much comfort when the sentence you need keeps leaving the screen.
Comments
For a small business using a chat widget it can't easily change, I'd put the frequently requested instructions on an ordinary page as well, and link to it from the answer. Someone checking a return rule should be able to open it and read at their own pace without keeping the chat alive. Keep that page current rather than maintain two different versions of the rule. Fix the scrolling when you can; a readable route out would help in the meantime. That's a workaround I'd suggest, not one tested in the study.
Try leaving the chat to find your order number in an email, then coming back. Is the answer still there, at the sentence you were reading? I'd include that in a phone test before calling the support chat easy to use. Getting a return sorted often means moving between the shop, your email and a photo of the parcel. The customer shouldn't have to collect everything first just to keep the conversation from losing their place.
Put that page where someone can find it without opening the chatbot, too. A visible ‘Returns’ link shouldn't become ‘Ask our assistant about returns’. If the business counts fewer support contacts as success, a customer who gives up can look suspiciously like a customer who got an answer. I'd keep the direct link beside the widget and test whether someone can find the rule without starting a conversation. Nobody should have to chat their way into reading a shop's policy.
I'd time how long it takes someone to find the return deadline and explain the exception that applies to their order, including any waiting and rereading. Keep wrong answers and abandoned attempts in the results; the fastest readers aren't the whole audience. To compare scrolling behavior, show the same answer text in both versions, or a clearer answer could get mistaken for a better interface. That's a test I'd suggest, not a result from NN/g. It would give a shop a reason to change the widget beyond making the reply look faster.