AI rewrote your staff survey. Can you still compare it with last year's?
Before asking an AI assistant to make the annual staff survey shorter, save the questions and answer options you used last time. A cleaner form can leave you with a comparison you can no longer explain. Pew Research Center's 2019 methods report describes a randomized experiment among 4,581 US adults. People reported undesirable experiences more often when asked separately about each item than when given a ‘select all that apply’ list. Across the items tested, the difference averaged 8 percentage points. This was not a workplace survey or an AI test, and that figure is not a correction to apply to your own results. It shows why the format deserves checking. Imagine last year's form asking staff to tick the problems they experienced, while the AI rewrite asks yes or no about each problem. A rise in reported problems could reflect the new questions, not a worse year at work. I'd ask the assistant to mark every change to the wording, answer options and format, then flag changed items in the report rather than draw an uninterrupted trend line. People who took time to describe their working day deserve a response to what they said, not a celebration or a crackdown built on a changed form.
Comments
Keeping the old form forever would be the wrong lesson, too. Pew adopted separate questions wherever possible after this research. Its accuracy argument rests on a specific assumption: people are more likely to conceal those undesirable experiences than invent them. The researchers did not independently verify each person's history. I wouldn't carry ‘more yes answers means more accurate’ into a staff question about feeling valued or trusting management. Ask the writing assistant to distinguish reports of events from judgments about work before recommending a format. A better survey may deserve a new baseline; staff shouldn't be stuck with a poor question just because it gives management a tidy trend line.
Decide which comparisons survive before anyone sees the answers. Otherwise a manager can discover a deep concern about question design just as the morale score falls, then forget it when the next chart looks flattering. I'd have the AI assistant list changed questions and proposed trend breaks before launch, with the same rule applying to good news and bad. That is a reporting rule I'd want, not a finding from Pew. Improving the form is reasonable. Letting the results decide whether the old form was comparable is rather convenient.
I wouldn't let a broken year-on-year comparison become a reason to postpone dealing with the answers. Imagine staff describing a late shift with nobody available to approve a refund. You may not know whether that problem became more common, but you can still check the coverage and give that shift a way to help the customer. I'd keep ‘can we claim an improvement?’ separate from ‘what needs fixing now?’ in the manager's review. An AI summary should preserve the concrete problems people raised even when it has to drop the trend chart. Otherwise staff get asked for feedback, then told the form wasn't good enough to act on.