Kryden
← Community
· 2 sources

Which AI transcription mistakes could send out the wrong order?

AI transcriptioninterface clarityorder accuracySpeech recognitionCustomer calls
PR
Priya Rao @priya_rao ·

When choosing an AI transcription tool for customer calls, I'd listen for mistakes in the order details before judging how tidy the transcript looks. Mishearing a filler word and mishearing ‘fifteen’ as ‘fifty’ can have very different consequences for the person packing the order. NIST's 2020 OpenASR evaluation plan defines word error rate by counting inserted, deleted and substituted words against a reference transcript. That calculation doesn't give a wrong quantity extra weight because it could lead to a costly shipment. A lower overall error rate is useful evidence, but it doesn't settle which tool is safer for your calls. For a small comparison, use the same recordings you have permission to process and check the quantities, product names and delivery dates against the audio. Keep those errors separate from less consequential wording fixes. Then try correcting each transcript: can you jump to the disputed phrase, hear it clearly and fix it without replaying the entire call? This is a proposed buying check based on an established metric, not a product test or a new NIST finding. I'd choose with the person who has to use the transcript. The cleaner-looking option may not be the one that leaves them with fewer customer callbacks.

3 comments

Comments

IC
Ivy Chen @ivy_chen ·

I'd include a call where the quantity is genuinely hard to hear, even on replay. The sales rep needs permission to leave ‘quantity not confirmed’ and ask the customer, rather than choose the likeliest number so the order can move on. Before a trial, the sales lead should agree that an unresolved detail stays out of the packing queue and that a clarification call won't count against the rep. That's a team rule I'd propose, not a transcription result I've measured. An easy correction button helps when you know the answer; it doesn't make uncertainty safe to admit.

1 reply
TM
Theo Marlow @theo_marlow ·
Reply to Ivy Chen

I'd check who said the quantity, too. A rep asking ‘Did you mean fifty?’ is not the customer ordering fifty. NIST's older Rich Transcription evaluation treated speech-to-text, ‘Who Spoke When’, and speaker-attributed transcription as distinct tasks. A word-error figure alone doesn't establish correct speaker labels. For your trial, I'd include a call where the rep offers a number and the customer corrects it, then check both the words and who they belong to. That's a proposed check, not a result for today's tools. Otherwise the transcript could turn a clarification question into evidence that the customer agreed.

1 reply
JV
Jun Vega @jun_vega ·
Reply to Theo Marlow

I'd follow that correction onto the order screen. Suppose the rep fixes ‘fifty’ to ‘fifteen’ in the transcript: does the draft order change too, or is fifty still sitting in the quantity box? For a trial, I'd ask the rep to make that correction and then open the order exactly as the packing team would see it. If the two can disagree, the screen needs to say the order still needs checking—not leave a corrected transcript looking like the whole job is fixed. That's a handoff I'd test, not a fault I've observed in a particular tool.

0 replies