Scriber — AI Medical Scribe for Behavioral Health Clinicians

AI bias in transcription is real. Studies show that transcription engines are less accurate on speakers with non-native English accents, Black English dialect speakers, and women's voices compared to men's. This directly affects documentation quality for patients from marginalized communities.

Scriber's AI should be tested and monitored for fairness. Here's how to evaluate whether bias is affecting your practice.

What AI Bias Looks Like in Transcription

Scenario 1: Accent-related errors

Patient with Spanish accent describes depression symptoms:

Patient: "I feel constant sadness and hopelessness about the future."
Scriber transcription: "I feel constant sadness and 'opelessness about the future." (missed the "h")

This is a minor error in isolation, but combined with dozens of minor errors, the transcript becomes harder to read and clinical meaning can shift.

Scenario 2: Dialect-related errors

Patient using African American English describes their experience:

Patient: "I been dealing with this anxiety for months, man."
Standard transcription: "I been dealing with this anxiety for months, man."
But if the engine "corrects" to standard English: "I have been dealing with this anxiety for months, man."

This subtly changes how the patient's voice is represented, and can introduce errors if the clinical note relies on exact wording.

Scenario 3: Cultural language interpretation

Patient speaks about cultural spiritual practice:

Patient: "I pray every morning, that's what keep me centered."
Scriber might transcribe: "I pray every morning, that's what keeps me centered." (grammatical correction)
Or: "I pray every morning, that's what kit me centered." (misheard "keep")

Either way, the exact patient language is altered, potentially changing how their coping strategy is understood.

How to Test Your Transcription Engine for Bias

Step 1: Record diverse voices

Create test recordings with speakers of different backgrounds:

Native English speakers (white, standard American accent)
Non-native English speakers (Spanish, Mandarin, African, Indian accents)
Speakers using African American English
Speakers who are hard of hearing or speak quietly
Women and men (transcription engines often perform worse on women)

You can recruit colleagues, friends, or family members, or find publicly available datasets.

Step 2: Use the same clinical content for all speakers

Have each person read the same 2–3 minute clinical passage:

"I've been feeling depressed for the past three weeks. I wake up in the morning and I don't 
have energy to do anything. I used to enjoy spending time with my friends, but now I don't 
want to go out. I'm also having trouble sleeping. Some nights I lie awake for hours. My 
appetite has decreased and I've lost about five pounds. I think my medications might not be 
working anymore. I'm worried about my job because I've been calling in sick."

Step 3: Transcribe each recording

Record each speaker reading the passage. Use your chosen transcription engine (xAI, Azure, Speechmatics) to transcribe each.

Step 4: Compare transcriptions for accuracy

Count errors for each speaker:

Word-level errors (misheard words, omissions)
Grammar or dialect "corrections"
Meaning-changing errors

Calculate error rate: (Errors / Total words) × 100

Example results:

Speaker 1 (native English, standard accent): 2 errors / 150 words = 1.3% error rate
Speaker 2 (Spanish accent): 8 errors / 150 words = 5.3% error rate
Speaker 3 (African American English): 6 errors / 150 words = 4% error rate
Speaker 4 (Indian accent): 10 errors / 150 words = 6.7% error rate

If error rates vary by more than 3–4%, your transcription engine has bias.

What Counts as a Bias Error?

Not all errors are bias. Some are just accuracy issues. Bias errors specifically:

Accent-related: Errors that happen for non-native speakers but not native speakers
Dialect "correction": The engine "corrects" dialect to standard English
Systematic under-performance: One demographic group consistently has higher error rates
Meaning-changing: Errors that alter clinical meaning

Not bias errors:

Random transcription errors that affect all speakers equally
Mishearing quiet speakers (everyone's transcript is quieter)
Errors on specialized terminology (everyone struggles with medical jargon)

Monitoring Bias in Your Practice

After you implement Scriber, monitor for bias over time:

Quarterly review:

Pull a random sample of 10–15 notes from diverse patients
Review transcripts for accuracy across groups
Note any patterns: Are transcripts for non-native English speakers consistently worse?
Are edits for patients of color more extensive?

Track metrics:

Average review time per patient: Is it longer for patients with accents?
Error types: Are errors random, or do they cluster by demographic?
Patient feedback: Have any patients reported that their words were misrepresented?

Annual fairness audit:

Repeat the test recording exercise (diverse voices, same clinical content)
Compare to baseline
Has the engine improved? Gotten worse? Remained the same?

What to Do If You Find Bias

If your transcription engine is biased:

Switch engines: Try Speechmatics (better at international accents) or Azure with accent-specific settings
Implement compensatory measures:
- Spend extra review time on transcripts from patients with accents
- Provide patient with transcript copy and chance to correct
- Use manual transcription for high-stakes content (safety assessments, trauma disclosures)
Report to Scriber: If you're using Scriber, report the bias you've identified
Document the bias: Keep records showing transcription disparities for your practice's quality improvement file

Long-term solution:

Advocate for Scriber to improve its transcription engines
Consider using multiple engines and comparing them
Contribute to bias mitigation research if you're able

Bias Beyond Transcription Accuracy

Bias can also show up in the AI-generated note itself:

Example 1: Stereotype-confirming language

Patient (Black male, low-income) describes substance use history:

Scriber transcription: [accurate]
AI-generated note: "Patient presents with long-standing alcohol abuse and poor insight into consequences, consistent with a pattern of poor self-care and limited motivation."

This language reinforces stereotypes (Black men, substance abuse, poor motivation). A less biased version: "Patient describes history of alcohol use. Assessed for readiness to change; patient ambivalent about treatment but agreed to consider outpatient alcohol counseling."

Example 2: Over-pathologizing cultural practices

Patient describes cultural family involvement:

Patient: "My family is very involved in my care. My mom and aunts help with my kids, and we talk about everything."
Biased note: "Patient has enmeshed family dynamics and poor boundaries with extended family."
Less biased note: "Patient describes strong family support system with extended family involvement in childcare and emotional support."

The same behavior is interpreted negatively if the clinician is biased against that cultural pattern.

How to check your own notes:

Review AI-generated notes for stereotype-confirming language
Would the language differ if the patient were from a different demographic?
Are cultural practices pathologized or normalized?
Edit the note to remove biased language

Patient Communication About AI Bias

Be transparent with patients:

"I use an AI transcription tool to capture our session accurately. Like all AI systems, it's been trained on diverse voices, but it can make occasional mistakes. I always review the transcript and note for accuracy, especially to make sure your words are captured correctly. If you ever feel that your words were misrepresented or your meaning was changed, please let me know."

This acknowledges potential bias and invites patient feedback.

The Broader Context

AI transcription bias is part of larger systemic inequities in healthcare:

Clinical notes written by biased human clinicians (confirmation bias, stereotyping)
Diagnostic and treatment disparities across racial/ethnic groups
Underrepresentation in training data for AI systems

AI scribing can reduce clinician bias (the AI captures patient words objectively), but it can also perpetuate or amplify existing biases if the training data is biased.

Stay critical. Monitor your tool. Advocate for fairness.

Practical Checklist for Fairness

Test your transcription engine with diverse speakers (accent, dialect, gender)
Calculate error rates by demographic group
If error rates differ by >3–4%, consider switching engines or monitoring closely
Quarterly: Review sample of notes from diverse patients; check for bias in transcription and AI-generated note language
Monitor review time: Is it longer for patients with accents? (If yes, investigate why)
Gather patient feedback: Ask if words were captured fairly
Edit AI-generated notes for stereotype-confirming language
Document your fairness monitoring process

Evaluating AI Scribes for Bias and Fairness | Scriber Wiki