How AI Scribes Handle Multi-Speaker Sessions

Group therapy, couples sessions, family visits, and appointments with a patient plus a support person all involve multiple speakers. An AI scribe that just generates one SOAP note is useless in these scenarios — you need distinct notes for each participant, or a properly attributed group summary.

Scriber handles this through diarization: separating speaker turns, attributing them to participants, and generating participant-specific notes.

What diarization is and why it matters

Diarization is the process of figuring out who is speaking when in a recording. It answers the question: "Who said that?"

Without diarization, an AI scribe treating a couples therapy session might generate a note that says:

"Patient presented with concerns about communication. Reports feeling unheard by partner. Discussed coping strategies. Agreed to practice active listening."

This is a blended summary, not a clinical note for either person. It does not capture:

What each person actually said.
Who initiated the conversation.
Whether the "coping strategy" was embraced by both or just one person.
How the partner responded.

With diarization, Scriber separates the speakers and can generate:

Note for Patient A: "Patient reports feeling unheard by partner. Therapist and partner discussed communication patterns. Patient agreed to practice active listening. Partner seemed receptive to this approach."

Note for Patient B (the partner): "Patient engaged in discussion of communication patterns with partner. Acknowledged partner's feelings. Agreed to practice active listening with partner."

This is accurate, clinically distinct, and captures what each person actually said and contributed.

How Scriber's diarization works

Step 1: Speaker identification (clinician provides the key)

Before or at the start of the recording, the clinician identifies each speaker. This can happen a few ways:

Option A: Voice intro at the start of the session The clinician says: "This is a couples session. We have Patient A (Mary), Patient B (John), and me as the clinician."

Scriber learns the voices from this intro and attributes subsequent turns.

Option B: Scriber's speaker-identification UI When you start a recording in Scriber, you see a prompt: "Who is in this session?" You enter:

Speaker 1: Mary (Patient)
Speaker 2: John (Patient)
Speaker 3: Dr. Smith (Clinician)

Scriber then listens for these speakers and attributes turns.

Option C: Speaker labels during the session As the session unfolds, if a new speaker appears (e.g., a parent joins a teen session midway), you can tell Scriber: "This is Sarah, the mother."

Once Scriber has a baseline for each speaker's voice, diarization continues automatically.

Step 2: Automatic speaker turn attribution

As the session proceeds, Scriber's speech-to-text engine identifies when a speaker changes (a new voice starts speaking) and attributes each turn to the right person based on voice characteristics.

This is not perfect — identical twins or very similar voices can confuse the system — but it is accurate 95%+ of the time in normal clinical environments (quiet room, clear speech).

Step 3: Transcript with speaker labels

The transcript shows:

[Dr. Smith] "Today we're going to talk about communication patterns. Mary, would you start?"

[Mary] "Sure. I feel like John doesn't listen when I try to tell him about my day."

[John] "That's not fair. I listen, but sometimes you're talking about so many things at once, I lose track."

[Mary] "See, that's what I mean. You make it about me, not about you listening."

[Dr. Smith] "Let's pause here. John, what I'm hearing is..."

Each turn is labeled with the speaker's name. The clinician can review this and correct any misattribution if needed.

Step 4: Note generation per speaker

When you generate the clinical note, Scriber asks: "What type of session is this?" and "Who are the participants?"

You select "Couples therapy" as the template, and Scriber generates:

A note for Mary (the patient).
A note for John (the patient).
A summary of the session from the clinician's perspective.

Each note reflects what that person said, their concerns, their responses, and their agreements.

Accuracy and clinician review

Diarization is not magic. It can fail in a few scenarios:

Poor audio quality: If the recording is garbled, background noise is loud, or someone is off-microphone, diarization struggles. Speaker attribution might be wrong or incomplete.

Similar voices: Twins or people with very similar speech patterns can confuse the system.

Overlapping speech: When two people talk at the same time, diarization has to guess who said what.

Unknown speakers: If someone joins the session without being identified at the start, the system may attribute their words to the wrong person.

This is why clinician review is mandatory. Scriber diarizes and generates the draft; you review and correct. If the note misattributes a quote or merges two speakers' comments, you catch it and fix it.

The review step is not a bug — it is the safety mechanism. You are responsible for the accuracy of the note before you sign it, regardless of how it was generated.

Specific session types and note generation

Group therapy

Setup: Group with 6-8 participants. Clinician opens Scriber and identifies each participant by first name or initials.

During the session: Scriber listens to the group, separates speakers, and labels each contribution.

Note generation: Scriber produces:

A per-participant progress note for each member (capturing their concerns, contributions, and the clinician's observations).
A group-level summary capturing themes, curriculum delivered, group dynamics, and clinical observations.

Length handling: If one participant talks a lot and another is quiet, their notes reflect that. The engaged person's note is longer and more specific; the quiet person's note honestly says they were present but silent, or made minimal contributions.

Review: You review each per-participant note for accuracy. If Scriber attributed a comment to the wrong person, you correct it. If a person's note seems incomplete, you add observations.

The result: a 90-minute group session produces 8 clinician-reviewable notes plus a group summary, ready for signature.

Couples or family therapy

Setup: Couples session: Patient A + Patient B + Clinician. Family session: Parent + Parent + Teen + Clinician. Scriber identifies each speaker.

During the session: Scriber separates all speakers and labels turns.

Note generation: You can generate:

Individual notes per person: Each participant gets a note reflecting what they said, their perspective, and the clinician's observations specific to them.
Joint summary: One document capturing the shared content, agreements, and homework.
Or both.

Review: Each note reflects one person's participation and perspective. You verify accuracy and sign.

Support person in a solo session

Setup: Patient attends session with a parent, partner, or advocate. Scriber identifies both speakers.

During the session: Scriber labels turns for both the patient and the support person.

Note generation: Your options:

Patient-focused note with observations about the support person ("Patient attended with mother, who provided supportive comments about patient's progress").
Separate notes if both are patients (e.g., therapy for both an adolescent and their parent).

Review: You review the note and ensure it captures the dynamic appropriately.

Common diarization challenges and solutions

Challenge 1: Speaker joins late or leaves early

Scenario: You start a group with 6 people. Midway through, person 7 joins.

Solution: Pause the recording, tell Scriber "New speaker: Alex, participant," and resume. Subsequent speakers are correctly attributed.

If you forget and Alex speaks unidentified, you catch it during the transcript review and either relabel the turns or add a note explaining the speaker.

Challenge 2: Very similar voices or accents

Scenario: Two participants have similar speech patterns, and the system confuses them.

Solution: Review the transcript and correct the misattribution. Scriber's UI lets you click on a turn and re-label it: "This was Alex, not Sam."

Challenge 3: Overlapping or simultaneous speech

Scenario: Two people start talking at the same time, and the system cannot tell who said what.

Solution: The transcript may show ambiguous attribution or request clarification. You review and correct. In the actual note, if the overlap was not critical to the clinical narrative, you might merge or simplify. If it was important, you add a note: "Both participants expressed concern about this topic."

Challenge 4: Quiet or soft-spoken participants

Scenario: One participant speaks very quietly and the system misses some words or sometimes does not detect they spoke at all.

Solution: The transcript is incomplete for that person. You review and either:

Add the words manually if you remember them.
Note that the person attended but their contributions were inaudible.
Request a better recording setup (mic closer, etc.) for future sessions.

Recording setup for multi-speaker sessions

To maximize diarization accuracy:

Best practice:

Single, high-quality microphone in the center of the room (omnidirectional or multi-channel).
All speakers at the same distance from the mic.
Minimal background noise.
Clear speech (ask participants to speak clearly).

For telehealth (Doxy.me, Zoom, etc.):

Each participant has their own audio channel, so diarization is very clean.
This is actually better than in-person sessions because speaker separation is built-in.

For in-person groups:

Use a conference-style microphone (e.g., Shure MX series or equivalent).
Avoid multiple microphones pointing in different directions (they create phase issues).
If possible, ask participants to introduce themselves at the start so Scriber learns their voices.

Privacy and confidentiality in multi-speaker notes

When you generate notes for a group session, each participant's note is part of their individual record:

Mary's note is in Mary's chart.
John's note is in John's chart.
The group summary may be in all charts (or just in the group record, if you maintain one).

If John is referred to another provider and you send his notes, his note from the couples session is included, not Mary's. Mary's privacy is maintained.

For group therapy, each member's progress note is stored in their individual record, protected by the same confidentiality and access controls as any other clinical note.

Limitations and when NOT to use multi-speaker diarization

Diarization works well for:

Couples and family therapy.
Group therapy and psychoeducation groups.
Medical appointments with a patient plus family member.
IOP and PHP group sessions (including curriculum-based groups).

Diarization may not be appropriate for:

Confidential individual therapy where recording with a third-party present raises consent or privacy issues.
Crisis or emergency sessions where speed matters more than perfect attribution.
Sessions where participants request anonymity (though this is rare in clinical settings).

In these cases, you can disable diarization and generate a single clinician-focused note instead.

Bottom line

Diarization is what makes AI scribes usable for multi-speaker sessions. Without it, a group therapy note would be useless — you would get a blended narrative instead of individual progress notes. Scriber's diarization separates speakers, attributes contributions, and lets you generate participant-specific notes that actually capture what each person said and did.

The key is clinician review. Diarization is automatic, but verification is manual. You review the transcript, correct any speaker misattributions, and sign the notes. This takes a few minutes for a group session and is far faster than writing the notes from scratch.

For group-based practices, IOP programs, family therapy clinics, and any modality with multiple participants in a single session, diarization is a game-changer — it lets one clinician generate 8 distinct, accurate progress notes in the time it currently takes to write 2.

How AI Scribes Handle Multi-Speaker Sessions | Scriber Wiki