AI Scribes and Chart Quality Improvement | Scriber Wiki
Last updated
Most clinicians think of notes as documentation burden: capture the session, document for liability, move on. But AI-generated notes create a secondary product: practice data.
Every note is now structured, standardized, and machine-readable. This data can be analyzed to identify patterns, spot gaps in your clinical process, and improve how you document and deliver care.
What Data is Available from AI-Scribed Notes?
When Scriber generates notes using consistent templates, you have:
Per-note data:
- Session date, time, duration
- Patient demographics (if available in your system)
- Chief complaint / presenting problem
- Interventions used
- Clinical outcome / progress
- Medication changes
- Referrals or care coordination
Aggregate data (when analyzed across many notes):
- Most common presenting problems in your practice
- Frequency of specific interventions
- Which interventions correlate with better outcomes
- Diagnostic patterns
- Gap areas (diagnoses you never document, interventions you rarely use)
- Quality metrics (completeness of notes, inclusion of required sections)
This data is gold for quality improvement.
Quality Improvement Project Ideas
Project 1: Identify Gaps in Your Documentation
Goal: Are you consistently documenting things you should be documenting?
How: Review a sample of notes (30–50) and ask:
- Are safety assessments in every note? (Should be for patients with risk factors)
- Do notes include cultural context? (Is this relevant to your practice?)
- Is trauma history documented when relevant? (Or are you missing historical context?)
- Are diagnoses clearly stated? (Or are they vague?)
- Is the plan specific? (Or do you write "continue treatment" without detail?)
Analysis:
- Count: 35 of 50 notes have explicit safety assessment = 70% completion
- Goal: 95% (near 100% for high-risk patients)
- Gap: 25% of notes are missing safety assessment
Improvement:
- Add safety assessment to your template as a required field
- Retrain clinicians (if a group practice)
- Retest in 30 days
Project 2: Standardize Treatment for Common Problems
Goal: Are you using evidence-based interventions consistently?
How: Pick a common diagnosis (e.g., depression, anxiety, PTSD). Review how you're documenting treatment:
- What interventions are you using? (CBT? DBT? Psychodynamic? Psychoeducation?)
- How often do you use each? (CBT in 60% of depression cases, psychodynamic in 40%)
- Are patients improving? (Which intervention correlates with better outcomes?)
Example:
Depression interventions in your practice (review 30 depression cases):
- CBT techniques: 18 cases (60%) — 70% showed symptom improvement
- Psychoeducation: 12 cases (40%) — 50% showed symptom improvement
- Referral for medication: 8 cases (27%) — 88% showed significant improvement
Insight: Patients with medication + therapy do better than therapy alone. Recommendation: Increase referral rate for psychiatric evaluation in depression cases.
Project 3: Track Outcome Data by Intervention Type
Goal: Which clinical approaches work best for your patient population?
How: Use your note data to track outcomes. If you're using standardized measures (PHQ-9, GAD-7, PCL-5), document scores at baseline and end of treatment.
Scriber can capture these scores from the session dialogue:
Transcript: "Your PHQ-9 score today is 14, which is moderate depression. Last month it was 19,
so you've improved by 5 points."
Note: "Patient's PHQ-9: 19 (4 weeks ago) → 14 (today). Improvement of 5 points, indicating
good response to current treatment plan. Patient attributes improvement to consistent CBT work
on negative thoughts."
Over time, you have a database:
- 30 depression cases
- Average baseline PHQ-9: 18
- Average PHQ-9 after 8 weeks: 11
- Average improvement: 7 points (39% reduction in symptoms)
- Outcomes by intervention: CBT showed 42% improvement; psychoeducation 35%; medication 48%
This data tells you what's working.
Project 4: Identify Over-Pathologizing or Under-Pathologizing
Goal: Are you seeing patterns in how you diagnose or describe patients?
How: Systematically review how you characterize presentations. Do you have favorite words or diagnoses that you overuse?
Example red flags:
- Every anxiety case is "GAD," even when presentations vary (situational anxiety, social anxiety, health anxiety)
- You frequently use language like "personality disordered" when patients are from marginalized groups
- You describe cultural practices as "pathological" (enmeshed family, spiritual beliefs)
- You assume deficits in intelligence or capacity based on demographics
Analysis tool: Search your notes for patterns:
- How often do you use diagnosis X?
- What language do you use to describe different patients?
- Would your language differ if the patient were from a different demographic?
Improvement:
- Broaden your diagnostic conceptualization
- Use more precise language (instead of "borderline traits," describe specific behaviors)
- Remove culture-specific pathologizing language
- Train yourself on cultural humility
Project 5: Care Coordination and Referral Patterns
Goal: Are you coordinating care effectively?
How: Review notes for referrals and coordination:
- When you refer to psychiatry, do you document it? (Yes: 100%, No: 0%)
- When patients need other services (substance abuse, medical, legal), are you referring? (Yes: 40%, Should be: 70%)
- When you have a team (therapist + psychiatrist), are you documenting coordination? (Yes: 50%, Should be: 90%)
Analysis:
Referral patterns (review 50 notes):
- Psychiatry referral: 18 cases (36%)
- Follow-up: Documented that patient went: 12/18 (67%)
- No follow-up documented: 6/18 (33%) — gap
- Substance abuse services: 4 cases (8%)
- Patient need present in transcript: 12/50 (24%)
- Referral rate should be: ~24%, actual: 8%
- Gap: You're missing substance abuse in 16% of cases where it's relevant
Improvement: Add substance abuse screening to your templates; make referral pathways explicit.
Data Collection and Analysis Tools
You don't need special software. Basic tools work:
Option 1: Spreadsheet analysis
- Export your notes (or summaries)
- Create columns: Date, Patient ID, Diagnosis, Interventions, Outcome, Follow-up Planned
- Use spreadsheet functions to calculate frequencies, percentages, outcomes
- Simple but manual
Option 2: Scriber reporting (if available)
- Some AI scribe platforms generate reports on note content (diagnoses, interventions, referrals)
- Check if Scriber has a reporting feature for your Scriber workspace
- More automated, but depends on platform
Option 3: Qualitative coding
- If you want deep analysis, read through notes and manually code them
- Example codes: "Uses trauma-informed approach," "Explicitly discusses cultural context," "Documents safety assessment"
- Count frequency of codes
- More labor-intensive but reveals nuanced patterns
Option 4: Work with a quality improvement specialist
- If your organization has a quality team, ask them to analyze your notes
- They can create dashboards, track trends, identify opportunities
- More sophisticated analysis but requires organizational support
Monthly Quality Reviews
Make quality review a routine practice:
Monthly (1st Friday):
- Review a random sample of 5–10 notes
- Check for completeness (all required sections? Clear plan?)
- Spot-check for bias or gaps
- Note any patterns
- 30 minutes per month
Quarterly (end of each quarter):
- Deeper analysis: diagnoses, interventions, outcomes
- Review trends from past quarter
- Identify one improvement area for next quarter
- 2–3 hours per quarter
Annually:
- Full quality improvement project (pick one goal from above)
- Analyze practice-wide patterns
- Set goals for next year
- 4–8 hours per year
Most practices can sustain this rhythm. It becomes part of your continuous improvement culture.
Group Practice Quality Improvement
If you're a group practice, this becomes even more valuable:
Practice-wide analysis:
- Do all clinicians document safety assessments? (Identify clinician-level gaps)
- Are outcomes similar across clinicians? (Does one therapist have better outcomes?)
- Are there diagnostic or treatment disparities across clinicians? (One therapist diagnoses PTSD 50% of the time; another 10%)
Peer comparison (constructive):
- Not: "Your documentation is worse."
- Instead: "Here's how documentation differs across our practice. Let's learn from the strongest examples."
Standardization:
- If group has 5 therapists, 5 templates, you have variation
- Use group data to create one standardized template that everyone uses
- Individual customization is fine, but the core structure should be consistent
Training:
- If analysis shows gaps (e.g., 30% of clinicians don't document safety), provide training
- Use examples from actual notes (anonymized) to show best practices
- Reassess after training
Linking Notes to Outcomes and Satisfaction
The gold standard for quality improvement is outcome tracking:
If you use outcome measures (PHQ-9, GAD-7, etc.):
- Document baseline and follow-up scores
- Link to note content
- Analyze: Do patients with documented treatment plans have better outcomes?
- Result: "Patients with explicit behavioral activation homework had 1.8x better depression improvement"
If you track patient satisfaction (NPS, patient surveys):
- Compare notes from high-satisfaction patients to low-satisfaction patients
- What's different? (Language? Type of plan? Engagement style?)
- Insight: "Patients who reported feeling heard had notes that included direct quotes of their concerns"
If you track re-admission or re-hospitalization:
- Do discharged patients who had safety plans readmit less often?
- Do patients with documented trauma assessment have better treatment engagement?
This creates a virtuous cycle: better notes → trackable outcomes → continuous improvement.
Privacy and Compliance in Data Analysis
When analyzing your notes for quality improvement:
Protect patient privacy:
- Don't include patient names in analysis
- Use patient ID numbers or de-identified codes
- If sharing examples with team, remove identifiers
Comply with regulatory requirements:
- Keep your quality improvement records (notes, analysis, action plans)
- Document that you reviewed for quality; it's evidence of compliance
- If licensing board asks "How do you ensure quality?", you can show your QI process
HIPAA consideration:
- Your internal quality improvement is permitted use of patient data
- External research requires IRB approval and patient consent
- Sharing data with third parties requires patient authorization
Closing the Loop: From Insight to Action
Quality data only matters if you act on it. Create a cycle:
Identify gap (30% of notes missing safety assessment)
↓
Analyze root cause (template doesn't prompt; clinician forgetfulness)
↓
Plan improvement (Add safety to template; train team)
↓
Implement (New template live; team meeting to review)
↓
Measure (Retest in 30 days: 95% completion)
↓
Sustain (Monthly spot-check to ensure compliance)
↓
Identify new gap (Now work on another area)
This cycle, repeated over time, creates continuous improvement.
Related questions
See it in action with Scriber
Start with a 7-day trial — HIPAA-compliant, no audio retention, BAA included at signup.