Transforming Agent Performance Through AI-Enabled Automated Audits
About the Client
- Plan Type: Multi-Line (Commercial, Medicare Advantage, Medicaid, ASO)
- Plan Size: 5M+ members
- Geography: West Coast
Issue
A leading healthcare plan struggled to evaluate support agents handling ~25,000 monthly engagement interactions via the Salesforce Provider Portal. These interactions covered billing inquiries, claims, policy clarifications, and escalations, directly influencing provider satisfaction, reimbursement accuracy, and efficiency in a regulated environment.
Manual audits sampled only 500 interactions monthly on a legacy platform, forcing QA teams to spend 30 minutes per transcript scoring rigid checklists, which resulted in inconsistent interpretations of key areas, or core parameters, such as legal/regulatory, experiential quality, procedural adherence, and compliance standards.
This process left massive gaps in spotting trends, errors, and risks across full volumes, causing delayed coaching, compliance threats with fines and reputational damage, and inability to scale amid rising interactions. Leadership lacked insights for targeted improvements like response templates or training.
Action
Sagility addressed these pain points with SensAI, an AI-driven automated audit platform using finetuned Large Language Models (LLMs), an LLM orchestrator, and Azure cloud services for scalable processing. Hosted on Azure, SensAI ingests domain-specific Salesforce Provider Portal chat data, outperforming general models via targeted finetuning on anonymized historical audits to grasp healthcare jargon, acronyms, and rubrics.
Development involved partnering with QA experts to define the core parameters listed above. SensAI’s interactive UI lets auditors review LLM predictions, metadata, and scores across 19 areas, processing ~25,000 monthly interactions in batches. It automates parsing, multi-parameter scoring, and anomaly detection (e.g., non-compliant language).
Impact
SensAI revolutionized quality assurance, turbocharging audit precision across the healthcare plan’s high-volume chat operations. The interactive UI enables auditors to collaborate seamlessly with LLM predictions and metadata, scoring agents with unmatched reliability validated across 19 critical parameters.
Highlights Include:
- 97-100% accuracy on across key areas
- 81-84% on website promotions, recaps, and self-service education
- 66% on excessive hold time
- 90% average accuracies, minimizing false positives for reliable insights
Business Transformation Results Were Also Dramatic:
- 25,000 monthly interactions (50x scale) exploded from 500
- 60% audit handling time reduced
AI-powered precision identified error patterns, slashed compliance risks, boosted provider satisfaction scores, and enabled scalability. SensAI eliminated audit bottlenecks, delivering actionable intelligence that drove competitive dominance for this plan in healthcare’s high-stakes arena.
(50x scale) exploded from 500
