Resource
Voice fraud detection benchmark methodology and metrics
Sonotheia uses transparent evaluation protocols. We measure spoof detection accuracy using standard scientific metrics—Equal Error Rate (EER) and minimum Detection Cost Function (minDCF)—benchmarked on the ASVspoof5 evaluation partition (spoof attack types A17-A32).
While raw database benchmarks establish baseline detection accuracy, real-world deployment requires blind, in-situ calibration on the partner's actual telephony codec path to control false-positive rates.
ASVspoof5 dataset validation scope
We validate our physics-based sensors against the ASVspoof5 evaluation partition. This benchmark includes advanced spoofing methods such as neural codec laundering, state-of-the-art TTS generation, and voice conversion algorithms. Performance sweeps map out-of-domain generalization limits across different noise and codec conditions.
What we do not claim yet
We do not claim universal, out-of-the-box accuracy below a 2% equal error rate across arbitrary networks. Telephony codecs, room acoustics, and microphone profiles shape the signal. Performance is subject to local channel characteristics and in-situ calibration. Real-time claims apply only to evaluated pilot and demo integrations.
Regulatory compliance and model validation expectations
Interagency guidelines and compliance bodies demand that technology verification remains auditable. The FINRA 2026 Annual Regulatory Oversight Report cautions that firms using AI-driven verification systems must implement vendor due-diligence and model performance audits. The FinCEN deepfake media alert (FIN-2024-DEEPFAKEFRAUD) and the NCUA AI resource point to the necessity of structured validation documentation in vendor risk assessments.
Vendor comparison at a glance
| Evaluation Dimension | Validated Telephony (G.711/AMR-NB) | Out-of-Scope Channels (WebRTC/Zoom) |
|---|---|---|
| Validation Metric | Equal Error Rate (EER) & minDCF tracked | Not validated under active regimes |
| Calibration Method | Blind in-situ calibration sweeps | Standard default thresholds only |
| Generalization Baseline | ASVspoof5 A17-A32 evaluation | Opaque training set assumptions |
| Supervisory Trace | Explainable sensor outputs logged | Opaque score outputs only |
Frequently asked questions
- What is Equal Error Rate (EER)?
- Equal Error Rate (EER) is the rate at which the false acceptance rate matches the false rejection rate. It is the primary metric used in biometric security and liveness verification evaluations.
- Which codecs are included in the validation sweeps?
- Our testing includes wideband audio, G.711 (u-law and a-law), and AMR-NB (narrowband telephony).
- How does calibration protect against false-positives?
- Local sweeps analyze the channel's background noise floor and acoustic characteristics, adjusting sensor thresholds so that normal caller speech does not trigger spoof alerts.
Official regulatory references
Ready to evaluate Sonotheia for your voice channel?
Request a demo