The Challenge
Phonetik.ai wanted to extend EventEngine with a new capability: reliably determining who is speaking in a piece of video content, whether that speaker is on screen or off screen, and doing so with a measurable confidence score. Rather than build this directly into their production platform, Phonetik needed to validate the approach first, testing whether active speaker detection and attribution could be layered on top of their existing transcription, diarization and face-tracking pipeline before committing to a wider rollout across EventEngine.
This meant the engagement had to answer a specific question: can a new AI-driven attribution layer be introduced without disrupting Phonetik’s live platform, and can it meet the accuracy and confidence thresholds needed for real editorial workflows such as captioning and dialogue sheets?
The Solution
Silicon Overdrive designed and built a standalone MVP AWS environment, provisioned independently of Phonetik’s production footprint, to prove out active speaker detection and attribution end to end. The engagement began with architecture and design work, validating Phonetik’s existing platform and defining the target MVP workflow from transcription and diarization through face tracking, active speaker detection and identity resolution, including the decision model for on-screen/off-screen handling, confidence gating, and unknown-speaker or failure paths.
On the infrastructure side, the entire MVP environment was provisioned using Terraform for repeatability, covering IAM access patterns, Amazon S3 integration points, AWS Lambda execution paths, Amazon SQS queue interactions and Amazon CloudWatch monitoring. For the machine learning component, an active speaker detection model was deployed on Amazon SageMaker in batch mode, with a defined input/output contract, batch invocation pattern and confidence scoring for downstream attribution.
Amazon Bedrock AgentCore was then configured as the GenAI orchestration and reasoning layer, handling confidence assessment, exception handling and on-screen/off-screen decisions, with AgentCore Gateway registering tools for active speaker detection, metadata services and Phonetik’s own workflow services via Lambda and APIs.
Observability and handover were built in from the start, with Amazon CloudWatch logging, metrics and traceability across the workflow, supported by reference architecture diagrams, deployment guidance and a full handover pack. The team then ran end-to-end validation on agreed sample assets to confirm that Phonetik’s existing transcription, diarization and face-tracking outputs could flow cleanly through the new detection and orchestration layer. Security was addressed throughout, with Trend Vision One Endpoint Security and Cyber/Cloud Risk Exposure Management (CREM), CIS Benchmark hardened instances, and least-privilege IAM access patterns implemented across Lambda, SQS, SageMaker and Bedrock AgentCore, with the environment validated against Trend Vision One CREM.
The Outcome
The agreed sample assets were successfully processed through the existing Phonetik workflow and the new AWS MVP services into speaker attribution outputs for the selected use case, with the MVP producing decisions that met the agreed acceptance thresholds on Phonetik’s validation set. Unknown speakers, off-screen dialogue, overlapping dialogue and low-confidence cases were handled consistently according to the agreed business rules, and downstream outputs, including captions, dialogue sheets and selected accessibility workflows, consumed the attribution results in the expected format, supporting the intended editorial process.
The solution demonstrated a measurable reduction in manual speaker identification effort on validation content compared with Phonetik’s current process, and the responsibility boundary between Silicon Overdrive’s AWS-delivered components and Phonetik’s application components was proven out through successful end-to-end validation and handover.
Francois Schreuder, CEO, Phonetik said, “On the security and compliance side, Trend Vision One compliance validation scored 95–97% across the AWS Well-Architected Framework, NIST Cybersecurity Framework, ISO 27001:2022, HIPAA, PCI DSS and GDPR, meeting the proposal’s 95%+ GDPR/CREM success metric.”
