Intelligent AI Assistant MVP Development for Multidisciplinary Tumor Boards

• 60%+ reduction in documentation time
• Literature lookup in seconds instead of minutes
• Improved decision traceability across multidisciplinary teams
• Analytics-driven governance that strengthens compliance and care quality
• A scalable architecture ready for EMR, research registry, and analytics integrations

Multidisciplinary tumor boards bring specialists together to review patient cases, align on best practices, and make informed treatment decisions while ensuring consistent standards of care. Our client aimed to reduce administrative load like manual documentation, literature research, and post-meeting analysis, and improve decision quality, supporting equitable, continuous care across teams and institutions.

The envisioned solution needed to capture multi-speaker discussions in real time, integrate federated medical literature, generate structured summaries, and ensure PHI compliance. Our team faced the challenge of transforming these requirements into a streamlined, low-latency, highly secure software solution capable of scaling across hospital systems.

• Capture and structure multidisciplinary discussions to reduce documentation time.
• Integrate evidence-based literature in real time, ensuring immediate access to the latest guidelines and clinical trials.
• Automate summaries, notes, and analytics to accelerate decision-making and support institutional learning.
• Ensure compliance with PHI, consent, and auditability within clinical environments.
• Promote continuity and supportive care, aligned with oncology quality frameworks such as SECON, PCAT, and EORTC.

We delivered a modular Proof-of-Concept, working closely with oncology experts to capture, structure, and summarize tumor board discussions in real time.

The system leveraged Microsoft Teams, Azure AI, and federated Retrieval-Augmented Generation (RAG) to connect transcription, embeddings, summarization, and literature retrieval in a single workflow. Features were prioritized using a MoSCoW matrix, focusing first on real-time transcription, structured summaries, and literature integration, with predictive surfacing and trial matching designed for future expansion.

A governance-first mindset guided development, embedding audit trails, PHI redaction, and human-in-the-loop validation to ensure transparency, compliance, and clinical trust.

The ChaturAI PoC comprised seven major capabilities, prioritized via a MoSCoW matrix.

The solution was built as a modular, Azure-native platform that connects multiple services and data layers in a streamlined workflow. Its architecture supports real-time data flow from audio capture to storage, embedding, summarization, and literature retrieval, with end-to-end observability, audit logging, and low-latency performance.

By structuring the system into independent, interoperable modules, it ensures scalability, maintainability, and the ability to extend functionality, such as predictive surfacing and trial matching, without disrupting core operations. The following choices highlight the key technologies and design decisions that bring this architecture to life.

• Bot + Graph API – integrates seamlessly with Teams meetings for audio capture.
• Media Workers on AKS – handle audio normalization, voice activity detection (VAD), and transcription at scale.
• Event Hubs + PostgreSQL + Blob Storage – support streaming, persistence, and real-time access to transcripts.
• Web PubSub – provides live transcript updates in the Teams sidebar.
• PHI masking with Azure AI Language – ensures compliance during data export.
• Latency (P95) – end-to-end processing completes in 0.75–1.37 seconds, supporting real-time workflows.

The solution functionality is delivered through a set of interconnected technical components, each designed to handle a specific part of the tumor board workflow. This section breaks down the key components, explaining how data flows through the platform and how each module contributes to the overall solution.

The bot passively joins meetings to capture in-room and virtual audio from multiple speakers. Media Workers on AKS handle normalization, noise reduction, and voice activity detection before transcription with Azure AI Speech (Medical).

PHI is automatically redacted before storage in Cosmos DB and Blob Storage, and Web PubSub streams live updates to the Teams sidebar. The system achieves end-to-end latency (P95) of 0.75–1.37 seconds and supports multi-tenant isolation for secure, scalable deployment.

This layer converts finalized utterances into versioned vector embeddings, supporting real-time retrieval and governance. Finalized transcripts are transformed into versioned vector embeddings to support fast, semantic search and retrieval. Real-time embeddings use text-embedding-3-small for immediate lookup, while post-meeting transcripts are chunked and processed with text-embedding-3-large for batch indexing.

Embeddings are stored in Azure AI Search (HNSW) with Redis caching for low-latency access, while the original and processed transcripts persist in PostgreSQL and Blob Storage. The system maintains rollback-safe indexing, lineage tracking, and audit-ready logs, ensuring both performance and governance across hospital tenants.

The Synopsis module compresses meeting transcripts into structured, review-ready summaries. It operates in two modes: post-meeting summaries, which generate concise reports within two minutes of the meeting’s end, and in-meeting auto-fill (beta), which updates summaries in near real-time within five seconds of each finalized speech-to-text event.

The module achieves over 50% compression, ≥90% decision recall, and an average completeness rating above 4/5, with full PHI redaction and audit trails for compliance. Its modular, microservices architecture ensures scalability, allowing future sources and features to be added without disrupting core workflows.

Contextual queries are generated for PubMed, PMC, and ClinicalTrials.gov. Federated retrieval is handled by microservice agents, with results ranked by BM25, embeddings, and recency. Top literature is surfaced in under 10 seconds via Teams UI cards. Performance goals included generating contextual queries in ≤400 ms, retrieving the top 10 results in under 5 seconds, and surfacing the full result from utterance to Teams card within 10 seconds.

Features:

• Contextual query generation with synonyms and relevance weighting.
• Federated retrieval via source agents (PubMed, ClinicalTrials) each hosted on AKS endpoints.
• Real-time progress streaming via Service Bus → WebPubSub → Teams UI cards.
• Inline surfacing in Teams.

Human-in-the-loop validation ensures transparency, and audit trails capture all system actions, supporting compliance and governance requirements across hospital systems.

All storage and processing are partitioned by tenant to maintain strict isolation. Access is controlled via Entra ID (Azure AD) SSO with per-tenant RBAC, and workloads run in namespaced AKS clusters with separate Cosmos DB and Blob Storage containers for each tenant.

• Participant consent banners and audit-tracked exports.
• PHI minimization through layered redaction and data partitioning.
• Zero PHI in vector indexes.
• Human-in-the-loop oversight for summaries and decisions.
• Equity audits on trial matching and literature sourcing.

The solution analytics layer monitors both system performance and clinical process outcomes. FHIR-compatible event schemas capture actions like meeting start, synopsis creation, decision logging, and exports, enabling end-to-end observability.

Dashboards provide insights into latency, transcript relevance, decision recall, trial quality, and time-to-finalization, while also tracking care continuity and quality. Key clinical KPIs include referral completeness, cross-team handoffs, patient satisfaction (PCAT, PCCQ), and supportive care coverage across physical, emotional, social, practical, informational, and spiritual domains.

The system identifies barriers such as delays, fragmentation, and coordination gaps, supporting data-driven improvements in oncology workflows.

Building on the successful pilot, the SaaS roadmap outlines the next phases for expanding functionality, improving clinical impact, and preparing the platform for enterprise-scale deployment. Each phase introduces new capabilities while maintaining security, compliance, and scalability.

• Phase II – Fusion Engine: Integrate EMR/FHIR data to enable automated, context-aware treatment recommendations.
• Phase III – Trial Matching: Enhance precision of clinical trial matching and improve interoperability across institutions.
• Phase IV – Predictive Surfacing: Preemptively surface relevant literature and references before meetings to support decision-making.
• Phase V – Enterprise Rollout: Deploy a SOC2-compliant, multi-region system with advanced governance and full scalability.
• Phase VI – Expansion. Zoom integration.