TM Evaluator
Assistant

The problem

Toastmasters clubs rely on member-to-member evaluations to help speakers grow. Evaluators listen to a speech and deliver structured verbal feedback. In practice, especially in smaller clubs, evaluators often struggle with:

• Recall — forgetting specific moments from the speech
• Generic feedback — saying "good job" without referencing what was actually good
• Misalignment — not knowing what the speaker's Pathways project was trying to achieve
• On-the-spot pressure — delivering an evaluation within minutes with limited preparation time

Can AI assist evaluators without replacing the human connection that makes Toastmasters valuable?

The guiding principle

AI should elevate the Toastmasters experience, not replace it.

Toastmasters serves three core purposes — community, people connection, and practice. Any AI integration must preserve all three. The tool assists the evaluator. The evaluator still delivers the feedback. The speaker still receives it from a human who was in the room.

Key product decisions

Architecture decisions

Speech-to-text: Browser Web Speech API

Browser Web Speech API was chosen for the prototype phase. It requires zero setup, costs nothing, and keeps the barrier to adoption as low as possible. Accuracy is sufficient because the AI analysis layer is robust enough to work with imperfect transcripts — it looks for patterns and themes, not word-for-word precision.

AI analysis: Claude Sonnet via Anthropic API

The evaluation requires nuanced understanding of Toastmasters culture — constructive framing, Commend, Recommend, Commend (CRC) structure, encouragement-first language. Sonnet provided the best balance of quality, speed (evaluators need results in under 10 seconds), and cost (~$0.03 per evaluation).

Prompt engineering as the core IP

The most valuable component of the system is not the code — it's the system prompt. This prompt encodes:

• General Toastmasters evaluation criteria (structure, vocal variety, language, engagement, timing, purpose)
• The Commend, Recommend, Commend (CRC) framework with specific output structure
• Pathways project-specific objectives dynamically injected based on the evaluator's selection
• Cultural guidelines — warm, encouraging, growth-oriented language

The prompt produces structured JSON output that the frontend renders into scannable talking points.

Security: serverless proxy for API key protection

A browser-based app calling an AI API directly would expose the API key in network requests. A lightweight serverless function sits between the browser and the API — the browser sends the transcript to the function, the function adds the API key (stored as a server-side environment variable) and forwards the request. The API key never touches the browser.

Access control: simple password gate

Rather than building user authentication, the app uses a single club access code. This provides a sense of security and prevents casual visitors from using the API credits, without the complexity of an auth system — appropriate for an experiment with a small, trusted group.

Data flow

Speaker's voice
    ↓
Phone microphone → Browser speech recognition (on-device)
    ↓
Text transcript (browser memory only)
    ↓
Evaluator taps "Done — Analyze"
    ↓
Transcript → Serverless proxy → AI API (encrypted)
    ↓
Structured evaluation (JSON) → Rendered as Commend, Recommend, Commend (CRC) talking points
    ↓
Transcript cleared from memory
    ↓
Tab closed → Everything gone

Key property: No data persists anywhere after the session. No database, no file storage, no cookies, no analytics tracking of speech content.

Pathways knowledge integration

The app contains a structured knowledge base of all six current Toastmasters Pathways with their levels and required projects. When an evaluator selects a pathway, level, and project, the system prompt is dynamically constructed to include the specific objectives for that project.

This means the same underlying AI produces different evaluation criteria for different speech types — an Ice Breaker evaluation focuses on self-introduction and personal storytelling, while a Persuasive Speaking evaluation focuses on evidence quality, emotional appeal, and call to action.

The knowledge base is structured as data (not hardcoded logic), making it straightforward to update when Toastmasters modifies their program.

Deployment and cost model

A $5 API credit sustains a small club for approximately 1.5–2 years. Usage tracking is handled through the hosting platform's built-in function invocation logs — each log entry represents one evaluation and one API call. No custom analytics infrastructure was needed.

Scaling considerations

The serverless architecture scales automatically — there's no server to upgrade. The cost scales linearly with usage. The constraint at scale is organizational (who manages the tool, who pays) rather than technical.

Key takeaways