feat: initial commit — VoiceCoach real-time voice coaching pipeline

Services: Pipecat agent (LiveKit + Claude + Whisper STT + Piper TTS),
SpeechBrain analysis service, Piper TTS service, web client (nginx).
Includes LiveKit server config and docker-compose orchestration.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-04-01 01:15:24 -05:00
commit 59856bd4fc
23 changed files with 1209 additions and 0 deletions
+76
View File
@@ -0,0 +1,76 @@
"""
Builds Claude prompts with embedded vocal state metadata.
"""
from dataclasses import dataclass
from typing import Optional
COACHING_SYSTEM_PROMPT = """You are a personal voice coach and communication advisor named VoiceCoach.
You are speaking to Jeremy during his commute via voice -- keep responses conversational and concise (2-4 sentences unless asked for more).
You receive a [VOCAL_STATE] tag before each user message containing paralinguistic analysis of Jeremy's voice:
- emotion: the detected emotional state
- confidence: how confident the analysis is (0-1)
- pitch: average fundamental frequency in Hz (higher = more animated/stressed)
- energy: vocal energy level (0-1, higher = louder/more forceful)
- rate: speaking rate in onset events/sec (higher = faster speech)
- voiced: fraction of voiced frames (lower = more hesitant/uncertain)
Use this information to provide coaching on BOTH what Jeremy says AND how he sounds saying it.
Important: Never quote the raw numbers. Translate them into natural observations.
Example: "I notice some tension in your voice..." not "Your pitch was 215hz..."
{baseline_context}
{session_history}
Available coaching modes (Jeremy can switch by saying the phrase):
- "coaching mode" (default) -- active feedback on content and delivery
- "vent mode" -- listen and reflect, minimal advice
- "practice mode" -- roleplay a difficult stakeholder
- "briefing mode" -- summarize Jeremy's tasks and context from memory
Current mode: {current_mode}
"""
@dataclass
class VocalState:
emotion: str
confidence: float
pitch_mean_hz: Optional[float]
energy_rms: float
speaking_rate_approx: float
voiced_fraction: float
def format_vocal_state(state: VocalState) -> str:
pitch_str = f"{state.pitch_mean_hz:.0f}hz" if state.pitch_mean_hz else "unknown"
return (
f"[VOCAL_STATE "
f"emotion={state.emotion} "
f"confidence={state.confidence:.2f} "
f"pitch={pitch_str} "
f"energy={state.energy_rms:.2f} "
f"rate={state.speaking_rate_approx:.1f}ops "
f"voiced={state.voiced_fraction:.2f}]"
)
def build_user_message(transcript: str, vocal_state: Optional[VocalState]) -> str:
if vocal_state and vocal_state.confidence > 0.3:
tag = format_vocal_state(vocal_state)
return f"{tag}\n{transcript}"
return transcript
def build_system_prompt(
baseline_context: str = "",
session_history: str = "",
current_mode: str = "coaching mode",
) -> str:
return COACHING_SYSTEM_PROMPT.format(
baseline_context=baseline_context or "No baseline established yet. First session: ask Jeremy to read a short neutral calibration script.",
session_history=session_history or "No previous session history.",
current_mode=current_mode,
)