// transcription.go - the cross-provider audio transcription (speech-to-text) // capability contract. Modeled on vision.go: TranscriptionProvider is // OPTIONAL -- not every ModelProvider implements it. A provider exposes // transcription by implementing Transcribe; consumers type-assert // (provider, ok := p.(flyto.TranscriptionProvider)) and degrade when absent. // Like vision, this is deliberately SEPARATE from the streaming // Provider.Stream path: transcription is a single-shot RPC against an // OpenAI-compatible /v1/audio/transcriptions endpoint (e.g. a self-hosted // oMLX ASR model), not the engine's multi-turn orchestration. // // The request shape is the OpenAI transcription API plus the extension // fields self-hosted servers accept (diarization, long-audio chunking, // word timestamps). The contract stays scenario-neutral (CLAUDE.md rule 9): // speaker labels are caller-supplied strings, nothing here assumes a // call-center. Fields a provider does not support are simply not sent. // // transcription.go - 跨 provider 的语音转写 (语音转文字) 能力契约. 仿 // vision.go: TranscriptionProvider 是**可选**的 -- 不是每个 ModelProvider 都 // 实现. provider 实现 Transcribe 即暴露转写; 消费者 type-assert 并在缺失时 // 降级. 与 vision 一样, 刻意与 streaming Provider.Stream 路径**分开**: 转写 // 是对 OpenAI 兼容 /v1/audio/transcriptions 端点 (如自托管 oMLX ASR 模型) // 的单次 RPC, 不是引擎的多轮编排. // // 请求形状 = OpenAI 转写 API + 自托管服务端接受的扩展字段 (说话人分离 / // 长音频切块 / 词级时间戳). 契约保持场景中性 (CLAUDE.md 原则 9): 说话人标签 // 由调用方传入, 这里不假设呼叫中心. provider 不支持的字段就不发. package flyto import ( "context" "io" ) // TranscriptionRequest is a single-shot audio transcription request. Audio // is a STREAM (io.Reader), not []byte: recordings run to hundreds of MB // (upload caps are server-side, e.g. 300 MB), so the provider pipes it to // the wire without buffering the whole file in memory. Filename hints the // container format to the server (multipart part filename; e.g. "call.wav"). // Model is REQUIRED (per-request selection, mirrors vision). All optional // string/bool fields are omitted from the wire when zero-valued, so the // server's own defaults apply. // // TranscriptionRequest 是单次语音转写请求. Audio 是**流** (io.Reader) 而非 // []byte: 录音可达数百 MB (上限在服务端, 如 300 MB), provider 边读边上线, // 不整文件进内存. Filename 向服务端提示容器格式 (multipart part 文件名, 如 // "call.wav"). Model **必填** (per-request 选, 对齐 vision). 可选字段零值时 // 不上线, 服务端自身默认生效. type TranscriptionRequest struct { Audio io.Reader // raw audio stream (wav/mp3/m4a/...); required Filename string // multipart filename hint; empty -> "audio" Model string // provider model id; required Language string // BCP-47-ish code, e.g. "zh"; empty -> server auto-detect Prompt string // bias context (domain terms, product names) ResponseFormat string // json / verbose_json / text / srt / vtt; empty -> server default (json) // Long-audio handling (server extension; empty/0 -> server default). // LongAudio: "auto" (single pass, chunk on failure) / "chunk" (always // chunk) / "off". ChunkMinutes: target minutes per chunk when chunking. // // 长音频处理 (服务端扩展; 空/0 -> 服务端默认). LongAudio: "auto" (单趟, // 崩了切块救回) / "chunk" (总是切块) / "off". ChunkMinutes: 切块时每块 // 目标分钟数. LongAudio string ChunkMinutes float64 // Diarization + word timestamps (server extension). DiarizeBackend: // "energy_tripass" (stereo, one speaker per channel) / "pyannote" (mono // multi-speaker) / "none". LeftSpeaker/RightSpeaker label the stereo // channels (both required by energy_tripass); caller-supplied strings, // scenario-neutral. // // 说话人分离 + 词级时间戳 (服务端扩展). DiarizeBackend: "energy_tripass" // (立体声, 每声道一人) / "pyannote" (单声道多说话人) / "none". // LeftSpeaker/RightSpeaker 标注立体声左右声道 (energy_tripass 两个都要); // 调用方自定字符串, 场景中性. // WordTimestamps is TRISTATE: nil sends nothing (server default applies), // &true / &false are sent explicitly. Explicit false matters: servers // that default word-level alignment ON hit per-request aligner limits on // long audio (e.g. 270s), and turning it off is the documented escape // hatch -- a plain bool cannot express that. // // WordTimestamps 三态: nil 不上线 (服务端默认生效), &true / &false 显式 // 上线. 显式 false 有实义: 默认开 word 级对齐的服务端在长音频上会撞 // per-request aligner 上限 (如 270s), 关掉它是文档化的出路 -- 普通 bool // 表达不了. WordTimestamps *bool DiarizeBackend string LeftSpeaker string RightSpeaker string // Extra carries forward-compatible extension form fields (e.g. sampling // params) sent verbatim as multipart values. Keys colliding with the // typed fields above are ignored (typed fields win). // // Extra 携带向前兼容的扩展表单字段 (如采样参数), 原样作 multipart 值上线. // 与上方类型化字段撞 key 时忽略 (类型化字段优先). Extra map[string]string } // TranscriptionWord is one word with its start/end time in seconds. // Present only when word-level alignment ran on the server. // // TranscriptionWord 是一个词及其起止秒. 仅服务端跑了 word 级对齐时有. type TranscriptionWord struct { Word string `json:"word"` Start float64 `json:"start"` End float64 `json:"end"` } // TranscriptionSegment is one contiguous transcript span. Speaker is set // only when diarization ran; Words only when word timestamps were asked for. // // TranscriptionSegment 是一段连续转写. Speaker 仅分离时有; Words 仅要了词级 // 时间戳时有. type TranscriptionSegment struct { Start float64 `json:"start"` End float64 `json:"end"` Text string `json:"text"` Speaker string `json:"speaker,omitempty"` Words []TranscriptionWord `json:"words,omitempty"` } // TranscriptionResponse carries the transcript. For json / verbose_json the // structured fields are populated; for text / srt / vtt the raw body lands // in Text and the rest stay zero. // // TranscriptionResponse 携带转写结果. json / verbose_json 填结构化字段; // text / srt / vtt 时原始 body 放 Text, 其余零值. type TranscriptionResponse struct { Text string `json:"text"` Language string `json:"language,omitempty"` Duration float64 `json:"duration,omitempty"` Segments []TranscriptionSegment `json:"segments,omitempty"` } // TranscriptionProvider is the optional audio transcription capability. // Consumers depend on this interface (not a concrete provider) so the ASR // backend is swappable / injectable / mockable. Single-shot + stateless: // safe to share across goroutines. // // TranscriptionProvider 是可选语音转写能力. 消费者依赖此接口 (非具体 // provider), 让 ASR 后端可换 / 可注入 / 可 mock. 单次 + 无状态: 可跨 // goroutine 共享. type TranscriptionProvider interface { Transcribe(ctx context.Context, req *TranscriptionRequest) (*TranscriptionResponse, error) }