Documentation
¶
Overview ¶
Package capability 是引擎模型能力探测的可复用核心.
它把原 cmd/capability-probe 的纯逻辑 (向 provider/model 发最小化探测 请求, 实测 streaming / thinking / tool_use / structured_output / caching / schema_ref / max_tools, 给每条能力打来源戳, 输出 ModelCapabilities 矩阵) 抽成包, 供 server / registry / pricing 等 引擎内消费者复用. cmd/capability-probe 退成薄 CLI 壳.
Package capability is the reusable core of the engine's model capability probing. It extracts the pure logic from the original cmd/capability-probe (sending minimal probe requests to a provider/model to empirically test streaming / thinking / tool_use / structured_output / caching / schema_ref / max_tools, stamping each capability with its Source, and emitting a ModelCapabilities matrix) into a package reusable by in-engine consumers (server / registry / pricing). cmd/capability-probe becomes a thin CLI wrapper.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func IsFullyProbed ¶
func IsFullyProbed(mc *ModelCapabilities) bool
IsFullyProbed 是 isFullyProbed 的公共导出版 (缓存 / skip 消费者用的完整性检查).
IsFullyProbed is the exported wrapper of isFullyProbed (a completeness check for cache / skip consumers).
Types ¶
type Capability ¶
type Capability struct {
Value any `json:"value"`
Source Source `json:"source"`
// Exhaustive 表示 Value 是否是穷尽测试得到的真上限.
// 升华改进(ELEVATED): 原先 max_tools 这种字段写 value=128 时,下游消费者无法区分
// "测了 128 都过(其实可能更多)"和"在 129 处确认拒绝"这两种语义.
// nil = 概念不适用(如 streaming/thinking 这种 bool 字段);
// false = 测试未触顶,Value 是已知下界,真值可能更大;
// true = 测试穷尽,Value 是确认上限.
// 替代方案:<把 value 写成 null + 文本 note 描述> - 否决:文本不可结构化解析,
// 未来 loader/registry/UI 等程序消费者要正则提取数字才能用,违背"数据驱动行为"原则.
Exhaustive *bool `json:"exhaustive,omitempty"`
Evidence map[string]any `json:"evidence,omitempty"`
Note string `json:"note,omitempty"`
}
Capability 是一个带来源标签的能力字段.
精妙之处(CLEVER): Value 用 any 而非泛型-- 一个能力可能是 bool(streaming),int(context_window),float64(price), string(note),统一成 any 让 JSON 序列化自然处理类型差异. Evidence 是可选的原始证据(如 token 计数,错误信息,API 响应摘要), 让下游能重现 probe 的判断过程.
type CapabilityReport ¶
type CapabilityReport struct {
SchemaVersion string `json:"schema_version"`
GeneratedAt string `json:"generated_at"`
Models map[string]*ModelCapabilities `json:"models"`
}
CapabilityReport 是整个 probe 运行输出的聚合报告.
type CapabilityResult ¶
type CapabilityResult struct {
Provider string
Model string
Streaming tristate
Thinking tristate
ToolUse tristate
StructuredOut tristate
Caching tristate
SchemaRef tristate
ToolCount int // 实际可处理工具数,0 表示未探测或探测失败
ToolCountExhaustive bool // true=Value 是确认上限;false=测试未触顶,Value 是已知下界
ToolCountNote string // 探测诊断信息
MaxOutputTokens int // 单次响应实测输出 token 数 (含 thinking + text + tool_use), 0 表示未探测或探测失败
MaxOutputTokensExhaustive bool // true=stop_reason=max_tokens/length 触底; false=stop_reason=end_turn/stop_sequence model 自停, 实际上限可能 ≥ Value
MaxOutputTokensNote string // 探测诊断信息
SchemaFeatures map[string]bool // L1174: schema 特性 → 是否被 API 接受
// ADR-0007 capability tracking 实测三件套 (TD-20).
//
// ToolNameRegex: 模型实际接受的工具名 regex 实证. 当前 prober 只测一个
// 触发字符 (`.`), 输出 "strict" (拒含 `.`) / "permissive" (接受) / 空 (未测).
// 完整 regex inference 留 follow-up — 单点探测足以暴露 r22 类业务 bug.
//
// ReasoningPassbackMode: 模型在多轮 tool calling 中是否要求 prior assistant
// 的 reasoning_content 在下一轮 messages 中 passback. "string" / "none" / 空.
// 实证方法: 2 round-trip, round 2 不回传 reasoning_content, 看是否 4xx.
//
// ProviderKind: 静态标 — 注册时按 target.providerKind 直接落到 result,
// 不实测 (direct vs aggregator 是注册期人类知识, 不需 API 调用).
//
// ADR-0007 capability tracking probed triplet (TD-20).
// ToolNameRegex: empirical evidence of the model's accepted tool-name
// pattern. Current prober tests one trigger char (`.`) and reports
// "strict" (rejects `.`) / "permissive" (accepts) / empty (untested).
// Full regex inference deferred — single-point probe already covers
// the r22-class business bug surface.
// ReasoningPassbackMode: whether the model requires the prior
// assistant's reasoning_content to be echoed in the next request
// during multi-turn tool calling. "string" / "none" / empty.
// ProviderKind: registration-time static label — never probed (direct
// vs aggregator is human knowledge, no API call needed).
ToolNameRegex string
ToolNameRegexNote string
ReasoningPassbackMode string
ReasoningPassbackNote string
ProviderKind string // 静态来自 target.providerKind, 无 prober
Notes []string
}
CapabilityResult 是单个 provider+model 的能力探测结果.
type ModelCapabilities ¶
type ModelCapabilities struct {
Provider string `json:"provider"`
Model string `json:"model"`
ContextWindow Capability `json:"context_window"`
MaxOutputTokens Capability `json:"max_output_tokens"`
InputPricePer1M Capability `json:"input_price_per_1m"`
OutputPricePer1M Capability `json:"output_price_per_1m"`
CacheReadPricePer1M Capability `json:"cache_read_price_per_1m,omitempty"`
CacheWritePricePer1M Capability `json:"cache_write_price_per_1m,omitempty"`
Streaming Capability `json:"streaming"`
Thinking Capability `json:"thinking"`
ToolUse Capability `json:"tool_use"`
StructuredOut Capability `json:"structured_output"`
Caching Capability `json:"caching"`
SchemaRef Capability `json:"schema_ref"`
MaxTools Capability `json:"max_tools"`
Vision Capability `json:"vision"`
PDF Capability `json:"pdf"`
Batch Capability `json:"batch"`
ParallelToolCalls Capability `json:"parallel_tool_calls"`
StrictJSON Capability `json:"strict_json"`
// ADR-0007 capability tracking 三件套 (TD-20).
// 不纳入 isFullyProbed (避免 ADR-0007 之前的缓存全部失效, 让新字段
// 渐进 backfill 而非 hard-cutover).
//
// ADR-0007 capability tracking triplet (TD-20). Not included in
// isFullyProbed to keep pre-ADR-0007 caches valid; new fields
// backfill incrementally rather than hard-cutover.
ToolNameRegex Capability `json:"tool_name_regex,omitempty"`
ReasoningPassbackMode Capability `json:"reasoning_passback_mode,omitempty"`
ProviderKind Capability `json:"provider_kind,omitempty"`
// SchemaFeatures 是 JSON Schema 特性的详细支持数据 (L1174).
// 不纳入 isFullyProbed 7 字段 -- 补充探测数据, 已缓存 target 需 --force 重跑.
SchemaFeatures map[string]Capability `json:"schema_features,omitempty"`
ProbedAt string `json:"probed_at"`
ProbeErrors []string `json:"probe_errors,omitempty"`
}
ModelCapabilities 是一个 provider+model 组合的完整能力画像.
字段分三组:
- 基础规格(Context/Tokens/Price):从 provider.Models() 查,标 SourceDocumented
- 可实测能力(Streaming/Thinking/Tool/...):probe 结果,标 SourceProbed
- 文档能力(Vision/PDF/Batch/Strict/Parallel):暂不实测,标 SourceDocumented/Untested
func ProbeModel ¶
ProbeModel 跑完整 probe + merge 流水线, 返回一个 model 的 probed + Models() + documented 合并能力画像.
它从 opts 构造内部 target, 运行 probe() + buildModelCapabilities, 返回 *ModelCapabilities. probe() 本身从不 hard-fail (诊断记进 Note 后继续), 故 error 仅保留给 nil-Provider 校验 / ctx 取消.
ProbeModel runs the full probe + merge pipeline for one model and returns the merged probed + Models() + documented capability picture. It builds the internal target from opts, runs probe() + buildModelCapabilities, and returns *ModelCapabilities. probe() itself never hard-fails (it records diagnostics in Notes and continues), so error is reserved for nil-Provider validation / ctx cancellation.
type ProbeOpts ¶
type ProbeOpts struct {
// ProviderName 是结果标注用的 provider 名 (如 "anthropic").
// ProviderName is the provider label for results (e.g. "anthropic").
ProviderName string
// Provider 是必需的基础 provider 句柄.
// Provider is the REQUIRED base provider handle.
Provider flyto.ModelProvider
// ThinkingProvider 是配置了 ThinkingBudget 的实例; nil -> 跳过 thinking.
// ThinkingProvider is a ThinkingBudget-configured instance; nil -> skip thinking.
ThinkingProvider flyto.ModelProvider
// CachingClient 是直连 internal/transport 的 client (Anthropic / MiniMax
// cache_control 直路); nil -> 跳过该路径.
// CachingClient is a direct internal/transport client for the
// Anthropic / MiniMax cache_control path; nil -> skip that path.
CachingClient *api.Client
// CachingProvider 是 EnableCaching=true 实例 (OpenRouter) 或同一实例
// (DeepSeek long-prefix 阶梯); nil -> 走 generic 路径.
// CachingProvider is an EnableCaching=true instance (OpenRouter) or the
// same instance (DeepSeek long-prefix ladder); nil -> generic path.
CachingProvider flyto.ModelProvider
// ProviderKind 是静态 "direct" / "aggregator" 标签.
// ProviderKind is the static "direct" / "aggregator" label.
ProviderKind string
// --- 成本控制: 贵探针可跳过 (见 target 注释) ---
// Cost controls: the expensive probes can be skipped (see target docs).
MaxProbeTools int // 0 -> 默认 128 (替换原写死值). 0 -> default 128 (replaces hardcode).
SkipCaching bool
SkipMaxOutputTokens bool
SkipToolCount bool
SkipReasoningPassback bool
SkipSchemaFeatures bool
}
ProbeOpts 携带 probe 需要的四个 provider 句柄 + 静态 providerKind 标签 + per-probe skip 控制.
升华改进(ELEVATED): 单 provider 签名不足 — caching 与 thinking 需各自 单独配置的实例 (ThinkingBudget / EnableCaching / cache_control 直路). 把四个句柄折进 ProbeOpts, 既保留 cmd 原有的 key-dependent 构造灵活性, 又给 server 等消费者一个干净的公共入口.
ProbeOpts carries the four provider-ish handles the probes need plus the static providerKind label and per-probe skip controls. A single provider signature is insufficient -- caching and thinking each need separately configured instances (ThinkingBudget / EnableCaching / the cache_control direct path). Folding the four handles into ProbeOpts keeps cmd's key-dependent construction flexibility while giving server-side consumers a clean public entry.