// serverside_multiscope is a reference wiring example for server-side // consumers that run the engine's Dream consolidation across many isolated // scopes -- e.g. a WeChat customer-service backend where each chat group // (or tenant, or account) gets its own private memory and its own Dream // loop. It shows the seam the engine provides (ScopeRoot + SessionProvider // + Dream().CheckAndRun) and, crucially, the part the consumer must build // itself (a central scheduler with a global concurrency cap so that a few // hundred groups do not fan out into a few hundred simultaneous LLM calls). // // This file uses a zero-API-key mock provider and an in-memory mock session // provider, so it compiles and runs the full structure without ever calling // a real LLM. A real deployment swaps the two mocks for a real provider // (anthropic.New / openai.New / ...) and a DB-backed SessionProvider. // // What the engine gives you (neutral, per-scope primitives): // - Config.ScopeRoot -- a single isolation anchor; memory lands under // /memory and Dream state under /dream.{lock,state}. // - engine.SessionProvider -- ListSince(t) returns the scope's session IDs // touched since t; the engine has no idea where they come from. // - engine.Dream().CheckAndRun(ctx) -- a per-scope hook that fires Dream // consolidation asynchronously IF its internal gates are satisfied. // // What the consumer must build (not in the engine, on purpose): // - the DB query behind SessionProvider.ListSince, // - the central scheduler: which scopes to poll, how often, and the global // concurrency cap that prevents an LLM fork storm, // - the scope_id data model, and GC of idle scopes. // // Run: // // cd core && go run ./examples/serverside_multiscope/ // // serverside_multiscope 是服务端消费者跨多个隔离 scope 跑引擎 Dream 巩固的 // 接线参考 -- 例如微信客服后端, 每个客服群 (或租户, 或账号) 有自己私有的 // memory 和自己的 Dream loop. 它展示引擎给的 seam (ScopeRoot + SessionProvider // + Dream().CheckAndRun), 以及消费者必须自建的部分 (一个带全局并发上限的 // 中央调度器, 防止几百个群同时 fan out 成几百个并发 LLM 调用). // // 本文件用零 API key 的 mock provider 加内存 mock session provider, 所以编译 // 和运行整个结构都不会真调 LLM. 真实部署把两个 mock 换成真 provider // (anthropic.New / openai.New / ...) 加 DB-backed SessionProvider. package main import ( "context" "fmt" "log" "os" "path/filepath" "sync" "time" "git.flytoex.net/yuanwei/flyto-agent/core/pkg/engine" "git.flytoex.net/yuanwei/flyto-agent/core/pkg/execenv" "git.flytoex.net/yuanwei/flyto-agent/core/pkg/flyto" ) // ── Block 1: mockProvider -- a zero-API-key flyto.ModelProvider ─────────────── // mockProvider is a minimal flyto.ModelProvider that satisfies engine.New's // non-nil Provider check and returns one stub model. Stream returns an // immediately-closed channel, so any code path that "talks to the LLM" simply // sees an empty event stream and finishes -- no network, no API key. This // mirrors the stubProvider pattern used in the engine's own tests. // // mockProvider 是最小 flyto.ModelProvider, 满足 engine.New 的非 nil Provider // 校验, 返回一个 stub 模型. Stream 返回一个立即 close 的 channel, 所以任何 // "跟 LLM 对话" 的代码路径只会看到空 event 流然后结束 -- 无网络, 无 API key. // 这与引擎自身测试里的 stubProvider 模式一致. type mockProvider struct{} func (mockProvider) Name() string { return "mock" } func (mockProvider) Models(_ context.Context) ([]flyto.ModelInfo, error) { return []flyto.ModelInfo{{ ID: "stub-model", Provider: "mock", ContextWindow: 200000, MaxOutputTokens: 64000, ProviderKind: "direct", ToolNameRegex: "^[a-zA-Z0-9_-]+$", }}, nil } func (mockProvider) Stream(_ context.Context, _ *flyto.Request) (<-chan flyto.Event, error) { ch := make(chan flyto.Event) close(ch) return ch, nil } // ── Block 2: mockDBSessionProvider -- the SessionProvider seam ──────────────── // mockDBSessionProvider implements engine.SessionProvider for one scope. Here // it returns a fixed in-memory slice of session IDs; in a real server-side // deployment this method runs a per-scope DB query, e.g.: // // SELECT session_id FROM sessions WHERE scope_id = $1 AND updated_at > $2 // // (with scopeID bound to $1 and sinceTime to $2). The engine never sees the // query -- it only calls ListSince and consumes the returned IDs, so the // storage backend (Postgres, Mongo, an internal API, ...) is entirely the // consumer's choice. // // mockDBSessionProvider 为单个 scope 实现 engine.SessionProvider. 这里它返回 // 一个固定的内存 session ID 切片; 真实服务端部署里这个方法跑一条 per-scope // 的 DB 查询, 例如: // // SELECT session_id FROM sessions WHERE scope_id = $1 AND updated_at > $2 // // (scopeID 绑 $1, sinceTime 绑 $2). 引擎从不看这条查询 -- 它只调 ListSince // 然后消费返回的 ID, 所以存储后端 (Postgres / Mongo / 内部 API / ...) 完全 // 由消费者自己选. type mockDBSessionProvider struct { scopeID string } func (m *mockDBSessionProvider) ListSince(sinceTime time.Time) ([]string, error) { // Real impl: run the SELECT above and return the rows. The mock ignores // sinceTime and returns two synthetic IDs so the wiring is observable. // // 真实实现: 跑上面的 SELECT 返回行. mock 忽略 sinceTime, 返回两个合成 ID // 让接线可观测. _ = sinceTime return []string{ m.scopeID + "-sess-1", m.scopeID + "-sess-2", }, nil } func main() { // dataDir is the shared Cwd for every scope's engine. Per-scope state is // kept disjoint via ScopeRoot, NOT via Cwd -- Cwd stays shared. // // dataDir 是每个 scope 引擎共享的 Cwd. per-scope 状态靠 ScopeRoot 互相 // 隔离, 而非靠 Cwd -- Cwd 保持共享. dataDir, err := os.MkdirTemp("", "serverside-multiscope-") if err != nil { log.Fatalf("mkdir temp: %v", err) } defer os.RemoveAll(dataDir) fmt.Printf("data dir: %s\n", dataDir) ctx := context.Background() // ── Block 3: build one engine per scope, each with its own ScopeRoot ────── // // We simulate 3 customer groups. In production this list comes from the // scope_id data model (one row per active chat group / tenant / account) // and is refreshed continuously; idle scopes get GC'd by the consumer. // // 我们模拟 3 个客户群. 生产环境这个列表来自 scope_id 数据模型 (每个活跃 // 客服群 / 租户 / 账号一行), 并持续刷新; 闲置 scope 由消费者 GC. groupIDs := []string{"group-a", "group-b", "group-c"} engines := make(map[string]*engine.Engine, len(groupIDs)) for _, groupID := range groupIDs { // scopeRoot is the per-scope isolation anchor. memory -> /memory, // Dream lock + state -> /dream.lock + dream_state.json. // // scopeRoot 是 per-scope 隔离锚点. memory 落 /memory, Dream // 的 lock 加 state 落 /dream.lock 加 dream_state.json. scopeRoot := filepath.Join(dataDir, "scopes", groupID) if err := os.MkdirAll(scopeRoot, 0o755); err != nil { log.Fatalf("mkdir scope root %s: %v", scopeRoot, err) } eng, err := engine.New(&engine.Config{ // 4 required fields. // 4 个必填字段. Provider: mockProvider{}, Model: "stub-model", Cwd: dataDir, Executor: execenv.DefaultExecutor{}, // Per-scope seam: isolation anchor + this scope's session source. // per-scope seam: 隔离锚点 + 本 scope 的 session 源. ScopeRoot: scopeRoot, SessionProvider: &mockDBSessionProvider{scopeID: groupID}, }) if err != nil { log.Fatalf("engine.New for %s: %v", groupID, err) } // Every engine owns OS resources (file handles, the Dream lock); close // them all when main returns. // // 每个引擎持有 OS 资源 (文件句柄, Dream 锁); main 返回时全部关闭. defer eng.Close() engines[groupID] = eng } printSection(fmt.Sprintf("Block 3: built %d per-scope engines", len(engines))) // ── Block 4: central scheduler skeleton (the consumer-side part) ────────── // // The engine gives a PER-SCOPE hook (engine.Dream().CheckAndRun). It does // NOT decide which scopes to poll, how often, or how many may run at once. // That orchestration is the consumer's job, because only the consumer knows // its fleet size and its LLM budget. Without a global cap, polling N scopes // would let up to N Dream consolidations -- and thus up to N LLM calls -- // fire at the same instant: a fork storm. The semaphore below is that cap. // // 引擎给的是 PER-SCOPE 钩子 (engine.Dream().CheckAndRun). 它不决定该轮询 // 哪些 scope, 多久一次, 同时能跑几个. 这个编排是消费者的活, 因为只有消费者 // 知道自己的舰队规模和 LLM 预算. 没有全局上限, 轮询 N 个 scope 会让多达 N // 个 Dream 巩固 -- 也就是多达 N 个 LLM 调用 -- 同一瞬间 fire: fork storm. // 下面这个 semaphore 就是那个上限. const globalConcurrencyCap = 2 sem := make(chan struct{}, globalConcurrencyCap) var wg sync.WaitGroup for groupID, eng := range engines { wg.Add(1) go func(groupID string, eng *engine.Engine) { defer wg.Done() sem <- struct{}{} // acquire a global slot / 占一个全局名额 defer func() { <-sem }() // release / 释放 // CheckAndRun checks its internal gates (session count, time since // last scan, multi-process file lock) and only then async-fires // Dream. It returns quickly either way -- it is safe to call on a // schedule. NOTE: in this mock harness the gates are NOT tripped // (no real sessions recorded, no time elapsed), so CheckAndRun // returns immediately and no Dream actually runs. On a real server // with traffic and elapsed time it would async-fire Dream, which is // where the mockProvider would be replaced by a real LLM. // // CheckAndRun 检查它的内部门槛 (session 数 / 距上次扫描时间 / 多进程 // 文件锁), 满足才异步 fire Dream. 无论如何都快速返回 -- 可以安全地 // 按计划调度. 注意: 在这个 mock harness 里门槛没被触发 (没记真 session, // 没经过时间), 所以 CheckAndRun 立即返回, 没有 Dream 真正运行. 真实 // 服务端有流量加时间流逝时它会异步 fire Dream, 那里 mockProvider 会被 // 换成真 LLM. eng.Dream().CheckAndRun(ctx) fmt.Printf(" scheduled CheckAndRun for %s (gates checked, mock = no-op)\n", groupID) }(groupID, eng) } wg.Wait() printSection(fmt.Sprintf("Block 4: dispatcher polled all scopes (global cap=%d)", globalConcurrencyCap)) // ── Block 5: verify isolation -- each scope's paths are disjoint ────────── // // memory lives at /memory; Dream state lives directly under // . Printing them shows the three scopes never share a directory. // // memory 在 /memory; Dream state 直接在 下. 打印出来 // 可见三个 scope 从不共享目录. printSection("Block 5: per-scope isolation (disjoint paths)") for _, groupID := range groupIDs { scopeRoot := filepath.Join(dataDir, "scopes", groupID) memDir := filepath.Join(scopeRoot, "memory") fmt.Printf(" %-8s scopeRoot=%s\n", groupID, scopeRoot) fmt.Printf(" %-8s memoryDir=%s\n", "", memDir) } fmt.Println() fmt.Println("Wiring complete. Engine seam exercised:") fmt.Println(" Config.ScopeRoot + engine.SessionProvider + engine.Dream().CheckAndRun") fmt.Println("Consumer-built (swap in for production):") fmt.Println(" DB-backed SessionProvider + scheduler cadence/cap + scope_id model + idle GC") } func printSection(title string) { fmt.Println() fmt.Printf("-- %s --\n", title) }