zhizhi-agent-runtime

中文文档

Architecture and package boundaries

Production-grade Go runtime for agents that plan, act, recover, and remain auditable.

Version 1.0.7 adds a realtime duplex speech API, kept separate from ordinary LLM calls, plus a function-calling Doubao adapter. See Doubao realtime speech.

Zhizhi is a model-agnostic execution layer for real applications. It turns ordinary Go functions into validated tools, builds dependency-safe plans, handles conditional branches and bounded replanning, and makes side effects explicit.

Star History

What You Get

• Typed tools with generated JSON schemas and runtime input/output validation
• Chat, streaming, capability routing, and model-planned complex tasks
• Dependency graphs, parallel batches, conditions, and evidence bindings
• Bounded replanning after required-step failures
• Retries, fallback, deadlines, budgets, and concurrency limits
• Guards, confirmation gates, action receipts, and side-effect safety
• MCP integration with allow-lists and trust-boundary controls
• JSONL traces, evidence, warnings, and execution statistics

The lifecycle is explicit:

request -> route -> plan -> execute -> observe evidence -> replan -> answer

Requirements

• Go 1.25 or newer
• An OpenAI-compatible Chat Completions endpoint

The built-in adapter calls {LLM_BASE_URL}/chat/completions. It accepts standard Chat Completions responses and Responses-style text responses (output_text or output[].content[].text). Internal tool IDs may contain dots; they are normalized only on the provider wire format.

Hybrid reasoning models such as DeepSeek and Doubao can be configured explicitly:

model := openaicompat.New(openaicompat.Config{
    BaseURL:         os.Getenv("LLM_BASE_URL"),
    APIKey:          os.Getenv("LLM_API_KEY"),
    Model:           os.Getenv("LLM_MODEL"),
    Thinking:        openaicompat.ThinkingEnabled, // or ThinkingDisabled / ThinkingAuto
    ReasoningEffort: "high",
})

The adapter sends thinking: {"type":"..."} and, when explicitly configured, the top-level reasoning_effort field. When thinking mode is combined with tool calls, returned reasoning_content is automatically preserved in same-turn continuation requests. If omitted, thinking defaults to ThinkingDisabled, while reasoning_effort is not sent and remains under caller/provider control. Supported values still depend on the provider and model version. Example 10 also reads LLM_THINKING and LLM_REASONING_EFFORT.

Enable detailed JSONL when diagnosing plans or tool arguments:

zhizhi.WithObserver(observe.NewJSONL(os.Stdout, observe.WithDetails()))

Detailed mode records planner/replanner model requests, raw responses and repair attempts, actual per-step inputs/outputs/errors, the final patch, and final-composer requests/responses. It may contain user or business data, so it is disabled by default; example 10 enables it explicitly.

The runtime no longer hand-parses structured model output: jsonrepair-go extracts and repairs non-standard JSON, mapstructure/v2 handles common weak scalar drift, and go-openapi/jsonpointer resolves RFC 6901 evidence paths. Repaired syntax must still pass plan, JSON Schema, and patch-application validation before execution.

The planner and replanner follow Eino's structured-output pattern on their primary path: derive JSON Schema from the Go result type, bind Plan or Patch as the only synthetic output tool, force tool_choice, and consume only that tool's arguments. Plain JSON text remains a compatibility path for models that do not advertise tool calling; domain tools are catalog metadata and cannot execute during planning.

Install

go get github.com/cocoyes/zhizhi-agent-runtime

Quickstart

package main

import (
    "context"
    "fmt"
    "log"
    "os"

    zhizhi "github.com/cocoyes/zhizhi-agent-runtime"
    "github.com/cocoyes/zhizhi-agent-runtime/adapter/model/openaicompat"
)

func main() {
    model := openaicompat.New(openaicompat.Config{
        BaseURL: os.Getenv("LLM_BASE_URL"),
        APIKey:  os.Getenv("LLM_API_KEY"),
        Model:   os.Getenv("LLM_MODEL"),
    })
    agent, err := zhizhi.New(zhizhi.WithModel(model))
    if err != nil { log.Fatal(err) }
    defer agent.Close(context.Background())

    response, err := agent.Run(context.Background(), zhizhi.Request{
        Input: "Summarize the latest support request",
    })
    if err != nil { log.Fatal(err) }
    fmt.Println(response.Text)
}

export LLM_BASE_URL="https://your-provider.example/v1"
export LLM_API_KEY="your-api-key"
export LLM_MODEL="your-model"
go run ./examples/00-chat

The endpoint can return a standard Chat Completions response:

{"choices":[{"message":{"role":"assistant","content":"..."}}]}

Responses-style text is also accepted:

{"output_text":"..."}

P0 execution contract

Messages support ordered text and image parts through model.UserMessage, model.TextPart, and model.ImageURLPart. Model inputs support auto, none, required, and named tool choice plus text, json_object, and strict json_schema response formats. Adapters declare their capabilities and reject unsupported content instead of silently dropping it.

Each run can add or filter an immutable tool snapshot and attach scoped options:

response, err := agent.Run(ctx, request,
    zhizhi.WithRunTools(requestTools...),
    zhizhi.WithRunToolFilter(allowForRequest),
    zhizhi.WithRunObserver(requestObserver),
    zhizhi.WithRunMetadata(metadata),
    zhizhi.WithRunResponseFormat(format),
)

Global and run-scoped middleware are available for agent, model, model stream, tool, tool stream, planner, and replanner boundaries. Stream emits stable run/model/tool/plan/step lifecycle events across every tool round and complex mode. Complex plans may use deterministic tool steps or bounded agentic steps with an explicit capability allowlist and tool-call budget. Final and suspended responses contain cumulative model/tool usage and execution statistics. Optional OpenTelemetry integration lives in adapter/otel; its instrumentation provides middleware for each execution boundary plus an event observer, keeping the core runtime independent of OTel APIs.

For local setup, copy .env.example to your shell environment and set a real provider endpoint, API key, and model. The runtime never fabricates model responses: planning, tool selection, replanning, and final composition all use the configured model.

Typed Tools

Tools are normal typed Go functions. Schemas are reflected automatically and arguments/results are validated at runtime.

type WeatherInput struct {
    City string `json:"city" jsonschema:"required"`
}
type WeatherOutput struct {
    City string `json:"city"`
    Condition string `json:"condition"`
    Temperature int `json:"temperature"`
}

weather := tool.Func(
    "weather.current",
    "Get the current weather for a city",
    func(ctx context.Context, in WeatherInput) (WeatherOutput, error) {
        return WeatherOutput{City: in.City, Condition: "sunny", Temperature: 27}, nil
    },
    tool.WithCapabilities("weather.current"),
)
agent, err := zhizhi.New(zhizhi.WithModel(model), zhizhi.WithTools(weather))

Tool descriptions are part of the model interface. Describe what a tool does and when it should be used. Keep IDs stable and specific; the runtime resolves the selected capability to the registered implementation.

Complex Agent Tasks

For multi-step work, ask the runtime to use complex mode. The built-in model planner chooses tools and builds the dependency graph.

ctx := context.Background()
mode := zhizhi.RunModeAgentComplex
response, err := agent.Run(ctx, zhizhi.Request{
    Input: "Plan tonight's activities in Shenzhen based on the weather",
    Mode: &mode,
})

Here context.Background() can be replaced with a request-scoped context carrying your deadline and cancellation policy.

A planner can express result-based branches:

{
  "version": 1,
  "steps": [
    {"id":"weather", "capability":"weather.current", "input":{"city":"Shenzhen"}},
    {"id":"outdoor", "capability":"activity.outdoor", "depends_on":["weather"], "condition":{"source_step":"weather","source_path":"outdoor_suitable","equals":true}, "bindings":[{"source_step":"weather","source_path":"temperature","target_path":"temperature"}]},
    {"id":"indoor", "capability":"activity.indoor", "depends_on":["weather"], "condition":{"source_step":"weather","source_path":"outdoor_suitable","equals":false}},
    {"id":"compose", "capability":"itinerary.compose", "depends_on":["outdoor","indoor"]}
  ]
}

The condition source must be a dependency. A non-matching step is skipped and never calls its tool. Bindings pass evidence into the next typed input. If a required step fails, one bounded replan can replace that step while preserving successful work.

The complete workflow is runnable and tested:

go run ./examples/10-full-agent
go test ./examples/10-full-agent

Side-Effect Safety

Declare risk and authorization requirements for writes:

sendEmail := tool.Func(
    "mail.send", "Send an email to a recipient", sendEmailImpl,
    tool.WithCapabilities("mail.send"),
    tool.WithSideEffect(tool.SideEffectWriteNonIdempotent),
    tool.WithIdempotency(tool.IdempotencyNonIdempotent),
    tool.WithRisk(tool.RiskHigh),
    tool.WithConfirmation(tool.ConfirmationAlways),
)
agent, err := zhizhi.New(
    zhizhi.WithModel(model), zhizhi.WithTools(sendEmail),
    zhizhi.WithGuard(authorize),
    zhizhi.WithConfirmation(askUser),
)

Without confirmation, the side effect is not executed. The response contains an ActionReceipt with CONFIRMATION_REQUIRED. Unknown external outcomes are represented explicitly and must not be treated as success.

Suspend and resume

Every agent has a process-local checkpoint store by default. Inject a durable store in production and resume the exact suspended state after approval:

agent, err := zhizhi.New(
    zhizhi.WithModel(model),
    zhizhi.WithTools(sendEmail),
    zhizhi.WithCheckpointStore(durableStore),
)

pending, err := agent.Run(ctx, request)
if pending.Suspended {
    completed, err := agent.Resume(ctx, zhizhi.ResumeRequest{
        RunID: pending.RunID, Checkpoint: pending.Checkpoint, Decision: true,
    }, runOptions...) // provide the same request-scoped tools, if any
    _ = completed
    _ = err
}

Checkpoints are versioned and bound to the tool-catalog fingerprint. Completed steps and successful write receipts are replayed without executing their tools; UNKNOWN receipts are never retried. A resumed checkpoint is invalidated even when execution suspends again, preventing stale approval tokens from replaying a write.

Reliability Controls

zhizhi.WithBudget(zhizhi.Budget{
    MaxSteps: 12, MaxBatches: 8, MaxModelCalls: 6,
    MaxToolCalls: 20, MaxReplans: 1, MaxParallelSteps: 4,
    HardTimeout: 30 * time.Second,
})

The default policy retries only transient, safe, idempotent failures. Writes are never silently repeated.

MCP

import runtimemcp "github.com/cocoyes/zhizhi-agent-runtime/mcp"

agent, err := zhizhi.New(
    zhizhi.WithModel(model),
    zhizhi.WithMCP(runtimemcp.Config{
        ID: "internal-tools", Transport: runtimemcp.TransportStreamableHTTP,
        Endpoint: "https://tools.example.com/mcp",
        ConnectionStrategy: runtimemcp.ConnectionLazy,
        Required: false,
		MaxConnectAttempts: 3,
		ReconnectBackoff: 200 * time.Millisecond,
        AllowTools: []string{"customer.lookup", "ticket.search"},
        DenyTools: []string{"admin.delete"},
    }),
)

Lazy connection is the default. Concurrent first use is single-flight; optional servers are isolated from agent creation, while Required servers fail the run if unavailable. Providers support catalog refresh, fingerprint/version drift, bounded reconnect backoff, cancellation, and context-bounded close. MCP tools are treated as an untrusted boundary. See SECURITY.md.

For a large MCP fleet, use a capability provider so the model first sees only server names and descriptions. The runtime connects and calls ListTools only for the selected server or servers (one request may select several):

capabilities, err := runtimemcp.NewCapabilityProvider(
    runtimemcp.Config{ID: "maps", Description: "Maps, weather, and routing", Endpoint: mapsURL},
    runtimemcp.Config{ID: "knowledge", Description: "Encyclopedia search", Endpoint: knowledgeURL},
)
agent, err := zhizhi.New(
    zhizhi.WithModel(model),
    zhizhi.WithMCPCapabilityProvider(capabilities),
)

Observability

import (
    "os"
    "github.com/cocoyes/zhizhi-agent-runtime/observe"
)

observer := observe.NewJSONL(os.Stdout)
agent, err := zhizhi.New(zhizhi.WithModel(model), zhizhi.WithObserver(observer))

Responses expose Evidence, Actions, Warnings, and statistics for model calls, tool calls, plan steps, batches, and replans.

Examples

Development

go fmt ./...
go vet ./...
go test ./...
go test -race ./...
go build ./...

make verify runs the standard local checks. Real-provider tests are opt-in; the default suite is offline and deterministic.

Security and License

Report vulnerabilities privately with the affected version, a minimal reproduction, and impact. See SECURITY.md. See LICENSE for licensing terms.