Review Studio 2.0

Yao Meta Skill

Create, improve, or evaluate an existing skill from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install/trust checks, 优化已有 skill, 补 trigger 评测. Exclude summary/translation/docs, brainstorming, one-off prompts, copy-only edits, outline-only requests.

审查结论 review Score 86/100

核心指标

Skill IR2.0.0

5 targets in platform-neutral contract

Compiler5/5

target contracts compiled from Skill IR

Output Delta100.0

5 cases; 1 file-backed

Exec Runs10

command 10; model 10; recorded 0

Blind A/B5

review pairs hide baseline vs with-skill labels

Review Kit0/5

pending 5; answer key hidden

Review A/B0/5

adjudication decisions; pending 5

Public Claimblocked

3 blockers; local reproducible true

Blueprint21/21

2.0 coverage; extensions partial 0, planned 0; evidence pending 4

Runtime5/5

target conformance pass rate

Perm Probe4/4

0 native; 4 installer-enforced

Trust0

174 scripts scanned; secrets found

Py Compat0

262 files scanned for Python 3.11

Arch Debt0

719 largest lines; 0 watchlist; 7 early; 74 CLI handlers; 17 in entrypoint

Atlas5

11 scanned skills; route collisions

Driftno-data

0 metadata events; 0 missed triggers

Daily Ops5

proposal-review; approval 0; release lock true

Weekly Queue5

curator-review; ready 1; top score 88

Waivers0

0 gates covered; human risk decisions

Intake4/4

0 valid submissions; 0 invalid

Claim Guard0

232 public surfaces scanned

Notes0/0

0 open blocker annotations

Registry2.1.0

5 targets; MIT license

Archivepass

658 zip entries; package verification

Installpass

4 adapters; 12 permissions enforced; 0 permission failures

Upgrademinor

declared major; 0 breaking changes

审查闸门

通过

意图画布

intent confidence 100/100; clarification proceed; assumptions 0; stop clear; Intent is clear enough to package the first routeable version.

reports/intent-confidence.json 证据
通过

触发实验

13 trigger cases; 0 misroutes; 0 ambiguous; frozen 30 P=1.0 R=0.917; hard-negative FP 0

reports/route_scorecard.json + reports/phase1_trigger_holdout.json 证据
关注

输出实验

5/5 cases; with-skill 100.0; baseline 0.0; file-backed 1; near-neighbor 1; blind A/B 5; exec 10; command 10; model 10; recorded 0; reviewed 0/5; review pending 5; provider matrix completed; phase1 review 3/3; promotion eligible

reports/output_quality_scorecard.json 证据
通过

上下文

initial load 945/1000; deferred 595301/120000; top deferred scripts 527937; resource governance governed; quality density 137.6

reports/context_budget.json 证据
通过

运行矩阵

5 / 5 targets pass

reports/conformance_matrix.json 证据
通过

信任报告

0 secrets; 174 scripts; 3 network-capable scripts; 0 help smoke failures

reports/security_trust_report.json 证据
通过

Python 兼容

Python 3.11; 262 files; 0 compatibility issues; 0 syntax; 0 f-string 3.11 hazards

reports/python_compatibility.json 证据
通过

架构维护

259 Python files; 0 hotspots; 0 watchlist files; 7 early watch files; 0 blockers; largest 719 lines; 74 CLI handlers; 17 in entrypoint

reports/architecture_maintainability.json 证据
通过

权限批准

3/3 permissions approved; gaps 0; required file_write, network, subprocess

reports/security_trust_report.json + security/permission_policy.json 证据
通过

权限探针

4/4 targets probed; native 0; metadata fallback 4; installer 4; residual risks 4

reports/runtime_permission_probes.json 证据
关注

组合治理

11 skills, 1 actionable; 0 actionable route collisions; 0 actionable owner gaps; 0 actionable stale; 1 actionable drift; 23 scoped non-actionable issues

reports/skill_atlas.json 证据
关注

运营回路

0 metadata events; adoption 0; missed 0; bad-output 0; risk no-data; daily proposals 5; daily decision proposal-review; daily release lock true; weekly queue 5 unique; weekly ready 1; weekly top 88; weekly release lock true

reports/adoption_drift_report.json + reports/skillops/daily + reports/skillops/weekly 证据
关注

人工批准

0 active waivers; 3 warning gates still need reviewer decision

reports/review_waivers.json 证据
关注

世界证据

4 pending world-class evidence entries; 1 human pending; 3 external pending; source checks 11/14 pass; 3 blocked; overclaim guard true

reports/world_class_evidence_ledger.json 证据
通过

注册审计

yao-meta-skill 2.1.0; 6/6 compatibility entries pass; install pass with 4 adapters; installer permissions 12 enforced / 0 failures

reports/registry_audit.json + reports/install_simulation.json 证据
通过

发布路线

0 promote; 3 keep current; 0 blocked; upgrade major declared / minor recommended

reports/promotion_decisions.json + reports/upgrade_check.json + docs/migration-v2.md 证据

闸门契约

Ok
是
期望 Gate
16
实际 Gate
16
缺失 Gate
无
未知 Gate
无
重复 Gate
无
未加权 Gate
无

阻断事项

无。

关注事项

  • 输出实验5/5 cases; with-skill 100.0; baseline 0.0; file-backed 1; near-neighbor 1; blind A/B 5; exec 10; command 10; model 10; recorded 0; reviewed 0/5; review pending 5; provider matrix completed; phase1 review 3/3; promotion eligible
  • 组合治理11 skills, 1 actionable; 0 actionable route collisions; 0 actionable owner gaps; 0 actionable stale; 1 actionable drift; 23 scoped non-actionable issues
  • 运营回路0 metadata events; adoption 0; missed 0; bad-output 0; risk no-data; daily proposals 5; daily decision proposal-review; daily release lock true; weekly queue 5 unique; weekly ready 1; weekly top 88; weekly release lock true
  • 人工批准0 active waivers; 3 warning gates still need reviewer decision
  • 世界证据4 pending world-class evidence entries; 1 human pending; 3 external pending; source checks 11/14 pass; 3 blocked; overclaim guard true

修复动作

关注

输出实验

补足 output eval 覆盖、execution evidence、blind A/B 和 reviewer adjudication。同步补足盲审声明。

没有输出质量和人工盲评证据时，Skill 只能证明会触发，不能证明输出真的更好且经得起审查。
修复位置
evals/output/cases.jsonl + reports/output_quality_scorecard.md + reports/output_review_kit.html + reports/output_review_adjudication.md + reports/output_review_decisions.json
验证命令
python3 scripts/adjudicate_output_review.py --write-template && python3 scripts/yao.py output-review --self
关注

组合治理

处理 portfolio 里的路由冲突、owner 缺口、stale skill 和重复能力。

单个 Skill 质量很高仍可能在团队 skill library 中被相邻 Skill 冲突削弱。
修复位置
reports/skill_atlas.html + skill_atlas/catalog.json
验证命令
python3 scripts/build_skill_atlas.py --workspace-root .
  • skill_atlas/catalog.json:8skill atlas catalog · atlas · pattern: summary
    "description": "Create, improve, or evaluate an existing skill from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install/trust check…
  • skill_atlas/policy.json:3atlas scope policy · policy · pattern: scope
    "scope_rules": [
  • reports/skill_atlas.html:6skill atlas report · report · pattern: Skill Atlas
    <title>Skill Atlas</title>
关注

运营回路

记录 metadata-only 使用事件，或明确当前 release 缺少真实使用信号。

没有运营回路时，reviewer 无法判断采用率、误触发、坏输出和 review overdue 的真实影响。
修复位置
reports/adoption_drift_report.md
验证命令
python3 scripts/render_adoption_drift_report.py . --record-event skill_activation --activation-type explicit --outcome accepted
reports/adoption_drift_report.json + reports/skillops/daily + reports/skillops/weekly 打开证据
关注

人工批准

对保留的 warning 写入 reviewer、理由、范围和到期时间，或修掉 warning。

warning 可以被接受，但必须可审计、会过期，并且不能掩盖 blocker。
修复位置
reports/review_waivers.md
验证命令
python3 scripts/render_review_waivers.py .
关注

世界证据

补齐 provider、真人盲评、原生权限执行和真实客户端遥测证据，或明确本次发布不声明 world-class 完成。

世界级结论必须来自已接受的外部/人工证据；计划、metadata fallback、待评审和本地命令都不能替代完成证据。
修复位置
reports/world_class_operator_runbook.html + reports/world_class_evidence_ledger.md + reports/world_class_evidence_intake.md + reports/world_class_submission_review.md
验证命令
python3 scripts/yao.py world-class-runbook . --submissions-dir evidence/world_class/submissions --self && python3 scripts/yao.py world-class-ledger . --submissions-dir evidence/world_class/submissions --self && python3 scripts/yao.py review-studio . --self

证据采集

以下条目仍需真实外部或人工证据；提交文件、校验命令和阻断检查必须同时闭环。

pending · external

Provider Holdout

phase1 model-executed 40/40; calls 40/40; status completed

提交
evidence/world_class/submissions/provider-holdout.json
模板
evidence/world_class/templates/provider-holdout.intake.json
阻断
0 blocked / 4 pass
修复
1 repair rows; counts as completion: false
阶段
1 blocked phases; counts as completion: false
下一步
Run evidence-build with DEEPSEEK_API_KEY and keep raw outputs in the isolated run directory.
阻断检查

暂无阻断检查。

阶段队列
  • #20 · unblock-accessUnblock access1 / 1 blockedowners: operator with provider credentialsevidence: provider-holdoutSet DEEPSEEK_API_KEY in the operator shell; never commit or print the value.python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions --self
修复清单
  • #20 · unblock-access · precheckprovider-api-keyoperator with provider credentialsRequired env precheck is missing.Set DEEPSEEK_API_KEY in the operator shell; never commit or print the value.python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions --self
操作命令
  • 准备提交python3 scripts/yao.py world-class-submission-kit . --evidence-key provider-holdout --output-dir evidence/world_class/submissions --self
  • 校验入口python3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions --self
  • 审查提交python3 scripts/yao.py world-class-submission-review . --submissions-dir evidence/world_class/submissions --self
  • 刷新台账python3 scripts/yao.py world-class-ledger . --submissions-dir evidence/world_class/submissions --self
首要步骤
  1. Set DEEPSEEK_API_KEY in the operator shell; never commit or print the value.
  2. python3 scripts/yao.py evidence-build . --run-id <PROVIDER_RUN_ID> --self
  3. Keep the generated private answer key and role-neutral review materials inside .yao/runs/<PROVIDER_RUN_ID>.
采集契约
来源要求
  • provider-backed model run
  • observed timing
  • observed token metadata
通过条件
  • reports/provider_output_evaluation.json summary.call_count == 40
  • reports/provider_output_evaluation.json summary.model_executed_count == 40
  • reports/provider_output_evaluation.json summary.failure_count == 0
  • reports/provider_output_evaluation.json summary.total_tokens <= 250000
  • reports/skill_os2_audit.json item provider-holdout status becomes pass
证据资产
  • evals/output/provider_matrix.json
  • reports/provider_output_evaluation.json
  • reports/provider_output_blind_pack.json
  • reports/provider_output_answer_commitment.json
  • reports/skill_os2_audit.json
资产角色

source: world-class-submission-kit; counts as evidence: false; prefill counts as evidence: false

  • submission-ref1 / 1 readyartifact_refs: trueRows marked submission-ref are the aggregate paths expected in artifact_refs.
  • supporting-evidence7 / 8 readyartifact_refs: falseSupporting-evidence rows help reviewers audit the packet but do not all need to be copied into artifact_refs.
隐私边界
  • Do not commit provider credentials or environment dumps.
  • The output execution report records output hashes and aggregate run metadata, not raw provider prompts.
pending · human

Human Adjudication

phase1 reviewers 3/3; pairs 20/20; promotion eligible

提交
evidence/world_class/submissions/human-adjudication.json
模板
evidence/world_class/templates/human-adjudication.intake.json
阻断
0 blocked / 4 pass
修复
1 repair rows; counts as completion: false
阶段
1 blocked phases; counts as completion: false
下一步
Collect three controlled reviewer packets and adjudicate them against the private run answer key.
阻断检查

暂无阻断检查。

阶段队列
  • #20 · unblock-accessUnblock access1 / 1 blockedowners: human reviewerevidence: human-adjudicationAssign three independent controlled reviewer identities before claiming human adjudication.python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions --self
修复清单
  • #20 · unblock-access · precheckhuman-reviewerhuman reviewerRequired human precheck is human-required.Assign three independent controlled reviewer identities before claiming human adjudication.python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions --self
操作命令
  • 准备提交python3 scripts/yao.py world-class-submission-kit . --evidence-key human-adjudication --output-dir evidence/world_class/submissions --self
  • 校验入口python3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions --self
  • 审查提交python3 scripts/yao.py world-class-submission-review . --submissions-dir evidence/world_class/submissions --self
  • 刷新台账python3 scripts/yao.py world-class-ledger . --submissions-dir evidence/world_class/submissions --self
首要步骤
  1. Give each registered reviewer an independent copy of the matching provider_review_reviewer-*.json template and the role-neutral blind pack.
  2. Collect all 20 A/B choices, reasons, controlled submission ids, timestamps, and truthful independent-review attestations.
  3. Export an access-controlled reviewer registry that binds each reviewer id to the exact packet SHA256.
采集契约
来源要求
  • real reviewer identity
  • blind A/B decisions
  • answer key unopened until decisions exist
通过条件
  • reports/provider_output_adjudication.json summary.reviewer_count == 3
  • reports/provider_output_adjudication.json summary.pair_count == 20
  • reports/provider_output_adjudication.json summary.failure_count == 0
  • reports/provider_output_adjudication.json evidence_binding.blind_pack_sha256 matches the source run
  • reports/skill_os2_audit.json item human-adjudication status becomes pass
证据资产
  • reports/provider_output_blind_pack.json
  • reports/provider_reviewer_registry.json
  • reports/provider_output_adjudication.json
  • reports/provider_review_lineage.json
  • scripts/adjudicate_multi_reviewer.py
资产角色

source: world-class-submission-kit; counts as evidence: false; prefill counts as evidence: false

  • submission-ref1 / 2 readyartifact_refs: trueRows marked submission-ref are the aggregate paths expected in artifact_refs.
  • supporting-evidence7 / 8 readyartifact_refs: falseSupporting-evidence rows help reviewers audit the packet but do not all need to be copied into artifact_refs.
隐私边界
  • Reviewer packets contain choices, reasons, hashes, and controlled submission metadata without raw prompts or answer-key roles.
  • The private answer key remains under .yao/runs and is opened by the finalizer after all controlled packets are fixed.
  • The adjudication and lineage artifacts preserve blind_pack_sha256 and answer_key_sha256 commitments.
pending · external

Native Permission Enforcement

native-enforced targets 0; installer-enforced targets 4

提交
evidence/world_class/submissions/native-permission-enforcement.json
模板
evidence/world_class/templates/native-permission-enforcement.intake.json
阻断
1 blocked / 2 pass
修复
2 repair rows; counts as completion: false
阶段
2 blocked phases; counts as completion: false
下一步
Integrate a real target-client or external installer runtime guard before claiming native permission enforcement.
阻断检查
  • Native enforcementnative_enforcement_count: 0 / >0Collect real target-client or external runtime guard proof.
阶段队列
  • #20 · unblock-accessUnblock access1 / 1 blockedowners: target client or installer integratorevidence: native-permission-enforcementAttach a real target-client or external installer runtime guard; metadata fallback is not enough.python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions --self
  • #40 · collect-sourceCollect source1 / 1 blockedowners: target client or installer integratorevidence: native-permission-enforcementCollect real target-client or external runtime guard proof.python3 scripts/yao.py runtime-permissions . --package-dir dist --self && python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions --self
修复清单
  • #20 · unblock-access · prechecknative-guardtarget client or installer integratorRequired external precheck is external-required.Attach a real target-client or external installer runtime guard; metadata fallback is not enough.python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions --self
  • #40 · collect-source · source-checknative_enforcement_counttarget client or installer integratorCurrent value 0 does not satisfy '>0'.Collect real target-client or external runtime guard proof.python3 scripts/yao.py runtime-permissions . --package-dir dist --self && python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions --self
操作命令
  • 准备提交python3 scripts/yao.py world-class-submission-kit . --evidence-key native-permission-enforcement --output-dir evidence/world_class/submissions --self
  • 校验入口python3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions --self
  • 审查提交python3 scripts/yao.py world-class-submission-review . --submissions-dir evidence/world_class/submissions --self
  • 刷新台账python3 scripts/yao.py world-class-ledger . --submissions-dir evidence/world_class/submissions --self
首要步骤
  1. Implement or connect a real target client or external installer runtime guard that blocks undeclared network, file_write, or subprocess capabilities.
  2. Update the generated target adapter only when the guard is actually enforced by that target.
  3. python3 scripts/yao.py package . --platform openai --platform claude --platform generic --platform vscode --output-dir dist --zip --self
采集契约
来源要求
  • real target client or external installer runtime guard
  • native enforcement flag or externally accepted guard proof
  • residual risk retained for fallback targets
通过条件
  • reports/runtime_permission_probes.json summary.native_enforcement_count > 0
  • reports/runtime_permission_probes.json summary.failure_count == 0
  • reports/runtime_permission_probes.json summary.installer_enforcement_pass_count records local installer enforcement but does not replace native evidence
  • reports/skill_os2_audit.json item native-permission-enforcement status becomes pass
证据资产
  • dist/targets/*/adapter.json
  • reports/runtime_permission_probes.json
  • reports/runtime_permission_probes.md
  • reports/install_simulation.json
  • reports/install_simulation.md
资产角色

source: world-class-submission-kit; counts as evidence: false; prefill counts as evidence: false

  • submission-ref2 / 2 readyartifact_refs: trueRows marked submission-ref are the aggregate paths expected in artifact_refs.
  • supporting-evidence11 / 11 readyartifact_refs: falseSupporting-evidence rows help reviewers audit the packet but do not all need to be copied into artifact_refs.
隐私边界
  • Do not mark native_enforcement true for metadata-only fallbacks.
  • Keep residual risks visible for targets that still rely on operator enforcement.
pending · external

Native Client Telemetry

external source events 0; adoption samples 0

提交
evidence/world_class/submissions/native-client-telemetry.json
模板
evidence/world_class/templates/native-client-telemetry.intake.json
阻断
2 blocked / 1 pass
修复
3 repair rows; counts as completion: false
阶段
2 blocked phases; counts as completion: false
下一步
Install a real client against the native host and import production metadata-only events.
阻断检查
  • External eventsexternal_source_events: 0 / >0Import at least one metadata-only event from a real client.
  • Adoption sampleadoption_sample_count: 0 / >0Telemetry must include adoption outcome evidence.
阶段队列
  • #20 · unblock-accessUnblock access1 / 1 blockedowners: Browser/Chrome/IDE/provider client integratorevidence: native-client-telemetryInstall a real Browser, Chrome, IDE, or provider client that emits metadata-only events.python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions --self
  • #40 · collect-sourceCollect source2 / 2 blockedowners: Browser/Chrome/IDE/provider client integratorevidence: native-client-telemetryTelemetry must include adoption outcome evidence.python3 scripts/yao.py telemetry-import . --input-jsonl .yao/telemetry_spool/external_events.jsonl --self && python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions --self
修复清单
  • #20 · unblock-access · precheckexternal-clientBrowser/Chrome/IDE/provider client integratorRequired external precheck is external-required.Install a real Browser, Chrome, IDE, or provider client that emits metadata-only events.python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions --self
  • #40 · collect-source · source-checkadoption_sample_countBrowser/Chrome/IDE/provider client integratorCurrent value 0 does not satisfy '>0'.Telemetry must include adoption outcome evidence.python3 scripts/yao.py telemetry-import . --input-jsonl .yao/telemetry_spool/external_events.jsonl --self && python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions --self
  • #40 · collect-source · source-checkexternal_source_eventsBrowser/Chrome/IDE/provider client integratorCurrent value 0 does not satisfy '>0'.Import at least one metadata-only event from a real client.python3 scripts/yao.py telemetry-import . --input-jsonl .yao/telemetry_spool/external_events.jsonl --self && python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions --self
操作命令
  • 准备提交python3 scripts/yao.py world-class-submission-kit . --evidence-key native-client-telemetry --output-dir evidence/world_class/submissions --self
  • 校验入口python3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions --self
  • 审查提交python3 scripts/yao.py world-class-submission-review . --submissions-dir evidence/world_class/submissions --self
  • 刷新台账python3 scripts/yao.py world-class-ledger . --submissions-dir evidence/world_class/submissions --self
首要步骤
  1. python3 scripts/telemetry_native_host.py . --write-launcher /tmp/yao-telemetry-host.sh --write-manifest /tmp/yao-telemetry-host.json --allowed-origin chrome-extension://<extension-id>/
  2. Install the generated native messaging manifest for the real client and send at least one accepted skill_activation or skill_output event.
  3. python3 scripts/yao.py telemetry-import . --input-jsonl .yao/telemetry_spool/external_events.jsonl --self
采集契约
来源要求
  • real external client source
  • metadata-only event
  • local-first import path
通过条件
  • reports/adoption_drift_report.json summary.source_types.external > 0
  • reports/adoption_drift_report.json summary.adoption_sample_count > 0
  • reports/skill_os2_audit.json item native-client-telemetry status becomes pass
证据资产
  • reports/adoption_drift_report.json
  • reports/adoption_drift_report.md
  • reports/telemetry_hook_recipes.json
  • scripts/telemetry_native_host.py
  • evidence/world_class/intake.schema.json
资产角色

source: world-class-submission-kit; counts as evidence: false; prefill counts as evidence: false

  • submission-ref2 / 2 readyartifact_refs: trueRows marked submission-ref are the aggregate paths expected in artifact_refs.
  • supporting-evidence6 / 6 readyartifact_refs: falseSupporting-evidence rows help reviewers audit the packet but do not all need to be copied into artifact_refs.
隐私边界
  • Telemetry must remain metadata-only and local-first.
  • Do not package reports/telemetry_events.jsonl or any raw prompt, output, transcript, note, or message field.

审查批注

当前批注：0 条批注；开放 0；阻断 0

当前没有 reviewer 批注。

意图画布

Turn repeated workflows, prompts, transcripts, runbooks, documents, or existing skill packages into routeable, evaluable, packageable, and governable agent skills for personal, team, library, or governed reuse. Primary output: A working skill package with lean SKILL.md, aligned agents/interface.yaml, justified references, scripts only when useful, eval evidence, reports, packaging metadata, and clear next iteration recommendations.. Exclusions: one-off writing, translation, explanation, or brainstorming requests that do not need a reusable skill, general code review or debugging unless the user is packaging that workflow as a skill, raw private material that was not intentionally supplied as skill evidence, platform-specific plugin creation when the request is not about an agent skill package.

追问决策

动作：proceed；停止原因：clear

Intent is clear enough to package the first routeable version.

结构化假设

  • none当前没有结构化假设。

证据路径

  • skill_overviewreports/skill-overview.html
  • review_viewerreports/review-viewer.html
  • phase1_trigger_holdoutreports/phase1_trigger_holdout.md
  • output_evalreports/output_quality_scorecard.md
  • output_executionreports/output_execution_runs.md
  • provider_output_evaluationreports/provider_output_evaluation.json
  • provider_output_adjudicationreports/provider_output_adjudication.json
  • output_blind_reviewreports/output_blind_review_pack.md
  • output_review_kitreports/output_review_kit.md
  • output_review_kit_htmlreports/output_review_kit.html
  • output_review_decisionsreports/output_review_decisions.json
  • output_review_adjudicationreports/output_review_adjudication.md
  • benchmark_reproducibilityreports/benchmark_reproducibility.md
  • skill_os2_coveragereports/skill_os2_coverage.md
  • runtime_conformancereports/conformance_matrix.md
  • trust_reportreports/security_trust_report.md
  • python_compatibilityreports/python_compatibility.md
  • architecture_maintainabilityreports/architecture_maintainability.md
  • permission_policysecurity/permission_policy.md
  • runtime_permissionsreports/runtime_permission_probes.md
  • skill_atlasreports/skill_atlas.html
  • compiled_targetsreports/compiled_targets.md
  • adoption_driftreports/adoption_drift_report.md
  • review_waiversreports/review_waivers.md
  • review_annotationsreports/review_annotations.md
  • adaptation_proposalsreports/adaptation_proposals.md
  • adaptation_approval_ledgerreports/adaptation_approval_ledger.json
  • adaptation_regressionreports/adaptation_regression_report.md
  • world_class_evidence_planreports/world_class_evidence_plan.md
  • world_class_evidence_ledgerreports/world_class_evidence_ledger.md
  • world_class_evidence_intakereports/world_class_evidence_intake.md
  • world_class_evidence_preflightreports/world_class_evidence_preflight.md
  • world_class_evidence_preflight_htmlreports/world_class_evidence_preflight.html
  • world_class_submission_reviewreports/world_class_submission_review.md
  • world_class_operator_runbookreports/world_class_operator_runbook.md
  • world_class_operator_runbook_htmlreports/world_class_operator_runbook.html
  • world_class_claim_guardreports/world_class_claim_guard.md
  • registry_auditreports/registry_audit.md
  • package_verificationreports/package_verification.md
  • install_simulationreports/install_simulation.md
  • upgrade_checkreports/upgrade_check.md
  • migrationdocs/migration-v2.md
  • skill_irskill-ir/examples/yao-meta-skill.json
  • daily_skillopsreports/skillops/daily/2026-08-17.md
  • weekly_curatorreports/skillops/weekly/2026-W34.md

触发实验

13 trigger cases; 0 misroutes; 0 ambiguous; frozen 30 P=1.0 R=0.917; hard-negative FP 0

组合治理

Skill 数
11
纳入审查
1
待处理冲突
0
待处理 owner
0
待处理过期
0
Telemetry Report Count
1
Actionable Drift Signal Count
1
Drift Signal Count
1
非行动项
23

输出实验

案例数
5
With Skill
100
Baseline
0
增益
100
Gate
是

执行证据

运行数
10
命令执行
10
模型执行
10
记录样本
0
计时样本
10
估算 Token
0

盲评包

Pair Count
5
Answer Key Separate
是
With Skill Hidden Count
5

审定报告

Pair Count
5
Judgment Count
0
待审
5
Agreement Count
0
Disagreement Count
0
Invalid Decision Count
0
答案揭示
0
答案隐藏
5
Reviewer Checklist Count
5
Reviewer Checklist Pending Count
5

评审清单

先打开 reports/output_review_kit.md；每张卡片对应一个 blind A/B case，说明当前是否待判、答案是否仍隐藏、应填写的决策文件、有效字段和复跑命令。

awaiting-decision · 答案隐藏

skill-package-contract

Reviewer has not selected A or B yet; answer key remains hidden.

盲评包
reports/output_blind_review_pack.json
决策表
reports/output_review_decisions.json
提示哈希
95405d805bd516d0998b69d31ece2c1a465b0aa99f04d603628a3939028133b5

操作命令

  • prepare_review_kitpython3 scripts/yao.py output-review-kit --self
  • write_templatepython3 scripts/adjudicate_output_review.py --write-template
  • import_decisionspython3 scripts/yao.py output-review-import --input <reviewer-decisions.json> --blind-review-attested --run-adjudication --self
  • adjudicatepython3 scripts/yao.py output-review --self
  • refresh_review_studiopython3 scripts/yao.py review-studio . --self

字段要求

  • winner_variantA or B after reading only the blind review pack.
  • confidenceOptional number from 0 to 1.
  • reasonRequired rationale; do not reveal baseline or with-skill labels before adjudication.
  • reviewerHuman reviewer name or review group at the decision-file top level.
  • reviewed_atReview date or timestamp at the decision-file top level.
  • reviewer_attestation.blind_review_completed_before_answer_keyTrue only after the reviewer has completed choices before opening the answer key.
  • reviewer_attestation.answer_key_not_opened_before_decisionsTrue only when the answer key was not opened before decisions were recorded.

隐私边界

  • Do not paste raw private user data into the decision reason.
  • Do not open the answer key before reviewer choices are recorded.
  • Leave winner_variant blank when the reviewer is not ready to decide.
awaiting-decision · 答案隐藏

output-eval-expectation

Reviewer has not selected A or B yet; answer key remains hidden.

盲评包
reports/output_blind_review_pack.json
决策表
reports/output_review_decisions.json
提示哈希
3829e97814ca269ad8e400ec984c56ddcd54b79448d3846c15e7f5ff44c2edc4

操作命令

  • prepare_review_kitpython3 scripts/yao.py output-review-kit --self
  • write_templatepython3 scripts/adjudicate_output_review.py --write-template
  • import_decisionspython3 scripts/yao.py output-review-import --input <reviewer-decisions.json> --blind-review-attested --run-adjudication --self
  • adjudicatepython3 scripts/yao.py output-review --self
  • refresh_review_studiopython3 scripts/yao.py review-studio . --self

字段要求

  • winner_variantA or B after reading only the blind review pack.
  • confidenceOptional number from 0 to 1.
  • reasonRequired rationale; do not reveal baseline or with-skill labels before adjudication.
  • reviewerHuman reviewer name or review group at the decision-file top level.
  • reviewed_atReview date or timestamp at the decision-file top level.
  • reviewer_attestation.blind_review_completed_before_answer_keyTrue only after the reviewer has completed choices before opening the answer key.
  • reviewer_attestation.answer_key_not_opened_before_decisionsTrue only when the answer key was not opened before decisions were recorded.

隐私边界

  • Do not paste raw private user data into the decision reason.
  • Do not open the answer key before reviewer choices are recorded.
  • Leave winner_variant blank when the reviewer is not ready to decide.
awaiting-decision · 答案隐藏

ir-before-packaging

Reviewer has not selected A or B yet; answer key remains hidden.

盲评包
reports/output_blind_review_pack.json
决策表
reports/output_review_decisions.json
提示哈希
d00333fa41f2e28dc7505f25f259aefb82e65b2df1abda2822686c9a6f07bb80

操作命令

  • prepare_review_kitpython3 scripts/yao.py output-review-kit --self
  • write_templatepython3 scripts/adjudicate_output_review.py --write-template
  • import_decisionspython3 scripts/yao.py output-review-import --input <reviewer-decisions.json> --blind-review-attested --run-adjudication --self
  • adjudicatepython3 scripts/yao.py output-review --self
  • refresh_review_studiopython3 scripts/yao.py review-studio . --self

字段要求

  • winner_variantA or B after reading only the blind review pack.
  • confidenceOptional number from 0 to 1.
  • reasonRequired rationale; do not reveal baseline or with-skill labels before adjudication.
  • reviewerHuman reviewer name or review group at the decision-file top level.
  • reviewed_atReview date or timestamp at the decision-file top level.
  • reviewer_attestation.blind_review_completed_before_answer_keyTrue only after the reviewer has completed choices before opening the answer key.
  • reviewer_attestation.answer_key_not_opened_before_decisionsTrue only when the answer key was not opened before decisions were recorded.

隐私边界

  • Do not paste raw private user data into the decision reason.
  • Do not open the answer key before reviewer choices are recorded.
  • Leave winner_variant blank when the reviewer is not ready to decide.
awaiting-decision · 答案隐藏

near-neighbor-boundary

Reviewer has not selected A or B yet; answer key remains hidden.

盲评包
reports/output_blind_review_pack.json
决策表
reports/output_review_decisions.json
提示哈希
2d0585e9fa6d03e28786f5e8f0c090781eb209afd344d25e2f91203c49a656d0

操作命令

  • prepare_review_kitpython3 scripts/yao.py output-review-kit --self
  • write_templatepython3 scripts/adjudicate_output_review.py --write-template
  • import_decisionspython3 scripts/yao.py output-review-import --input <reviewer-decisions.json> --blind-review-attested --run-adjudication --self
  • adjudicatepython3 scripts/yao.py output-review --self
  • refresh_review_studiopython3 scripts/yao.py review-studio . --self

字段要求

  • winner_variantA or B after reading only the blind review pack.
  • confidenceOptional number from 0 to 1.
  • reasonRequired rationale; do not reveal baseline or with-skill labels before adjudication.
  • reviewerHuman reviewer name or review group at the decision-file top level.
  • reviewed_atReview date or timestamp at the decision-file top level.
  • reviewer_attestation.blind_review_completed_before_answer_keyTrue only after the reviewer has completed choices before opening the answer key.
  • reviewer_attestation.answer_key_not_opened_before_decisionsTrue only when the answer key was not opened before decisions were recorded.

隐私边界

  • Do not paste raw private user data into the decision reason.
  • Do not open the answer key before reviewer choices are recorded.
  • Leave winner_variant blank when the reviewer is not ready to decide.
awaiting-decision · 答案隐藏

file-backed-governed-package

Reviewer has not selected A or B yet; answer key remains hidden.

盲评包
reports/output_blind_review_pack.json
决策表
reports/output_review_decisions.json
提示哈希
c79a554b573517682da48879c1ef54399841d9ebf570088a2174dc38de39cc7d

操作命令

  • prepare_review_kitpython3 scripts/yao.py output-review-kit --self
  • write_templatepython3 scripts/adjudicate_output_review.py --write-template
  • import_decisionspython3 scripts/yao.py output-review-import --input <reviewer-decisions.json> --blind-review-attested --run-adjudication --self
  • adjudicatepython3 scripts/yao.py output-review --self
  • refresh_review_studiopython3 scripts/yao.py review-studio . --self

字段要求

  • winner_variantA or B after reading only the blind review pack.
  • confidenceOptional number from 0 to 1.
  • reasonRequired rationale; do not reveal baseline or with-skill labels before adjudication.
  • reviewerHuman reviewer name or review group at the decision-file top level.
  • reviewed_atReview date or timestamp at the decision-file top level.
  • reviewer_attestation.blind_review_completed_before_answer_keyTrue only after the reviewer has completed choices before opening the answer key.
  • reviewer_attestation.answer_key_not_opened_before_decisionsTrue only when the answer key was not opened before decisions were recorded.

隐私边界

  • Do not paste raw private user data into the decision reason.
  • Do not open the answer key before reviewer choices are recorded.
  • Leave winner_variant blank when the reviewer is not ready to decide.

发布标准

Governed 和 Library 至少需要 5 个 output eval cases，并覆盖 file-backed、near-neighbor、boundary case、execution evidence 和 blind A/B review pack。

人工结论

没有 reviewer 决策时只显示 pending；只有真实决策文件会进入一致率和分歧统计。

评审方式

先打开 reports/output_review_kit.html 做盲评，填入 reports/output_review_decisions.json，再用 reports/output_review_adjudication.md 核对答案 key。

运行方式

reports/output_execution_runs.md 会标明 recorded fixture、command run 或 model run；只有 provider runner 返回 model metadata 时才算 model-executed。

运行矩阵

目标数
5
通过数
5
Fail Count
0

目标编译

目标数
5
通过数
5
Warn Count
0
Block Count
0
失败数
0

上下文

initial load 945/1000; deferred 595301/120000; top deferred scripts 527937; resource governance governed; quality density 137.6

编译证据

Review reports/compiled_targets.md before packaging to inspect target adapter modes, generated files, preserved semantics, warnings, and unsupported features.

信任报告

Secret
0
脚本数
174
网络脚本
3
Help 失败
0
包体哈希
54d1fb49b5de9d73a5a312204ffdfad76937ed8e877571521c569139d9630bd8

安全边界

高风险 secret、远程 inline execution、缺失依赖策略或无法解释的脚本接口应阻断 governed release。

Python 兼容

目标 Python
3.11
文件数
262
问题数
0
语法错误
0
F-string 3.11
0

解释器边界

CI 和发布审查以 Python 3.11 兼容为底线；本地更高版本允许的新语法不能绕过兼容门禁。

权限批准

3/3 permissions approved; gaps 0; required file_write, network, subprocess

批准策略

高权限能力需要 reviewer、scope、reason、expires_at 和 openai/claude/generic 目标端 enforcement 说明。

权限探针

4/4 targets probed; native 0; metadata fallback 4; installer 4; residual risks 4

运行边界

目标数
4
通过数
4
原生执行
0
Metadata fallback
4
安装拦截
4
安装权限失败
0
残余风险
4
失败数
0

组合治理

11 skills, 1 actionable; 0 actionable route collisions; 0 actionable owner gaps; 0 actionable stale; 1 actionable drift; 23 scoped non-actionable issues

下一动作

优先处理真实 portfolio 中的 duplicate names、stale skills、owner gaps，再用运营回路判断真实影响。

运营回路

0 metadata events; adoption 0; missed 0; bad-output 0; risk no-data; daily proposals 5; daily decision proposal-review; daily release lock true; weekly queue 5 unique; weekly ready 1; weekly top 88; weekly release lock true

漂移信号

事件数
0
采用率
0
漏触发
0
Bad Output Count
0
风险带
no-data

日常运维

决策
proposal-review
提案数
5
Approval Count
0
Pending Review Count
0
发布锁
是
世界级
否
Writes Source Files
否
Auto Patch Enabled
否

周度队列

决策
curator-review
Week Id
2026-W34
日报数
10
唯一机会
5
待批准复核
1
提案复核
4
Top Score
88
发布锁
是
Writes Source Files
否
Auto Patch Enabled
否

人工批准

warning 可以被有边界地接受，但必须写入 reviewer、理由、范围和到期时间；blocker 与 world-class 完成证据不能通过 waiver 变成通过。

批准概况

0 active waivers; 3 warning gates still need reviewer decision

批准台账

Waiver Count
0
Active Count
0
Expired Count
0
Invalid Count
0
覆盖 Gate
0

批准候选

可批准 · needs-reviewer-decision

Output Lab

review pending 5; model-executed 10; output failures 0

Gate
output-lab
证据
reports/output_review_adjudication.md
选项
accepted-risk, false-positive, temporary-exception
边界
Does not count as provider, human, or public world-class completion evidence.

审查条件

  • Reviewer confirms this release does not claim provider-backed or human-adjudicated output superiority.
  • Reviewer names the release scope and expiry date.
  • Reviewer links output_review_adjudication or output_execution evidence.
建议命令python3 scripts/yao.py review-waivers . --add-waiver --gate-key output-lab --reviewer "<reviewer>" --reason "Output Lab has pending human/provider evidence; accepted only for this bounded review scope." --expires-at 2027-08-17 --evidence reports/output_review_adjudication.md --self
不可批准 · cannot-waive

World-Class Evidence

4 pending evidence entries; 1 human pending; 3 external pending

Gate
world-class-evidence
证据
reports/world_class_evidence_ledger.md
选项
无
边界
Non-waivable completion boundary.

审查条件

  • Do not use a waiver to claim public world-class readiness.
  • Either submit accepted ledger evidence or state that this release does not claim world-class completion.
  • Keep claim guard active until ledger summary.ready_to_claim_world_class is true.
建议命令python3 scripts/yao.py world-class-ledger . --submissions-dir evidence/world_class/submissions --self && python3 scripts/yao.py world-class-claim-guard . --self

世界证据

这里列出每个 world-class 证据项的当前状态、完成定义、证据来源、隐私约束和下一步；计划、metadata fallback、待评审和本地命令不会被当成完成证据。

待补证 · external

Provider Holdout

Complete the fixed 10-case DeepSeek Flash+Pro matrix with 40 real calls and governed budget evidence.

负责人
operator with provider credentials
当前状态
phase1 model-executed 40/40; calls 40/40; status completed
下一步
Run evidence-build with DEEPSEEK_API_KEY and keep raw outputs in the isolated run directory.
观测值
contract_version: phase1; call_count: 40; model_executed_count: 40; failure_count: 0; total_tokens: 40938; accepted: True
提交态
status: missing; path: evidence/world_class/submissions/provider-holdout.json; attested_real_evidence: False; privacy_contract_satisfied: False

执行步骤

  • Set DEEPSEEK_API_KEY in the operator shell; never commit or print the value.
  • python3 scripts/yao.py evidence-build . --run-id <PROVIDER_RUN_ID> --self
  • Keep the generated private answer key and role-neutral review materials inside .yao/runs/<PROVIDER_RUN_ID>.
  • python3 scripts/yao.py skill-os2-audit . --generated-at <YYYY-MM-DD> --self
  • Copy evidence/world_class/templates/provider-holdout.intake.json to evidence/world_class/submissions/provider-holdout.json and fill only real evidence fields.
  • python3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions --self

完成定义

  • reports/provider_output_evaluation.json summary.call_count == 40
  • reports/provider_output_evaluation.json summary.model_executed_count == 40
  • reports/provider_output_evaluation.json summary.failure_count == 0
  • reports/provider_output_evaluation.json summary.total_tokens <= 250000
  • reports/skill_os2_audit.json item provider-holdout status becomes pass

证据来源

  • evals/output/provider_matrix.json
  • reports/provider_output_evaluation.json
  • reports/provider_output_blind_pack.json
  • reports/provider_output_answer_commitment.json
  • reports/skill_os2_audit.json
  • evidence/world_class/intake.schema.json
  • evidence/world_class/templates/provider-holdout.intake.json
  • reports/world_class_evidence_intake.json
  • reports/world_class_evidence_intake.md

隐私约束

  • Do not commit provider credentials or environment dumps.
  • The output execution report records output hashes and aggregate run metadata, not raw provider prompts.

源证据检查

  • Provider callscall_count: 40 / ==40Complete all 40 fixed DeepSeek calls.
  • Provider model runsmodel_executed_count: 40 / ==40Require model identity on all 40 calls.
  • Provider failuresfailure_count: 0 / ==0Resolve every fixed-matrix failure.
  • Token budgettotal_tokens: 40938 / <=250000Keep the matrix within the 250000-token ceiling.
待补证 · human

Human Adjudication

Collect three controlled, independent reviews of the same 20-pair provider blind pack.

负责人
human reviewer
当前状态
phase1 reviewers 3/3; pairs 20/20; promotion eligible
下一步
Collect three controlled reviewer packets and adjudicate them against the private run answer key.
观测值
contract_version: phase1; reviewer_count: 3; pair_count: 20; failure_count: 0; blind_pack_bound: True; accepted: True
提交态
status: missing; path: evidence/world_class/submissions/human-adjudication.json; attested_real_evidence: False; privacy_contract_satisfied: False

执行步骤

  • Give each registered reviewer an independent copy of the matching provider_review_reviewer-*.json template and the role-neutral blind pack.
  • Collect all 20 A/B choices, reasons, controlled submission ids, timestamps, and truthful independent-review attestations.
  • Export an access-controlled reviewer registry that binds each reviewer id to the exact packet SHA256.
  • python3 scripts/yao.py evidence-finalize-review . --source-run <PROVIDER_RUN_ID> --decisions <reviewer-a.json> --decisions <reviewer-b.json> --decisions <reviewer-c.json> --reviewer-registry <registry.json> --run-id <FINAL_RUN_ID> --self
  • python3 scripts/yao.py skill-os2-audit . --generated-at <YYYY-MM-DD> --self
  • Copy evidence/world_class/templates/human-adjudication.intake.json to evidence/world_class/submissions/human-adjudication.json and fill only real evidence fields.
  • python3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions --self

完成定义

  • reports/provider_output_adjudication.json summary.reviewer_count == 3
  • reports/provider_output_adjudication.json summary.pair_count == 20
  • reports/provider_output_adjudication.json summary.failure_count == 0
  • reports/provider_output_adjudication.json evidence_binding.blind_pack_sha256 matches the source run
  • reports/skill_os2_audit.json item human-adjudication status becomes pass

证据来源

  • reports/provider_output_blind_pack.json
  • reports/provider_reviewer_registry.json
  • reports/provider_output_adjudication.json
  • reports/provider_review_lineage.json
  • scripts/adjudicate_multi_reviewer.py
  • scripts/finalize_provider_review.py
  • evidence/world_class/intake.schema.json
  • evidence/world_class/templates/human-adjudication.intake.json
  • reports/world_class_evidence_intake.json
  • reports/world_class_evidence_intake.md

隐私约束

  • Reviewer packets contain choices, reasons, hashes, and controlled submission metadata without raw prompts or answer-key roles.
  • The private answer key remains under .yao/runs and is opened by the finalizer after all controlled packets are fixed.
  • The adjudication and lineage artifacts preserve blind_pack_sha256 and answer_key_sha256 commitments.

源证据检查

  • Registered reviewersreviewer_count: 3 / ==3Collect reviewer-a, reviewer-b, and reviewer-c.
  • Blind pairspair_count: 20 / ==20Complete all 20 blind pairs.
  • Review failuresfailure_count: 0 / ==0Resolve packet, identity, or adjudication failures.
  • Blind pack bindingblind_pack_bound: True / trueBind adjudication to the reviewed blind pack SHA256.
待补证 · external

Native Permission Enforcement

Prove at least one real target client or external installer runtime guard enforces approved high-permission capabilities.

负责人
target client or installer integrator
当前状态
native-enforced targets 0; installer-enforced targets 4
下一步
Integrate a real target-client or external installer runtime guard before claiming native permission enforcement.
观测值
native_enforcement_count: 0; metadata_fallback_count: 4; installer_enforcement_pass_count: 4; installer_permission_failure_count: 0; installer_enforcement_ready: True; residual_risk_count: 4; failure_count: 0; accepted: False
提交态
status: missing; path: evidence/world_class/submissions/native-permission-enforcement.json; attested_real_evidence: False; privacy_contract_satisfied: False

执行步骤

  • Implement or connect a real target client or external installer runtime guard that blocks undeclared network, file_write, or subprocess capabilities.
  • Update the generated target adapter only when the guard is actually enforced by that target.
  • python3 scripts/yao.py package . --platform openai --platform claude --platform generic --platform vscode --output-dir dist --zip --self
  • python3 scripts/yao.py install-simulate . --package-dir dist --install-root dist/install-simulation --self
  • python3 scripts/yao.py runtime-permissions . --package-dir dist --self
  • python3 scripts/yao.py skill-os2-audit . --generated-at <YYYY-MM-DD> --self
  • Copy evidence/world_class/templates/native-permission-enforcement.intake.json to evidence/world_class/submissions/native-permission-enforcement.json and fill only real evidence fields.
  • python3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions --self

完成定义

  • reports/runtime_permission_probes.json summary.native_enforcement_count > 0
  • reports/runtime_permission_probes.json summary.failure_count == 0
  • reports/runtime_permission_probes.json summary.installer_enforcement_pass_count records local installer enforcement but does not replace native evidence
  • reports/skill_os2_audit.json item native-permission-enforcement status becomes pass

证据来源

  • dist/targets/*/adapter.json
  • reports/runtime_permission_probes.json
  • reports/runtime_permission_probes.md
  • reports/install_simulation.json
  • reports/install_simulation.md
  • security/permission_policy.json
  • evidence/world_class/intake.schema.json
  • evidence/world_class/templates/native-permission-enforcement.intake.json
  • reports/world_class_evidence_intake.json
  • reports/world_class_evidence_intake.md

隐私约束

  • Do not mark native_enforcement true for metadata-only fallbacks.
  • Keep residual risks visible for targets that still rely on operator enforcement.

源证据检查

  • Native enforcementnative_enforcement_count: 0 / >0Collect real target-client or external runtime guard proof.
  • Probe failuresfailure_count: 0 / ==0Runtime permission probes must stay clean.
  • Installer supportinstaller_enforcement_ready: True / trueInstaller enforcement is supporting evidence, not native proof.
待补证 · external

Native Client Telemetry

Import production metadata-only events from a real external client into the local drift loop.

负责人
Browser/Chrome/IDE/provider client integrator
当前状态
external source events 0; adoption samples 0
下一步
Install a real client against the native host and import production metadata-only events.
观测值
external_source_events: 0; adoption_sample_count: 0; raw_content_allowed: False; risk_band: no-data; accepted: False
提交态
status: missing; path: evidence/world_class/submissions/native-client-telemetry.json; attested_real_evidence: False; privacy_contract_satisfied: False

执行步骤

  • python3 scripts/telemetry_native_host.py . --write-launcher /tmp/yao-telemetry-host.sh --write-manifest /tmp/yao-telemetry-host.json --allowed-origin chrome-extension://<extension-id>/
  • Install the generated native messaging manifest for the real client and send at least one accepted skill_activation or skill_output event.
  • python3 scripts/yao.py telemetry-import . --input-jsonl .yao/telemetry_spool/external_events.jsonl --self
  • python3 scripts/yao.py skill-atlas --workspace-root . --self
  • python3 scripts/yao.py skill-os2-audit . --generated-at <YYYY-MM-DD> --self
  • Copy evidence/world_class/templates/native-client-telemetry.intake.json to evidence/world_class/submissions/native-client-telemetry.json and fill only real evidence fields.
  • python3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions --self

完成定义

  • reports/adoption_drift_report.json summary.source_types.external > 0
  • reports/adoption_drift_report.json summary.adoption_sample_count > 0
  • reports/skill_os2_audit.json item native-client-telemetry status becomes pass

证据来源

  • reports/adoption_drift_report.json
  • reports/adoption_drift_report.md
  • reports/telemetry_hook_recipes.json
  • scripts/telemetry_native_host.py
  • evidence/world_class/intake.schema.json
  • evidence/world_class/templates/native-client-telemetry.intake.json
  • reports/world_class_evidence_intake.json
  • reports/world_class_evidence_intake.md

隐私约束

  • Telemetry must remain metadata-only and local-first.
  • Do not package reports/telemetry_events.jsonl or any raw prompt, output, transcript, note, or message field.

源证据检查

  • External eventsexternal_source_events: 0 / >0Import at least one metadata-only event from a real client.
  • Adoption sampleadoption_sample_count: 0 / >0Telemetry must include adoption outcome evidence.
  • Raw content blockedraw_content_allowed: False / falseTelemetry must stay metadata-only.

提交清单

每张卡片给出模板、提交文件、准备命令、校验命令、收集要求、通过条件和隐私边界；只有真实 provider、真人、原生权限或真实客户端证据通过 intake 后，才进入 ledger review。

awaiting-submission · external

Provider Holdout

No real evidence submission has been provided yet.

负责人
operator with provider credentials
模板
evidence/world_class/templates/provider-holdout.intake.json
提交
evidence/world_class/submissions/provider-holdout.json
下一步
Run evidence-build with DEEPSEEK_API_KEY and keep raw outputs in the isolated run directory.

操作命令

  • prepare_submissionpython3 scripts/yao.py world-class-submission-kit . --evidence-key provider-holdout --output-dir evidence/world_class/submissions --self
  • validate_intakepython3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions --self
  • submission_reviewpython3 scripts/yao.py world-class-submission-review . --submissions-dir evidence/world_class/submissions --self
  • refresh_ledgerpython3 scripts/yao.py world-class-ledger . --submissions-dir evidence/world_class/submissions --self
  • guard_claimpython3 scripts/yao.py world-class-claim-guard . --self

收集要求

  • provider-backed model run
  • observed timing
  • observed token metadata

执行步骤

  • Set DEEPSEEK_API_KEY in the operator shell; never commit or print the value.
  • python3 scripts/yao.py evidence-build . --run-id <PROVIDER_RUN_ID> --self
  • Keep the generated private answer key and role-neutral review materials inside .yao/runs/<PROVIDER_RUN_ID>.
  • python3 scripts/yao.py skill-os2-audit . --generated-at <YYYY-MM-DD> --self
  • Copy evidence/world_class/templates/provider-holdout.intake.json to evidence/world_class/submissions/provider-holdout.json and fill only real evidence fields.
  • python3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions --self

通过条件

  • reports/provider_output_evaluation.json summary.call_count == 40
  • reports/provider_output_evaluation.json summary.model_executed_count == 40
  • reports/provider_output_evaluation.json summary.failure_count == 0
  • reports/provider_output_evaluation.json summary.total_tokens <= 250000
  • reports/skill_os2_audit.json item provider-holdout status becomes pass

隐私边界

  • Do not commit provider credentials or environment dumps.
  • The output execution report records output hashes and aggregate run metadata, not raw provider prompts.
awaiting-submission · human

Human Adjudication

No real evidence submission has been provided yet.

负责人
human reviewer
模板
evidence/world_class/templates/human-adjudication.intake.json
提交
evidence/world_class/submissions/human-adjudication.json
下一步
Collect three controlled reviewer packets and adjudicate them against the private run answer key.

操作命令

  • prepare_submissionpython3 scripts/yao.py world-class-submission-kit . --evidence-key human-adjudication --output-dir evidence/world_class/submissions --self
  • validate_intakepython3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions --self
  • submission_reviewpython3 scripts/yao.py world-class-submission-review . --submissions-dir evidence/world_class/submissions --self
  • refresh_ledgerpython3 scripts/yao.py world-class-ledger . --submissions-dir evidence/world_class/submissions --self
  • guard_claimpython3 scripts/yao.py world-class-claim-guard . --self

收集要求

  • real reviewer identity
  • blind A/B decisions
  • answer key unopened until decisions exist

执行步骤

  • Give each registered reviewer an independent copy of the matching provider_review_reviewer-*.json template and the role-neutral blind pack.
  • Collect all 20 A/B choices, reasons, controlled submission ids, timestamps, and truthful independent-review attestations.
  • Export an access-controlled reviewer registry that binds each reviewer id to the exact packet SHA256.
  • python3 scripts/yao.py evidence-finalize-review . --source-run <PROVIDER_RUN_ID> --decisions <reviewer-a.json> --decisions <reviewer-b.json> --decisions <reviewer-c.json> --reviewer-registry <registry.json> --run-id <FINAL_RUN_ID> --self
  • python3 scripts/yao.py skill-os2-audit . --generated-at <YYYY-MM-DD> --self
  • Copy evidence/world_class/templates/human-adjudication.intake.json to evidence/world_class/submissions/human-adjudication.json and fill only real evidence fields.
  • python3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions --self

通过条件

  • reports/provider_output_adjudication.json summary.reviewer_count == 3
  • reports/provider_output_adjudication.json summary.pair_count == 20
  • reports/provider_output_adjudication.json summary.failure_count == 0
  • reports/provider_output_adjudication.json evidence_binding.blind_pack_sha256 matches the source run
  • reports/skill_os2_audit.json item human-adjudication status becomes pass

隐私边界

  • Reviewer packets contain choices, reasons, hashes, and controlled submission metadata without raw prompts or answer-key roles.
  • The private answer key remains under .yao/runs and is opened by the finalizer after all controlled packets are fixed.
  • The adjudication and lineage artifacts preserve blind_pack_sha256 and answer_key_sha256 commitments.
awaiting-submission · external

Native Permission Enforcement

No real evidence submission has been provided yet.

负责人
target client or installer integrator
模板
evidence/world_class/templates/native-permission-enforcement.intake.json
提交
evidence/world_class/submissions/native-permission-enforcement.json
下一步
Integrate a real target-client or external installer runtime guard before claiming native permission enforcement.

操作命令

  • prepare_submissionpython3 scripts/yao.py world-class-submission-kit . --evidence-key native-permission-enforcement --output-dir evidence/world_class/submissions --self
  • validate_intakepython3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions --self
  • submission_reviewpython3 scripts/yao.py world-class-submission-review . --submissions-dir evidence/world_class/submissions --self
  • refresh_ledgerpython3 scripts/yao.py world-class-ledger . --submissions-dir evidence/world_class/submissions --self
  • guard_claimpython3 scripts/yao.py world-class-claim-guard . --self

收集要求

  • real target client or external installer runtime guard
  • native enforcement flag or externally accepted guard proof
  • residual risk retained for fallback targets

执行步骤

  • Implement or connect a real target client or external installer runtime guard that blocks undeclared network, file_write, or subprocess capabilities.
  • Update the generated target adapter only when the guard is actually enforced by that target.
  • python3 scripts/yao.py package . --platform openai --platform claude --platform generic --platform vscode --output-dir dist --zip --self
  • python3 scripts/yao.py install-simulate . --package-dir dist --install-root dist/install-simulation --self
  • python3 scripts/yao.py runtime-permissions . --package-dir dist --self
  • python3 scripts/yao.py skill-os2-audit . --generated-at <YYYY-MM-DD> --self
  • Copy evidence/world_class/templates/native-permission-enforcement.intake.json to evidence/world_class/submissions/native-permission-enforcement.json and fill only real evidence fields.
  • python3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions --self

通过条件

  • reports/runtime_permission_probes.json summary.native_enforcement_count > 0
  • reports/runtime_permission_probes.json summary.failure_count == 0
  • reports/runtime_permission_probes.json summary.installer_enforcement_pass_count records local installer enforcement but does not replace native evidence
  • reports/skill_os2_audit.json item native-permission-enforcement status becomes pass

隐私边界

  • Do not mark native_enforcement true for metadata-only fallbacks.
  • Keep residual risks visible for targets that still rely on operator enforcement.
awaiting-submission · external

Native Client Telemetry

No real evidence submission has been provided yet.

负责人
Browser/Chrome/IDE/provider client integrator
模板
evidence/world_class/templates/native-client-telemetry.intake.json
提交
evidence/world_class/submissions/native-client-telemetry.json
下一步
Install a real client against the native host and import production metadata-only events.

操作命令

  • prepare_submissionpython3 scripts/yao.py world-class-submission-kit . --evidence-key native-client-telemetry --output-dir evidence/world_class/submissions --self
  • validate_intakepython3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions --self
  • submission_reviewpython3 scripts/yao.py world-class-submission-review . --submissions-dir evidence/world_class/submissions --self
  • refresh_ledgerpython3 scripts/yao.py world-class-ledger . --submissions-dir evidence/world_class/submissions --self
  • guard_claimpython3 scripts/yao.py world-class-claim-guard . --self

收集要求

  • real external client source
  • metadata-only event
  • local-first import path

执行步骤

  • python3 scripts/telemetry_native_host.py . --write-launcher /tmp/yao-telemetry-host.sh --write-manifest /tmp/yao-telemetry-host.json --allowed-origin chrome-extension://<extension-id>/
  • Install the generated native messaging manifest for the real client and send at least one accepted skill_activation or skill_output event.
  • python3 scripts/yao.py telemetry-import . --input-jsonl .yao/telemetry_spool/external_events.jsonl --self
  • python3 scripts/yao.py skill-atlas --workspace-root . --self
  • python3 scripts/yao.py skill-os2-audit . --generated-at <YYYY-MM-DD> --self
  • Copy evidence/world_class/templates/native-client-telemetry.intake.json to evidence/world_class/submissions/native-client-telemetry.json and fill only real evidence fields.
  • python3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions --self

通过条件

  • reports/adoption_drift_report.json summary.source_types.external > 0
  • reports/adoption_drift_report.json summary.adoption_sample_count > 0
  • reports/skill_os2_audit.json item native-client-telemetry status becomes pass

隐私边界

  • Telemetry must remain metadata-only and local-first.
  • Do not package reports/telemetry_events.jsonl or any raw prompt, output, transcript, note, or message field.

蓝图覆盖

项目数
21
模块数
8
建议 PR
13
通过数
21
Warn Count
0
缺失数
0
Extension Track Count
4
Extension Covered Count
4
Extension Partial Count
0
Extension Planned Count
0
Adaptive Extension Ready
是
本地蓝图
是
世界级
否
待补证据
4

覆盖边界

蓝图覆盖只证明 2.0 模块、建议 PR、脚本、报告和测试在本地闭环；public world-class 仍以 world-class evidence ledger 的真人和外部证据为准。

公开声明

本地复现
是
发布锁
是
可公开声明
否
声明阻断
3
Provider 证据
是
人审完成
否
世界级就绪
否

声明阻断

  • 阻断human blind-review adjudication is incomplete
  • 阻断world-class evidence is not accepted yet (2 open gaps, 4 ledger pending)
  • 阻断world-class source checks are not all accepted (11/14 pass, 3 blocked)

世界证据

4 pending world-class evidence entries; 1 human pending; 3 external pending; source checks 11/14 pass; 3 blocked; overclaim guard true

证据台账

Ledger Entry Count
4
Accepted Count
0
待审
4
Human Pending Count
1
External Pending Count
3
Overclaim Guard Active
是
Ready To Claim World Class
否

证据入口

Schema
是
模板数
4
模板通过
4
提交数
0
有效提交
0
无效提交
0
可收证据
是
可审台账
否
Ready To Claim World Class
否

入口边界

intake 只校验证据包格式、来源、隐私和反过度声明；只有 ledger 看到真实 provider、真人、原生权限或真实客户端结果后，world-class 才能进入完成审计。

声明守卫

台账可声明
否
台账待补
4
声明面
232
违规数
0
Overclaim Guard Active
是

声明边界

claim guard 扫描 README、docs 和 reports 中的完成态表述；ledger 未 ready 时，任何英文完成断言、true 状态声明或中文完成态都会阻断发布审查。

注册审计

yao-meta-skill 2.1.0; 6/6 compatibility entries pass; install pass with 4 adapters; installer permissions 12 enforced / 0 failures

包体元数据

名称
yao-meta-skill
版本
2.1.0
Maturity
governed
Owner
Yao Team
License
MIT
信任级别
local
目标平台
openai, claude, generic, agent-skills-compatible, vscode
兼容通过
6/6

发布路线

0 promote; 3 keep current; 0 blocked; upgrade major declared / minor recommended

包体验证

目标数
4
Adapter
4
归档存在
是
Zip 条目
658
失败数
0
警告数
0

安装模拟

Archive Extracted
是
Entrypoint Loaded
是
Manifest Loaded
是
Interface Loaded
是
Adapter
4
安装权限
12
安装权限失败
0
权限目标
4
权限能力
3
失败数
0
警告数
0

权限覆盖

安装模拟会读取解压后的 target adapter，并按 declared_capabilities 核对 security/permission_policy.json 中的有效批准、证据、过期时间和目标端 enforcement 说明。