{ "reportVersion": "public-2026-10-13-v6", "complete": true, "protocol": { "date": "2026-10-13", "maxGenerationRequestsPerCleanAttempt": 8, "maxDirectToolExecutions": 40, "checks": "锁定需求、相关运行测试、隐藏验收；TypeScript真实仓库任务包含类型验收。XCR完整通过还要求自身报告完成。", "accounting": "XCR为模拟客户费用（API成本加20%）；对照为模型API费用。失败任务与相关中断重做费用保留；未知用量以费用区间表示。费用由测试时计价快照与返回用量估算，缓存、思考token与实际服务档位计入。未含外层AI编程工具调用、评测工具开发调试、税费及汇率差异；不是已确认账单，也不是当前官网报价。", "comparison": "比较完整工作流程；XCR与单模型工具循环流程不同，不能把差异全部归因于底层模型能力。内部实现不公开。" }, "benchmarkBuild": "XCR evaluation build 2026-10-13 / round 1", "baselines": { "Astra": { "provider": "OpenAI", "modelId": "gpt-6-astra", "reasoningEffort": "high", "servicePolicy": "Flex优先；明确容量拒绝后回退Standard，按实际返回档位计价" }, "Sol": { "provider": "OpenAI", "modelId": "gpt-6.1-sol", "reasoningEffort": "high", "servicePolicy": "Flex优先；明确容量拒绝后回退Standard，按实际返回档位计价" }, "Sonnet": { "provider": "Anthropic", "modelId": "claude-sonnet-5-5", "reasoningEffort": "high", "servicePolicy": "Standard" } }, "baselinePricingSnapshot": { "date": "2026-10-13", "unit": "USD per million billed tokens", "Astra": { "flexInput": 5, "flexOutput": 25, "standardInput": 10, "standardOutput": 50, "cacheReadInputMultiplier": 0.1, "source": "https://developers.openai.com/api/docs/models/gpt-6-astra" }, "Sol": { "flexInput": 1, "flexOutput": 5, "standardInput": 2, "standardOutput": 10, "cacheReadInputMultiplier": 0.05, "source": "https://developers.openai.com/api/docs/models/gpt-6.1-sol" }, "Sonnet": { "standardInput": 2, "standardOutput": 10, "cacheReadInputMultiplier": 0.05, "cacheWriteInputMultiplier": 1.25, "source": "https://platform.claude.com/docs/en/about-claude/pricing" } }, "publicationNotice": "本报告由XCR自行设计和执行，非第三方独立认证；未经OpenAI、Anthropic或所用开源项目赞助、认证或背书。相关名称仅用于准确识别对照和来源，商标属于各权利人。结果仅对应2026年10月13日的两轮评测构建、固定任务与工具流程，未验证正式部署版本。样本通过率不是总体准确率或统计等效结论；估算费用不构成实际账单、服务报价或任何节省保证。100倍示例仅作费用换算。实际效果和费用受任务、工作流程、服务档位、缓存及重做影响。发布者仍须遵守适用法律与合同，本说明不排除依法不得排除的责任或消费者权利。", "tasks": [ { "id": "f1", "title": "含时区的日期范围边界", "category": "bugfix", "lang": "ts", "prompt": "Fix `parseRange`. Date-only inputs (`YYYY-MM-DD`) mean whole UTC calendar days, with the end date inclusive. Inputs with a time or timezone remain exact instants. Reject invalid dates, malformed values, and ranges whose normalized start is after the normalized end with RangeError. Return Unix milliseconds.", "taskHash": "d8d34f8a21f6fe9eff949cf71c5ac54ea0ca3a034015c2d24396359e78366f7c" }, { "id": "f2", "title": "税额分厘舍入与输入校验", "category": "bugfix", "lang": "py", "prompt": "Fix `line_tax_cents`: validate non-negative integer cents and integer basis points from 0 through 10000, reject booleans, and round tax half up using integer arithmetic. Preserve the function name and raise ValueError for invalid input.", "taskHash": "e6da4efe4c7f6b0d1ec1fdc02ea94799e830cc005da6df994dfbfed806beb9ef" }, { "id": "f3", "title": "带引号字段的 CSV 联系人导入", "category": "feature", "lang": "ts", "prompt": "Add `importContacts(text: string, idHeader: string, nameHeader: string): Contact[]` in `src/importContacts.ts`. It must parse a header row and CSV records (quoted commas and doubled quotes supported), map the selected header names to `{id, name}`, trim values, skip rows with an empty id, and reject missing required headers or malformed quoted fields with SyntaxError. Keep `parseCsvRow` exported for focused testing.", "taskHash": "28429a04d0d9f670413adafb3f1e0c1e2ea8e65d6edcb7118574d63a6538c8eb" }, { "id": "f4", "title": "游标分页去重收集", "category": "feature", "lang": "py", "prompt": "Add paginated collection across `src/page_types.py` and `src/pager.py`. `collect_pages(fetch, start_cursor=None, max_pages=100)` calls `fetch(cursor)` and receives `{items, next_cursor}`. Stop at a null cursor, a repeated cursor, or max_pages. Deduplicate items by their `id`, preserving the first-seen item and page order. Reject max_pages unless it is a positive integer (booleans are invalid), with ValueError. Keep source pages untouched.", "taskHash": "752b9965ade51fde476934848f06a01b770d63350f04e3f6572fca63d84da3af" }, { "id": "f5", "title": "跨模块库存预留与订单提交", "category": "cross-file", "lang": "ts", "prompt": "Implement stock reservations across `src/inventory.ts` and `src/orders.ts`. `reserveStock` returns a new inventory map after reserving every requested SKU, and returns null if any SKU is missing or insufficient; it must not partially mutate the input. `placeOrder` returns `{inventory, status:'accepted'}` on success or `{inventory: original, status:'rejected'}` on failure. Duplicate SKUs in an order are additive. Negative or non-integer line quantities must reject the whole order.", "taskHash": "2b9088cc24c5c49b26718cccbf5bc7a1ea1c83c54cb54c6e680fe73e5b198cbb" }, { "id": "f6", "title": "请求配额跨模块校验", "category": "cross-file", "lang": "ts", "prompt": "Add a per-user quota across `src/quota.ts` and `src/service.ts`. `consumeQuota(state, user, amount, limit)` returns a new state with the amount added when the resulting usage is at most the limit, otherwise null. Reject non-positive/non-integer amounts and invalid limits. `runRequest` must call the quota helper first, return `{ok:false, state:original}` on rejection, and only invoke the supplied operation after quota is accepted; successful calls return its value and the updated state.", "taskHash": "00f60bb01bdf13d6290c311ee50b0d5178c6bf5f6ffbf7246b5c236d0de52045" }, { "id": "f7", "title": "事件监听器解绑语义重构", "category": "refactor", "lang": "ts", "prompt": "Refactor `Emitter` into named `addListener`, `removeListener`, and `dispatch` helpers while keeping the public `on` and `emit` behavior. `on` returns an idempotent unsubscribe function. Registering the same function twice creates two subscriptions; calling one unsubscribe removes only its own registration. Dispatch uses a snapshot so listeners added during an emit start on the next emit, while listeners removed during an emit still run if they were already in that snapshot.", "taskHash": "f11daa338f5557ed585c53647c9dfdfd40be25509c82b0904ec61eda83db8472" }, { "id": "f8", "title": "指标统计格式化重构", "category": "refactor", "lang": "py", "prompt": "Refactor `render_metrics` into `clean_samples`, `summarize`, and `render_summary`, preserving the current public output and function signature. The helpers are clean_samples(samples) returning normalized rows, summarize(cleaned_samples) returning a name-to-mean mapping, and render_summary(summary) returning the formatted string. Ignore samples with non-finite values, preserve insertion order for metric names, compute the arithmetic mean, and format to exactly two decimals. Never mutate the samples.", "taskHash": "8d1d710f85e2c076c7184b3c672e8a4c668d432fedddd96e93ecb9eb63fad373" }, { "id": "f9", "title": "多选状态与键盘导航", "category": "frontend", "lang": "ts", "prompt": "Implement selection helpers in `src/selection.ts`. `toggleSelected(selected, value)` adds or removes a value without mutating the array and removes all duplicate copies when deselecting. `moveActive(current: number, enabled: number[], key: string)` moves among enabled item indexes only: ArrowDown/ArrowUp wrap, Home/End select the first/last enabled index, and unknown keys preserve the current index. An empty enabled list returns the current index.", "taskHash": "7ed71f736a54cfd0a44917c8ce4718bbf5eb03aa1801e15b723d34312a069a67" }, { "id": "f10", "title": "可访问菜单焦点导航", "category": "frontend", "lang": "ts", "prompt": "Complete `menuFocus` in `src/menuFocus.ts`. Given menu items with `disabled`, `hidden`, and a stable `id`, return the next focusable item's index. ArrowDown/ArrowUp wrap; Home/End jump to the first/last focusable item. If the current index is not focusable, ArrowDown starts at the first and ArrowUp at the last. Unknown keys or no focusable items return the original current index.", "taskHash": "2cbd20464f0e2c86f7ef63d2b8562987fee551b01e59507c1a55403166208302" }, { "id": "f11", "title": "租约队列认领与过期回收", "category": "reliability", "lang": "ts", "prompt": "Implement the in-memory leased queue in `src/leaseQueue.ts`. Enqueue must reject duplicate IDs with RangeError. `claim(worker, now, leaseMs)` returns the oldest queued job or an expired leased job, sets its owner/expiry, and increments attempts; empty worker names and non-positive leases return null; it returns null if nothing is available. `ack(id, worker, now)` succeeds only for the current owner before or at expiry. `retry(id, worker, now)` returns the job to queued only for its current owner before or at expiry. Unknown IDs, stale owners, and expired leases return false. An expired job becomes available strictly after expiry. Never mutate job payloads or return internal mutable references; keep snapshot() as an isolated deep copy of all jobs.", "taskHash": "cc6533121dc15322f7b44889697793836a3a0d5cea4c0a84f77b7c29bc531a68" }, { "id": "f12", "title": "半开探针熔断器", "category": "reliability", "lang": "py", "prompt": "Implement `CircuitBreaker` in `src/breaker.py` with injected monotonic clock. It starts closed. After `failure_threshold` consecutive failures it opens for `cooldown_seconds`; after cooldown exactly one request may be admitted as the half-open probe. A probe success closes and resets failures; probe failure reopens for a full cooldown. `allow_request` and record methods must handle clock advancement deterministically. failure_threshold must be a positive integer, excluding booleans; cooldown_seconds must be non-negative. Invalid thresholds/cooldowns raise ValueError.", "taskHash": "e500a1566dcff09c6a61f8db95363f02a8494e532a1e05cb3d164fa3787c2256" } ], "rows": [ { "task": "f1", "group": "Astra", "passed": true, "customerCostLowerUsd": 0.11196224999999999, "customerCostUpperUsd": 0.11196224999999999, "seconds": 84 }, { "task": "f1", "group": "XCR", "passed": true, "customerCostLowerUsd": 0.06638493, "customerCostUpperUsd": 0.06638493, "seconds": 97, "falseAcceptance": false }, { "task": "f1", "group": "Sol", "passed": true, "customerCostLowerUsd": 0.04935195, "customerCostUpperUsd": 0.04935195, "seconds": 110 }, { "task": "f1", "group": "Sonnet", "passed": true, "customerCostLowerUsd": 0.038388, "customerCostUpperUsd": 0.038388, "seconds": 25 }, { "task": "f2", "group": "Astra", "passed": true, "customerCostLowerUsd": 0.02576, "customerCostUpperUsd": 0.02576, "seconds": 18 }, { "task": "f2", "group": "XCR", "passed": true, "customerCostLowerUsd": 0.000360225, "customerCostUpperUsd": 0.000360225, "seconds": 20, "falseAcceptance": false }, { "task": "f2", "group": "Sol", "passed": true, "customerCostLowerUsd": 0.008355999999999999, "customerCostUpperUsd": 0.008355999999999999, "seconds": 27 }, { "task": "f2", "group": "Sonnet", "passed": true, "customerCostLowerUsd": 0.016356, "customerCostUpperUsd": 0.016356, "seconds": 11 }, { "task": "f3", "group": "Astra", "passed": true, "customerCostLowerUsd": 0.074587, "customerCostUpperUsd": 0.074587, "seconds": 65 }, { "task": "f3", "group": "XCR", "passed": true, "customerCostLowerUsd": 0.00076404, "customerCostUpperUsd": 0.00076404, "seconds": 40, "falseAcceptance": false }, { "task": "f3", "group": "Sol", "passed": true, "customerCostLowerUsd": 0.028232499999999997, "customerCostUpperUsd": 0.028232499999999997, "seconds": 59 }, { "task": "f3", "group": "Sonnet", "passed": true, "customerCostLowerUsd": 0.032644, "customerCostUpperUsd": 0.032644, "seconds": 21 }, { "task": "f4", "group": "Astra", "passed": true, "customerCostLowerUsd": 0.045977750000000005, "customerCostUpperUsd": 0.045977750000000005, "seconds": 34 }, { "task": "f4", "group": "XCR", "passed": true, "customerCostLowerUsd": 0.0005492699999999999, "customerCostUpperUsd": 0.0005492699999999999, "seconds": 32, "falseAcceptance": false }, { "task": "f4", "group": "Sol", "passed": true, "customerCostLowerUsd": 0.0155729, "customerCostUpperUsd": 0.0155729, "seconds": 36 }, { "task": "f4", "group": "Sonnet", "passed": true, "customerCostLowerUsd": 0.022612, "customerCostUpperUsd": 0.022612, "seconds": 14 }, { "task": "f5", "group": "Astra", "passed": true, "customerCostLowerUsd": 0.039940249999999955, "customerCostUpperUsd": 0.45500274999999996, "seconds": 29 }, { "task": "f5", "group": "XCR", "passed": true, "customerCostLowerUsd": 0.021517289999999998, "customerCostUpperUsd": 0.021517289999999998, "seconds": 49, "falseAcceptance": false }, { "task": "f5", "group": "Sol", "passed": true, "customerCostLowerUsd": 0.015158, "customerCostUpperUsd": 0.015158, "seconds": 49 }, { "task": "f5", "group": "Sonnet", "passed": true, "customerCostLowerUsd": 0.028724, "customerCostUpperUsd": 0.028724, "seconds": 17 }, { "task": "f6", "group": "Astra", "passed": true, "customerCostLowerUsd": 0.03713525, "customerCostUpperUsd": 0.03713525, "seconds": 40 }, { "task": "f6", "group": "XCR", "passed": true, "customerCostLowerUsd": 0.00068079, "customerCostUpperUsd": 0.00068079, "seconds": 71, "falseAcceptance": false }, { "task": "f6", "group": "Sol", "passed": true, "customerCostLowerUsd": 0.0134702, "customerCostUpperUsd": 0.0134702, "seconds": 44 }, { "task": "f6", "group": "Sonnet", "passed": true, "customerCostLowerUsd": 0.030035999999999997, "customerCostUpperUsd": 0.030035999999999997, "seconds": 17 }, { "task": "f7", "group": "Astra", "passed": true, "customerCostLowerUsd": 0.03501175, "customerCostUpperUsd": 0.03501175, "seconds": 24 }, { "task": "f7", "group": "XCR", "passed": true, "customerCostLowerUsd": 0.000498975, "customerCostUpperUsd": 0.000498975, "seconds": 28, "falseAcceptance": false }, { "task": "f7", "group": "Sol", "passed": true, "customerCostLowerUsd": 0.0133719, "customerCostUpperUsd": 0.0133719, "seconds": 32 }, { "task": "f7", "group": "Sonnet", "passed": true, "customerCostLowerUsd": 0.020516, "customerCostUpperUsd": 0.020516, "seconds": 12 }, { "task": "f8", "group": "Astra", "passed": true, "customerCostLowerUsd": 0.0327055, "customerCostUpperUsd": 0.0327055, "seconds": 25 }, { "task": "f8", "group": "XCR", "passed": true, "customerCostLowerUsd": 0.0005187299999999999, "customerCostUpperUsd": 0.0005187299999999999, "seconds": 26, "falseAcceptance": false }, { "task": "f8", "group": "Sol", "passed": true, "customerCostLowerUsd": 0.0125882, "customerCostUpperUsd": 0.0125882, "seconds": 31 }, { "task": "f8", "group": "Sonnet", "passed": true, "customerCostLowerUsd": 0.02122, "customerCostUpperUsd": 0.02122, "seconds": 13 }, { "task": "f9", "group": "Astra", "passed": true, "customerCostLowerUsd": 0.027979999999999998, "customerCostUpperUsd": 0.027979999999999998, "seconds": 45 }, { "task": "f9", "group": "XCR", "passed": true, "customerCostLowerUsd": 0.000546435, "customerCostUpperUsd": 0.000546435, "seconds": 29, "falseAcceptance": false }, { "task": "f9", "group": "Sol", "passed": true, "customerCostLowerUsd": 0.012036999999999999, "customerCostUpperUsd": 0.012036999999999999, "seconds": 31 }, { "task": "f9", "group": "Sonnet", "passed": true, "customerCostLowerUsd": 0.020306, "customerCostUpperUsd": 0.020306, "seconds": 14 }, { "task": "f10", "group": "Astra", "passed": true, "customerCostLowerUsd": 0.031813, "customerCostUpperUsd": 0.031813, "seconds": 23 }, { "task": "f10", "group": "XCR", "passed": true, "customerCostLowerUsd": 0.000552525, "customerCostUpperUsd": 0.000552525, "seconds": 29, "falseAcceptance": false }, { "task": "f10", "group": "Sol", "passed": true, "customerCostLowerUsd": 0.012220499999999999, "customerCostUpperUsd": 0.012220499999999999, "seconds": 34 }, { "task": "f10", "group": "Sonnet", "passed": true, "customerCostLowerUsd": 0.019746, "customerCostUpperUsd": 0.019746, "seconds": 28 }, { "task": "f11", "group": "Astra", "passed": true, "customerCostLowerUsd": 0.05406275, "customerCostUpperUsd": 0.05406275, "seconds": 37 }, { "task": "f11", "group": "XCR", "passed": true, "customerCostLowerUsd": 0.000988875, "customerCostUpperUsd": 0.000988875, "seconds": 57, "falseAcceptance": false }, { "task": "f11", "group": "Sol", "passed": true, "customerCostLowerUsd": 0.01207995, "customerCostUpperUsd": 0.01207995, "seconds": 49 }, { "task": "f11", "group": "Sonnet", "passed": true, "customerCostLowerUsd": 0.030447999999999996, "customerCostUpperUsd": 0.030447999999999996, "seconds": 18 }, { "task": "f12", "group": "Astra", "passed": true, "customerCostLowerUsd": 0.09056499999999998, "customerCostUpperUsd": 0.09056499999999998, "seconds": 63 }, { "task": "f12", "group": "XCR", "passed": true, "customerCostLowerUsd": 0.0007531499999999998, "customerCostUpperUsd": 0.0007531499999999998, "seconds": 42, "falseAcceptance": false }, { "task": "f12", "group": "Sol", "passed": true, "customerCostLowerUsd": 0.03366925, "customerCostUpperUsd": 0.03366925, "seconds": 82 }, { "task": "f12", "group": "Sonnet", "passed": true, "customerCostLowerUsd": 0.027572, "customerCostUpperUsd": 0.027572, "seconds": 16 } ], "stats": [ { "group": "XCR", "total": 12, "passed": 12, "passRate": 1.0, "customerCostLowerUsd": 0.09411523499999999, "customerCostUpperUsd": 0.09411523499999999, "meanCostLowerUsd": 0.00784293625, "meanCostUpperUsd": 0.00784293625 }, { "group": "Astra", "total": 12, "passed": 12, "passRate": 1.0, "customerCostLowerUsd": 0.6075004999999999, "customerCostUpperUsd": 1.0225629999999999, "meanCostLowerUsd": 0.050625041666666655, "meanCostUpperUsd": 0.08521358333333333 }, { "group": "Sol", "total": 12, "passed": 12, "passRate": 1.0, "customerCostLowerUsd": 0.22610834999999999, "customerCostUpperUsd": 0.22610834999999999, "meanCostLowerUsd": 0.018842362499999998, "meanCostUpperUsd": 0.018842362499999998 }, { "group": "Sonnet", "total": 12, "passed": 12, "passRate": 1.0, "customerCostLowerUsd": 0.308568, "customerCostUpperUsd": 0.308568, "meanCostLowerUsd": 0.025714, "meanCostUpperUsd": 0.025714 } ], "limitations": [ "样本为12道小型任务和两个开源工具库的10道公开历史任务；并非随机抽取的行业任务。", "公开历史源码可能被模型记忆；未验证大型应用、原生AI编程工具或订阅额度。", "同样的通过数量不代表通过相同任务，也不证明总体准确率等效。", "每轮每组每题一次有效运行；生成请求上限包含阅读和工具决策。基础设施中断经修复重做，费用保留。" ] }