# Cloudflare cuts AI code-review cost 68% by fixing cache misses and testing model effort Cloudflare reports cutting average internal AI code-review spend from $2.43 to $0.77 while reducing median review time from 4m09s to 1m32s. The useful lesson is to inspect real provider requests and verify quality by review category, not simply buy a cheaper model. A detailed engineering case study links a 68% AI-review cost reduction to prompt-cache structure and effort tuning, with security-review trade-offs still unresolved. - Status: Active - Published: 2026-10-20T17:36:01+13:00 - Updated: 2026-10-20T17:36:01+13:00 - Categories: Artificial Intelligence, Web Development, SaaS, Indie Business, AI SaaS, AI Coding, Product Economics, Developer Tools - Tags: AI agents, AI coding, API pricing, Cloudflare - Canonical HTML: https://beyondthe.news/dossiers/cloudflare-ai-code-review-inference-cost-68-percent-prompt-cache ## What changed In an October 19, 2026 engineering post, Cloudflare described optimising its internal multi-agent code reviewer after average review cost climbed from about $1.19 in April to $2.46 in late August without major architecture changes. Comparing its two most expensive pre-optimisation weeks with the later system, it reports average cost falling from $2.43 to $0.77 per review (68%), and median runtime from 4m09s to 1m32s. Before optimisation, output tokens represented 23% of spend, cache writes 43%, and cache reads 34%. Work included stabilising the first request's prompt ordering for cache reuse, model/effort experiments and moving to OpenCode v2; vendor-reported shadow tests agreed on review approval decisions in 98% of cases. ## Why it matters Teams paying for repeated coding-agent or review-agent runs may have an avoidable prompt-cache bill even if their model and application architecture seem unchanged. Request ordering, cache hit/miss patterns and effort settings can dominate cost per completed review. Cloudflare's experience also shows why cheap tokens are not the only metric: reducing effort in performance and security reviews appeared to lose real findings, so category-specific quality checks remain essential. ## The cost leak was in the actual provider payload Cloudflare says it investigated what reached the provider on each session's first request, rather than trusting intended prompt structure. Inconsistent message ordering reduced cache reuse across commits and repositories. Making the first request stable enough to hit the cache was the largest reported saving. The pre-optimisation spend mix was unusually cache-heavy: 43% writes, 34% reads and 23% output. ## The workflow changed as well as the bill The reviewer moved to OpenCode v2 after a shadow-mode comparison on canary repositories. Cloudflare reports 98% agreement on approval decisions between versions and a roughly 13% speedup for full-tier reviews using the same models over two comparison days. Agreement on approval decisions is not proof that both versions catch every defect, and no independent reproduction of the system's cost figures is available. ## Model effort has different consequences by review type Cloudflare reports that high-effort code-quality review retained about as many real findings with less noise and fewer inflated severities. Lower effort on performance and security reviews appeared to miss findings, so the team plans to test those categories at high effort. A single effort setting for every agent role may therefore be a false economy. ## How to reproduce the useful investigation Instrument token usage, cache-write and cache-read spend, request ordering and completed-review costs; compare identical review sets in shadow mode; measure false negatives and useful findings separately for security, performance and code quality. The published numbers are Cloudflare's internal workload, not a guaranteed saving for other stacks. ## Key details - Engineering report dated October 19, 2026. - Average review cost $2.43 before optimisation to $0.77 after (68% lower), vendor-reported. - Median runtime 4m09s before to 1m32s after, vendor-reported. - Pre-optimisation spend: 23% output tokens, 43% cache writes, 34% cache reads. - OpenCode v2 shadow review approval decisions agreed 98% with v1 on canary repositories. - Performance/security lower-effort reviews may have missed findings; further high-effort testing pending. ## Builder takeaways - Audit the actual first provider request and prompt order before assuming caching is working. - Measure per-completed-task cost including cache writes, reads, retries and agent coordination. - Use shadow-mode review comparisons and evaluate actual defect findings, not just approval agreement. - Tune model and effort separately for code quality, security and performance instead of applying one global low-cost preset. - Treat the reported 68% reduction as one company's internal result, not a transferable benchmark. ## What to watch - Independent replication of prompt-cache optimisation in other code-review workloads. - Cloudflare's planned high-effort security/performance comparisons and false-negative rates. - Changes to OpenCode v2 and its internal Durable Objects experiments. ## Uncertainties - The cost and latency comparisons are company-reported and not independently reproduced. - Review mix, token prices and baseline may differ between comparison windows. - 98% approval agreement is not a measured defect-detection recall rate. - Security/performance review quality under lower effort remains unresolved. ## Sources - [Cutting our inference bill by 68% by being obsessive about optimization](https://blog.cloudflare.com/obsessive-inference-optimization/) — Cloudflare Engineering · primary/vendor. October 19 original before/after cost and runtime figures, token spend mix, shadow testing and quality caveats.