Interactive guide · Day 5 · Lee Boonstra (Google)

From vibe to production-grade

A Google engineer's daily routine did a 180-degree turn: coding agents produce a thousand lines of documented code before lunch. But writing speed is not shipped software. This guide summarizes the paper "Spec-Driven Production Grade Development in the Age of Vibe Coding" — the blueprint for turning AI-generated prototypes into reliable production systems: specs as source of truth, 3-tier code review, zero-trust guardrails and continuous evaluation.

📄 Paper: Day 5 · Lee Boonstra ⏱ ~20 min of study 🧪 7 code snippets ✅ 8-question quiz 🌐 EN · ES · PT-BR
👥 Who this guide is for
👩‍💻 Engineers using coding agents 🧭 Tech leads & managers 🛡️ Platform / security engineers 🏛️ Architects

The goal: scale agent output without turning the repository into a minefield — solid specs, review that scales, external guardrails and humans in the right loop. Assumes familiarity with modern development; not with ML.

dev@2026: ~/checkout — the agent writes, the reviewer blocks
dev@2026:~/checkout$ agent "implement payment retry per specs/payment_retry.md" ▸ agent: writing payment_retry.py… 1,000 lines generated ✓ ▸ agent: opening pull request… PR #482 ⚠ reviewer (code-check.md): unparameterized SQL at retry.py:142 — CRITICAL ⚠ reviewer: unmasked PII in log at line 87 — CRITICAL ✗ REQUEST CHANGES — writing speed ≠ shipping speed dev@2026:~/checkout$
0
lines of well-documented code a coding agent can generate before lunch — which alone do not mean shipped software.
SHIPPED?≠ shipped software
0%
performance drop with unoptimized generic Markdown (SkCC study)
0%
parsing accuracy of YAML on nested configs (vs 43.1% JSON · 33.8% XML)
0%
more likely to experience high burnout among frequent AI users (Quantum Workplace/CNBC)
01

The illusion of speed

A thousand lines before lunch is impressive — until you look at what reached production.

The routine has completely changed. Before, the day was digging through API docs, testing code line by line, figuring out whether the language uses substring in string, string.includes() or string.contains() — and debugging the gap between working code and the original intent. Today, coding agents like Antigravity and Gemini CLI don't just suggest text: they use tools, execute tasks and churn out a thousand documented lines in minutes.

⏪ Before: the craftsman

  • Dig through APIs and documentation manually
  • Test code line by line
  • Discover syntax by trial and error
  • Debug the gap between code and intention

⏩ Today: warp speed

  • Agents use tools and execute tasks
  • 1,000 lines of documented code, fast
  • AI writes tests, specs, roadmaps and analyses
  • The bottleneck moved to human review and integration

It's like hiring a legion of interns who never sleep and never complain.

— about coding agents · Lee Boonstra

The problem: the Illusion of Speed is real. The bug-to-code ratio is still a challenge — AI writes much faster, but it also generates potential errors at an unprecedented rate. And when an agent hallucinates (the model confidently invents something that isn't true), it doesn't create one bug: it creates a thousand lines of "vibe-consistent", functionally broken logic.

Writing speed (with agents)10×
Shipping speed (review + integration)~1.3×

If human reviewers are drowning in a sea of AI-generated PRs, writing speed becomes irrelevant: the process didn't get faster — it just created a bigger pile of "stuff" to triage later.

🎲 Vibe Coding

vague intent → generate fast → validate later (or never)
  • Great for prototypes and experiments
  • Unvalidated AI output
  • Failing is acceptable — it's a draft

🏭 Vibe in Production

solid spec → generate → verify → integrate
  • Everything intentional and controlled
  • Production-grade reliability
  • In enterprise: "Development with Agentic AI"
Hybrid Team Member

Agentic AI differs from standard Generative AI (smart autocomplete): the agent acts as a hybrid team member — it uses the LLM as a brain to generate and tools as hands to integrate: it reasons, writes specs and tests, uses the browser to test the UI, commits and merges to Git. There are ways to protect this process — but apply them from the start, not halfway through.

02

Spec-Driven Development

Most of your time is now spent writing specifications — code became a disposable byproduct.

In the traditional world, devs are taught to be "Code-First": vague idea → open the editor → type until something works. In the Agentic AI era, most of the time becomes writing high-quality specifications — detailed technical instructions that tell the AI exactly what to build. The dev's role moves closer to technical architect than to traditional coder.

🧱 Code-First

Vague idea → editor → type until it works. Emotional attachment to code that cost 12 hours of debugging.

📐 Spec-First

Solid spec → agent generates → regenerate whenever you want. The dev becomes an architect; the code, compiled output.

Code is disposable

With a solid spec, the entire codebase can be regenerated repeatedly — an agent can even convert an entire project from Python to JavaScript in an afternoon. No emotional attachment: since you didn't spend 12 hours debugging a semicolon, there's no fear of throwing it away and starting over if requirements change.

Coding agents use the LLM as the brain (reasoning) and tools as the hands (execution). The direct consequence:

🎲 Vibe instead of blueprint

vibe → the brain GUESSES → Rogue Agent
  • The model fills gaps with guesses
  • In enterprise, guessing = "Rogue Agent" incidents
  • An agent acting without verifying anything
vs

📐 Blueprint

spec → precise execution → production
  • Every requirement is written and reviewed
  • Code regenerable at any time
  • Auditable by humans and AI
03

Anatomy of a good spec

The spec is the architectural North Star — and the antidote to the digital "telephone game".

A production-grade spec works as the Architectural North Star: it prevents "context fragmentation" — the digital equivalent of the telephone game, where the AI loses the thread because it looks at outdated snapshots of files. The AI can co-author or review the spec; it lives in the codebase (a specs/ folder, in Markdown or YAML) and acts as the source of truth for humans and machines.

What a spec for a new project contains

📦 Full Technical Design

No "make a login page". Break it down: requirements, database schemas (the structure of the data) and API specifications (the "contracts" that let the parts of the software talk to each other).

🎨 Visual Aids

Diagrams + a list of specific tools and libraries with version numbers — without versions, the agent may suggest old releases.

🧭 Background Information

Give the agent the "why" behind the "what". Knowing the goal, it thinks ahead and anticipates the steps that will likely be needed.

🧪 Scenarios

What "good" looks like, what is wrong — and the edge cases. Scenarios are the raw material of tests.

Better a human catches a logic flaw in the design than waiting until the AI has already generated thousands of lines of broken code.

— author's tip: write technical designs in Google Docs, let many people review them, then File → Download → Markdown → specs/
🐴 Jet engine on a horse carriage

Keeping old processes with modern tools is trying to bolt a jet engine onto a horse-drawn carriage: technology cannot be screwed onto a 20-year-old workflow and expected to fly. The spec is the first screw of the new workflow.

04

The right format: YAML wins

LLMs are extremely sensitive to the format of instructions — up to a 40% performance drop with generic Markdown.

The SkCC study (Ouyang et al., 2026 — "Portable and Secure Skill Compilation for Cross-Framework LLM Agents") showed that agents exhibit extreme sensitivity to how instructions are formatted: up to a 40% performance drop with unoptimized generic Markdown. The researchers created SkCC (Skill Compiler): an ultra-fast tool that compiles the single-source instruction file into the model's optimal target format in under 10 milliseconds.

Parsing accuracy — deeply nested configurations

YAML
51.9%🏆 Winner for structured configurations and data schemas with nesting depth > 3
JSON
43.1%Heavy JSON inputs charge a reasoning and token "format tax"
XML
33.8%Maximum verbosity, minimum accuracy — avoid for agent instructions

Source: SkCC (Ouyang et al., 2026). For teams using Gemini, the absolute best strategy is the Markdown + Conditional YAML hybrid.

Hybrid strategy for Gemini

Use clean Markdown headers to anchor attention and switch to YAML for any structured configuration with nesting > 3. Rendering nested specs in YAML + narrative instructions in Markdown bypasses the "format tax" → Gemini operates at maximum accuracy and optimal token economy.

05

BDD & Gherkin

Turn vague human ideas into precise architectural design — no room for guessing.

A BDD spec is the ultimate tool for turning vague, ambiguous ideas into a precise design the agent can build without guessing. Behavior Driven Development uses simple, structured natural language to describe exactly how the system should behave from the user's perspective before any code is written. The standardized syntax is Gherkin: a declarative Scenario / Given / When / Then template that forces the LLM to think in State → Action → Outcome — completely eliminating "vibe coding" and keeping the agent on a strict track.

STATE · GivenACTION · WhenOUTCOME · Then
specs/payment_retry.featureexecutable spec — Gherkin syntax
Feature: Retry de pagamentoScenario: Cartão recusado, nova tentativa automáticaGiven um pedido "#8842" com pagamento "recusado"And o cliente tem 2 tentativas restantesWhen o webhook "payment.retried" é recebidoThen o sistema agenda nova tentativa em 30 minutosAnd o cliente recebe notificação por email

Executable specs beat prose: each scenario becomes a verifiable test, and the agent follows a strict track instead of interpreting ambiguous paragraphs.

⚛️ The physics of tokens

LLMs don't interpret data structures — they process tokenized text. Every character sent is broken into tokens; each token consumes budget, time and context capacity. Writing production-grade specs means treating tokenization as a hard physical constraint: every newline and indentation space translates directly into development budget and latency. Even generous platforms like Antigravity are bounded by the token physics of the underlying models — every unnecessary space in nested YAML and every repetitive Given/When/Then consumes cycles and attention-heads in multi-turn reasoning loops. Treat /specs not as documentation, but as a compiled, lean instruction set: human-readable Markdown + flat, highly targeted YAML blocks.

06

Where instructions live

Three layers with different scopes and lifetimes — dumping everything into the chat exhausts the context.

To practice SDD, understand how coding tools consume instructions: they are not written in a single place. Dumping a massive 100-page system design document straight into the chat window exhausts the short-term context budget, increases latency and fragments the context. Instructions live in three layers:

Chat Interface — short-lived, session-specific layer 1

The IDE's ephemeral conversational box (Gemini side-panel or terminal). It lives with the dev's active session — use it purely for high-level orchestration and instant feedback loops.

  • Example: "Review the design in specs/payment_retry.md and generate the failing unit tests defined in Scenario 3."
  • Never: entire specs pasted into the prompt (manual prompt-stuffing)

Spec Folder — task-specific, versioned layer 2

A static folder committed directly to the repository: technical design, BDD scenarios, API contracts, structural YAML schemas. The agent indexes the directory dynamically to build and verify code without manual prompt-stuffing.

  • Example: ./my-app/specs/my_spec.md
  • Source of truth shared by humans and agents

Agent Skills — reusable, feature-focused layer 3

Structured Markdown files with trigger-based specialized workflows. They teach repeatable engineering habits (e.g.: automatically maintaining CHANGELOG.md when code changes are detected). The skills folder can also contain data assets and scripts.

  • Example: ./my-app/.agent/skills/docs-maintenance/SKILL.md
  • They should live in the .agent directory so the Antigravity workspace manager recognizes them
+ Global layer: System Prompts

Gemini CLI and Antigravity scan and concatenate context hierarchically, from global overrides down to local configurations: Global Profile (~/.gemini/GEMINI.md — universal persona, default style and core principles, project-independent) → shared AGENTS.md (a cross-tool shared foundation for teams with multiple AI clients; the local GEMINI.md keeps priority for Google-specific configs) → Project Spec (./my-app/.gemini/GEMINI.md — the project's DNA, detected and read automatically).

07

The 5 execution modes

Five characters, five mindsets: pick the prompt by the job, not by habit.

There is no single way to turn a spec into code — each job calls for a different execution mode. Click the characters to see each one's playbook:

🏛️
Architect
Project Generation
🔨
Builder
Feature Generation
🔬
Forensic Specialist
Bug Fixing
✍️
Author
Documentation
📚
Librarian
Data Engineering
Cross-cutting golden rule

Version numbers for every library, always. The model's knowledge cutoff is in the past: without an explicit version, the agent suggests old releases — and even suggests lower versions of models (e.g. gemini-1.5-flash) simply because newer ones don't exist in its training. Proposed versions must always be double-checked; use the editor's RAG or download documentation as Markdown into specs/, skills or profile prompts.

08

MCP: one integration, every framework

The "USB-C of AI tools" — build one server, connect any agent.

The Model Context Protocol (created by Anthropic, now an open standard) is nicknamed "the USB-C for AI tools" — an exaggeration, but the analogy captures the idea: build one MCP server for your database, API or file system, and any compatible agent can use it without writing a custom integration.

🗄️

1 MCP server

mcp_server.py · "knowledge-base"
🔌
🤖 Antigravity
⌨️ Gemini CLI
🧠 Your ADK agent
🔧 Any MCP client
mcp_server.pySnippet 1 — exposing a SQLite database as 2 tools (~40 lines)
from mcp.server import Serverfrom mcp.server.stdio import stdio_serverimport sqlite3 server = Server("knowledge-base") conn = sqlite3.connect("knowledge.db")@server.list_tools()async def list_tools():return [{"name": "query_knowledge","description": "Query the knowledge base with SQL","inputSchema": {"sql": "SQL query to execute (SELECT only)"}},{"name": "add_knowledge","description": "Add a new knowledge entry","inputSchema": {"title": ..., "content": ..., "tags": "Comma-separated tags"}}, ]@server.call_tool()async def call_tool(name, arguments):if name == "query_knowledge": sql = arguments["sql"]if not sql.strip().upper().startswith("SELECT"):return "Error: Only SELECT queries allowed"rows = conn.execute(sql).fetchall()return [dict(zip(cols, r)) for r in rows]   # → TextContentif name == "add_knowledge": conn.execute("INSERT INTO knowledge (title, content, tags) VALUES (?, ?, ?)", ...) conn.commit(); return "Knowledge entry added."async def main():async with stdio_server() as (r, w):await server.run(r, w, server.create_initialization_options())
On the other side of the cable

The client (mcp_client.py, Snippet 2) is symmetrical: StdioServerParameters(command="python", args=["mcp_server.py"])session.initialize()session.list_tools()session.call_tool("query_knowledge", {"sql": "SELECT * FROM knowledge WHERE tags LIKE '%agent%'"}). One integration, every framework.

09

Team culture & process evolution

Huge PRs, merge conflicts and the telephone game: what changes when the whole team uses agents.

Working with modern coding agents demands a mindset and culture shift. Without it, the classic scenario: PRs become huge, merge conflicts multiply (devs hitting the same files) and the dependency chain becomes impossible to untangle — PR #1 can't merge without PR #2, which needs PR #3, blocked by a reviewer in another timezone. Some changes approved while related ones wait → even more conflicts. Suddenly: broken branch.

⚔️ Merge conflicts

Multiple devs (and their agents) hitting the same file within an hour.

🪆 Review gridlock

The massive PR becomes a "Russian doll" of nested sub-PRs, impossible to review in one go.

🧩 Context fragmentation

While you're away, a colleague renames a variable in a shared file; your agent, citing an outdated snapshot, generates code that calls a function that no longer exists.

Strategies for high-velocity integration

📋 Bundled Summaries & Risk Assessments

Every PR includes an AI-generated snapshot of what changed, potential breaking points and a risk assessment (Markdown or commit description) — the human reviewer focuses on architectural impact instead of getting lost in the lines.

🎯 Reimagined Ownership

Human review moves away from "style nitpicking" on disposable agent-written code and toward guaranteeing the integrity of architectural blueprints. Style is a job for automated tools: shared linters and stylebooks (SKILLS.md).

⏱️ The "Conditional LGTM"

Eliminates 12-hour delays in cross-timezone teams: the reviewer approves the PR contingent on all automated tests passing — if they go green, the code merges automatically.

🕊️ No-Blame Culture

In high-velocity environments, whoever produces the most code becomes the easy scapegoat for bugs and conflicts. Attribute those problems to broken integration processes — not to the individual dev using the agent.

And the uncomfortable question

If you can work with a squad of agents, do you really need to work as a team? If the answer is yes, split the work so members rarely touch the same files (clear ownership of APIs vs UX); when overlap is unavoidable, a designated "part owner" handles the final synchronization. And automate: you can write skills that do code review — and even skills that respond to code reviews (Snippet 3, code-check.md: analyzes critical vulnerabilities, logic, readability and edge cases, returning Description + Critical / Warnings / Best Practices / Quick Win), fired via GitHub Actions or Gemini Code Assist on GitHub.

code-check.mdSnippet 3 — code-review skill: critical vulnerabilities, logic & efficiency, readability and edge cases
Act as a Senior Software Engineer and Security Researcher. Review the provided code for this Github PR or Diff using these strict criteria: Use the command line to fetch the Github PR: `gh pr view <PR NUMBER>` First analyze the code, then code review: 1. **Critical Vulnerabilities:** Check for hardcoded secrets (API keys), SQL injection, XSS, or broken authentication. 2. **Logic & Efficiency:** Identify "off-by-one" errors, infinite loops, or redundant API calls. 3. **Readability:** Suggest better naming conventions or breaking down "megafunctions" into smaller pieces. 4. **Edge Cases:** What happens if the input is null? What if the network fails? Output Format: - **Description:** - What is this PR doing? Explain in details. ISSUES: -⚠ **Critical:** (Stop-ship issues) -⚠️ **Warnings:** (Code smells or style issues) -✅ **Best Practices:** (Specific lines to refactor for better performance) -💡 **Quick Win:** (One sentence summary of the biggest improvement) When there are no issues return - **Description:** - What is this PR doing? Explain in details. LGTM
0%
more likely to experience high burnout among frequent AI users (Quantum Workplace, via CNBC) — team culture is also a safety net.
10

The 3 tiers of code review

Who runs the reviewer on every PR, with no human pressing a button? A spectrum of control × simplicity.

You can write a great review prompt — but the skill only runs when invoked from inside the IDE. The next step is the continuous reviewer: services that watch the repository, react to events (PR opened, nightly cron) and post findings without anyone asking. They catch what tired reviewers miss on a Friday afternoon: a dependency with a new CVE, a 6-month-old TODO that became a silent breach. When the team ships AI-generated PRs at volume, the continuous reviewer is the only thing that scales with the output. The question is how custom you need to go — the answer is a 3-tier spectrum:

simplicitycontrol
CriterionTier 1 · ManagedTier 2 · HybridTier 3 · Custom
ExampleGemini Code Assist on GitHub · SaaS reviewerGitHub Action + coding agent CLI (Antigravity CLI)ADK agent on Gemini Enterprise Agent Engine
SetupEnable in the org · minutesSkill in the repo + CI action · ~1 dayOwn runtime + webhooks · weeks
RuntimeThe vendor's · pay per seatThe CI provider'sYours (Agent Engine: Sessions + Memory Bank)
Review criteriaThe vendor's (generic)Yours (skill committed to the repo)Yours + long-term memory
Memory across runsNoNoYes — cross-PR context, codebase memory
You ownNothing beyond the subscriptionPrompts, model, sandboxing, criteriaEverything: eval, observability, cost, on-call
Main tradeThe vendor's opinions, not yoursRight starting point for most teamsMaximum power · maximum operating cost

The 3 questions that tell you which tier you need

1️⃣ How specific are your criteria?

Generic → Tier 1. Team/repo-specific → Tier 2 or 3.

2️⃣ Does the agent need to remember across runs?

No → Tier 1 or 2. Yes (codebase memory, cross-PR context) → Tier 3.

3️⃣ What's the worst case if it goes wrong?

Noisy comment → any tier. Merged regression or leaked secret → Tier 3 with a Policy Server in front of every tool call.

How teams discover their own tier

The moment the managed reviewer misses something specific. Example from the paper: a platform team at a mid-size fintech started on Tier 1 and discovered the compliance reviewer was flagging boilerplate auditors had already approved — while missing the one pattern that mattered: unmasked PII in log statements. A 40-line compliance-check.md skill on a GitHub Action (Tier 2) crushed the false positives within a week. Tier 3 wasn't needed yet. Practical rule: pick the lowest tier that catches what matters.

11

Tier 3 at full scale: graph-native review

Not an agent that watches PRs — one that understands the entire system the PRs live in.

In hundred-million-line legacy codebases, loading code as plain text into the context window runs out of space, and standard RAG removes the structure that makes code readable (a class belongs to a file, a function call points to a requirements doc written a decade ago). Flattening everything into a vector store = the map disappears. The pattern that emerged from the biggest modernizations: build the agent on top of a knowledge graph — ingest code, docs, tickets and design PDFs into a graph database (e.g. Spanner Graph) and combine 3 retrieval modes:

🕸️
GRAPH TRAVERSAL (GQL)
Structural queries: "every function that transitively calls payment.process()"
🧬
VECTOR SEARCH (ANN)
Semantic queries over node embeddings: "find code that does what this paragraph describes"
🔎
FULL-TEXT SEARCH
Exact identifier matches
🗺️
IMPACT MAP
"What breaks if I change this?" answered with precision — not with a confident guess
🔍 Search agent📖 Story agent💥 Impact agent🧱 Task-breakdown⌨️ Coding agent

The second half is decomposition: a single agent instructed to "refactor this module" fails. Split into an ADK sub-agent pipeline — explore the graph, capture requirements, predict side effects, produce atomic units of work and ONLY THEN code — and the work becomes manageable.

Figure 1 — Graph-Native Code Understanding Architecture

2 wk → h
Production pilots on million-line codebases moved equivalent refactoring work from two weeks to a few hours.
100M+
lines of legacy code — the scale at which only graph-native retrieval survives (case study: Siemens).

Summary of the spectrum: Managed = generic reviewer in minutes · Hybrid = YOUR reviewer in a day · Custom = a reviewer that understands the ENTIRE system — at the cost of owning the runtime and the evaluation.

12

Approval fatigue: the sustainability of the process

If every tool call asks for approval, nobody really approves — they just click.

A new phenomenon: faced with a constant flow of micro-approvals (improve a single line, adjust a tool call), devs start clicking "Approve" reflexively. It's a form of low-grade exhaustion where the team stops verifying the machine's work just to keep up with the pace — and loses attention to detail. Constant supervision doesn't scale; structured boundaries do.

🌙 Digital Quiet Hours

Explicit boundaries so approval requests don't leak into nights and weekends. An agent that never sleeps cannot mean a human who never sleeps.

🤝 Agent Insight Sessions

Weekly sessions where devs share patterns identified by their AI counterparts — turning isolated findings into shared organizational knowledge.

Traffic lights + referee, not a traffic cop on every corner

The answer to fatigue is not removing guardrails — it's calibrating them: cheap deterministic rules (traffic lights) for the obvious, intelligent judgment (referee) for the nuanced, and humans only where risk truly demands it. That is exactly the Policy Server design from section 18.

13

The email incident: chain reaction

One innocent prompt, YOLO mode, and fifty colleagues receiving hallucinated content.

During a routine update, the author discovered the power — and the limits — of Antigravity's built-in browser: the feature lets the agent interact with applications under development without login credentials (invaluable for UX testing). But in YOLO mode (auto approve), the agent can act faster than a human can think. A simple prompt to create a button triggered the following chain:

🖱️
Simple prompt: "create a button"
🌐
The browser agent clicks the new button, autonomously
✉️
The button was meant for an email agent — no URL specified
🌀
With no data, the agent HALLUCINATES and connects to a deprecated legacy agent with no safeguards
💥
50 colleagues receive fake emails full of hallucinated content
⚠ INCIDENT — the agent fulfilled the directive with the available data, without ever verifying whether it SHOULD

The incident highlighted the risk of context hallucination: when the AI doesn't have enough data, it fills gaps using whatever strings exist in the context — including sensitive information like hardcoded email addresses or URLs. It may seem minor if it's "just an email". But consider what the agent was doing: fulfilling its directive with the available data, without any verification of whether it should. That is the core risk of autonomous systems.

Guardrails are not optional; they are what keeps a useful tool from becoming unpredictable.

— without human-in-the-loop or a policy engine, the agent optimizes for the goal using whatever it finds
14

Zero-Trust for agents

Never trust the model's self-policing — governance must be external and tamper-proof.

As the boundaries of Agentic AI expand, a paradox emerges: agents must be autonomous enough to solve complex problems, but you can't afford the risk of them going "rogue" in an enterprise environment. Imagine an agent tasked with "resolving customer disputes": to be effective, it needs access to customer data, email tools and internal systems — but the challenge is guaranteeing it doesn't accidentally email the entire database or share proprietary code.

🚫 The model policing itself

restrictions in the system prompt → fragile
  • LLMs are probabilistic, not deterministic
  • Contexts overflow; rules get lost
  • Prompt injection "convinces" the agent to bypass rules
vs

🛡️ External enforcement

external governance → tamper-proof
  • Policies outside the model, in the runtime
  • Every tool call intercepted before execution
  • The agent cannot edit its own rules
🧱

Sandboxing

A restricted execution environment that contains destructive actions (section 15).

HITL Checkpoints

Human sign-off for high-risk actions (section 16).

🚦

Policy Server

Structural + semantic gating before external systems (section 18).

📎 Companion paper

To go deeper on protecting and evaluating agents against malicious code, see the Day 4 — Vibe Coding Agent Security and Evaluation guide (link in the Companions section).

15

Sandboxing & blast radius

If the agent gets tricked, the damage must fit inside a disposable box.

Beyond sanitizing strings, real security requires a restricted execution environment to contain the agent's actions. Even with rigorous output filtering, the LLM can generate syntactically valid but logically malicious code. Running tasks in ephemeral, low-privilege containers — isolated from the main network and sensitive file systems — creates a "blast radius" that protects the core infrastructure: if the agent is tricked into running a destructive command, the damage is confined to a disposable instance, cleaned and reset with no consequences.

host untouched
ephemeral sandbox
🤖
agent

destructive command → hard permission error at kernel level → host completely untouched

⚙️ In Antigravity: one toggle

User Settings → enable "Terminal Sandboxing". Done: the agent's commands run contained.

🐳 For the team: portable cloud sandbox

Containerize the workspace: a custom Dockerfile (e.g. .gemini/sandbox.Dockerfile) starting from the official Gemini CLI sandbox image, inject scoped cloud credentials and force the mode with export GEMINI_SANDBOX=docker.

16

Human-in-the-loop & testing

Automation is the goal — but high-risk operations need a human at the checkpoint.

Although automation is the goal, high-risk operations require a Human-in-the-Loop (HITL) protocol as the final fail-safe: checkpoint gates for actions that match a specific risk profile. Presenting the agent's sanitized intent to a human supervisor for manual sign-off balances AI speed with the dev's nuanced judgment — and guarantees that final responsibility for architectural integrity stays in human hands.

🚀

Deploy to production

AI-generated code only goes up with explicit sign-off.

🗃️

Database schema changes

Migrations are too irreversible for auto-approve.

💸

Financial transactions

No agent initiates money movement alone.

The surge of AI-generated tests

The surge of AI-generated code pushes the process from manual testing to AI-generated test coverage — and here AI has a structural advantage: since implementation is no longer the bottleneck, it can write broader test coverage than any human in the same time, programmatically and powerfully. In a high-velocity environment, test-driven development becomes real: the machine writes the very tests that validate its output.

1 · FAILING test2 · agent fixes3 · suite green4 · merge with confidence

The process forces the agent to produce a failing unit test or a reproduction command (like a curl request) before attempting any fix. Embedding these tests in the codebase means every fast iteration is backed by a verifiable suite — bugs don't come back, and human reviewers can trust the automated "green light" for integration.

17

Continuous evaluation

Traditional tests are insufficient when output is GENERATED, not COMPUTED.

Why special quality checks for ML-driven systems? Because traditional software tests are insufficient for systems whose output is generated rather than computed. An agent (or any ML-driven component — classifier, summarizer, retriever) can pass 100 unit tests on its tools and still fail spectacularly by picking the wrong tool, paraphrasing a critical answer or hallucinating a fact. The error margin is not a defect to eliminate — it's an inherent property of the model, and the testing strategy must accommodate it.

🧪 Unit test

"Did the function return the right value?"
  • Binary answer: pass or fail
  • Catches deterministic regressions
  • Assert flips → gate fires

📊 Evaluation

"Is the agent's behavior at least as good as the baseline?"
  • 0–5 score from an LLM-as-judge (scorecard)
  • Trajectory verification that tolerates ordering variance in tool calls
  • The gate fires when quality drops below a configurable margin — not when an assertion flips
0–5
scored judgments and tolerance bands replace binary assertions
baseline
every round is compared against the previous one — behavioral drift becomes a number
loop
continuous eval in CI: generate → evaluate → compare → only then integrate

Tests catch deterministic regressions; evaluation catches behavioural drift.

— the golden rule of quality in ML systems
18

The Policy Server

Two gating layers intercept every action before it reaches external systems.

The paper's central guardrail example is the Hybrid Policy Server: middleware that intercepts actions before they reach external systems, operating in two complementary layers.

🚦

Structural Gating — the traffic lights

fast · deterministic · cheap
  • Deterministic rules based on roles and environments
  • Binary checks: role viewer cannot use send_email
  • Prevents architectural violations WITHOUT asking an LLM
🧑‍⚖️

Semantic Gating — the intelligent referee

nuanced · LLM judge · policies.yaml
  • A specialized secondary LLM (Gemini) inspects intent and content against natural-language policies
  • For when the tool IS ALLOWED, but the WAY it's used violates policy: an admin may use send_email, but not with unmasked PII
  • This is where structural rules fail — you can't regex every possible PII leak
policies.yamlSnippet 4 — deterministic rules
environments:localhost:blocked_tools:- send_emailroles:viewer:allowed_tools:- list_files- read_file
policy_server.pySnippet 5 — runtime interception
def is_tool_allowed(self, tool_name):# 1) Environment blocksif tool_name in env_blocked: return False# 2) Role permissionsreturn "*" in role_allows or \ tool_name in role_allowsdef check_action_semantic(self, action_description): client = Client(vertexai=True) prompt = "Evaluate if this action violates " \f"PII policies: {action_description}"response = client.ai.models.generate_content( model="gemini-3.1-pro", contents=prompt)return not response.text.startswith("VIOLATION")

🎛️ Simulate the Policy Server

When the agent decides to use a tool, the flow is intercepted: structural check (is the tool allowed for this role/env?) → semantic check (are the arguments safe?) → execution (if both pass) or a "Policy Violation" message returned to the agent for self-correction or graceful failure. Pick a scenario:

Intaketool call received
Structuralrole/env · YAML
SemanticLLM judge · PII
▸ pick a scenario above to see the verdict…
Critical separation of concerns

The Policy Server creates a safety net that separates execution logic from governance logic — the critical separation of concerns in enterprise software. The agent executes; the server decides what may be executed.

19

Context hygiene & the Context Resolver

The agent should never see real PII — only placeholders resolved at the last mile.

A significant danger of autonomous development is Context Hallucination: without specific data, the agent fills gaps with whatever strings are available in the context — potentially leaking hardcoded email addresses or private URLs. The mitigation is rigorous Context Hygiene via middleware: PII masking and placeholder injection, so the agent always operates on sterilized data. And every agent output must be sanitized against prompt injection and rogue UI interactions — the machine's "vibe" must never become an architectural vulnerability.

1 · Raw tool outputarguments with real PII: ana@corp.com
2 · PII scrubregex replaces it with a placeholder: [[COMMENTER_EMAIL]]
3 · Truncatecut to fit the context budget
4 · Injectsterilized context enters the prompt
context_resolver.pySnippet 6 — dynamic placeholders
def resolve_context(template_str, override_state):def replacement(match): var_name = match.group(1)# 1) Prioriza overrides de runtime stateif var_name in state_to_check \and state_to_check[var_name] is not None:return state_to_check[var_name]# 2) Fallback para env vars validadasif var_name in os.environ:return os.environ[var_name]# 3) Deixa não resolvido — sem falhas silenciosasreturn match.group(0)return re.sub(r'\[\[([^\]]+)\]\]', replacement, template_str)# ex.: resolve [[COMMENTER_EMAIL]] dinamicamente
tool_policy_engine.pySnippet 7 — middleware in the agent pipeline
def validate_tool_call(tool_call): args = tool_call.function_call.args resolved_args = for k, v in args.items():if isinstance(v, str): resolved_args[k] = resolve_context( v, override_state)elif isinstance(v, list): resolved_args[k] = [resolve_context(i, override_state)if isinstance(i, str) else ifor i in v]else: resolved_args[k] = v args.clear(); args.update(resolved_args)# intercepta TODA tool call ANTES de rodar

With the middleware wired directly into the execution pipeline, any attempt by the agent to run an action — sending email, querying a cloud presentation — is intercepted. The engine translates placeholders like [[COMMENTER_EMAIL]] or [[DEFAULT_PRESENTATION_ID]] into authorized test assets, safely and silently — eliminating hardcoded PII from test suites and system prompts.

20

Summary & where to start

The bottleneck moved — and the whole blueprint boils down to three commands.

In less than a year, development cycles became dramatically faster. But the speed revealed the important shift: AI eliminated the code-production bottleneck and moved the constraint downstream — to the humans who must review, test and integrate that output. This is shared cognitive load: humans act as architects (Test Specs, Integration Specs, MLOps/DevOps blueprints), while AI handles the heavy lifting (actual test code, integrations, granular operational details).

🏭 Old bottleneck: production

Writing code was the hard part. AI solved that — a thousand lines before lunch.

🧭 New bottleneck: integration

Verify, integrate and deliver. Better prompts and faster models alone won't fix this.

Success depends on evolving team dynamics, refining collaboration with agents and setting strict boundaries for tools that never sleep. The challenge changed from mere code production to orchestrating systems that verify, integrate and deliver work.

🚀 The patterns become commands you can run today

terminaluv google-agents-cli setup — installs the 7 skills on your coding agent (scaffolding, ADK code, evaluation, deployment, publishing, observability)
# geração de projeto spec-drivenagents-cli scaffold# gate de cobertura de testes gerada por IAagents-cli eval run# deployment com sandbox para Cloud Run ou Vertex AI Agent Engineagents-cli deploy

Vibes prototype. Specs ship.

— the paper summed up in four words
21

Quiz: do you survive production?

Eight questions covering the whole paper — from specs to zero-trust.

22

Copyable cheatsheets

Three artifacts ready to paste into your repository.

spec-template.mdCheat 1 — production-grade spec template (Markdown + YAML hybrid)
# Feature: Payment Retry## 1. Background (o "porquê")Contexto de negócio, restrições, links para docs de design.Payments falham em ~3% dos checkouts; retry automático recupera ~40%.## 2. Technical Design (o "quê")requirements:- idempotency_key em toda tentativa- máximo de 3 tentativas, backoff 30/120/480 mindatabase_schema:payment_retries: {id, order_id, attempt, next_at, status}api_contract:POST /v1/payments/{id}/retry → 202 Accepted## 3. Libraries (com versão SEMPRE)dependencies:fastapi==0.115.0sqlalchemy==2.0.36## 4. Scenarios (Gherkin — vira teste)Scenario: Cartão recusado, nova tentativa automáticaGiven um pedido "#8842" com pagamento "recusado"When  o webhook "payment.retried" é recebidoThen  o sistema agenda nova tentativa em 30 minutos## 5. Out of scopeO que NÃO construir — evita alucinação de features.
gherkin-quickref.mdCheat 2 — Gherkin/BDD quick reference
# Gherkin — State → Action → OutcomeFeature: Nome do comportamento (perspectiva do usuário)Scenario: Caminho felizGiven [STATE]  um estado inicial verificávelAnd   [STATE]  condições adicionaisWhen  [ACTION] o evento/ação que dispara o comportamentoThen  [OUTCOME] o resultado observável esperadoAnd   [OUTCOME] efeitos colaterais verificáveisScenario: Edge case — rede falhaGiven um pedido pendenteWhen  a gateway retorna timeoutThen  o sistema agenda retry e notifica o clienteRegras de ouro: Declarativo, nunca imperativo (descreva O QUÊ, não COMO) Cada Scenario = um teste executável Inclua o "bom", o "errado" e os edge cases Curto: cada token consome budget e attention-heads
zero-trust-checklist.mdCheat 3 — zero-trust checklist for agents
# Zero-Trust Checklist — agentes em produção[ ] Sandboxing[ ] Terminal Sandboxing habilitado (Antigravity) ou GEMINI_SANDBOX=docker [ ] Containers efêmeros e de baixo privilégio, isolados da rede principal [ ] Credenciais cloud limitadas por escopo, nunca amplas[ ] Human-in-the-Loop[ ] Checkpoint gates: deploy em produção [ ] Checkpoint gates: mudança de database schema [ ] Checkpoint gates: transações financeiras[ ] Policy Server (2 camadas)[ ] Structural: blocked_tools por ambiente, allowed_tools por role [ ] Semantic: LLM juiz contra policies.yaml (PII não mascarada) [ ] Toda tool call interceptada ANTES da execução[ ] Context Hygiene[ ] PII masking + placeholder injection ([[VAR]]) [ ] Context resolver conectado ao pipeline (tool_policy_engine) [ ] Outputs sanitizados contra prompt injection[ ] Verificação contínua[ ] Teste que falha ANTES de qualquer correção [ ] Eval com scorecard 0–5 vs baseline (behavioural drift) [ ] Revisor contínuo de PRs no tier adequado (1/2/3)[ ] Nunca[ ] Modo YOLO (auto approve) sem guardrails [ ] Confiar na auto-polícia do system prompt [ ] Versões de biblioteca sem verificação dupla
23

Adoption checklist

Check what your team already practices — progress is saved in the browser.

📐Adopt SDD

Create the specs/ folder in the repository
Write the first spec in the hybrid format (Markdown + YAML)
Use Gherkin (Given/When/Then) in the scenarios
Define GEMINI.md / AGENTS.md per layer (global → project)
Require version numbers for every library

🛡️Build the safety net

Enable Terminal Sandboxing
HITL checkpoints for deploy, schema and finance
Policy Server: structural layer (policies.yaml)
Policy Server: semantic layer (LLM judge)
Context resolver in the tool-call pipeline

🔄Scale review & culture

Choose the code-review tier (1, 2 or 3)
Bundled summaries + risk assessment on every PR
Adopt the "Conditional LGTM"
Continuous eval with scorecards vs baseline
Digital Quiet Hours + Agent Insight Sessions
24

Glossary

The minimum vocabulary to navigate the paper.

Vibe Coding
Generating code quickly from high-level intent/vibe. Great for prototypes — never for production without validation.
SDD (Spec-Driven Development)
Development guided by high-quality specifications; code becomes disposable, regenerable output.
Spec
A detailed technical instruction telling the AI exactly what to build; source of truth for humans and agents.
Gherkin
The standardized BDD syntax (Scenario/Given/When/Then) that forces the LLM to think in State → Action → Outcome.
MCP
Model Context Protocol — the "USB-C of AI tools": one server, any compatible agent.
Sandbox
A restricted execution environment (ephemeral container, low privilege) that contains the agent's actions.
Blast Radius
The maximum damage area if something goes wrong — the sandbox keeps it inside a disposable instance.
Zero-Trust
Never trust the model's self-policing: external, tamper-proof governance over every tool call.
HITL (Human-in-the-Loop)
Checkpoint gates with human sign-off for high-risk actions: deploy, schema, finance.
Eval (Evaluation)
Scored judgments (0–5, LLM-as-judge) and tolerance bands that catch behavioural drift — tests catch regressions.
Policy Server
Middleware that intercepts actions before external systems: structural gating (deterministic) + semantic (LLM judge).
Approval Fatigue
Exhaustion from micro-approvals that leads devs to click "Approve" reflexively — without checking.
Hallucination
The model confidently invents something that isn't true; without data, it fills gaps with strings from the context (context hallucination).
Context Resolver
A utility that resolves [[VAR]] placeholders via runtime state/env vars, keeping PII out of the agent's context.
25

Companion guides

The complete series — from the new SDLC to agent security and evaluation.

HUBstarting point

Series hub — all days

The navigable index of every study guide in the whitepaper series.

indexnavigation
D1foundations

The New SDLC with Vibe Coding

How the development lifecycle changed with agents — the foundation this guide builds on.

new SDLCvibe codingworkflow
D2tools

Agent Tools & Interoperability

The 5 open protocols that connect agents to tools and to each other.

MCPA2AA2UI
D3context

Context Engineering: Sessions, Memory & Skills

The direct complement to the context hygiene section: how sessions, memory and skills feed the agent.

sessionsmemoryskills
D4security

Vibe Coding Agent Security and Evaluation

Explicitly referenced in this paper: protecting and evaluating agents against malicious code, in depth.

securityevaluationquality gates
26

References

All 17 endnotes from the paper + the main citation.

[1]Google. Antigravity — agentic development platform (built-in browser, Terminal Sandboxing, workspace manager).
[2]Google. Gemini CLI — command-line agent with sandboxing and hierarchical system prompts (GEMINI.md).
[3]OUYANG et al. SkCC: Portable and Secure Skill Compilation for Cross-Framework LLM Agents, 2026 — format sensitivity (−40%) and skill compiler <10 ms.
[4]SkCC study (Ouyang et al., 2026) — parsing accuracy on nested configs: YAML 51.9% · JSON 43.1% · XML 33.8%.
[5]Cucumber. Gherkin Reference — Given/When/Then syntax for Behavior Driven Development.
[6]Google Cloud. Google Cloud Data Extension for IDEs — cloud data access straight from the editor.
[7]GitHub. GitHub Actions — CI automation to fire review skills on every PR.
[8]Google. Gemini Code Assist on GitHub — managed PR reviewer (Tier 1).
[9]Google. Antigravity CLI — coding agent CLI in non-interactive mode for CI pipelines (Tier 2).
[10]Google. Gemini Enterprise Agent Engine — managed runtime with durable Sessions and Memory Bank (Tier 3).
[11]Google. A2A (Agent2Agent) protocol — coordination between agents.
[12]Google Cloud. Spanner Graph — graph database for the code knowledge graph (GQL traversal).
[13]Google. ADK (Agent Development Kit) — sub-agent pipelines (Search, Story, Impact, Task-breakdown, Coding).
[14]Quantum Workplace (via CNBC) — frequent AI users are 45% more likely to experience high burnout.
[15]Google. Antigravity — Terminal Sandboxing (User Settings).
[16]Google. Gemini CLI sandbox image — official Docker image + GEMINI_SANDBOX=docker.
[17]Siemens — legacy modernization case study with agents (Tier 3 at scale).
Main citation

BOONSTRA, Lee. "Spec-Driven Production Grade Development in the Age of Vibe Coding: The Blueprint for Scalable Workflows and Team Evolution — From Vibe Prototypes to Production Reality". Google, May 2026.