Reprobe: reproduce the bug, verify the fix

Reprobe reads a bug report (steps, screenshots or a screen recording), follows it in a real browser inside a disposable gVisor sandbox on Vultr, and returns a verdict with proof: expected vs. observed, the step, and a recording. Run the same check again after the fix with one click. Every model call goes through Vultr Serverless Inference.

Live demoGitHub0

Overview

What it is:

Reprobe is an agent that reproduces web bugs and comes back with proof. You give it what a bug report usually has (written steps, screenshots or a screen recording) plus the app URL and what should have happened. It opens a real browser inside a disposable sandbox, follows the steps, checks the result on the page, and returns a verdict.

The problem:

Every bug gets checked by hand, twice or more. Once to find out if an old ticket is still real, and again after the fix, then again the next time someone touches that code. Same steps, by hand, every time.

How it works:

1. Reads the evidence. A vision model turns steps, screenshots and video frames into a numbered plan. Every step cites its source ("written step 5", "video 00:07").

2. Starts a sandbox. A root-owned helper starts a fresh gVisor container that runs only Chromium.

3. Runs the plan. The agent (built on LangChain deep agents and Playwright) drives that browser from the VM, types like a person, and reads form values as well as page text.

4. Checks the result. The verdict is reproduced, not reproduced or inconclusive, with expected vs. observed and the step where it showed. The verdict tool rejects any observed value that isn't on the page word for word, so the agent can't invent evidence.

5. Returns proof and destroys the sandbox. You get a recording of each tab with click markers and the full action log, and Run again repeats the same check later and keeps every result in the bug's history.

You can watch live, take over the browser, answer the agent's questions, or stop a run.

Real numbers from our demo:

On RoomPilot, a demo booking app, bug RP-214 (moving a meeting left it in both calendar slots) came back Reproduced in 47 seconds. After a ten-line fix, Run again came back Not reproduced in 39 seconds. Both runs sit side by side in the bug's history. A SauceDemo checkout bug reproduces in about 20 seconds, using about 120k input tokens (mostly cached) and 600 output tokens, roughly $0.10 at list price.

Containment:

Each run gets its own gVisor container with 1.5 vCPU, 1536 MB, 256 processes and a 900-second cap. It's removed after every run, and a reaper catches anything a crash leaves behind. The browser can reach only the app under test, plus read-only files from a fixed CDN list; everything else is blocked and logged. No API keys, env vars or mounts go into the sandbox. Saved test logins are AES-256-GCM encrypted on the server, never shown to the AI, and typed masked only into the allowed site.

Honest trade-offs:

- Everything runs on one VM, so concurrent runs are capped and there's no autoscaling yet.

- The firewall pins the target's IP, not its name. On shared hosting, a hand-built request to another site on the same IP could get through. DNS is off and Chromium's resolver rules allow only the target, so the browser can't make that request by name.

- Pages can load scripts, styles, fonts and images from a fixed list of public CDNs. That makes real apps render correctly, at the cost of a slightly wider allowlist.

- A saved test login does reach the sandboxed browser, as keystrokes into the target site's own login field. Nothing is stored there, and the browser is destroyed afterwards.

- Runs are capped at 10 minutes, and only ports 80 and 443 on the target are allowed.

Try it:

Live demo at https://144-202-90-160.sslip.io. Sign up with a real email, since account creation sends a verification code. The repository is named Navgent because that was the working name before the product became Reprobe. It's the same project.

How it uses Vultr

Reprobe runs entirely on Vultr, in the Seattle (sea) region. Vultr Cloud Compute: one VM (plan vhp-4c-8gb-amd, 4 vCPU, 8 GB, NVMe, Ubuntu 24.04) runs everything: Caddy for HTTPS, the FastAPI web app, the browser agent, and Docker with gVisor for the sandboxes. We created and manage it with vultr-cli. One VM keeps the design simple and makes the trust boundary clear: the agent, prompts and inference key stay on the VM, and only Chromium runs inside each sandbox. Sandboxes on that VM: each run gets a fresh Docker container under gVisor (runsc), with 1.5 vCPU, 1536 MB of memory, 256 processes, a read-only root and no capabilities. A root-owned helper (sandboxctl) is the only privileged step. It pins the target host, adds per-run iptables rules (target only, plus HTTPS to a fixed CDN list), and removes the container and its rules after every run. A systemd timer reaps anything left over every five minutes. Vultr Firewall: a firewall group that only exposes TCP 22, 80 and 443. The per-run iptables rules add the sandbox network limits on top. Vultr Serverless Inference: every LLM and vision call goes through the OpenAI-compatible endpoint (api.vultrinference.com/v1). glm-5.3 drives the agent loop, because it passed our tool-calling checks and reproduced our demo bugs reliably. glm-5.3-flash reads screenshots and video frames into the reproduction plan with structured output. Both run in Seattle, next to the VM, and no other model provider is configured. Wiring: browser → Caddy (HTTPS) → FastAPI on the VM → sandboxctl starts a gVisor container → the agent drives Chromium inside it over the Playwright protocol → model calls go to Vultr Serverless Inference → the report, recording and lifecycle are saved on the VM, and the container is destroyed.

No comments yet

Be the first to comment on this project.