76.9%
Fewer LLM calls
13 → 3 calls across eight constructed local agent briefs, versus an ordinary retry route.
Early failure detection + runtime control for AI agents
Stop paying
for agent runs
that fail.
Why wait for an agent to finish before discovering it was heading toward failure? We’re building the intelligence layer that learns from execution history and helps teams decide when to continue, reroute, repair, or stop—before more cost is incurred.
- Two-week read-only pilots
- 20,441 agent runs audited
- Measured local execution experiments
Illustrative workflow · not a production result
Without First Light
✕ failed
Example: 12 steps consumed before an unresolved outcome.
With First Light
Flagged
Illustration: a flag at step 4 could prompt a targeted repair.
Signal
Repeated tests have failed and edits are moving outside the task scope.
Possible next action: repair
Recommend re-scoping to the failing module and reviewing the full error trace.
01 / The problem
We started with the economics of failed agent runs.
Our audit of 20,441 public coding-agent attempts found that 42.1% went unresolved, yet they accounted for 52.9% of measured spend. That raised a simple question: could agents make better decisions before the expensive part of a bad run?
52.9%
of agent spend went to runs that failed
1.5×
what a failed run costs compared with a successful one
873K
model calls analyzed in our audit
What we found
Generic warning signs don't work.
8 in 10runs that hit failing tests or tool errors still went on to succeed
Kill switches and fixed step caps can discard runs that would have recovered. The useful signals depend on the workflow, the objective and the cost of being wrong.
Our thesis: learn from the outcomes of your own runs, then make earlier, more informed execution decisions.
Why nowAgents are moving to outcome-based pricing. When you charge per outcome, every failed run comes out of your margin.
02 / What we've actually tested
Early decisions. Measurably less wasted work.
We started with historical coding-agent traces and moved to executable local tests of agent routing, media generation and selective repair. These are measured reductions in specific operations—not claims of production savings.
80.5%
Fewer reported tokens
3,836 → 747 Ollama-reported tokens in the same local routing experiment.
50.0%
Fewer video encodes
48 → 24 FFmpeg section encodes across 16 local card-based videos, with byte-identical final files.
66.7%
Fewer repair encodes
12 → 4 section encodes across four controlled corrections, with identical repaired outputs.
2 / 8
Intervened before model generation. In the eight-brief risk-check experiment, two planned model-generation routes were redirected before the call was made.
Experiment scope matters: the call/token results use authored tasks and rule-based alternatives, with creative quality not independently approved. Video reuse and repair matched conventional caching approaches. These measurements do not establish a trained production failure predictor or customer cost savings. Read the methods and limitations →
03 / How it works
From execution history to better next actions.
We start by understanding each workflow’s true outcomes, identify opportunities for early intervention, and evaluate decisions in read-only shadow mode before any production control.
- 01
Measure
Use run traces, tool calls, retries, cost and final outcomes to establish the baseline cost and time per successful result.
- 02
Learn
Study the patterns in completed successes and failures. Evaluate workflow-specific risk signals against your own definition of done.
- 03
Detect in shadow mode
Score ongoing runs without changing execution. Track false positives and how much work was still ahead when a risk was flagged.
- 04
Intervene when proven useful
Move from read-only recommendations to controlled continue, reroute, repair or stop decisions—measuring actual outcomes and the cost of incorrect interruptions.
Start without a rewrite
Share redacted OpenTelemetry, LangSmith, Langfuse or JSONL trace exports. A proxy integration can be explored for a later controlled trial.
Where it fits
Observability shows what happened. Evals assess the outcome. First Light aims to help decide what should happen next, while execution is still underway.
Best suited to teams with repeated, costly, multi-step agent runs and measurable outcomes: coding, browser, deployment and other tool-using workflows.
04 / Team
We find signal in noise.
Our backgrounds span quantitative finance, credit risk modeling, prediction markets and astrophysics research. We’ve spent our careers reasoning about noisy signals, uncertain outcomes and the cost of being wrong. Now we’re bringing that discipline to AI agent execution.
Both founders trained in quantitative finance at Stony Brook University under former Renaissance Technologies researchers.
Aman Arya
Co-founder
LinkedIn
Evaluates AI-generated code for Microsoft Copilot. Built machine learning models on large astrophysical datasets at the Indian Institute of Astrophysics, with peer-reviewed research published in the Journal of Astrophysics and Astronomy. MS in Data Science from Stony Brook, with graduate coursework in computational finance, stochastic models and machine learning for quantitative finance. Author of the SWE-bench agent spend audit.
Kunal Gaurav, CQF
Co-founder
LinkedIn
Quantitative researcher at a hedge fund and an investment firm, building stochastic volatility, statistical arbitrage and portfolio risk models. Then senior data scientist, and now in AI operations research at Uber AI Solutions. Published in the International Journal of Financial Engineering on stress testing and tail risk. Previously founded a data and analytics venture. Studied at Stony Brook University.
05 / Design partners
Two weeks, read-only. One clear picture of wasted work.
Our free 14-day diagnostic is designed to fit around an existing workflow, without stopping or rerouting a single production run. We ask for two things.
01
Your definition of success and failure, in writing.
For each workflow, what counts as a successful run (an approved demo, a merged PR, a resolved ticket) and what counts as a failure (rejected, re-rendered, reverted, escalated, abandoned). Plus what a wrongly stopped run costs you. We agree on this before we see any data, and every result we report is scored against it.
02
Traces of your agents.
Past runs with every step and its outcome.
- For each step: its type (model call, tool call, render), timestamps, model, tokens or cost, and status or error.
- For each run: the final outcome, plus the reviewer's decision and reason if you have one.
OpenTelemetry, LangSmith or Langfuse exports, or plain JSONL. Redact what you need to.
What you get in the first 14 days
Days 1–3
A baseline of run outcomes, time and cost, with tracing gaps identified.
Days 4–10
Read-only shadow scoring of live runs and evaluation against held-out history.
Days 11–14
A report on early flags, false positives and potentially avoidable continuation costs—plus a proposal for a controlled intervention trial.
Your data: start with redacted, read-only traces. We agree on access, confidentiality, retention and deployment boundaries before any transfer. Potential avoidable costs are estimates, not realized savings.
Find out what your agent runs are costing you.
Apply for a free, read-only two-week evaluation.