Fly Labs
Ask FlyBot

Canary (YC W26) – AI QA that understands your code

Hey HN! We're Aakash and Viswesh, and we're building Canary (https://www.runcanary.ai). We build AI agents that read your codebase, figure out what a pull request actually changed, and generate and execute tests for every affected user workflow.Aakash and I previously built AI coding tools at Windsurf, Cognition, and Google. AI tools were making every team faster at shipping, but nobody was testing real user behavior before merge. PRs got bigger, reviews still happened in file diffs, and changes that looked clean broke checkout, auth, and billing in production. We saw it firsthand. We started Canary to close that gap. Here's how it works:Canary starts by connecting to your codebase and understands how your app is built: routes, controllers, validation logic. You push a PR and Canary reads the diff, understands the intent behind the changes, then generates and runs tests against your preview app checking real user flows end to end. It comments directly on the PR with test results and recordings showing what changed and flagging anything that doesn't behave as expected. You can also trigger specific user workflow tests via a PR comment.Beyond PR testing, tests generated from the PR can be moved into regression suites. You can also create tests by just prompting what you want tested in plain English. Canary generates a full test suite from your codebase, schedules it, and runs it continuously. One of our construction tech customers had an invoicing flow where the amount due drifted from the original proposal total by ~$1,600. Canary caught the regression in their invoice flow before release.This isn't something a single family of foundation models can do on its own. QA spans across many modalities like source code, DOM/ARIA, device emulators, visual verifications, analyzing screen recordings, network/console logs, live browser state etc. for any single model to be specialized in. You also need custom browser fleets, user ses

Hacker NewsToolAISource
0Sign in to voteCopy link

FL score

68

out of 100

Verdict

VALIDATE

high confidence

Competition

25

competitors found, growing market, big tech present, funded players

Trend

1 community mentions

An AI-powered QA agent that understands code changes to generate and execute end-to-end tests for affected user workflows, preventing critical production bugs.

The pain

Growing code complexity and faster shipping with AI tools lead to bigger PRs, inadequate manual reviews, and significant production bugs (e.g., in checkout, auth, billing, invoicing) causing financial losses and developer frustration.

The gap

While the AI code review/testing market is crowded, current tools don't fully address the need for AI that understands codebase changes to *generate and execute end-to-end tests for every affected user workflow*, beyond just code-level analysis or generic test generation.

Build angle

AI QA agents that connect to the codebase, read PR diffs, understand intent, then generate and run specific E2E tests on preview apps, commenting results directly on PRs and feeding tests into regression suites.

Strengths

  • Addresses a high-severity, well-understood problem (production bugs).
  • Strong value proposition with measurable impact (e.g., catching $1,600 bug).
  • Specific, unaddressed angle within a crowded market (E2E tests for affected user workflows).
  • Growing market for AI-powered software testing.
  • High willingness to pay for preventing critical business-impacting issues.

Risks

  • Extremely high technical complexity for a solo builder, requiring deep AI and dev tools expertise.
  • Crowded competitive landscape with funded startups and Big Tech, making differentiation and distribution challenging.
  • Significant assumption risk regarding the AI's accuracy, reliability, and low false positive rate in real-world, complex codebases.
  • High resource demands (team, funding) for development and go-to-market.
  • Potential for slow adoption if integration is not seamless or if value is not immediately apparent beyond existing, cheaper tools.

Next steps

  1. Deeply validate the exact pain points and current workarounds for 'affected user workflows' testing with target users.
  2. Focus on an even narrower niche (e.g., specific tech stack, industry, or type of critical flow) to prove value quickly.
  3. Conduct in-depth user interviews with engineering managers and QA leads to understand their current solutions, budget, and exact criteria for adopting a new E2E AI testing tool.
  4. Develop a minimal core feature that demonstrates the 'magic' of identifying changed user workflows and generating/executing relevant tests, rather than building the full vision upfront.
  5. Explore partnerships or integrations with existing CI/CD platforms or code review tools to reduce distribution friction.

Questions about this idea?

FlyBot reads the scoring and gives you a second opinion on “Canary (YC W26) – AI QA that understands your code”.

Open FlyBot