Haystack – Review the PRs that need human attention

Hey HN! We're building Haystack (https://haystackeditor.com/) to help teams deal with the explosion in the number of pull requests that need to be reviewed due to the rise of coding agents.Haystack replaces the GitHub PR review system with a queue that triages each PR before a human has to read any diffs. It looks at the diffs, the codebase, and the coding-agent conversation that produced the PR. Haystack then routes it into one of three buckets:1. Safe to merge. This means the PR has enough evidence behind it that the team can merge it without another human's review.Some examples:-- A small UI copy change that includes a screenshot showing the final state-- A backend change where the author clearly tested the important paths and ran the changes in a real environment2. Needs fixes. This means that the PR has bugs or violates a rule in your codebase and therefore the PR needs to be fixed by the author.Some examples:-- The agent was asked to make loading a large table faster by adding pagination, but the PR still loads every result at once and "implements" pagination in the UI-- The PR silently catches an error instead of logging, surfacing, or handling it. This violates the team's "no silent error swallowing" rule3. Needs human review. This means that the PR could not be sufficiently verified by the author or is touching a sensitive part of the codebase (determined by user-input guidelines) and thus requires human review.Some examples:-- The PR changes a significant amount of logic in billing-- The PR changes an important user flow like onboarding, but the author only ran unit tests and never opened the app to check the flow end-to-end. That violates the team's rule that high-impact user-facing changes need manual verification.Instead of starting with line-by-line diffs, Haystack immediately tells the reviewer the goal behind the PR, what design decisions the author made (informed by their coding-agent conversati

Hacker NewsToolAISource
0Sign in to voteCopy link

FL score

72

out of 100

Verdict

VALIDATE

high confidence

Competition

No competitor data yet

Trend

No signal yet

AI-powered PR triage system that filters out safe/fixable PRs before human review, solving the bottleneck created by coding agents flooding development workflows.

The pain

Coding agents (Claude, GPT-4, etc.) are generating massive volumes of PRs, overwhelming human reviewers. Teams face either: (1) review bottlenecks that slow shipping, (2) merging unreviewed code that introduces bugs, or (3) spending hours on low-value review work that could be automated.

The gap

GitHub's native PR system treats all PRs equally, no intelligent triage. Existing CI/CD tools catch syntax errors but miss logic bugs, security issues, and architectural violations. No system currently understands the *intent* behind a PR (from agent conversation) + *impact* (codebase context) + *verification quality* (what the author actually tested).

Build angle

Position as the 'quality gate for AI-generated code', not replacing human judgment but making it 40-60% more efficient by eliminating low-risk reviews and surfacing high-context summaries. Monetize via GitHub App SaaS with usage-based or per-seat pricing. Early TAM: engineering teams using Claude/GPT-4 agents (growing rapidly).

Strengths

  • Solves a *new* problem created by recent AI advances, timing is excellent
  • Clear ROI: saves 5-10 hours/week per senior engineer on review work
  • Defensible moat: requires deep codebase understanding + agent conversation context (hard to replicate)
  • Natural GitHub integration point with high switching costs once embedded
  • Three-bucket system is intuitive and reduces false negatives (errs toward 'needs review')
  • Addresses both speed (safe merges) and quality (catches bugs before review)

Risks

  • Accuracy risk: False negatives (unsafe merges) are catastrophic for trust; requires very high precision on 'safe to merge' bucket
  • Adoption friction: Requires teams to adopt coding agents *and* trust a new tool, chicken-egg problem
  • Competitive threat: GitHub/GitLab could build this natively; large AI companies (Anthropic, OpenAI) could integrate into their platforms
  • Scope creep: Temptation to add linting, security scanning, etc. dilutes focus, must stay narrowly on triage
  • Data sensitivity: Needs access to private code + agent conversations; privacy/security concerns for enterprise
  • Market timing: Coding agents still early adoption; may need to wait 12-18 months for critical mass
  • Validation gap: No evidence yet that teams actually want this vs. just accepting slower reviews

Questions about this idea?

FlyBot reads the scoring and gives you a second opinion on “Haystack – Review the PRs that need human attention”.

Open FlyBot