Browser Harness – Gives LLM freedom to complete any browser task

Hey HN,We got tired of browser frameworks restricting the LLM, so we removed the framework and gave the LLM maximum freedom to do whatever it's trained on. We gave the harness the ability to self correct and add new tools if the LLM wants (is pre-trained on) that.Our Browser Use library is tens of thousands of lines of deterministic heuristics wrapping Chrome (CDP websocket). Element extractors, click helpers, target managemenet (SUPER painful), watchdogs (crash handling, file downloads, alerts), cross origin iframes (if you want to click on an element you have to switch the target first, very anoying), etc.Watchdogs specifically are extremely painful but required. If Chrome triggers for example a native file popup the agent is just completely stuck. So the two solutions are to: 1. code those heuristics and edge cases away 1 by 1 and prevent them 2. give LLM a tool to handle the edge caseAs you can imagine - there are crazy amounts of heuristics like this so you eventually end up with A LOT of tools if you try to go for #2. So you have to make compromises and just code those heuristics away.BUT if the LLM just "knows" CDP well enough to switch the targets when it encounters a cross origin iframe, dismiss the alert when it appears, write its own click helpers, or upload function, you suddenly don't have to worry about any of those edge cases.Turns out LLMs know CDP pretty well these days. So we bitter pilled the harness. The concepts that should survive are: - something that holds and keeps CDP websocket alive (deamon) - extremely basic tools (helpers.py) - skill.md that explains how to use itThe new paradigm? SKILL.md + a few python helpers that need to have the ability to change on the fly.One cool example: We forgot to implement upload_file function. Then mid-task the agent wants to upload a file so it grepped helpers.py, saw nothing, wrote the function itself using raw DOM.setFileInputFiles (which we only noticed that later in a git diff). Thi

Hacker NewsToolAISource
0Sign in to voteCopy link

FL score

72

out of 100

Verdict

VALIDATE

high confidence

Competition

No competitor data yet

Trend

No signal yet

A minimal Chrome automation layer that lets LLMs handle their own edge cases instead of requiring developers to code thousands of heuristics.

The pain

Current browser automation frameworks force developers to anticipate and code around every edge case, file dialogs, cross-origin iframes, alerts, DOM mutations. This creates massive maintenance burden. Teams building AI agents spend weeks on deterministic heuristics that still fail on novel scenarios. The alternative (giving LLMs a tool for every edge case) explodes the tool count and makes agents slower and less reliable.

The gap

No existing framework lets the LLM itself write and modify its own capabilities mid-task. Everything is either locked-down (restrictive) or tool-heavy (slow). The insight that modern LLMs understand CDP well enough to self-correct and extend their own tooling is genuinely novel and underexploited.

Build angle

Start with teams building AI agents that interact with web apps (RPA, testing, data extraction). They're currently losing weeks to edge case handling. Sell them a harness that reduces their heuristic code from 10k+ lines to <500 lines, with the LLM handling the rest. Charge per task execution or per agent deployment. Build a marketplace for shared skills/helpers that agents discover and use.

Strengths

  • Solves a real, acute pain point for AI agent builders, edge case handling is genuinely the hardest part of browser automation
  • The core insight (LLMs can write their own tools) is defensible and hard to copy without deep CDP knowledge
  • Minimal surface area means fast iteration and fewer bugs than traditional frameworks
  • Self-extending capability creates a flywheel, more tasks = more skills discovered = better agent performance
  • Concrete proof of concept (agent writing upload_file function) shows the idea works in practice

Risks

  • LLM reliability is still inconsistent, agents may write broken CDP code or get stuck in loops. Needs robust error handling and rollback mechanisms
  • Security risk if LLMs can modify their own tools, malicious prompts could inject code. Requires sandboxing and audit trails
  • Market timing: AI agent adoption is still early. Most companies aren't yet building multi-step browser automation systems at scale
  • Competitive pressure from larger frameworks (Anthropic, OpenAI) adding native browser control. They have distribution and resources
  • Skill discovery and sharing is a network effect problem, needs critical mass of users to be valuable. Chicken-and-egg problem
  • Debugging is harder when the agent writes its own code. Support burden could be high
  • Chrome/CDP API changes could break agent-written code. Requires versioning and compatibility layers

Questions about this idea?

FlyBot reads the scoring and gives you a second opinion on “Browser Harness – Gives LLM freedom to complete any browser task”.

Open FlyBot