Fly Labs
Ask FlyBot

We post-trained a model that pen tests instead of refusing

Anthropic and OpenAI's publicly available models are explicitly guard-railed so that they refuse offensive tasks. And their cyber-focussed models are gated for enterprises. This leaves SMEs and mid market open to major vulnerabilities.AI can be used as both an adversarial and defensive tool in the world of cyber. A worst case outcome is if only the adversaries have access.Meanwhile, most existing AI cyber tools are just wrappers. The problem is that they still have all the guardrails on from the foundation model where they will inherit its refusals.For this project we've post-trained a specific model on a decade of capture-the-flag contests. This won't be made available to anyone and everyone, but we do believe that responsible SMEs and midmarket companies also need access to these tools in order to identify key vulnerabilities in their systems; not just enterprises.We have developed two modes that run over a CLI:• Security scan: a read-only audit of your local codebase for vulnerabilities. It only reports what it can tie to a specific file and line, so you're not wading through vibes-based findings.• Pen test: an active adversarial mode that will try to break a live system in a sandboxed environment. It proves each vulnerability by running the exploit and showing the request it sent and the response your code gave back, not a confidence score. Currently gated.To show what the scan does, we pointed it at Bank of Anthos and it found an integer overflow in the transfer path: amount is an int, and amount + fee can overflow negative, so the balance check passes and you move funds you don't have. Plus the usual auth and secrets issues. (Bank of Anthos is Google's open-source bank. It's a known app and some of it is intentionally weak, which is the point: you can clone it and re-run the scan yourself instead of trusting a screenshot)The base model is a Kimi K2.6 (open weights). We didn't pretrain from scratch. We post-trained it our

Hacker NewsToolAISource
0Sign in to voteCopy link

FL score

72

out of 100

Verdict

VALIDATE

medium confidence

Competition

No competitor data yet

Trend

No signal yet

An unrestricted AI model trained to find real vulnerabilities in code, positioned for mid-market companies locked out of enterprise security tools.

The pain

SMEs and mid-market companies run vulnerable code but can't access specialized pen testing tools (gated to enterprises) or afford traditional security firms. Generic AI models refuse to help with offensive security work. Existing 'AI security tools' are just wrappers around guardrailed models that still refuse the hard problems.

The gap

The market has a capability gap: enterprises have gated access to offensive security AI, while smaller companies either use generic tools that won't actually attack their systems, or hire expensive consultants. No one is selling an unrestricted but responsible offensive AI tool to the mid-market.

Build angle

You've built the technical proof (post-trained model that actually pen tests, concrete findings on Bank of Anthos). Now validate: (1) Do mid-market companies actually want this enough to pay? (2) Can you gate access responsibly without it becoming a liability? (3) What's your defensible moat if you're just post-training an open model? Start with 10-20 paying customers doing security scans, measure if they renew, then expand to active pen test mode.

Strengths

  • Concrete technical differentiation: model actually executes exploits and proves vulnerabilities with request/response pairs, not confidence scores
  • Trained on real data (CTF contests) rather than synthetic or theoretical attacks
  • Dual-mode approach (read-only scan + sandboxed pen test) lets you start low-risk and scale trust
  • Reproducible demo (Bank of Anthos) that anyone can verify themselves
  • Real market gap: mid-market companies are underserved and have budget

Risks

  • Regulatory and legal exposure: selling offensive security tools invites scrutiny from regulators, law enforcement, and liability if misused. You need clear terms of service and customer vetting.
  • Moat is weak: you post-trained an open model. Competitors can do the same. Your defensibility is speed to market and customer relationships, not IP.
  • Customer acquisition is hard: security buyers are risk-averse and slow. You need to prove this works at scale and that you're trustworthy.
  • Responsible gating is vague: you say 'responsible SMEs' but how do you actually vet customers? What stops a bad actor from signing up with a fake company?
  • Liability tail risk: if someone uses your tool to attack a third party, you could face lawsuits even if your terms forbid it.
  • Product-market fit unclear: do mid-market companies actually want an AI pen tester, or do they want a consultant who can explain findings and help fix them? You're selling a tool, not a service.

Questions about this idea?

FlyBot reads the scoring and gives you a second opinion on “We post-trained a model that pen tests instead of refusing”.

Open FlyBot