Fly Labs
Ask FlyBot

Swift-Qwen3.8-27B, -58.3% thinking, x1.95 speed, accuracy of xhigh

Hi everybody, we post-trained Qwen 3.8 27B to be more efficient by figuring out which tokens were linked to overthinking and penalizing them without "attacking" the reasoning length directly then fixed the accuracy with a bit of secret sauce (hint On-Policy Distillation) and achieved great results (-58% length, 1.95x speed up, <1% accuracy loss) so we wanted to open-source it and hear the feedback of the community.This is the link to the model: https://huggingface.co/ukisai/Swift-Qwen3.8-27b / It's at 80k downloads in 3 days with independent evals here: https://www.reddit.com/r/LocalLLaMA/comments/1wg7dd5/ukisai_...We are also providing a Free Research Purpose API (OpenAI compatible), courtesy of Nvidia who were kind enough to provide us with the GPUs. It's limited at 5RPM. https://ukisai.com/api/swift/v1/modelsWe also made a GGUF (Q1-Q8): https://huggingface.co/ukisai/Swift-Qwen3.8-27B-GGUF and there's also a few nice community quants with even lower/higher precision (Bartowski: https://huggingface.co/bartowski/ukisai_Swift-Qwen3.8-27b-GG...). The community also created amazing MLX, NVFP4, W4A16 and Uncensored versions you can find on Huggingface.IMPORTANT: Our training approach is not a replacement for the reasoning effort settings, chat templates or token caps but is complementary and targets a completely separate issue (overthinking and "anxiety-like" reasoning loops prior seen in PTQ, but as far as we identified also prominent in BF16 of this size class LLMs as well). Contrary to popular belief, these specific patterns do not contribute to answer quality when properly targeted. (our thesis being: reasoning length IS extremely important and should NOT be shortened by force, but rather optimized). This is also demonstrated bellow in our xhigh vs medium effort benchmark. The goal is to keep xhi

Hacker NewsToolAISource
0Sign in to voteCopy link

FL score

62

out of 100

Verdict

VALIDATE

high confidence

Competition

No competitor data yet

Trend

No signal yet

A technically strong open-source LLM optimization that gained fast traction but lacks a clear monetization path or defensible market position.

The pain

Running large language models locally or in production is slow and expensive because models waste compute on redundant reasoning patterns and overthinking loops that do not improve answer quality. Developers and companies running inference at scale lose money to wasted tokens and latency.

The gap

Existing solutions either force shorter reasoning (losing quality) or do nothing. This team identified and targeted the specific token patterns causing wasted computation without cutting reasoning length, achieving 1.95x speedup with less than 1% accuracy loss. No competitor has published this exact optimization.

Build angle

The founders should pick one paying customer segment (e.g., inference providers, enterprise self-hosted deployments, or edge device manufacturers) and build a commercial product around this optimization rather than relying on a rate-limited free API. They could license the training methodology, offer managed inference, or sell model weights with support contracts.

Strengths

  • Technical execution is real and verified by independent community testing on Reddit.
  • 80k downloads in 3 days shows genuine developer interest and product-market fit at the awareness stage.
  • The optimization targets a specific problem (overthinking loops) rather than generic compression, making it harder to copy.
  • Multiple distribution formats (GGUF, MLX, quantized versions) show thoughtful product design for different use cases.
  • Nvidia partnership for free API infrastructure removes initial capital barrier.

Risks

  • Open-source model with free API generates zero revenue and no clear path to paid tier that users will accept.
  • Larger model providers (OpenAI, Anthropic, Meta) can incorporate this optimization into their own models, eliminating the advantage.
  • The 5 requests per minute rate limit on free API is too restrictive to drive meaningful usage or data collection.
  • No defensible IP if the training approach becomes public knowledge or is reverse-engineered from the model weights.
  • Market for local LLM inference is shrinking as cloud inference becomes cheaper and easier, reducing total addressable market.
  • Founders have not articulated who specifically will pay for this or why they would choose this over alternatives.

Questions about this idea?

FlyBot reads the scoring and gives you a second opinion on “Swift-Qwen3.8-27B, -58.3% thinking, x1.95 speed, accuracy of xhigh”.

Open FlyBot