Postgres extension for BM25 relevance-ranked full-text search

Last summer we faced a conundrum at my company, Tiger Data, a Postgres cloud vendor whose main business is in timeseries data. We were trying to grow our business towards emerging AI-centric workloads and wanted to provide a state-of-the-art hybrid search stack in Postgres. We'd already built pgvectorscale in house with the goal of scaling semantic search beyond pgvector's main memory limitations. We just needed a scalable ranked keyword search solution too.The problem: core Postgres doesn't provide this; the leading Postgres BM25 extension, ParadeDB, is guarded behind AGPL; developing our own extension appeared daunting. We'd need a small team of sharp engineers and 6-12 months, I figured. And we'd probably still fall short of the performance of a mature system like Parade/Tantivy.Or would we? I'd be experimenting long enough with AI-boosted development at that point to realize that with the latest tools (Claude Code + Opus) and an experienced hand (I've been working in database systems internals for 25 years now), the old time estimates pretty much go out the window.I told our CTO I thought I could solo the project in one quarter. This raised some eyebrows.It did take a little more time than that (two quarters), and we got some real help from the community (amazing!) after open-sourcing the pre-release. But I'm thrilled/exhausted today to share that pg_textsearch v1.0 is freely available via open source (Postgres license), on Tiger Data cloud, and hopefully soon, a hyperscalar near you:https://github.com/timescale/pg_textsearchIn the blog post accompanying the release, I overview the architecture and present benchmark results using MS-MARCO. To my surprise, we were not only able to meet Parade/Tantivy's query performance, but exceed it substantially, measuring a 4.7x advantage on query throughput at scale:https://www.tigerdata.com/blog/pg-textsearch-bm25-fu

Hacker NewsToolAISource
0Sign in to voteCopy link

FL score

85

out of 100

Verdict

BUILD

high confidence

Competition

6

competitors found, emerging market, funded players

Trend

No signal yet

A high-performance, permissively licensed BM25 full-text search extension for Postgres, addressing key pain points of existing complex or restrictively licensed solutions.

The pain

Developers and companies using Postgres need scalable, relevance-ranked full-text search but face significant complexity, cost, and operational overhead with external solutions like Elasticsearch, or are limited by the restrictive AGPL license of Postgres extensions like ParadeDB. This results in fragile systems, slow queries, and rigid schemas.

The gap

There is a clear gap for a Postgres-native, high-performance BM25 full-text search solution that comes with a permissive open-source license. Existing alternatives are either over-engineered and costly (Elasticsearch) or have licensing that deters commercial adoption (ParadeDB AGPL). This leaves a specific unserved need for companies wanting to keep their search stack simpler and integrated within Postgres.

Build angle

Develop a Postgres extension (pg_textsearch) that provides BM25 relevance-ranked full-text search, with a permissive open-source license, focusing on superior performance and ease of integration over existing complex external systems or restrictively licensed alternatives.

Strengths

  • Creator has deep expertise and has already built a highly performant, open-source MVP.
  • Addresses a significant and specific pain point with documented user frustrations (Elasticsearch complexity, AGPL licensing).
  • Demonstrated superior performance (4.7x query throughput) compared to a leading competitor.
  • Clear market gap for a permissively licensed, Postgres-native solution.
  • Strong demand signals from internal company needs and general market trends towards simplified data stacks.
  • Growing market for AI-centric and hybrid search workloads.

Risks

  • Despite performance, convincing users to switch from deeply entrenched systems like Elasticsearch can be challenging.
  • Monetization strategy for an open-source project needs careful planning beyond cloud provider partnerships.
  • Ongoing maintenance and community management for an open-source database extension can be resource-intensive for a solo builder.
  • While the niche is clear, the broader search market is still crowded with well-funded players.
  • Dependency on the Postgres ecosystem for adoption and distribution.

Next steps

  1. Focus on aggressive marketing and education within the Postgres developer community to highlight performance and licensing advantages.
  2. Explore and formalize various monetization models (e.g., premium support, commercial features, cloud partnerships) beyond just the Tiger Data Cloud offering.
  3. Actively engage with the open-source community for contributions, feedback, and further validation of features and use cases.
  4. Create detailed migration guides and comparison benchmarks against more competitors to ease adoption for users currently on other platforms.

Questions about this idea?

FlyBot reads the scoring and gives you a second opinion on “Postgres extension for BM25 relevance-ranked full-text search”.

Open FlyBot