Ideas Lab

Real problems people complain about online, pulled every morning and scored out of 100. Build, validate, or skip. How scoring works

Ideas
4,820
BUILD
702
Sources
8
Last sync
8h ago
4,820 ideas

Respan (YC W24) – Open-source Rust AI gateway built around plugins

Hey HN, this is Andy from Respan (https://www.respan.ai). We are building an open-source Rust AI gateway. It is a decoupled gateway, so everything is a plugin, including different providers, protocols (MCP, voice), modalities (images, videos), and gateway features (fallbacks, PII redaction, smart routing).Instead of packaging tons of providers and features into one gateway, Respan provides a small core and public interfaces that teams can use to assemble the AI infrastructure their application needs. That way, users can build their own features as additional plugins without having to change the gateway's core.Here’s a video demo: https://www.youtube.com/watch?v=pVv2ibXiTXMThere are countless LLM gateways and most of them solve a common problem: calling different providers from a single endpoint. However, adding new features can become prohibitively difficult. So developers can end up building another gateway to solve a specific problem.We've been building our gateway based on LiteLLM since 2024, and now it's providing services for thousands of orgs. Each org needs different API key settings, routes, and limits. To support that, we kept wrapping LiteLLM, adding more features to it, and at one stage, our wrapper code exceeded 100,000 lines.This January, a routine LiteLLM upgrade caused intermittent connection errors, and we had to roll back. We still haven't figured out the cause, but since then, we no longer upgrade LiteLLM directly and instead manually backport upstream fixes. These experiences made us realize that continuing to patch LiteLLM is not a sustainable long-term solution, and we had to redesign the gateway ourselves.While redesigning our gateway, we wanted to make it easier to add features without breaking existing ones. Inspired by Nginx, we think an AI gateway should have a really small core, only responsible for dispatching events.We believe that everything should be a plugin in our gateway, including provid

Hacker News10d agoToolAI

78FL score
0Sign in to vote

How do you deal with AI output of your boss?

So, my boss goes on a rampage. You can clearly see the joy and at the same time the rage and impatience. AI is the tool he always wanted. We are too slow. Process is out of the window. No need for sprints anymore. No planning, just doing. All people downstream must somehow adapt.Whenever i see now his automated messages in slack, my natural reaction is: why do i even bother reading that? Thats for machines to read. He does not need me? Clearly i am not made for that. What is he expecting me to do? To confirm how great the AI did the job? Should i remember all these perfect presentations AI has created and integrate it into my doings into the coming weeks?I am torn. Should i really free my AI and deal with all this incoming text? Is this the way to react on this perfection?I am not mad at him, it clearly frees him of all the burder of dealing with people.I just dont know how to deal with this output personally. It is not for humans.I feel left alone. This is not a company anymore. Everyone is talking to AI now. This feels like chaos. Where everyone can do everything and stepping into each other's domain and it is perfect. The only way AI is stopped at are the ACLs.And because AI does everything for me, i don't remember things anymore, as i have no emotion connected with it. Feels like bad schooling and memoizing and get through it and you can't as it so dry and lifeless.So, how people deal with it in a company context? How have they setup their machines to react and still be up to date without reading the constant stream of changes coming from others?

Hacker News13d agoArticleAI

62FL score
0Sign in to vote