A new Python library called litelm has been getting attention on Hacker News this week with a simple pitch. The README describes it as "litellm without the bloat": LiteLLM's model routing and message translation core, rebuilt in roughly 2,900 lines with two dependencies, openai and httpx.
LiteLLM itself is an open-source AI gateway that fronts 100+ LLM providers with a single OpenAI-format interface, plus a deployable proxy server with virtual keys, spend tracking, guardrails, and load balancing. litelm's position is that the valuable part of that is the call path: model routing, message translation, streaming, tool use, and embeddings. The core, the README argues, is buried under 100k+ lines of proxy that most users never touch, so it kept just that path and nothing else. No Router class, no proxy, no caching.
The API is a find-and-replace
litelm mirrors LiteLLM's API: same function names, same arguments, same response types. If you already use LiteLLM, the README says switching is s/litellm/litelm/ in your imports.
import litelm
response = litelm.completion("openai/gpt-4o", messages=[{"role": "user", "content": "Hello!"}])
print(response.choices[0].message.content)import litelm
response = litelm.completion("openai/gpt-4o", messages=[{"role": "user", "content": "Hello!"}])
print(response.choices[0].message.content)Every function has an async variant (acompletion, aembedding, aresponses, atext_completion). The kept feature list covers streaming with stream_chunk_builder, tool calling, embeddings, text completions, the OpenAI Responses API, and mock responses, and models are addressed with the same provider/model syntax. Any OpenAI-compatible endpoint works by passing api_base, which is how local servers like vLLM, Ollama, and LM Studio fit in.
Tool calling looks the way it does in LiteLLM:
tools = [{"type": "function", "function": {
"name": "get_weather",
"parameters": {"type": "object", "properties": {"city": {"type": "string"}}},
}}]
response = litelm.completion(
"openai/gpt-4o", messages=[{"role": "user", "content": "Weather in Paris?"}],
tools=tools, tool_choice="required",
)tools = [{"type": "function", "function": {
"name": "get_weather",
"parameters": {"type": "object", "properties": {"city": {"type": "string"}}},
}}]
response = litelm.completion(
"openai/gpt-4o", messages=[{"role": "user", "content": "Weather in Paris?"}],
tools=tools, tool_choice="required",
)Provider errors map onto a small exception hierarchy, ContextWindowExceededError and friends, so the usual retry-and-truncate loop has something concrete to catch.
What got cut
The README keeps a blunt in/out list. Kept: model routing, message translation for Anthropic, Bedrock, Cloudflare, and Mistral, streaming, tool use, embeddings, text completions, the Responses API, and mocks. Cut: the load-balancing Router, the proxy server, caching, budgets and cost tracking, token counting, image gen, audio, OCR, fine-tuning, agents, guardrails, and the scheduler.
That list is where the HN discussion splits down the middle. Several commenters said cost tracking and caching are exactly why they deploy LiteLLM at work, and removing them takes away the point. Others said the full install is heavy enough that a lean alternative is overdue. LiteLLM's own repo now describes the project as a "Rust core with Python SDK", which suggests both projects are converging on the same idea: the core should be small.
Written with AI, audited against upstream
litelm is upfront that it is "human-directed, AI-assisted software". Most of the code was written with Claude Code using Claude Opus 4.6/4.7, and anything from 2026-05-14 onward was written through Pi using GPT-5.5. That disclosure drew its own share of the thread.
The project also does upstream attestation. On September 11 the maintainer reviewed LiteLLM's routing and formatting changes from commit 649eb2d through 9a715df2, triaged 360 core path commits, and fixed the compatibility gaps test first. The 0.5.2 changelog reports 256 unit tests and 45 live provider tests passing, and DSPy is verified as a drop-in across all seven of its execution paths.
Status
litelm is at version 0.5.2 on PyPI, released September 11, 2026; 0.1.0 came in mid-March. It routes to 19 providers, with custom handlers for Anthropic, Bedrock, Cloudflare, and Mistral, and the README marks OpenAI, Anthropic, Groq, Mistral, xAI, OpenRouter, and Azure as verified live. It needs Python 3.10 or newer, is MIT licensed, and installs with optional extras for Anthropic, Bedrock, or everything:
pip install litelmpip install litelmWhether it fits your stack comes down to what you want from an LLM client. If you need completion("anthropic/claude-...") to just work without standing up a gateway, litelm is a small, readable option with a test suite to back it. If your team lives on per-customer spend numbers and virtual keys, LiteLLM still has plenty to offer.