The IA gateway: one entry point for every model

13 October 2026aillminfra

Content created with the help of AI

This is the experiment running at alfa-safety: IA, but not the flashy demo — the plumbing. People ask me in a thousand ways (“what is IA at the office, actually?”) and the short answer fits in one image: a single tap, connected to several sources. Here is the long version.

The problem, before even choosing a model

Every idea for using IA in a small company starts with the same three concrete questions: where does the model run, who holds the API keys, and how will we know it works.

Without answers, what happens is predictable: every team that tries an IA tool connects it their own way, API keys live in config files, and the monthly bill arrives without anyone having been able to predict it. Nothing looks broken — the domain just assembles itself through small copies, and stops being controlled from anywhere. It is a gap of the same kind as the one that called for Toolbox: craft where you need industry.

What a gateway is

A gateway is a single entry point. Instead of every application calling a model provider directly (OpenAI, Mistral, others…), it calls one address — and that address sorts it out: it routes to the right provider, keeps the keys in one place, and logs who asked for what.

The detail that makes it: the format. The gateway exposes an API in the OpenAI format — not out of taste for OpenAI, but because that dialect has become the common language of model providers. With the gateway in front, an application written for one model becomes compatible with everything connected behind it, without a rewrite.

Two open-source components: LiteLLM / Bifrost

LiteLLM and Bifrost AI do the same job: expose an OpenAI-compatible entry point and route to several providers. The POC tests both — and that is a method choice: the test bench deliverable is not “we picked the winning tool”, it is “we know what matters”: how it routes, what it logs, what it costs to run. The component can change later without the decision suffering.

Two providers from the start

Connected behind the gateway from day one: OVH AI Endpoints and the Mistral API. Two providers at POC stage is not luxury — it is what lets you verify the central promise: change providers without touching the application. If the switch works in the POC, it will work in production.

Ollama first — local as the workshop

The experiment actually started before the gateway, locally: Ollama and open-weights models quantized in GGUF. Local has three virtues for getting started: no per-request cost, no data leaving the house, and above all you learn what a model is — latency, memory footprint, what quantization changes. When the gateway arrived, local became the first “provider” on the list: same format, same tests. The gateway then serves as the open door to the cloud, where it is useful — and nowhere else.

A home-made harness

To judge, a harness: a simple test bench — frozen prompts, expected answers, measured latencies. No grand public benchmark; real cases from the house, replayable identically against any model connected behind the gateway. That bench is what allows scoping: comparing two models on our cases is a table; without a harness, it is a feeling.

What changed

Concretely, connecting an IA use now means: one URL, one key, and a key that does what it must — rather than a project. The harness reports were enough to scope the potential IA projects: which case to try first, with which model, and what it looks like in cost.

What remains

It is a POC, internal — no published numbers, no production. Next: move the test bench from the workshop to the real thing, and turn the scoping into a plan. It is also the first concrete step of the announced trajectory: MLOps and LLMOps — making IA reproducible, monitorable, industrializable.