Hetzner Inference: the new AI API serving Qwen3.6, OpenAI-compatible on EU servers

My servers have been running on Hetzner for years, including the one serving this website. So I paid attention when Hetzner quietly switched on a new experiment in July 2026, an AI inference API of its own. Language models behind an API, the way you know it from OpenAI, except on servers in Germany and Finland.

For a company like Hetzner this is a remarkable move. Until now, running open-source AI on European infrastructure almost always meant renting a GPU server and operating the model yourself. That only pays off under constant load and requires real operational knowledge. An API where you only pay per request lowers the bar considerably. That is exactly the gap this experiment aims at.

What Hetzner actually offers

The scope is deliberately small. There is exactly one model right now, Qwen3.6-35B-A3B in FP8 quantisation. It is a mixture-of-experts model with 35 billion parameters, of which only around 3 billion actively compute per request. Put simply, the model carries a lot of knowledge around but uses only a small, fitting slice of it for each answer. That makes it fast and cheap to run, with limited depth as the trade-off.

Access works through the experiments dashboard at experiments.hetzner.com. You create an API token there, and from then on the interface behaves like OpenAI’s. It is free, too, because Hetzner simply has not built billing yet. The other side of that coin comes with it: no SLA, no operational guarantees, explicit experiment status. Hetzner visibly wants to find out first whether and how the offering gets used.

OpenAI-compatible means changing two lines

The most important practical point is compatibility. The API speaks the OpenAI format, which means existing code keeps running once you swap the base URL and the key:

from openai import OpenAI

client = OpenAI(
    base_url="https://<endpoint-from-the-dashboard>",
    api_key="<your-hetzner-api-token>",
)

response = client.chat.completions.create(
    model="Qwen/Qwen3.6-35B-A3B-FP8",
    messages=[
        {"role": "user", "content": "Summarise this request in one sentence: ..."}
    ],
)

print(response.choices[0].message.content)

Anyone who has ever migrated between classic cloud providers knows what a difference this is. No new SDK, no rewritten calls, no weeks of migration work. The OpenAI format has become the de-facto standard, and Hetzner is right to simply adopt it. For testing purposes it means you can point an existing application at Hetzner in minutes and compare directly.

Photo: pankajpatel / Unsplash

How fast is it?

First independent measurements from 23 July 2026 give usable reference points, with 153 milliseconds to first token in the median and around 224 tokens per second on longer outputs. For context, that makes a chat feel fluid, and it is more than enough for background jobs like summarisation or classification. It is one measurement at one point in time, things can look different under load, but as a first impression it is a respectable number.

On the model itself, expectations need to be set correctly. A 35-billion-parameter MoE model does not play in the league of the large models from OpenAI, Anthropic or Google. In early tests Qwen3.6 even stumbled over simple arithmetic. For the tasks you would realistically give a small, fast model, that is acceptable: summarising text, classifying emails, extracting data from documents, internal search and RAG building blocks, simple assistants. In practice these tasks make up most of what companies actually automate, and none of them needs a frontier model.

The real point: EU infrastructure, but no DPA yet

What makes the offering interesting is the location. The servers are in Germany and Finland, operated by a German company. The US CLOUD Act, which can force American providers to hand over data even from EU data centres, does not apply here. For anyone who wants to add AI features and gets an uneasy feeling about every US provider, this sounds like exactly the missing piece.

Photo: albertstoynov / Unsplash

Except the experiment is simply not built for that yet. Hetzner offers no data processing agreement, and without a DPA the contractual basis that Art. 28 GDPR requires for processing personal data through a service provider is missing. Server location alone does not make processing lawful. Add the missing SLA, which turns any production use into a gamble, and you get a clear boundary you should know before getting too excited.

What already works cleanly today: prototypes and feasibility tests, internal tools without personal data, processing of synthetic or anonymised data, development and staging environments. That is less than one would wish for and more than it sounds. Most AI projects I see do not fail in production, they fail before that, because nobody could cheaply find out whether the use case holds up at all. Exactly that experiment is free now.

Why I take this experiment seriously

You could dismiss the whole thing, one model, no SLA, no billing. I read it differently, because of the pattern behind it. Hetzner has shown in the classic cloud business how it enters markets: late, unspectacular, and then with prices a factor of 4 to 10 below the US hyperscalers. The GPU capacity exists, the data centres are built, the cost structure is famously aggressive.

If this experiment becomes a regular product with billing, an SLA and a DPA, something emerges that barely exists in this form so far, an AI inference API on European infrastructure at Hetzner prices. For small and mid-sized companies in the EU that would answer the most common question I hear about AI automation, namely where the data actually ends up. Whether it happens will presumably depend on how much the experiment gets used. One more reason to try it.

Until then, a pragmatic split applies. Develop and test against the Hetzner API, run production workloads with personal data where DPAs and SLAs exist. Thanks to the OpenAI compatibility, the later move really is just a change of the base URL.

My conclusion

Hetzner Inference today is a free experiment with one small model and clear limits, but it is the most strategically interesting infrastructure signal to come out of Germany in a long time. The interface is there, the speed is there, the location is right. What is missing are the contracts, and for a provider of this size those are a matter of decision, not capability.

If you are currently weighing where the AI features of your website or internal workflows should run, with a US provider, on your own hardware, or eventually on an EU API like this one, feel free to get in touch. I build these automations regularly and will give you a sober comparison of what fits your case.