Hetzner Inference: the new AI API serving Qwen3.6 and Qwen3.8, OpenAI-compatible on EU servers

My servers have been running on Hetzner for years, including the one serving this website. So I paid attention when Hetzner quietly switched on a new experiment in July 2026, an AI inference API of its own. Language models behind an API, the way you know it from OpenAI, except on servers in Germany and Finland.

For a company like Hetzner this is a remarkable move. Until now, running open-source AI on European infrastructure almost always meant renting a GPU server and operating the model yourself. That only pays off under constant load and requires real operational knowledge. An API where you only pay per request lowers the bar considerably. That is exactly the gap this experiment aims at.

Update September 2026

A lot has happened since this first look at the end of July. Here is the current state as of 21 September 2026, queried directly from the API and checked against the official documentation. Hetzner offers two models, both free of charge and still explicitly an experiment with no announced end date:

Model	Architecture	Context window
Qwen/Qwen3.6-35B-A3B-FP8	mixture of experts, 35B parameters, about 3B active	262,144 tokens
Qwen3.8-27B	dense model, 27B parameters	262,144 tokens

According to the documentation both models accept text and images. The service stays free as long as it is an experiment, and Hetzner says it will announce any change by email beforehand. The limits per API key are now documented. Per 60 seconds you get at most 10 requests, 4 million input tokens and 100,000 output tokens, and anything above that returns HTTP status 429. On privacy, Hetzner states that it only keeps usage data such as timestamps and token counts, not the content of requests and responses.

Getting here took a few turns. On 24 July the documentation went online and a personal key gave access to exactly one model, Qwen3.6. On 7 August Hetzner added three large models, Kimi-K2.7-Code, GLM-5.2 and DeepSeek-V4-Flash, and started promoting the experiment to its customers on 10 August. By Hetzner’s own account, demand exceeded its most optimistic expectations within hours, and the 99th percentile time to first token peaked at almost ten minutes. In a detailed write-up from 17 August Hetzner announced it would keep serving only small models such as Qwen 3.6 and 3.8 for now and pause the large ones. That is still the line-up today.

On 21 September I repeated the speed test with the same method as in July and August, five runs for time to first token and an explanation of roughly 400 words for throughput, this time repeated three times. Throughput is counted from the first token so that queueing does not distort it.

Metric	24 July	23 August	21 September
Qwen3.6, time to first token (median)	244 ms	257 ms	0.45 to 2.3 s
Qwen3.6, throughput	181 tokens/s	75 tokens/s	24 to 50 tokens/s
Qwen3.8, time to first token (median)	not yet offered	364 ms	9.7 s to over 2 min
Qwen3.8, throughput	not yet offered	28 tokens/s	4 to 7 tokens/s

The September figures come from three measurement runs between 4 and 5 pm, and even those differed noticeably. Depending on the run, Qwen3.6 wrote at between a seventh and just over a quarter of its July speed, and the median wait for the first token ranged from half a second to just over two seconds. In the slowest run the 400-word explanation took 22 to 28 seconds instead of just under 4. Qwen3.8 was hit much harder that afternoon. Time to first token ranged from 1.2 to just over 60 seconds in the five short tests and reached up to two minutes for the three longer texts. Test calls shortly before waited almost three minutes. The output itself ran at 4 to 7 tokens per second, so including the wait the same text took between two and a half and four and a half minutes. These are snapshots and may look different at another time of day. They do show that a free service used by a great many people at once has no predictable response time. The July versus August comparison with all individual values is in my re-measurement after four weeks.

Photo: Magda Ehlers / Pexels

The reasoning default has not changed. Without enable_thinking: false, Qwen3.6 still thinks before answering, and in my test 284 of 290 output tokens went to reasoning even though the answer was a single number. The code example below shows how to switch it off.

What Hetzner actually offers

The scope was deliberately small at launch. At the end of July there was exactly one model, Qwen3.6-35B-A3B in FP8 quantisation. It is a mixture-of-experts model with 35 billion parameters, of which only around 3 billion actively compute per request. Put simply, the model carries a lot of knowledge around but uses only a small, fitting slice of it for each answer. That makes it fast and cheap to run, with limited depth as the trade-off. A second model has been added since, see the September update above for the current line-up.

Access works through the experiments dashboard at experiments.hetzner.com. You create an API token there, and from then on the interface behaves like OpenAI’s. It is free, too, because Hetzner simply has not built billing yet. The other side of that coin comes with it: no SLA, no operational guarantees, explicit experiment status. Hetzner visibly wants to find out first whether and how the offering gets used.

OpenAI-compatible means changing two lines

The most important practical point is compatibility. The API speaks the OpenAI format, which means existing code keeps running once you swap the base URL and the key:

from openai import OpenAI

client = OpenAI(
    base_url="https://inference.hetzner.com/api/v1",
    api_key="<your-hetzner-api-token>",
)

response = client.chat.completions.create(
    model="Qwen/Qwen3.6-35B-A3B-FP8",
    messages=[
        {"role": "user", "content": "Summarise this request in one sentence: ..."}
    ],
    extra_body={"chat_template_kwargs": {"enable_thinking": False}},
)

print(response.choices[0].message.content)

The extra_body line switches off the reasoning mode that Hetzner runs by default. Without it the model thinks at length before the actual answer, and with a tight token limit the answer comes back empty. I explained why in my German language test of the model.

Anyone who has ever migrated between classic cloud providers knows what a difference this is. No new SDK, no rewritten calls, no weeks of migration work. The OpenAI format has become the de-facto standard, and Hetzner is right to simply adopt it. For testing purposes it means you can point an existing application at Hetzner in minutes and compare directly.

Photo: pankajpatel / Unsplash

How fast is it?

First independent measurements from 23 July 2026 gave usable reference points, with 153 milliseconds to first token in the median and around 224 tokens per second on longer outputs. For context, that makes a chat feel fluid, and it is more than enough for background jobs like summarisation or classification. It is one measurement at one point in time, things can look different under load, but as a first impression it is a respectable number. That things really do look different under load is shown by my own measurements from August and September, listed in the September update above.

On the model itself, expectations need to be set correctly. A 35-billion-parameter MoE model does not play in the league of the large models from OpenAI, Anthropic or Google. In early tests Qwen3.6 even stumbled over simple arithmetic. For the tasks you would realistically give a small, fast model, that is acceptable: summarising text, classifying emails, extracting data from documents, internal search and RAG building blocks, simple assistants. In practice these tasks make up most of what companies actually automate, and none of them needs a frontier model.

The real point: EU infrastructure, but no DPA yet

What makes the offering interesting is the location. The servers are in Germany and Finland, operated by a German company. The US CLOUD Act, which can force American providers to hand over data even from EU data centres, does not apply here. For anyone who wants to add AI features and gets an uneasy feeling about every US provider, this sounds like exactly the missing piece.

Photo: albertstoynov / Unsplash

Except the experiment is simply not built for that yet. Hetzner offers no data processing agreement, the documentation as of September 2026 does not mention one either, and without a DPA the contractual basis that Art. 28 GDPR requires for processing personal data through a service provider is missing. Server location alone does not make processing lawful. Hetzner’s FAQ saying that no request or response content is stored is a good sign, but it does not replace the contract. Add the missing SLA, which turns any production use into a gamble, and you get a clear boundary you should know before getting too excited.

What already works cleanly today: prototypes and feasibility tests, internal tools without personal data, processing of synthetic or anonymised data, development and staging environments. That is less than one would wish for and more than it sounds. Most AI projects I see do not fail in production, they fail before that, because nobody could cheaply find out whether the use case holds up at all. Exactly that experiment is free now.

Why I take this experiment seriously

You could dismiss the whole thing, small models, no SLA, no billing. I read it differently, because of the pattern behind it. Hetzner has shown in the classic cloud business how it enters markets: late, unspectacular, and then with prices a factor of 4 to 10 below the US hyperscalers. The GPU capacity exists, the data centres are built, the cost structure is famously aggressive.

If this experiment becomes a regular product with billing, an SLA and a DPA, something emerges that barely exists in this form so far, an AI inference API on European infrastructure at Hetzner prices. For small and mid-sized companies in the EU that would answer the most common question I hear about AI automation, namely where the data actually ends up. Whether it happens will presumably depend on how much the experiment gets used. One more reason to try it.

Until then, a pragmatic split applies. Develop and test against the Hetzner API, run production workloads with personal data where DPAs and SLAs exist. Thanks to the OpenAI compatibility, the later move really is just a change of the base URL.

My conclusion

Hetzner Inference today is a free experiment with two small models and clear limits, but it is the most strategically interesting infrastructure signal to come out of Germany in a long time. The interface is there and the location is right. Speed, on the other hand, depends noticeably on load, as the September measurements make clear. What is missing are the contracts, and for a provider of this size those are a matter of decision, not capability.

If you are currently weighing where the AI features of your website or internal workflows should run, with a US provider, on your own hardware, or eventually on an EU API like this one, feel free to get in touch. I build these automations regularly and will give you a sober comparison of what fits your case.