In late July I introduced Hetzner’s new AI inference API and then tested the model it serves against real German office tasks. Back then the offering was a few days old, there was exactly one model, and the numbers were respectable.
Four weeks on, I wanted to know what had become of it. So I sent the same prompts to the same endpoint again, with the same method and the same token. The result is unchanged in one respect, considerably worse in another, and surprisingly better in a third.
The model line-up has turned over twice
At launch there was exactly one model, Qwen3.6-35B-A3B in FP8 quantisation. In early August three more were added, and prominent ones at that: Kimi-K2.7-Code, GLM-5.2 and DeepSeek-V4-Flash. For a free offering that was a considerable list.
Today none of that is left. The model endpoint returns exactly two entries:
Qwen/Qwen3.6-35B-A3B-FP8
Qwen3.8-27BThe three large models are gone, replaced by a new, smaller Qwen. Hetzner explained why on 17 August in a post of its own. Demand exceeded even the most optimistic expectations within six hours of launch. DeepSeek produced the most demanding requests with very large input and output volumes, while Qwen drew a high number of smaller ones. The result was latency that rose to almost ten minutes to first token at the 99th percentile. The service was effectively unusable at times.
Hetzner scaled back in response: fewer concurrent requests per instance, lower rate limits, a focus on small models. That is a sensible decision, and it explains the measurements.
Photo: KoolShooters / Pexels
Response time unchanged, throughput halved
I measure two things separately because they say different things. Time to first token describes how long you wait until anything happens at all. Throughput describes how fast text is produced after that.
| Measurement | 24 July (Qwen3.6) | 23 August (Qwen3.6) | 23 August (Qwen3.8-27B) |
|---|---|---|---|
| Time to first token, median | 244 ms | 257 ms | 364 ms |
| Throughput | 181 tokens/s | 75 tokens/s | 28 tokens/s |
The first row is the good news. At 257 milliseconds, Qwen3.6 sits practically where it sat in late July. Anyone using the API in a chat will notice no difference, the answer starts just as promptly as before. As last time, the outlier was the first call after a pause, 1.2 seconds back then, 1.25 seconds now. That looks like a cold start and disappears from the second call onwards.
The second row is the bad news. 75 instead of 181 tokens per second means 41 percent of the original speed is left. In the test run the same task, an explanation of roughly 400 words, took 8.0 seconds instead of 3.8. On a short classification that goes unnoticed. On longer outputs, and especially with an agent making several calls in sequence, it adds up noticeably.
What is interesting is that the two figures point in different directions. A quick start alongside slow output is exactly the picture you get when more requests share the same hardware. You get served immediately, but the compute time is divided.
Photo: MART PRODUCTION / Pexels
The new model writes better German
During the first test I found a fault that remains the most interesting finding of the whole experiment to me: Qwen3.6 dropped Chinese characters into a formal German business email. Specifically 预计, the Chinese word for “expected”, appeared mid-sentence exactly where the German word belonged. In four out of seven runs.
I repeated the same test, seven runs again, same prompt.
| Model | Runs containing Chinese characters |
|---|---|
| Qwen3.6 in July | 4 of 7 |
| Qwen3.6 today | 2 of 7 |
| Qwen3.8-27B today | 0 of 7 |
With Qwen3.6 the fault still occurs, at the same semantic spot and with the same word. Two out of seven instead of four out of seven is not an improvement worth relying on at this sample size, it is ordinary variance. The point is that after four weeks the fault is unchanged.
The new Qwen3.8-27B showed it in no run at all. Seven clean German business emails. That is the difference between a model you can let draft a text and one where somebody has to read every output before it reaches a customer.
The price sits in the table above. At 28 tokens per second, Qwen3.8-27B is considerably slower again than Qwen3.6 and needs 23 seconds instead of 8 for the same task. So you trade speed for language quality, and depending on the use case that trade is worth it or it is not.
Photo: panumas nikhomkhai / Pexels
What this means in practice
The measurements suggest a fairly clear split. For anything a human will read in the end, there is a strong case for the new model. For behind-the-scenes work where a schema secures the output, meaning classification, extraction and pre-sorting, the older model with nearly three times the throughput is the more pragmatic choice. I recommended that same split after the first test, it still holds, except the line now runs between two models rather than between two types of task.
What has not changed are the conditions. There is still no service level agreement and no data processing agreement. For personal data in production the contractual basis is therefore still missing, regardless of the servers sitting in Germany and Finland. Hetzner itself advises against production use, and the past four weeks show why: models came and went, limits were adjusted, performance fluctuates.
Even so, I find the experiment more interesting than before rather than less. A provider that publicly writes up how its own demand blew past its projections, and then visibly rebuilds, says more about the seriousness of the effort than a polished announcement would. The interesting question is what it turns into. If the experiment becomes a product with billing, commitments and contracts, something emerges that barely exists in this form on European infrastructure.
Until then it stays what it is: a very good playground for building and experimenting, and no foundation for a client project.
If you are weighing which tasks in your business a small language model can actually handle, and where a stronger model or a human needs to stay in the loop, feel free to get in touch. Drawing that line is usually what decides whether an AI project succeeds or frustrates.
FAQ
Is the Hetzner Inference API still free to use?+
Yes. Access still runs through the experiments platform and billing still does not exist. Hetzner has however lowered the rate limits and reduced the number of concurrent requests to give more users a chance. There is still no service level agreement, and Hetzner continues to advise against production use.
Why has throughput dropped while response time stayed the same?+
The two figures measure different things. Time to first token mostly depends on how quickly a request gets picked up at all. Throughput describes how fast text is produced after that. When more requests share the same hardware, the start stays quick while the output drags. That is exactly the picture the measurements show.
Which of the two models should I pick?+
For anything a human will read, there is a strong case for Qwen3.8-27B, because it produced clean German in my runs. For behind-the-scenes tasks where a schema secures the output and speed matters, Qwen3.6 with nearly three times the throughput is the more pragmatic choice. Both are subject to the same experimental limitations.
Can I rely on this API for a project?+
No, and that is not a criticism but the stated status of the offering. Hetzner calls it an experiment, there is no service level agreement, no data processing agreement and no availability commitment. The past four weeks show exactly that: models appeared and vanished again, limits were adjusted. For development and experimentation this is usable, as the foundation of a client project it is not.
Want to know more?
In a free intro call we discuss how you can use these topics for your company. Not a sales pitch, but an honest assessment.
Book a free intro call
