MC-2, our GPU cluster in Melbourne
We own and operate MC-2, a bare-metal AMD MI355X cluster under Australian law, and we train and serve Matilda on it.
MC-2 at a glance
Melbourne, Australia
MC-2 is bare metal, so our code runs on the GPUs with no hypervisor or cloud layer in between.
A rented API is quicker to start on, but its provider sets these five. On hardware you own, you set them.
- Latency
- Pricing
- Availability
- Privacy
- Data residency
- What
- Bare-metal training and inference
- Where
- Melbourne, in a facility we operate
- Serving
- Private, OpenAI-compatible streaming API
- Runs
- Matilda, in production
Benchmarks
9,526 output tokens a second from one MI355X node.
We measured it in May 2026 with Qwen3.6 35B serving 256 concurrent streams of 10,000 tokens in and about 1,500 out, on plain vLLM with no speculative decoding or pruning.
We don’t claim to beat hosted APIs on per-request speed, and per-stream speed falls as concurrency rises, from 11.46 to 5.26 tokens a second on Kimi K2.6 between 64 and 256 streams. The run shows these models serving at production throughput on hardware we control. Full method and per-stream results.
Runtime
Our own inference engine, in Rust.
AMD’s RCCL has years of work behind it and has to handle every job a runtime might give it, while our engine only serves inference on the MI355X.
- 1.3×
- Faster than RCCL’s fastest configuration on the small all-reduce that decode runs for each token, across 8 GPUs from one host thread.
- ~20 µs
- The latency of that all-reduce, which is the fabric’s floor.
Economics
Owning the hardware changes the cost per token.
Public APIs need no upfront capital and scale on demand, but on a steady long-context workload an owned cluster costs less per token.
87%lower cost per token than public AI APIs, at best
Each request is 8,000 tokens in and 1,000 out, against public APIs from Azure, AWS, Fireworks and Mistral, with our hardware amortised over three years and data-centre costs included (Dec 2025).
9months
At 100,000 multi-page documents a day, the monthly bill fell from $77,254 on public APIs to $26,706, so the hardware paid for itself in nine months.
16GPUs
Moonshot AI’s deployment guide puts the smallest cluster for Kimi K2 at 16 GPUs, so frontier open weights need hardware at cluster scale.
Why open weights →The ROI of an AI Token Factory sets out the method, comparison set and assumptions.
The stack
We run each layer, from the GPUs to Matilda.
Because we run each layer ourselves, we can test a change to Matilda anywhere in the stack and ship it when it passes.
Published work