05Sizing and hardware

Size it before you sign it.

Pick a machine, a model and a context window, and see how many people it will serve. The numbers start from a deployment we measured, and everything else is derived from it conservatively.

Which machine

Which model

How much context

Frontier quality, strongest for agent work. A bigger context window means more memory per conversation and smaller batches.

Measured on this configuration

~120people chattingconcurrent conversations at 256k

~40developers on agentslong-running agent sessions

Configuration8x H200 · 1,128 GB

ModelKimi K2.7 · 1T at FP8

Weights in VRAM~1,000 GB, leaving ~83 GB for cache

Throughput~93 tokens per second per request

01What drives the answer

Three numbers, and the third one surprises people.

The weights have to fit firstEvery parameter sits in VRAM whether the model activates it or not. If the weights do not fit the node, nothing else matters.

What is left over holds the usersThe memory that remains after the weights is the key-value cache. That is what a concurrent conversation actually consumes.

Context window is the expensive choiceGoing from 32k to 256k multiplies the memory each conversation holds. The same box then serves a fraction of the people. Choose the window the work needs, not the largest one available.

02The machines

Five configurations, from a workstation to two nodes.

We supply, install and support the box, or you buy the hardware and we deploy onto it. Either way the model, the data and the logs stay on your side of the network.

2x RTX 6000 Ada96 GB VRAMA workstation under a desk. A department, a pilot, or a site with no data centre.

4x H100320 GB VRAMOne node. Mid-size models at full precision, or a large model quantised.

8x H100640 GB VRAMThe common enterprise node. Most of the model list fits, several at full precision.

8x H2001,128 GB VRAMTrillion-parameter models on one node. This is the configuration we measured.

16x H2002,256 GB VRAM, two nodesWhere one model has to serve the whole organisation, or where two models run side by side for chat and coding.

03Where the numbers come from

One measurement, and honest arithmetic from there.

We served Kimi K2.7 on 8x H200 with vLLM, following the official recipe, and measured 93 tokens per second per request at 256k context. Every other figure on this page is derived from that benchmark, deliberately on the low side.

These are first numbers, before any optimisation. Quantisation, prefix caching and a tuned batch size all move them upward, and we tune after the box is in place rather than promising it beforehand.

MeasuredKimi K2.7 on 8x H200 at 256k context, served with vLLM. 93 tokens per second per request, about 120 people chatting or 40 developers running agents.

EstimatedEvery other combination. Weights come from the parameter count at the serving precision, cache from the context window, and throughput scales with the node's compute against the measured baseline.

A developer is not a chat userAn agent run holds a long context and generates for minutes. One box serves roughly a third as many developers as it serves people chatting.

04Beyond the GPUs

What your facilities team will ask about.

Power and coolingA single eight-GPU node draws in the order of 10 kW under load and needs the rack cooling to match.

NetworkInternal only. No outbound path is required for the model to serve, and we can install with the interface disabled.

StorageFast local NVMe for the weights and the index. A trillion-parameter model is roughly a terabyte on disk.

ResilienceOne node is a single point of failure. Where the service must not stop, size for two and load balance across them.

05Where it sits

Your rack, your tenancy, or ours.

Residency, sovereignty and access are three separate questions, and the deployment you choose answers them differently. The Trust page sets out what each one does and does not guarantee.

Deployment options compared

01In your data centre, on hardware you own, with no outbound network path.

02In your own cloud tenancy, in the region your regulator requires.

03Hosted by us, for a pilot, with a defined path to move it in-house.

Model comparisonEvery model we serve, its published scores, and what it costs in memory.Senpiper CodebaseWrite and change software without your codebase leaving your environment.Assess your needsTell us the workload and we will size it against your own numbers.