What one record of the Mooncake tool-agent trace looks like

gpu-tips, note 2. November 20th, 2026.

The Mooncake paper (FAST '25) published three request traces from a production LLM service. Many KV cache and routing benchmarks replay them. This note shows what one record of the tool-agent trace contains, what the whole trace looks like, and how to turn a record into a request.

The trace has no text. Each record gives the lengths of a request and a list of block hashes. The files are in the Mooncake repository.

One record

The file is JSON Lines. One line is one request. This is the first line of toolagent_trace.jsonl.

{"timestamp": 0, "input_length": 6758, "output_length": 500,
 "hash_ids": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]}
FieldMeaning
timestampArrival time in milliseconds from the start of the trace.
input_lengthPrompt length in tokens.
output_lengthNumber of generated tokens.
hash_idsOne ID for each 512-token block of the prompt, in order. The number of IDs is the input length divided by 512, rounded up.

The example has 6,758 input tokens. That is 13 full blocks and one partial block, so the list has 14 IDs.

How the block IDs show prefix sharing

An ID identifies a block together with all the blocks before it. Two requests that start with the same IDs have the same tokens in that part of the prompt. Thus the KV cache of one request can serve the shared part of the other.

The figure shows the first, second and fourth records of the trace. All three start with block 0. After that block, each request has its own IDs.

The shared block is 512 tokens. It is probably a system prompt that many agents use. A later request of the same agent session repeats a longer list, for example blocks 0 to 13 and then new blocks.

The trace has no session ID. A session is visible only as a sequence of requests whose block lists extend each other.

The whole trace

I computed these values from the file.

PropertyValue
Requests23,608
Duration59 minutes
Mean rate6.7 requests per second
Blocks per requestMedian 13, maximum 247
Distinct block IDs183,300 of 409,616 block references
Blocks that repeat the prefix of an earlier request55%
Tokensp10p50Meanp90p99Max
Input1,8146,3468,59616,80461,525126,195
Output3301825078972,000

What the values tell

How to turn a record into a request

The trace gives no text, so the benchmark must make the tokens. This procedure keeps the lengths and the prefix sharing exact.

  1. Use each block ID as the seed of a random generator. Draw 512 token IDs from the vocabulary of the model. The same block ID always gives the same 512 tokens.
  2. Join the blocks in the order of hash_ids. Cut the result to input_length tokens.
  3. Send the list of token IDs as the prompt of /v1/completions. vLLM accepts token IDs there, so no tokenizer changes the length.
  4. Set max_tokens to output_length.
  5. Send the request at its timestamp.

Keep the random token IDs below the special tokens of the model. Then no random token ends the turn or starts a tool call.

If a request is longer than the context of the model, keep the leading whole blocks that fit. Do not drop the request. A dropped request also removes the cache entries that later requests of the session use.

Is the prompt readable?

No. The prompt is random token IDs. If you decode it, you get unrelated word pieces from many languages.

This is not a problem for cache measurements. The KV cache, the prefix cache of vLLM and LMCache all use the token IDs as the key. They do not read the meaning. The prefill time and the cache load time depend on the number of tokens, not on their content.

What this method cannot measure

Back to gpu-tips or Home

My past life... here