ive been watching the ai gateway, and

ive been watching the ai gateway, and most errors are either capacity or something like

AiError: AiError: unexpected tokens remaining in message header: [\"……………...]???…….……...…..\", \"………...…ż…………..………………….………………………..\", \"...??…\",

AiError: AiError: unexpected tokens remaining in message header: [\"……………...]???…….……...…..\", \"………...…ż…………..………………….………………………..\", \"...??…\",

AiError: AiError: unexpected tokens remaining in message header: [\"……………...]???…….……...…..\", \"………...…ż…………..………………….………………………..\", \"...??…\",

AiError: AiError: unexpected tokens remaining in message header: [\"……………...]???…….……...…..\", \"………...…ż…………..………………….………………………..\", \"...??…\",

.... and it goes on for a while until it looks like some of the strings are words from thoughts?

samjs•9/17/25, 7:38 PM

Huh, I haven't seen that before. Thanks for raising!

samjs•9/17/25, 7:57 PM

My findings so far:
- I'm not seeing other accounts impacted by this, are you able to share your prompts with me so I can try and reproduce it?
- There's this related vLLM issue that might be the underlying cause: https://github.com/vllm-project/vllm/issues/23567

GitHub

[Bug]: openai_harmony.HarmonyError: unexpected tokens remaining in ...

Your current environment I keep hitting this error “ openai_harmony.HarmonyError: unexpected tokens remaining in message header” In multi-turn conversations when using gpt-oss-120b and with both vl...

ajgeiss0702OP•9/17/25, 11:18 PM

I can't even reproduce it reliably. I do retries on errors, and often this error will appear for the first few requests, then it will randomly succeed with nothing changing

ajgeiss0702OP•9/17/25, 11:20 PM

I should note that this is mainly just used for fun in a discord server, so mainly just an annoyance for me and the people in the server that use it.
the context is usually pretty long, since it sends at least the past 100 messages from the channel (more if the context window allows it and the bot has been running long enough) so im thinking that might have something to do with it? I use gpt-oss 20b on cf for another usecase that doesnt have as long context and that hasnt failed once as far as ive seen

ajgeiss0702OP•9/17/25, 11:22 PM

yeah i just checked that other usecase. in 203 requests, only 2 errored and they were just the capacity error

ajgeiss0702OP•9/17/25, 11:24 PM

with that long context, i should note it is well under the 128k token context window shown on cf's docs for the model. (the biggest appear to be around 12k tokens)

ajgeiss0702OP•9/17/25, 11:25 PM

I should note that it looks like the unexpected tokens remaining in message headerunexpected tokens remaining in message header also happens with the 120b model. Possibly less often though? hard to tell if its less often because that model is called less often though

ajgeiss0702OP•9/17/25, 11:27 PM

if its helpful, i might be able to send the full request (in dms) that the error happened on

samjs•9/18/25, 1:31 AM