Skip to content
Docs

Troubleshooting — Tool Calls

Tool calling fails in three distinct ways, and they need different fixes. Identify yours first.

The three symptoms

Section titled “The three symptoms”

Send one non-streaming request with tools set, and look at the response:

What you see Meaning
tool_calls present Tool calling works. If it fails when streaming, it is a streaming issue — see below.
Raw <tool_call> or <function= visible as text No parser is set, or it does not recognise the model’s format.
content: "", tool_calls: null, finish_reason: "stop" The dangerous one. A parser matched the opening marker but could not read the body, so the output was swallowed.

That third case is worth understanding: the model generated a perfectly good tool call, a mismatched parser took ownership of it, failed to parse, and emitted nothing. The client sees an empty reply and waits for a tool call that can never arrive — so an agent hangs rather than erroring.

The fix, usually automatic

Section titled “The fix, usually automatic”

Lifeboat reads the model’s own chat template — which is the definition of the format and is name-independent — and corrects a mismatched or missing tool-call parser at server start, recording it in the audit log as server_tool_call_parser_corrected.

So the first thing to check is the audit log for that server. If the correction fired, the parser is right.

It is deliberately narrow: it fires only when the template proves a format and the configured parser is one known to fail silently on it, or is unset. Any other operator choice is left alone rather than second-guessed.

If it persists

Section titled “If it persists”
  • Re-apply the launch recommendation for that model, which sets the parsers from the model itself.
  • Check the server log for repeated parse failures — the engine warns when a parser fails consecutively while its buffer grows, which is the signature of a mismatch rather than ordinary partial parsing.
  • Renaming a model can matter if the template is unavailable. When weights are not local, the format is guessed from the name; once downloaded, the template is authoritative and the start-time correction fires.

Works unstreamed, fails streamed

Section titled “Works unstreamed, fails streamed”

A protocol-level ordering problem rather than a model problem, and it shows up on the block-structured Anthropic endpoint far more than the OpenAI one — clients there accumulate arguments by index and tolerate ordering slack, while a block must be well-formed.

Typical reports: “could not be parsed as JSON” with all bytes accounted for, or “no such tool available: (empty name)”. Both are the same root cause seen through two different client behaviours. Upgrade; this is fixed in current releases. See Anthropic-compatible API.

Seeing what the model actually emits

Section titled “Seeing what the model actually emits”

Send the same prompt with tools omitted and ask for the markup as text. No parser runs, so the raw form is visible — which tells you what format the parser needs to handle.

Thinking text in the reply

Section titled “Thinking text in the reply”

A different setting. <think> blocks in content mean no reasoning parser is set; set it and the thinking moves to its own field. See Configuration reference.

Sources and references

Section titled “Sources and references”