LLaVA image freeze after Third inference

[KLL535]

Description:

When using the LLaVA multimodal chat handler Llava15ChatHandler or Llava16ChatHandler, the model stops updating images starting from the third request. After this point, all responses reference the image from the second request, regardless of what new image or prompt is provided. This issue does not occur with Qwen-VL models using the same setup.

Environment:

llama-cpp-python version: 0.3.32 (latest commit)
llama.cpp version: latest commit
Model: llama-joycaption-beta-one-hf-llava-q8_0.gguf + llama-joycaption-beta-one-llava-mmproj-model-f16.gguf
https://huggingface.co/concedo/llama-joycaption-beta-one-hf-llava-mmproj-gguf/tree/main
GPU: NVIDIA RTX 5080 (16 GB VRAM)
OS: Windows 10
Python: 3.13.2

Steps to Reproduce

Start a chat session with a LLaVA model
Send a message with Image A + text prompt → ✅ Model correctly describes Image A
Send a message with Image B + same text prompt → ✅ Model correctly describes Image B
Send a message with Image C + same text prompt → ❌ Model describes Image B
Send a message with Image D + different text prompt → ❌ Model still describes Image B

Recovery:

❌ Changing the text prompt does not help
❌ Sending more requests does not help
✅ Only a model reload resolves the issue

Request

Please investigate why LLaVA image caching behaves differently from Qwen-VL
I'm provide additional logs if needed to help diagnose this issue.

[JamePeng]

Does the code maintain multiple rounds of sessions? Also, please provide the relevant runtime code.

[KLL535]

I use create_chat_completion without storing dialogue history, that is, 1 question 1 answer.
Without resetting llama cache.

The code is here:
https://github.com/KLL535/ComfyUI_Simple_Qwen3-VL-gguf/blob/main/qwen3vl_run.py#L9

[JamePeng]

If you're not maintaining multi-round sessions and only keeping the llama object for a single round of sessions, you should clear the key-value cache and reset n_tokens in each round. It would be best to print out the logs for review.

[KLL535]

If image A is repeated many times, only the prompt changes? Does the image cache not work if cache reset each round? And why isn't there any freezing for qwen?

[JamePeng]

It's best to set verbose=True to see what the caching mechanism is doing.