LLM responses not in JSON format: instructions instead of data

[ericsun2]

Version: 0.9.46

Reproduction pattern:

1. graphify extract <source_code_dir> --backend (openai|gemini|nvidia|...)

$ graphify provider show nvidia
{
  "nvidia": {
    "base_url": "https://integrate.api.nvidia.com/v1",
    "default_model": "nvidia/nemotron-3.5-lightning-30b-a3b",
    "env_key": "NVIDIA_API_KEY",
    "context_length": 65536,
    "pricing": {
      "input": 0.0,
      "output": 0.0
    },
    "temperature": 0
  }
}

The same errors are not observed for gemma4 because of the leak of CoT.

[graphify extract] semantic extraction on 4908 files via ...


[graphify] LLM returned invalid JSON, skipping chunk (first 200 chars: 'Let me analyze the provided source files to extract a knowledge graph fragment. I need to follow the rules carefully:\n\n1. EXTRACTED: relationship explicit in source\n2. INFERRED: reasonable inference\n3')
[graphify] chunk of 19 truncated at depth 2, splitting into halves of 9 and 10
[graphify] LLM returned invalid JSON, skipping chunk (first 200 chars: "Here's a thinking process:\n\n1.  **Analyze User Input:**\n   - User provided several `<untrusted_source>` blocks with file paths and content.\n   - The task is to extract a knowledge graph fragment from ")
[graphify] chunk of 9 still truncated at recursion depth 3 (max 3) — partial result kept (not cached as complete)
[graphify] LLM returned invalid JSON, skipping chunk (first 200 chars: "Here's a thinking process:\n\n1.  **Analyze User Input:**\n   - User is asking me to extract a knowledge graph fragment from provided files.\n   - There are images attached with source files listed: `docs")
[graphify] chunk of 3 still truncated at recursion depth 3 (max 3) — partial result kept (not cached as complete)
[graphify] LLM returned invalid JSON, skipping chunk (first 200 chars: 'Here\'s a thinking process:\n\n1.  **Analyze User Input:**\n   - User is providing image files as visual input.\n   - Instructions: Emit one node per image with `"file_type":"image"` and the listed source_')
[graphify] chunk of 6 truncated at depth 2, splitting into halves of 3 and 3
[graphify] LLM returned invalid JSON, skipping chunk (first 200 chars: 'Here, I need to extract a knowledge graph from the provided files. The user has given me a set of image files with their source_file paths. I need to output valid JSON with nodes and edges according t')

Root cause: Nemotron has/uses different responses.

Request: add certain harness to deal with Nemotron.

Workaround: Use other models or modify ~/.graphify/providers.json