Skip to content

[Bug] First debate speaker fabricates the opponent's argument (current_response starts empty) #1176

Description

@taro0915

[Bug] First debate speaker fabricates the opponent's argument (current_response starts empty)

Summary

In both the investment debate (Bull Researcher / Bear Researcher) and the risk debate (Aggressive / Conservative / Neutral Analyst), the graph fixes a strict speaking order, and the agent that speaks first is given an empty "last argument from the other side" (current_response / current_conservative_response / current_neutral_response all start as ""). On a local run (AAPL, 2026-07-22), the first speaker routinely writes as though it is rebutting a specific opposing argument that was never made. This reproduces regardless of which side the graph puts first, and persists even where the prompt already contains an explicit instruction telling the agent what to do when the other side hasn't spoken yet.

Code evidence

investment_debate_state is initialized with current_response as an empty string:

# TradingAgents/tradingagents/graph/propagation.py, create_initial_state()
"investment_debate_state": InvestDebateState(
    {
        "bull_history": "",
        "bear_history": "",
        "history": "",
        "current_response": "",   # L46
        "judge_decision": "",
        "count": 0,
    }
),

The graph fixes a strict speaking order — Bull Researcher always runs before Bear Researcher:

# TradingAgents/tradingagents/graph/setup.py, L135
workflow.add_edge(current_clear, "Bull Researcher")

bull_researcher.py and bear_researcher.py both interpolate current_response directly into the prompt as "the other side's last argument," with no guard for the empty case:

# TradingAgents/tradingagents/agents/researchers/bull_researcher.py, L43
Conversation history of the debate: {history}
Last bear argument: {current_response}
# TradingAgents/tradingagents/agents/researchers/bear_researcher.py, L45
Conversation history of the debate: {history}
Last bull argument: {current_response}

When count == 0, whichever of these runs first receives current_response = "" and is still instructed ("Engagement: ... engaging directly with the [other]'s points") to rebut it.

Reproduction: same pattern in both speaking orders

  • Default order (Bull → Bear), smoke test on AAPL 2026-07-22 (logs/debate_full.md): the Bull Analyst's first and only line opens with

    "Look, I hear what you're saying about the 'higher-for-longer' interest rate environment. It's the standard playbook for bears: point to the Fed, point to the discount rates, and try to paint a picture of a stagnant ceiling..."

    — even though no Bear argument exists yet at that point in the run (current_response was "").

  • Reversed order (Bear → Bull), produced by editing setup.py L135 to workflow.add_edge(current_clear, "Bear Researcher"), rerunning, then reverting (logs/debate_reversed.md): the Bear Analyst, now speaking first with an empty current_response, opens with

    "Look, I understand why the bulls are cheering right now — the charts look 'clean,' and Apple is still the darling of the tech world..."

    — again rebutting a Bull argument that has not been produced yet.

The same pattern (opening by rebutting a specific claim from the side that hasn't spoken) was also observed by direct inspection in the Bull-first transcripts of expC_C1_iter2.json, iter5.json, iter9.json, and iter13.json (not an exhaustive audit of all 20 C1 runs, but every transcript sampled showed it).

The prompt-level avoidance instruction exists elsewhere in the codebase and is demonstrably ineffective

The risk-debate agents (aggressive_debator.py, conservative_debator.py, neutral_debator.py) — unlike the bull/bear researchers — do have an explicit guard for the empty case:

# TradingAgents/tradingagents/agents/risk_mgmt/aggressive_debator.py, L35
Here is the current conversation history: {history} Here are the last arguments from
the conservative analyst: {current_conservative_response} Here are the last arguments
from the neutral analyst: {current_neutral_response}. If there are no responses from
the other viewpoints yet, present your own argument based on the available data.

Aggressive Analyst is structurally the first risk-debate speaker (setup.py L145: workflow.add_edge("Trader", "Aggressive Analyst")), so current_conservative_response and current_neutral_response are both "" when this instruction fires. Despite the explicit instruction, the observed output still fabricates both other analysts' positions (logs/risk_debate_excerpt.txt):

"Aggressive Analyst: Look, I understand the hesitation from both sides — the conservative analyst is terrified of a 'valuation trap' because of these high interest rates, and the neutral analyst wants to wait for some kind of 'clarity' before moving..."

At the time this text was generated, current_conservative_response and current_neutral_response were both empty strings — the specific positions attributed to the Conservative and Neutral analysts ("terrified of a valuation trap," "wants to wait for clarity") were invented, not read from state. This shows the fabrication is not simply a missing-instruction problem: an explicit "don't assume, argue from data if no response yet" instruction is present in the prompt and does not prevent it, at least not reliably, for this model.

Reproduction

Environment

  • Repo: TauricResearch/TradingAgents, commit a33fd4c0f134485a43553a2c23a63cb14adbd88f (2026-07-18, v0.3.1)
  • Python 3.12.8, Windows
  • TRADINGAGENTS_LLM_PROVIDER=openai_compatible
  • TRADINGAGENTS_DEEP_THINK_LLM=google/gemma-4-12b-qat
  • TRADINGAGENTS_QUICK_THINK_LLM=google/gemma-4-12b-qat (same model for both roles)
  • TRADINGAGENTS_LLM_BACKEND_URL=http://localhost:1234/v1 (local LM Studio)
  • TRADINGAGENTS_MAX_DEBATE_ROUNDS=1, TRADINGAGENTS_MAX_RISK_ROUNDS=1
  • Temperature: unset (provider/model default — not fixed to 0, so run-to-run wording varies)
  • selected_analysts=("market", "news"), ticker AAPL, trade date 2026-07-22

Steps

  1. Default-order run: python smoke_test.py (repo root) — produces the full graph output including investment_debate_state; inspect bull_history/bear_history/current_response in the resulting state dump. → logs/debate_full.md, logs/full_dump_raw.txt.
  2. Reversed-order run: edit TradingAgents/tradingagents/graph/setup.py L135 from workflow.add_edge(current_clear, "Bull Researcher") to workflow.add_edge(current_clear, "Bear Researcher"), rerun step 1, then revert the edit. → logs/debate_reversed.md.
  3. Risk-debate transcript: same smoke-test run as step 1; inspect risk_debate_state (aggressive_history, current_conservative_response, current_neutral_response). → logs/risk_debate_excerpt.txt.

Limitations

  • Single ticker, single date: all results are from AAPL on 2026-07-22 only. No other ticker, sector, or market regime was tested.
  • Single model: both deep_think_llm and quick_think_llm were the same local quantized model (google/gemma-4-12b-qat) served via LM Studio. Behavior of the hosted models the project defaults to (e.g. GPT-5.x) was not tested and may differ substantially.
  • Temperature not fixed: sampling temperature was left at the provider/model default rather than pinned (e.g. to 0), so some run-to-run variance is expected.
  • Order-reversal test ran once per direction: the controlled speaking-order swap (setup.py L135) was run once for the default order and once for the reversed order, not repeated n times; the additional four C1 transcripts checked (iter2/5/9/13) used the default order only, so the reversed-order observation itself is a single data point corroborated qualitatively rather than statistically.
  • Fabrication was assessed qualitatively: identified by manually reading transcripts against the known-empty state fields, not by an automated large-n fabrication detector, so no fabrication rate is reported — only that the pattern was observed in every transcript checked.
  • No claim about trading outcomes: this issue is about prompt/state-handling behavior (what the agent asserts about a nonexistent prior turn), not about whether resulting ratings were profitable or otherwise correct in hindsight.

This should be read as a reproducible local observation on one configuration, not as a general claim about the framework's behavior across models, tickers, or dates.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions