Skip to content

[Bug] Rating parser silently converts unrecognized output to Hold #1170

Description

@lx7720

Problem

The shared rating parser uses Hold as its default when it cannot extract a recognized five-tier rating. Because Hold is a valid rating, this conflates "no parseable rating" with an actual neutral decision.

The parser first looks for a canonical Rating: X-style label and then scans whitespace-separated tokens for the five-tier vocabulary. The fallback is sensitive to formatting details such as Unicode punctuation and localized labels. For example, output such as the following may contain a recognizable English rating but still fail exact token matching:

  • Rating:Overweight
  • Final rating: Sell(bearish)
  • 评级:Overweight(超配)

This request does not require the project to become Chinese-first or to support a particular output language. The agent can remain English-default. The issue is language- and format-agnostic: when free-text fallback produces a non-canonical response, an unparseable result should not be silently converted into a valid Hold signal.

Impact

When structured output falls back to free text, the machine-readable rating can disagree with the human-readable Portfolio Manager decision. Downstream reports, memory entries, or signal consumers cannot distinguish a genuine Hold from a parser failure.

The five-tier rating and the separate three-tier trade action should also remain distinct. For example, Overweight is not interchangeable with Buy merely because both are bullish.

Possible direction

One possible direction would be to return an explicit REVIEW, None, or equivalent parse-status value when no rating is recognized, while preserving the raw text and parsing method. The parser could normalize harmless Unicode punctuation and support explicitly documented aliases, but the exact normalization policy is up to the maintainers.

For structured-output fallback, another option is to preserve a typed rating field or attach an explicit "unparseable free-text result" status instead of relying on a valid rating as the fallback value.

Acceptance criteria

  • An unrecognized or missing rating never silently becomes Hold.
  • Canonical English output, Markdown-wrapped output, and harmless punctuation variants are covered by tests.
  • A non-canonical or localized example is either parsed according to a documented policy or surfaced as an explicit review state; multilingual prompting is not required.
  • Rating and trade-action fields remain separate and are not inferred from each other without an explicit mapping.
  • A mismatch between the human-readable final decision and the machine signal is visible to downstream consumers.
  • No broker connection, order execution, account data, or private analysis reports are required.

This issue concerns deterministic parsing and failure handling in tradingagents/agents/utils/rating.py.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions