Skip to content

fix: replace ast.literal_eval with json.loads for eval CLI parsing [AE-1071] - #1400

Closed
mjnovice wants to merge 2 commits into
mainfrom
fix/eval-json-parsing-AE-1071
Closed

mjnovice wants to merge 2 commits into
mainfrom
fix/eval-json-parsing-AE-1071

Conversation

@mjnovice

@mjnovice mjnovice commented Mar 3, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Replaced ast.literal_eval with json.loads in the LiteralOption CLI option class used by --input-overrides and --eval-ids
  • ast.literal_eval cannot parse JSON values like null, true, false — causing eval runs with file input overrides containing these values to crash
  • Added 13 regression tests covering JSON parsing with null, true, false, nested structures, and error cases

Root Cause

Part of the fix for AE-1071 — eval sets with file inputs crash because:

  1. Backend serializes input overrides as JSON (with null/true/false)
  2. Python CLI's LiteralOption uses ast.literal_eval which only understands Python literals (None/True/False)
  3. Parsing fails → entire eval batch crashes

Test plan

  • All 1591 existing tests pass
  • New test_literal_option.py with 13 test cases covering JSON null/true/false parsing
  • Lint passes (ruff check)

🤖 Generated with Claude Code

…ing [AE-1071]

ast.literal_eval cannot parse JSON values like null, true, and false,
causing eval --input-overrides to fail when the backend sends JSON with
these values. Switching to json.loads ensures proper JSON parsing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@github-actions github-actions Bot added test:uipath-langchain Triggers tests in the uipath-langchain-python repository test:uipath-llamaindex Triggers tests in the uipath-llamaindex-python repository labels Mar 3, 2026
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@mjnovice

Copy link
Copy Markdown
Contributor Author

not needed anymore

@mjnovice mjnovice closed this Mar 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test:uipath-langchain Triggers tests in the uipath-langchain-python repository test:uipath-llamaindex Triggers tests in the uipath-llamaindex-python repository

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant