Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Making Data interpreter able to repair llm output, similar to what is done in ActionNode #1216

Open
usamimeri opened this issue Apr 22, 2024 · 1 comment
Assignees

Comments

@usamimeri
Copy link
Contributor

usamimeri commented Apr 22, 2024

Feature description
Using open-source models in the data interpreter often leads to issues with incapability of generating correct JSON format . I tried to add repair_llm_output: true in the config2.yaml but it didn't work.

by checking the source code, I noticed that the repair work is conducted in ActionNode, while my issue occurs in WriteAnalysisCode:

  1. The model generated incorrect JSON response and passed it into the CodeParser
  2. The parser did not match the regex
  3. the logger reported an error, then directly returned the incorrectly formatted text.
  4. The incorrectly formatted text was passed back to the action, where it failed during decoding with reflection = json.loads(CodeParser.parse_code(block=None, text=rsp))

The repair mechanism was not triggered.
20240422-180524

Therefore, it is ideal to adapt the repair functionality for the data interpreter as well.

Your Feature
Enabling Data Interpreter to postprocess llm output

@seehi
Copy link
Contributor

seehi commented Apr 23, 2024

FYI: @garylin2099

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants