-
Notifications
You must be signed in to change notification settings - Fork 382
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
bug: Langfuse craps out when there is an error upstream #2001
Comments
do you use groq via the openai integration and host |
Not sure but I have seen this happen with OpenAI as well. |
Can confirm that I receive the same error
when OpenAI errors out too, due to rate limiting. |
@maxdeichmann yes. |
@maxdeichmann I'm not using the langchain integration. We have our own function that calls an LLM API that's wrapped in |
@marcklingen `Giving up execute_task_with_backoff(...) after 3 tries (langfuse.request.APIError: Invalid JSON (400): None) During handling of the above exception, another exception occurred: Traceback (most recent call last): |
Describe the bug
If there is an upstream error with the service like OpenAI or grok, Langfuse says invalid JSON error
Langfuse craps out when there is an error with groq.RateLimitError: Error code: 429 - {'error': {'message': 'Rate limit reached for model llama3-70b-8192 in organization org_BLAH on tokens per minute (TPM): Limit 7000, Used 1607, Requested ~47333. Please try again in 5m59.484999999s. Visit https://console.groq.com/docs/rate-limits for more information.', 'type': 'tokens', 'code': 'rate_limit_exceeded'}}
Giving up execute_task_with_backoff(...) after 3 tries (langfuse.request.APIError: Invalid JSON (400): None)
Giving up execute_task_with_backoff(...) after 3 tries (langfuse.request.APIError: Invalid JSON (400): None)
Giving up execute_task_with_backoff(...) after 3 tries (langfuse.request.APIError: Invalid JSON (400): None)
Another example groq.BadRequestError: Error code: 400 - {'error': {'message': 'messages.1.content: user message cannot be an null or empty', 'type': 'invalid_request_error'}}
Giving up execute_task_with_backoff(...) after 3 tries (langfuse.request.APIError: Invalid JSON (400): None)
Giving up execute_task_with_backoff(...) after 3 tries (langfuse.request.APIError: Invalid JSON (400): None)
Giving up execute_task_with_backoff(...) after 3 tries (langfuse.request.APIError: Invalid JSON (400): None)
To reproduce
See above
SDK and container versions
No response
Additional information
No response
Are you interested to contribute a fix for this bug?
No
The text was updated successfully, but these errors were encountered: