Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable query tracing & limits in Query Node #5088

Open
kdembler opened this issue Mar 1, 2024 · 1 comment
Open

Enable query tracing & limits in Query Node #5088

kdembler opened this issue Mar 1, 2024 · 1 comment
Assignees

Comments

@kdembler
Copy link
Member

kdembler commented Mar 1, 2024

Recently we've had a lot of issues with our QN instances being unavailable or very slow to respond. This is most likely caused by some expensive queries running for a long time basically blocking all the other queries. To fix this situation I think we should do 2 things:

  1. Add ability to track queries that took more than X seconds to execute - query body, request headers so we can see what apps are making those. Ideally we would save that to some log file so it's easy to inspect later.
  2. Ability to abort queries that take longer than X seconds
@thesan
Copy link
Member

thesan commented Mar 4, 2024

@kdembler @mnaamani

I addressed (2) with Joystream/hydra#538

About (1) as I wrote on Discord the current open telemetry implementation should already enough that. Can the prod QN instance be run with open telemetry on ? should I change the command to also run it with pm2 ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Review
Development

No branches or pull requests

2 participants