Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature]: Health check for restart policy #4867

Open
pseudotensor opened this issue May 16, 2024 · 1 comment 路 May be fixed by #4868
Open

[Feature]: Health check for restart policy #4867

pseudotensor opened this issue May 16, 2024 · 1 comment 路 May be fixed by #4868

Comments

@pseudotensor
Copy link

馃殌 The feature, motivation and pitch

A small change to get alot of reliability back....

We see vLLM crash or hang in various ways, e.g.:

#4108
#4344

And manually managing that is a hassle.

vLLM team could easily add a HEALTHCHECK line in the Dockerfile so tools like autoheal can function.

https://hub.docker.com/r/willfarrell/autoheal/
https://docs.docker.com/reference/dockerfile/#healthcheck

Would looks like:

HEALTHCHECK --interval=5m --timeout=10s curl -f http://localhost/health || exit 1

This would allow one to use the other docker image to manage the vLLM images.

Alternatives

Manual labor

Additional context

No response

@robertgshaw2-neuralmagic
Copy link
Collaborator

@pseudotensor

Can you open a PR?

@pseudotensor pseudotensor linked a pull request May 16, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants