[FEAT]: Limiting Token Usage

[chalitbkb]

What would you like to see?

Would it be possible to add a feature to limit the number of tokens that users can access per "workspace"? This would allow for more granular control over usage based on the cost rate of tokens, helping to reduce costs in more detail. In cases where users input long prompts and receive long responses, this would allow for more granular control. It could also provide a summary displaying the remaining tokens the user has available for that workspace. I think this would be extremely useful.​​​​​​​​​​​​​​​​ Setting limits on usage by number of messages sent per day may not be as granular, so I think this would be a valuable feature.​​​​​​​​​​​​​​​​

[atljoseph]

Second this. Even just showing the usage would be a huge value add. Break it down by total, workspace, user. If you could do a bar graph bucketed by the day that’d be great too.

@chalitguza are you referring to tokens per workspace per day? And in what timezone does a day start? I’d assume UTC?

[chalitbkb]

Second this. Even just showing the usage would be a huge value add. Break it down by total, workspace, user. If you could do a bar graph bucketed by the day that’d be great too.

@chalitguza are you referring to tokens per workspace per day? And in what timezone does a day start? I’d assume UTC?

If possible, I would like to have an option to set a daily token limit and a per-use token specification. The per-use limit means that within the "workspace," users can operate without daily or monthly limits. However, once they reach the specified token limit, for example, if an admin sets User: abc123 with a total of 50k tokens, the user cannot exceed this amount but will not be restricted by daily/monthly usage limits.

And yes, UTC can be set. Administrators can specify custom time zones such as UTC+1 or UTC-4, etc.

This may also be relevant: #1404