Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(imports/logger): Loki not indexing varargs #583

Merged
merged 1 commit into from
May 20, 2024

Conversation

SeaLife
Copy link
Contributor

@SeaLife SeaLife commented May 20, 2024

Hey!

i dont know if this is expected behaviour and the loki provider probably needs more rework but i played around with it a bit and currently its submitting the data in a wrong format. The code i used to submit logs (test):

RegisterCommand('t_log', function(source, args, rawCommand)
    lib.logger(-1, 'LogTest', 'Test 123', 'test123:test123', 'rpname:Lena Meier', 'job:police')
end, true)

(before)
image
(after my changes)
image

I tried the following:

RegisterCommand('t_log', function(source, args, rawCommand)
    lib.logger(-1, 'LogTest', 'Test 123', { rpname = "Lena Meier", job = "police" })
end, true)

Which ended up in this:
image

Probably this need some more rework, i think submitting the meta data in this format key:value is inconvenient. But this is out-of-scope for this "fix" 🗡️

@thelindat thelindat merged commit 7e67b57 into overextended:master May 20, 2024
@SeaLife SeaLife deleted the patch-1 branch May 21, 2024 13:53
@Cata-a1138
Copy link
Contributor

Hello, I would like to report a situation where the code is not working properly in my environment.
loki version 2.8.11
promtail version 2.8.11

Test code
lib.logger(-1, 'LogTest', 'Test 123', 'test123:test123', 'rpname:Lena Meier', 'job:police')

postBody
{"streams":[{"stream":{"server":"[QBCore Framework] QBServer","resource":"pg_test","event":"LogTest"},"values":[["1716564324000000000","Test 123",{"job":"police","test123":"test123","rpname":"Lena Meier"}]]}]}

PerformHttpRequest callback function
[400,null,[],"HTTP 400: loghttp.PushRequest.Streams: []*loghttp.Stream: unmarshalerDecoder: Value is string, but can't find closing '"' symbol, error found in #10 byte of ...|Meier"}]]}]}|..., bigger context ...|ice","test123":"test123","rpname":"Lena Meier"}]]}]}|...\n"]

Code to print data

local postBody = json.encode({streams = tempBuffer})
print(postBody)
PerformHttpRequest(endpoint, function(...)
    print(json.encode({...}))
end, 'POST', postBody, headers)

@SeaLife
Copy link
Contributor Author

SeaLife commented May 25, 2024

Hello, I would like to report a situation where the code is not working properly in my environment. loki version 2.8.11 promtail version 2.8.11

Test code lib.logger(-1, 'LogTest', 'Test 123', 'test123:test123', 'rpname:Lena Meier', 'job:police')

postBody {"streams":[{"stream":{"server":"[QBCore Framework] QBServer","resource":"pg_test","event":"LogTest"},"values":[["1716564324000000000","Test 123",{"job":"police","test123":"test123","rpname":"Lena Meier"}]]}]}

PerformHttpRequest callback function [400,null,[],"HTTP 400: loghttp.PushRequest.Streams: []*loghttp.Stream: unmarshalerDecoder: Value is string, but can't find closing '"' symbol, error found in #10 byte of ...|Meier"}]]}]}|..., bigger context ...|ice","test123":"test123","rpname":"Lena Meier"}]]}]}|...\n"]

Code to print data

local postBody = json.encode({streams = tempBuffer})
print(postBody)
PerformHttpRequest(endpoint, function(...)
    print(json.encode({...}))
end, 'POST', postBody, headers)

image

The generated JSON works by posting it via HTTPie directly to loki. I think this is not related to this?

Im using Loki 3.0.0, maybe there is a difference in the API?
image

@SeaLife
Copy link
Contributor Author

SeaLife commented May 25, 2024

Just investigated a bit around the API and it seems that they added posting meta variables in the values-Part in 3.0.0 and therefore this fails on 2.8.x

https://grafana.com/docs/loki/v2.8.x/api/#push-log-entries-to-loki
https://grafana.com/docs/loki/latest/reference/loki-http-api/#ingest-logs

If possible, can you upgrade to Loki 3.0.0 ?

Can you check if the error persist on ox_lib v3.21.0 ?

@Cata-a1138

@Cata-a1138
Copy link
Contributor

Cata-a1138 commented May 25, 2024

Just tested in loki 3.0.0 and ox_lib v3.21.0, get the same results as you, thank you for your reply.

@Cata-a1138
Copy link
Contributor

And I found the reason why you get "table":0x7fe593f2ca50".
That is, { rpname = "Lena Meier", job = "police" } in string.tostringall(...) is then converted to a string, so json.encode doesn't work as expected.
I have opened a pull request to solve this problem. #588

thelindat added a commit that referenced this pull request May 30, 2024
FilipeCuco added a commit to FilipeCuco/ox_lib that referenced this pull request May 30, 2024
Revert "fix(imports/logger): loki indexing for varargs (overextended#583)"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants