Make customizing chart context information simpler, consistent and collaborative #14107
Replies: 7 comments 15 replies
-
Should be a list, not a dictionary. There are cases when you want to create N gauges using the same metric - e.g. Net Inbound/Outbound gauges. A lot of them. |
Beta Was this translation helpful? Give feedback.
-
|
@shyamvalsan how about if you start be extending the existing metadata to what we might want to have. eg what would we need to add here: https://london.my-netdata.io/api/v1/chart?chart=system.cpu {
"id": "system.cpu",
"name": "system.cpu",
"type": "system",
"family": "cpu",
"context": "system.cpu",
"title": "Total CPU utilization (system.cpu)",
"priority": 100,
"plugin": "proc.plugin",
"module": "/proc/stat",
"units": "percentage",
"data_url": "/api/v1/data?chart=system.cpu",
"chart_type": "stacked",
"duration": 138995,
"first_entry": 1670364870,
"last_entry": 1670503864,
"update_every": 1,
"dimensions": {
"guest_nice": { "name": "guest_nice" },
"guest": { "name": "guest" },
"steal": { "name": "steal" },
"softirq": { "name": "softirq" },
"irq": { "name": "irq" },
"user": { "name": "user" },
"system": { "name": "system" },
"nice": { "name": "nice" },
"iowait": { "name": "iowait" }
},
"chart_variables": {
},
"green": null,
"red": null,
"alarms": {
"20min_steal_cpu": {
"id": 1650007870,
"status": "CLEAR",
"units": "%",
"update_every": 300
},
"10min_cpu_iowait": {
"id": 1650007869,
"status": "CLEAR",
"units": "%",
"update_every": 60
},
"10min_cpu_usage": {
"id": 1650007868,
"status": "CLEAR",
"units": "%",
"update_every": 60
}
},
"chart_labels": {
"_collect_plugin":"proc.plugin",
"_collect_module":"/proc/stat",
"_instance_family":"cpu"
}
}Then we would have a sort of end state goal that building on something we already have. And then from there we could try work backwards to see how we might then centrally define such a global master /charts type lookup in yaml somewhere central. I think if we can introduce as little new terms like I think all we really need is to extend whats in the chart json above in a few ways - like you are suggesting and then just define it globally in one place and move to that then being the central source of truth for it all. |
Beta Was this translation helpful? Give feedback.
-
|
One part of this is that a user, who sees a particular chart and based on their subject matter expertise recognizes that either the "info" or "group-by" or "aggregation" should be different, should be able to easily - with minimal effort - open a PR for this. YAML is easier to grok than code (especially if our collector code is going to be primarily Go based, and the user is not a Go-dev). How easy would it be for such a user to update our existing metadata format? The other part is regarding the summary dashboard - I agree we should minimize new terms but we need some level of flexibility as well. Need to find right balance. It's debatable whether the summary dashboard definition needs to live in standard metadata or it can be its own thing. |
Beta Was this translation helpful? Give feedback.
-
@shyamvalsan : I am not sure if this should be a global setting based on "one" user recommendation?
And the above considerations to be made for each chart as well as the summary dashboards. |
Beta Was this translation helpful? Give feedback.
-
|
This is not per space, it's for Netdata as a whole. The YAML is a centralized file, only one copy of it in production. So there's really only 1 user journey:
I don't think we need to be as flexible as to do this per space at this moment. Lots of bigger fish to fry :) |
Beta Was this translation helpful? Give feedback.
-
|
@shyamvalsan can you make the full yaml for the |
Beta Was this translation helpful? Give feedback.
-
|
i also wonder if we should maybe just drop all down straight to the metric level and then have a separate place that defines a chart as a collection of metrics. |
Beta Was this translation helpful? Give feedback.
-
|
this below seems to mix chart and metric stuff: collector.summary:
section: <optional field, text>
row: <optional field, number>
collector.metric:
chart-type: <mandatory, options are gauge|pie|number and any summary chart types we support in future>
title: <mandatory>
units: <mandatory>
textbox: <optional, supports markdown> |
Beta Was this translation helpful? Give feedback.
-
|
think will be easier (as much as possible) to just define all metrics separate and then charts separate and just use the metric key's in them eg can we not have equivalent to a spreadsheet - one row per metric, a col for each metadata of importance - could something like that work (not suggesting a spreadsheet - just to help us conceptualize if we could make one row per metric work). |
Beta Was this translation helpful? Give feedback.
-
|
This is primarily about charts, yes. Me mentioning the word " I took my cue from So I guess I am suggesting something similar to what you are suggesting :) metric metadata can remain in the place where it is today and have a separate place to define chart properties. |
Beta Was this translation helpful? Give feedback.
-
|
I made some updates above to reflect that we're talking about contexts and charts. |
Beta Was this translation helpful? Give feedback.
-
|
agreed with all said so far, the centralized place to keep chart properties definitions per context summary sections, default options, etc. makes total sense.
I think this has come in the past, to allow definitions per Space / War Room / User to be used on the Overview but we focused first on providing this on the custom dashboards first. |
Beta Was this translation helpful? Give feedback.
-
|
I think we should also consider the TOC sections here - I've updated the YAML description to include.
|
Beta Was this translation helpful? Give feedback.
-
|
on this one I'm not so sure... the goal is for this stuff to be driven from the context itself and OpenTelemetry standards netdata/netdata-cloud#354 and having two things commanding the creation on the TOC seems to lead to a possible big mess I think you're probably mentioning this because of this netdata/netdata-cloud#550, I would probably keep the TOC creation as it is and allow the definition of a chart label to be used as a group by/breakdown to create further sub-sections on the TOC. that would probably deliver what is on #550 |
Beta Was this translation helpful? Give feedback.
-
|
Fair point @hugovalente-pm But the question is should collector really be the only place we can modify the sections a chart shows up in. For example, if a user who's an SME wants to propose a new section or ordering, is collector code the best place for it? |
Beta Was this translation helpful? Give feedback.
-
|
in a situation like that, it would probably mean our naming of the metric is not the most appropriate, so I think we should really review it from the collector. @ilyam8 is into this in more depth so he can probably share his view |
Beta Was this translation helpful? Give feedback.
-
|
I'm removing |
Beta Was this translation helpful? Give feedback.
-
|
I still think this would probably be valuable:
here we would be just using the available metadata associated with the context to allow a further breakdown on TOC sub-sections |
Beta Was this translation helpful? Give feedback.
-
|
Feedback from call with @cakrit @sashwathn @amalkov :
|
Beta Was this translation helpful? Give feedback.
-
|
More feedback based on feedback, discussions and research:
An updated version of the Until FE has implemented the changes (section separator, multiple rows in summary, color coded thresholds, info box supporting markdown) the following fields will be ignored: |
Beta Was this translation helpful? Give feedback.
-
Objective
When creating a new collector or editing an existing one, I should be able to specify the below details per chart context in a single code location in a consistent, simple manner
group-bythe chart should useAVG, SUM, MIN or MAXby defaultI should also be able to define what charts show up in the summary dashboard for a collector, and in what form and what order these charts should be visualized.
Problem(s)
On Netdata cloud to edit the context information (1) I need to edit
contexts.jsfile in thenetdata/cloud-frontendrepo (This is a private repo and this file cannot be edited by the community)netdata/netdatarepo) I need to edit calleddashboard_info.jsbut changes done here will have to be manually ported over to the previously mentionedcontexts.jsfile in the netdata/cloud-frontend repoThe other details such as default group-by, aggregation function etc. cannot currently be modified by someone working on the collector.
group by databaseby default) it is hardcoded in thecloud-frontendprivate repo by the FE teamTo create summary dashboards for a collector (section) I need to edit the same
contexts.jsfile in the privatenetdata/cloud-frontendrepo (Hence cannot be edited by the community, changes not propagated automatically to local dashboard etc.)Idea
contexts.yaml, in thenetdata/netdata-cloudpublic repo which is accessible to communityHere's a short example
And a more verbose example
summary.yaml, dedicated to summary dashboards with a YAML definition per collectorThe
section,row,textboxfields are for future proofing, we do not support these in Netdata today. But to do full page useful summary dashboards per section we need these three fields.sectiondefines a header where charts can be grouped together,rowdefines how many rows there are per section and how many charts per row.textboxallows us to create a helpful info box with hyperlinks or any other text.An example:
netdata/cloud-frontendwill always pull the latest version of the YAML when doing a build and use it to auto-generate thecontexts.jsfileThis will ensure that the problems mentioned above are solved, and there's a simple, consistent and collaborative way to do this.
Beta Was this translation helpful? Give feedback.
All reactions