Adding Grafana annotations using HTTP API

• Authenticating API requests
• Adding annotations via HTTP
• Application in Node-RED

Grafana is a really flexible visualization tool for home and laboratory automation, data analysis, industrial plant monitoring, etc. and it’s pretty easy to fetch annotations into a dashboard using annotation queries. But sometimes it’s even more convenient to use an HTTP request to add annotations to a dashboard or even to a single panel instead of going the full route through the database (in my opinion - if it’s possible to use the route via database one should do this to remove any dependencies of the backend from frontends such as Grafana). Some examples for what this might be useful:

• Adding annotations about alert messages
• Opening or closing of doors or valves
• Motion alarms
• Changing system state
  • Electron or ion beam source status switches
  • High voltage supply status in particle accelerators and physics experiments
  • air conditioning setpoint changes
• … and many more

Authenticating API requests

First of Grafana API requests require an API key to be used (I won’t ever suggest using basic authentication using user credentials - just don’t do it).

One can create API keys either as an administrator for the whole installation or for organizations. Unfortunately it’s not possible to restrict API keys to single dashboards or allow just pushing no annotations so they’re rather high risk from a security point of view - just threat them like precious passwords to your most valuable information.

To create an API key first select Configuration and the menu option API keys

There you can select Add API key. Enter a reasonable name - this will be used to identify the key later on in the configuration and potentially revoke it - and either Editor or Admin role to allow write access to annotations. Since I’m a huge fan of least possible privileges use Editor is possible - this already has way enough permissions. Also select a reasonable lifetime for the key. If you really don’t want the key to expire set to 0.

After creation of the key immediately copy and store it at a secure location (like your password manager).

Fortunately Grafana even shows us how to use the key using curl:

curl
   -H "Authorization: Bearer eyJrIjoiSXI1UnYwNzRJQ0haS3hrdFE4aFpzelVlanB2cU9sNmsiLCJuIjoiTm90aWZpY2F0aW9uIEJsb2cgQXJ0aWNsZSIsImlkIjoxfQ==" \
   https://www.example.com/api/dashboards/home

As one can see this is a request for the home dashboard - the -H argument simple adds an Authorization header that includes a bearer token - this is a token that anyone can pass to proof their identity.

Adding annotations via HTTP

Adding annotations is done using POST requests. To do this one requires some additional information:

• The title of the annotation. This is required and shown as - for obvious reasons - annotation title
• The text that is also required.
• There are additional optional arguments:
  • The dashboardId. If one does not supply a dashboard identification (id - note that this is not the uid) the annotation affects all dashboards
  • The panelId specifies a single panel that is annotated. If no panel is specified all panels of the given dashboard are annotated.
• Optionally one can set the event time - and for timespans the start and end timestamp (note that timestamps usually have to be supplied in milliseconds instead of seconds)
  • The event time - or start time in case one supplies a region - is added numerically as timestamp using time (example: "time":1627501786)
  • In case one specifies a region one should set "isRegion":true as well as supply the timeEnd parameter (ex.: "timeEnd":1512548410517)

In case one does not want to use the API to determine the dashboardId the most efficient method I personally found has been to use the JSON display of the dashboard or panel id.

For the dashboard this is located in the dashboard settings - one can simply locate the id:

The same method can be used for the panel ID by selecting Inspect and panel JSON:

Then one just has to send the JSON describing the annotation the the /api/annotations endpoint. Using curl this would look like the following:

curl \
   -H "Authorization: Bearer eyJrIjoiSXI1UnYwNzRJQ0haS3hrdFE4aFpzelVlanB2cU9sNmsiLCJuIjoiTm90aWZpY2F0aW9uIEJsb2cgQXJ0aWNsZSIsImlkIjoxfQ==" \
   -X POST \
   -H "Content-Type: application/json" \
   -d '{ "text" : "Test annotation pushed via curl", "dashboardId" : 6, "panelId" : 2 }' \
   https://www.example.com/api/annotations

If everything turns out to work correct Grafana will respond with a JSON containing the annotation ID:

{"id":1,"message":"Annotation added"}

An example range annotation would look like the following command:

curl \
   -H "Authorization: Bearer eyJrIjoiSXI1UnYwNzRJQ0haS3hrdFE4aFpzelVlanB2cU9sNmsiLCJuIjoiTm90aWZpY2F0aW9uIEJsb2cgQXJ0aWNsZSIsImlkIjoxfQ==" \
   -X POST \
   -H "Content-Type: application/json" \
   -d '{ "text" : "Watering plants", "dashboardId" : 5, "panelId" : 2, "time" :  1627500813000, "isRegion" : true, "timeEnd" : 1627504413000 }' \
   https://www.example.com/api/annotations

Application in Node-RED

How could one use this API to push out notifications from a Node-RED flow? One could simply use the HTTP-Request node. This node already allows one to configure the endpoint as well as the bearer token that will be used during authentication:

The message that’s pushed into the http node contains the payload to be sent to the endpoint. One could also use the msg.topic parameter to specify the endpoint URI during runtime - and it’s also possible to pass the bearer token dynamically (which should of course only be done if really necessary - if it’s possible to use Node-RED’s store for passwords this should be done).

For example in case one wants to generate an event with timespan one could use the following function node in front of the HTTP request:

/* Somehow generate start time and end time */

return {
    headers : {
		/* If required we could also set the bearer token here */
        /* Authorization : "Bearer eyJrIjoiSXI1UnYwNzRJQ0haS3hrdFE4aFpzelVlanB2cU9sNmsiLCJuIjoiTm90aWZpY2F0aW9uIEJsb2cgQXJ0aWNsZSIsImlkIjoxfQ==", */
        "Content-type": "application/json"
    },
    payload : JSON.stringify(
        {
            "text" : "Example Grafana region event",
            "dashboardId" : 5,
            "panelId" : 2,
            "time" : startTime,
            "timeEnd" : endTime,
            "isRegion" : true
        }
    )
};

As above a single event just lacks timeEnd and isRegion. And also as mentioned above one should really consider using the database backend and annotation queries as an alternative to decouple the dependency of one’s backend from Grafana on the frontend side to follow a more microservice like approach.

This article is tagged:

• Programming
• Tutorial
• Web
• System administration
• Computer
• Home automation
• Shellscript
• Node Red