Skip to main content

Data Flow

Version

Relevant since version 3.2.0.

The Data Manipulation plugin offers to convert any Grafana dashboard into a fully functional web application. No wonder getting lost in all option parameters and their interconnections is easy.

The schema below is created to illustrate the major parts and their roles.

Data flow diagram for the Data Manipulation panel.
Data flow diagram for the Data Manipulation panel.

Initial Request

The initial request is executed after the browser finishes a page-loading step. You can configure your Initial request as Query, Data Source, and REST API.

Query

The query is short for the native Grafana query. Any initial request carries data from a data source into Grafana and is what the core Grafana does already with excellence. The Query method works with the data frames prepared by the Data Source, the one you specified on the left.

This method was created to allow leveraging the existing Grafana data extraction mechanism. To make it work, map every Data Manipulation form element to the appropriate data frame field.

Version

Since version 4.2.0. the Query Field and Field Name fields have been relocated from the Elements category to the Initial Fields options category.

Configure the Initial Request for Query.
Configure the Initial Request for Query.
New location of `Query Field` fields.
New location of `Query Field` fields.

Data source

Do not confuse this data source with the one you can select on the left-hand side. This is a different parameter located in the Update Request category and is visible when the Data source type is selected.

The Data source type is more flexible than the Query type. However, it means the developer must also take more responsibility by creating the Initial Payload Request.

The Initial Payload Request is a mandatory piece of code that needs to contain the database request at the bare minimum. More complex data parsing functionality could be added if required.

Steps to configure the Initial Request for Data Source.
Steps to configure the Initial Request for Data Source.
Steps to configure the Initial Request for Data Source. New location of `Field Name` fields.
Steps to configure the Initial Request for Data Source. New location of `Field Name` fields.

REST API

The REST API type works with an external API server. When specifying the API URL, you can use global and dashboard variables. Header parameters are available as separate options.

Steps to configure the Initial Request for REST API.
Steps to configure the Initial Request for REST API.

You can find ready-to-use API servers for Deno, InfluxDB, JSON API, MySQL, Node-Red, and PostgreSQL in servers section.

Find more information, tips and tricks in our documentation.

Initial Request Custom Code

To create post-processing logic, you can access panel options, API responses, form elements, Grafana services, dashboard, and global variables. Find the code snippet here.

Initial Request Custom Code exists for every Initial Request type.
Initial Request Custom Code exists for every Initial Request type.

Ball is on the user's side

The Data Manipulation form rests at this step, awaiting the user's actions.

Highlight Changes

You can enable the Highlight Changes feature, which displays all modifications made by the user in a specified color. For the Query and REST API Initial request types, simply enabling the feature in the plugin edit mode is enough.

For the Data Source type, you must use the SetInitial() function in the Initial Request Custom Code. See the illustration for the Initial Request, Data Source type above (on the picture). Also, find code examples here.

Enable Highlight Changes to see the new values on the form.
Enable Highlight Changes to see the new values on the form.

Reset button

The reset button is not shown by default. To review all available options, switch the visibility parameter to anything, but Hidden. Primary, Secondary and Destructive are pre-set visualization templates. In the Custom, you can specify foreground color, background color, icon and the text to replace the 'Reset' label if needed.

When this button is not hidden (any other mode is selected), you will also get the Reset Request category. There, you can go with either Custom Code, Initial Request or Data Source option.

The Initial request is selected by default. In that event, the plugin executes the Initial Request again as if the form is being loaded for the first time.

With the Data Source reset action you can specify a data source for the Reset Request and set payload in the following Custom code area.

In the Custom code, you can have any custom logic required. For instance, set all form elements to their defaults.

Reset button can call the Initial Request or execute any custom code.
Reset button can call the Initial Request or execute any custom code.

Update request

The Update Request carries data from the Grafana dashboard to your data source. There are two ways of doing so - Data Source and Rest API. This process is independent of the Initial Request, which means you can use any combination of Initial and Update Request types.

Data Source

For the Data Source type, start with the Data Source parameter. Then go to the Update Request Payload, select Custom code and create a payload script there. Use JavaScript and the language of your data source to create a code that will update your data source data.

Steps to configure the Update Request for Data Source.
Steps to configure the Update Request for Data Source.

Query editor and Frontend Data Source

Breaking change

Query editor and Frontend Data Sources are available starting from version 4.0.0.

This feature is related to the:

  • Initial Request -> Initial Action -> Data Source,
  • Update Request -> Update Action -> Data Source.
The payload area relates to the Data Source (Initial and Update requests).
The payload area relates to the Data Source (Initial and Update requests).

We simplified work with the payload creation. The Business Forms panel has a designated area to enter the payload request.

Payload for the Initial Request example.
Payload for the Initial Request example.
Payload for the Update Request example.
Payload for the Update Request example.

Use Text Area with multiple lines

Text Area element allows you to create text with multiple lines:

Text Area element with multiple lines.
Text Area element with multiple lines.

After entering the data and submitting the form, you may see a data source error. The issue is caused by not properly processing and converting multiple lines in the payload.

Data source update error with multiple lines.
Data source update error with multiple lines.

To avoid this behavior, please add following steps in the Create Payload code editor:

  • Find element refer to the TextArea element.
  • Add replace logic to value payload[element.id] = element.value.replaceAll("\n", "\\n");

Code example

const payload = {};

context.panel.elements.forEach((element) => {
if (!element.value) {
return;
}

/**
* Required logic to update the value
*/
if (element.id === "description") {
payload[element.id] = element.value.replaceAll("\n", "\\n");
return;
}

payload[element.id] = element.value;
});

/**
* Check payload using developer tools
*/
console.log("update payload:", payload);

/**
* Data Source payload
*/
return payload;
Create payload code editor with custom payload to update multiple lines.
Create payload code editor with custom payload to update multiple lines.

REST API

See the description for the Initial Request REST API. Both Initial and Update requests have the same parameters. Also similarly, you can use global and dashboard variables in the API server call along with header parameters.

Submit button

You can enable the Confirmation step if needed. This step looks like a popup window with old and new values. For the Query and REST API Initial request types, simply enabling the feature in the plugin edit mode, Update Request category, is enough.

For the Data Source type, you must use the SetInitial() function in the Initial Request Custom Code. See the illustration for the Initial Request, Data Source type above (on the picture).

Confirmation step looks like a table with old and new values.
Confirmation step looks like a table with old and new values.

Following Confirmation, the Submit button initiates the Update Request.

Update Request Custom Code

Or in other words, the post-processing step. Similarly to the Initial Request Custom Code, this step exists in all Update Request Custom Code types. You can access the same entities from the code: panel options, API responses, form elements, Grafana services, dashboard, and global variable. Find the code snippet here.

For example, during the post-processing, you can output a status message, transition the user to another dashboard, reload a page, call an initial request, etc.

Save Default Button

In addition to the Submit and Reset button, there is a third one - Save Default. It saves the current form values in the dashboard. You can choose an icon and text instead of the pre-set ones.

Save Default Button configuration.
Save Default Button configuration.

Initial Request vs Update Request

Below is the comparative table showing the similarities and differences between the initial and update requests.

Similarities and differences between the initial and update requests.
Similarities and differences between the initial and update requests.

Buttons

The below schema summarises the information about buttons available in the Data Manipulation form.

Three buttons are available in the Data Manipulation form.
Three buttons are available in the Data Manipulation form.