Display Large PDF documents in Grafana

We discussed the benefits of using Docker containers and initial Provisioning in the recent article about creating our panel plugin template for Grafana. While working on the current feature request for the Base64 image/PDF panel, this technique helped us quickly deliver solutions and improve the long-term support for the panel.

When we created a Base64 image/PDF panel to display images for one of our projects, support for PDF documents was added as a feature. I am glad the panel is being used to display PDF documents stored in the databases like PostgreSQL.

You can learn more about the plugin in the video on our YouTube channel.

Images, PDFs, Video, Live Camera Feed on Grafana Dashboard.

PDF documents

Depending on the size, PDF documents are classified as

• Small (10-100kb)
• Medium (100-1MB)
• Large (1-16 MB)
• Huge (16-128 MB).

Our panel supported Small and Medium-sized PDF documents, which are easy to test and do not require any database to store. We used the Static Data Source, and data was held in the dashboard.

To test support for Large documents, we

• Installed PostgreSQL.
• Loaded PDF documents into the database.
• Created a data source and dashboard to validate and check performance.

Also, it has to be scripted, and easy to deploy for continuous integration and development.

PostgreSQL

From Grafana's point of view, all data sources are the same while they return data frames. The panel's feature request was opened to support Large PDF documents retrieved from PostgreSQL.

Large PDF document retrieved from the PostgreSQL database and displayed in Grafana.

To install PostgreSQL, we added a container with the latest version of the image postgres. We specified a volume for the folder /docker-entrypoint-initdb.d, which takes care of creating necessary tables and permissions when starting.

postgres:
  image: postgres
  restart: always
  environment:
    POSTGRES_PASSWORD: postgres
  ports:
    - 5432:5432
  volumes:
    - ./postgres:/docker-entrypoint-initdb.d

In our case, it was a table with bytea fields to store binary data with unique names. Images and PDF documents can be stored in base64 format or converted using the command encode() when retrieved from the database.

CREATE TABLE images (name text, img bytea, UNIQUE(name));

Load PDF into the database

To load PDF documents and a test image into the database, we created a Node.js script:

postgres/load.ts

loading...

Provisioning

Our Panel plugin template already has the Initial Provisioning configuration in place to add a new dashboard for PostgreSQL.

provisioning/dashboards/default.yaml

loading...

And we additionally added configuration to provision PostgreSQL data source with SSL disabled, specific username, password, and URL.

provisioning/datasources/postgres.yaml

loading...

You can learn more about the PostgreSQL data source in the official Documentation.

Load data

The last step is to start containers and load data, when starting, Grafana and PostgreSQL will be auto-provisioned and ready to go.

Overall, a 13Mb PDF document with 2990 pages takes 1-2 seconds to display. The user confirmed the feature request, and the plugin was sent to the Grafana team to review and update in the Grafana Catalog.

Take a look at the repository and let us know if you have any questions or future requests.