JarCasting
JarCasting JarCasting

storm-metrics-reporter-prometheus

Storm metrics reporter module that supports Prometheus Gateway

License

License
Categories

Categories

Metrics Application Testing & Monitoring Monitoring Prometheus ORM Data
GroupId

GroupId

com.wizenoze
ArtifactId

ArtifactId

storm-metrics-reporter-prometheus
Last Version

Last Version

0.0.1
Release Date

Release Date
Type

Type

jar
Description

Description

storm-metrics-reporter-prometheus
Storm metrics reporter module that supports Prometheus Gateway
Project URL

Project URL

https://github.com/wizenoze/storm-metrics-reporter-prometheus
Project Organization

Project Organization

WizeNoze BV
Source Code Management

Source Code Management

https://github.com/wizenoze/storm-metrics-reporter-prometheus

Download storm-metrics-reporter-prometheus

Filename Size
storm-metrics-reporter-prometheus-0.0.1.pom
storm-metrics-reporter-prometheus-0.0.1.jar 84 KB
storm-metrics-reporter-prometheus-0.0.1-sources.jar 9 KB
storm-metrics-reporter-prometheus-0.0.1-javadoc.jar 52 KB
Browse

How to add to project

<!-- https://jarcasting.com/artifacts/com.wizenoze/storm-metrics-reporter-prometheus/ -->
<dependency>
    <groupId>com.wizenoze</groupId>
    <artifactId>storm-metrics-reporter-prometheus</artifactId>
    <version>0.0.1</version>
</dependency>
// https://jarcasting.com/artifacts/com.wizenoze/storm-metrics-reporter-prometheus/
implementation 'com.wizenoze:storm-metrics-reporter-prometheus:0.0.1'
// https://jarcasting.com/artifacts/com.wizenoze/storm-metrics-reporter-prometheus/
implementation ("com.wizenoze:storm-metrics-reporter-prometheus:0.0.1")
'com.wizenoze:storm-metrics-reporter-prometheus:jar:0.0.1'
<dependency org="com.wizenoze" name="storm-metrics-reporter-prometheus" rev="0.0.1">
  <artifact name="storm-metrics-reporter-prometheus" type="jar" />
</dependency>
@Grapes(
@Grab(group='com.wizenoze', module='storm-metrics-reporter-prometheus', version='0.0.1')
)
libraryDependencies += "com.wizenoze" % "storm-metrics-reporter-prometheus" % "0.0.1"
[com.wizenoze/storm-metrics-reporter-prometheus "0.0.1"]

Dependencies

provided (1)

Group / Artifact Type Version
org.apache.storm : storm-core jar 1.2.2

test (4)

Group / Artifact Type Version
org.junit.jupiter : junit-jupiter-api jar 5.1.0
org.junit.jupiter : junit-jupiter-engine jar 5.1.0
org.mockito : mockito-junit-jupiter jar 2.23.4
org.hamcrest : hamcrest-library jar 1.3

Project Modules

There are no modules declared in this project.

storm-metrics-reporter-prometheus

Storm metrics reporter module that supports Prometheus Push Gateway

Motivation

Apache Storm supports the following metrics reporters at the time of writing.

  • Console Reporter (org.apache.storm.metrics2.reporters.ConsoleStormReporter): Reports metrics to System.out.
  • CSV Reporter (org.apache.storm.metrics2.reporters.CsvReporter): Reports metrics to a CSV file.
  • Ganglia Reporter (org.apache.storm.metrics2.reporters.GagliaStormReporter): Reports metrics to a Ganglia server.
  • Graphite Reporter (org.apache.storm.metrics2.reporters.GraphiteStormReporter): Reports metrics to a Graphite server.
  • JMX Reporter (org.apache.storm.metrics2.reporters.JmxStormReporter): Exposes metrics via JMX.

The closest which could be used to push data to Prometheus is org.apache.storm.metrics2.reporters.JmxStormReporter. That could be put together with Prometheus JMX exporter in theory, there's one pitfall thought.

Prometheus provides a Java agent which spins up a lightweight HTTP server. That doesn't fit well with Storm's architecture, because the supervisor might create multiple worker processes on a single node and those workers would try to open the same HTTP port.

Having looked into org.apache.storm.metrics2.reporters.GraphiteStormReporter, it was pretty close what we actually need, but with Prometheus.

Installation

Download storm-metrics-reporter-prometheus-0.0.1-SNAPSHOT.jar from here and put into underneath {STORM_DIR}/extlib and/or ${STORM_DIR}/extlib-daemon depending upon the metrics of which process(es) you want to send to Prometheus.

Add enable Prometheus Metrics Reporter in storm.yaml.

storm.metrics.reporters:
  # Prometheus Reporter
  - class: "com.wizenoze.storm.metrics2.reporters.PrometheusStormReporter"
    daemons:
        - "supervisor"
        - "nimbus"
        - "worker"
    report.period: 60
    report.period.units: "SECONDS"
    filter:
      class: "org.apache.storm.metrics2.filters.RegexFilter"
      expression: "storm\\.worker\\..+\\..+\\..+\\.(?:.+\\.)?-?[\\d]+\\.\\d+-(emitted|acked|disruptor-executor.+-queue-(?:percent-full|overflow))"

    prometheus.scheme: "http"
    prometheus.host: "localhost"
    prometheus.port: 9091

Point prometheus.host and prometheus.port to your Prometheus Push Gateway. You may adjust report.period and report.period.units to make it aligned with Prometheus' scrape interval, as well as the filter expression according to your needs.

com.wizenoze

Wizenoze

Versions

Version
0.0.1