Prometheus metrics

Squasher accepts metrics over OTLP, not the Prometheus protocols. Run an OpenTelemetry Collector that scrapes your targets, or that receives remote write, and forwards the metrics to Squasher. On a Linux host, the Squasher Collector can also scrape a list of targets.

Prerequisites

An OpenTelemetry Collector Contrib build (otelcol-contrib) that can reach your targets.
An ingest key in the Collector’s environment as SQUASHER_API_KEY. See Create an ingest key.

Scrape your targets

Use the Prometheus receiver when the Collector can reach each target’s metrics endpoint. It takes standard Prometheus scrape configuration, service discovery, and relabeling.

otel-collector-config.yaml

receivers:
  prometheus:
    config:
      scrape_configs:
        - job_name: checkout-api
          scrape_interval: 30s
          static_configs:
            - targets: ["checkout-api:9464"]

processors:
  resource:
    attributes:
      - { key: deployment.environment.name, value: production, action: upsert }
  batch: {}

exporters:
  otlphttp/squasher:
    endpoint: https://ingest.squasher.ai
    headers:
      x-squasher-key: "${env:SQUASHER_API_KEY}"

service:
  pipelines:
    metrics:
      receivers: [prometheus]
      processors: [resource, batch]
      exporters: [otlphttp/squasher]

The receiver sets service.name from the scrape job. Use metric_relabel_configs to drop series and labels that you do not query before you add more jobs.

Scrape with the Squasher Collector

The Squasher Collector can also scrape Prometheus endpoints on the same host or on its private network. It scrapes only the targets that you list. It opens no port: every scrape is an outbound request. It has no service discovery and no relabeling. Add a prometheus receiver to the Collector configuration, /etc/squasher-collector/config.yaml. Then add it to the receivers of the metrics pipeline, and keep the receivers that are there:

config.yaml

receivers:
  prometheus:
    scrape_interval: 60s # whole seconds, 10s or more
    timeout: 10s # not longer than scrape_interval
    max_body_size: 10MiB # a larger response fails the scrape
    sample_limit: 50000 # more samples fail the scrape
    max_concurrent_scrapes: 4 # scrapes that run at the same time
    max_in_flight_bytes: 64MiB # all response bodies in memory together
    label_limit: 64 # a sample with more labels is skipped
    label_name_length_limit: 128 # bytes; a longer label name skips the sample
    label_value_length_limit: 1024 # bytes; a longer label value skips the sample
    max_series_per_target: 10000 # samples of more series are dropped
    targets: # 1 to 256 targets
      - job: node
        url: http://127.0.0.1:9100/metrics
      - job: checkout-api # becomes service.name
        url: http://10.0.0.12:9464/metrics
        timeout: 5s
        labels:
          team: payments # added to every data point
      - job: secured-app
        url: https://10.0.0.20:8443/metrics
        bearer_token_file: /etc/squasher-collector/secured-app.token # read at each scrape

service:
  pipelines:
    metrics:
      receivers: [host_metrics, prometheus]

Check the file, then restart the Collector:

sudo /usr/local/bin/squasher-collector validate --config=file:/etc/squasher-collector/config.yaml
sudo systemctl restart squasher-collector

Each target becomes one resource with service.name from job, and service.instance.id as host:port. The receiver values in the example are the defaults. The Collector reads the Prometheus text format and the OpenMetrics text format.

Prometheus type	Squasher
Counter	Sum
Gauge	Gauge
Histogram	Histogram
Summary	A gauge with a `quantile` attribute, plus `<name>_sum` and `<name>_count` sums

Each scrape also sends these metrics for the target, also when the scrape fails:

Metric	Value
`squasher.collector.scrape.up`	`1`, or `0` when the scrape fails. A failed point has `error.type`, for example `timeout`, `connection_refused`, `body_too_large`, `sample_limit`, or `scrape_budget`
`squasher.collector.scrape.duration`	The scrape time in seconds
`squasher.collector.scrape.samples`	The number of samples
`squasher.collector.scrape.parse_errors`	The number of lines that the Collector skipped, including samples over a label limit
`squasher.collector.scrape.dropped_series`	The number of samples dropped because the target has more than `max_series_per_target` series

A scrape fails when the response is larger than max_body_size, or has more samples than sample_limit. It also fails with scrape_budget when the response bodies of the scrapes that run at the same time would use more than max_in_flight_bytes. Other targets wait until fewer than max_concurrent_scrapes scrapes run. To use service discovery or relabeling, use the OpenTelemetry Collector above.

Receive remote write

If a Prometheus server must push its data, use the Prometheus Remote Write receiver from Collector Contrib. It is alpha and accepts Remote Write 2.0 only.

otel-collector-config.yaml

receivers:
  prometheusremotewrite:
    endpoint: 0.0.0.0:9090

processors:
  batch: {}

exporters:
  otlphttp/squasher:
    endpoint: https://ingest.squasher.ai
    headers:
      x-squasher-key: "${env:SQUASHER_API_KEY}"

service:
  pipelines:
    metrics:
      receivers: [prometheusremotewrite]
      processors: [batch]
      exporters: [otlphttp/squasher]

Then point Prometheus at the receiver with the Remote Write 2.0 message:

prometheus.yml

remote_write:
  - url: http://otel-collector:9090/api/v1/write
    protobuf_message: io.prometheus.write.v2.Request

Collector Contrib v0.141.0 and earlier works with Prometheus 3.7.x and earlier. v0.142.0 and later works with Prometheus 3.8.0 and later. Check the receiver’s compatibility table before you upgrade either one.

What Squasher keeps

Prometheus type	Squasher
Counter	Sum
Gauge	Gauge
Histogram	Histogram
Summary	Dropped. Squasher does not store summary metrics.
Labels	Attributes on each point

Verify

Start the Collector with SQUASHER_API_KEY set, and check its logs for export failures.
In Squasher, select Logs in the sidebar and choose Metrics in the source menu.
Find a known metric from checkout-api, and check its service, unit, and labels.

Limits

Squasher does not store summaries. Change them to histograms, or leave them out. The Squasher Collector changes them to gauges and sums for you.
The Remote Write receiver does not accept summaries or classic histograms.
The Prometheus Alertmanager connector receives alerts, not metric series.

Get started

Guides

Prometheus metrics

Prerequisites

Scrape your targets

Scrape with the Squasher Collector

Receive remote write

What Squasher keeps

Verify

Limits

Next steps

OpenTelemetry

Kubernetes

​Prerequisites

​Scrape your targets

​Scrape with the Squasher Collector

​Receive remote write

​What Squasher keeps

​Verify

​Limits

​Next steps

OpenTelemetry

Kubernetes

Prerequisites

Scrape your targets

Scrape with the Squasher Collector

Receive remote write

What Squasher keeps

Verify

Limits

Next steps