Skip to main content
An Amazon Data Firehose stream delivers CloudWatch Logs to Squasher. Squasher reads the Firehose request format and the CloudWatch Logs records inside it.

​
Prerequisites

  • Permission in AWS to create a Firehose stream, an IAM role, and CloudWatch Logs subscription filters.
  • An ingest key and the project ID. See Create an ingest key.

​
Set up the drain

1

Create the Firehose stream

In the Amazon Data Firehose console, create a stream with the source Direct PUT and the destination HTTP Endpoint. Use these settings:Keep the CloudWatch Logs decompression option of the stream turned off. Squasher decompresses the records itself.
2

Let CloudWatch Logs write to the stream

Create an IAM role that logs.amazonaws.com can assume, with firehose:PutRecord and firehose:PutRecordBatch on the stream. See the AWS guide to subscription filters with Firehose.
3

Subscribe a log group

An empty filter pattern sends every line. Repeat for each log group.

​
What Squasher keeps from each record

CloudWatch lines have no level field. Squasher reads the level from the message: the Lambda ERROR and WARN columns, a JSON "level" field, level=error, a leading word such as [ERROR], or a stack trace such as Traceback (most recent call last):. Lines at error level or above create issues. Other lines are stored at info level. Every line gets the service name aws-cloudwatch, unless the line is JSON with a service field. CloudWatch lines have no environment. To set one, add it to the HTTP endpoint URL, for example https://ingest.squasher.ai/v1/firehose/aws/logs/<project-id>?environment=production. Automatic AI triage starts only for issues with an enabled environment. For stack traces and spans from Lambda or ECS code, use the Node.js SDK or OpenTelemetry in the function or container.

​
Verify

  1. Write one line to a subscribed log group.
  2. In the Firehose console, check that the stream delivers to the HTTP endpoint without errors.
  3. In Squasher, select Logs in the sidebar. The line appears with the service aws-cloudwatch.

​
Limits

  • One Firehose request can hold up to 10,000 records and 12 MiB, or 64 MiB after decompression. One record can be at most 1,024,000 bytes.
  • Squasher accepts each Firehose request ID once. A retry of an accepted request is acknowledged without storing the records again.
  • When the project is above its rate limit, Squasher answers 503, and Firehose retries.

​
Let Squasher create the stream

A provider connection gives you a CloudFormation launch link for a stack named squasher-logs. The stack creates the Firehose stream, the IAM roles, an S3 bucket for failed records, and the subscription filters for the log groups that you choose. To remove it, delete the stack in AWS. Repeat the setup in each Region that has logs.

​
Next steps

Provider connections

Get a CloudFormation launch link from Integrations or through the API.

OpenTelemetry

Send errors from Lambda functions and containers.