Amazon CloudTrail contains logs from activities and audit events of an AWS account. Analyzing CloudTrail events can be essential to understanding what’s going on across your AWS accounts: unexpected changes, anomalies, and risks in the underlying computing infrastructure. Maybe EC2 instances were spawned with the wrong type or configuration, or the account have the maximum number of IPv4 adresses allocated, or someone is moving a production resource to another, less safe, security group?

CloudTrail logs are formatted as JSON and stored as files on S3.

This topic will walk you through forwarding CloudTrail logs to Unomaly’s ingestion API.


Instructions

This setup for forwarding CloudTrail logs to Unomaly uses two other AWS services, S3 and Lambda, for event-based processing:

  1. CloudTrail will write files into S3 at regular intervals, each containing an unknown number of CloudTrail records.

  2. An S3 event will trigger the lambda function.

  3. The lambda function will read the object containing the CloudTrail records from S3 and post them in a batch oriented fashion to the Unomaly ingestion API.

The script uses the AWS Serverless Application Model to define the lambda function and necessary AWS infrastructure. The SAM template contains specifications for the CloudTrail trail, S3 bucket and its corresponding policies, and the lambda function. When deploying, it will convert the SAM template to a CloudFormation template.

The code and templates are available on GitHub at https://github.com/unomaly/unomaly-blog/tree/master/cloudtrail-lambda-blog. Deployment instructions are in the README.md file.

The CloudTrail logs in Unomaly

When Unomaly receives the CloudTrail logs, you will see the different AWS services pop up as systems in the Unomaly UI. Read more about this in our blog post, Detecting Anomalies in AWS CloudTrail Using Unomaly.