Amazon CloudTrail contains logs from activities and audit events of an AWS account. CloudTrail logs are formatted as JSON and stored as files on S3.
Analyzing CloudTrail events can be essential to understanding what’s going on across your AWS accounts: unexpected changes, anomalies, and risks in the underlying computing infrastructure. Maybe EC2 instances were spawned with the wrong type or configuration, or the account have the maximum number of IPv4 addresses allocated, or someone is moving a production resource to another, less safe, security group?
This topic will walk you through forwarding CloudTrail logs to Unomaly’s ingestion API.
This setup for forwarding CloudTrail logs to Unomaly uses two other AWS services, S3 and Lambda, for event-based processing:
1. CloudTrail will write files into S3 at regular intervals, each containing an unknown number of CloudTrail records.
2. An S3 event will trigger the lambda function.
3. The lambda function will read the object containing the CloudTrail records from S3 and post them in a batch oriented fashion to the Unomaly ingestion API.
The script uses the AWS Serverless Application Model to define the lambda function and necessary AWS infrastructure. The SAM template contains specifications for the CloudTrail trail, S3 bucket and its corresponding policies, and the lambda function. When deploying, it will convert the SAM template to a CloudFormation template.
The code and templates are available on GitHub at: https://github.com/unomaly/unomaly-blog/tree/master/cloudtrail-lambda-blog. Deployment instructions are in the README.md file.
When Unomaly receives the CloudTrail logs, you will see the different AWS services pop up as systems in the Unomaly UI. Read more about this in our blog post, Detecting Anomalies in AWS CloudTrail Using Unomaly.