If your organization uses AWS services, you’d agree that it can be difficult for AWS administrators to stay on top of all activities and events that occur within your AWS account. If your organization heavily uses AWS services, there’d be a phenomenal amount of events and activities to track, thereby making governance, risk, and compliance audits more complicated than you’d imagine. Keeping this in mind, AWS developed AWS CloudTrail – an AWS service that helps you enable governance, risk, compliance, and operational auditing of your AWS account.
Using AWS CloudTrail logs, you can view and analyze actions that occur within your AWS account along with details of when the action happened, the user or process that kicked off the action, and the resources that were affected by it. You can also enable AWS CloudTrail Insights for trails you create to help you identify and respond to unusual account activity.
AWS CloudTrail is enabled on your AWS account by default. All events occurring within your AWS account are recorded in CloudTrail events that you can view, search, and download. You can also forward CloudTrail events to Amazon CloudWatch Logs for further analysis.
Forwarding AWS CloudTrails to CloudWatch
Amazon CloudWatch is an observability and monitoring service that provides actionable insights to monitor your applications, stay on top of performance changes, and optimize resource utilization while providing a centralized view of operational health. CloudWatch collects operational data of your AWS resources, applications, and services running on AWS and on-prem servers in the form of logs, metrics, and events.
CloudWatch then uses this data to help detect and troubleshoot issues and errors in your environments, visualize logs and metrics, set up and take automated actions, and uncover insights that help keep your applications and deployments running smoothly.
You can forward your CloudTrail events to CloudWatch, thereby enabling you to search through logs quickly, send alerts on events, and visualize logged events within CloudWatch. To forward CloudTrail events to CloudWatch, you’ll need:
- A CloudWatch log group and a log stream
- An IAM role and policy that grants CloudTrail access to CloudWatch and permissions to configure CloudWatch Logs integration
You can also configure CloudTrail forwarding to CloudWatch while creating new trials. To set up CloudTrail forwarding to CloudWatch Logs, first follow the instructions listed on the official AWS documentation site around configuring CloudWatch Logs monitoring with the AWS console.
Once you’ve set up and configured the CloudTrail and CloudWatch Logs integration, do the following.
Set up LOGIQ’s AWS CloudWatch Exporter Lambda function
LOGIQ’s AWS CloudWatch Exporter Lambda function acts as a trigger for a CloudWatch log stream and enables you to export CloudWatch logs to your LOGIQ instance. Whenever CloudWatch receives events from CloudTrail, it triggers the Lambda function to forward the CloudWatch logs to LOGIQ.
You can create the LOGIQ CloudWatch Exporter Lambda Function using the CloudFormation template available at https://logiqcf.s3.amazonaws.com/cloudwatch-exporter/cf.yaml.
Note: Alternatively, you can also use the code available in our client integrations Bitbucket repository to create the Lambda function.
This CloudFormation template creates a Lambda function along with the permissions it needs. Before using this template, you’ll need to configure the following attributes:
| Parameter | Description |
APPNAME | A readable application name for LOGIQ to partition logs by. |
CLUSTERID | A Cluster ID for LOGIQ to partition logs by. |
NAMESPACE | A namespace for LOGIQ to partition logs by. |
LOGIQHOST | IP address or hostname of your LOGIQ instance. |
INGESTTOKEN | JWT token to securely ingest logs into LOGIQ |
Creating and configuring the CloudWatch trigger
Once you’ve created the AWS Lambda function, it’s time to create and configure the CloudWatch trigger. On your AWS dashboard, do the following:
- Navigate to the AWS Lambda function you just created (CloudtrailExporter).
- Click Add Trigger.
- On the Add Trigger page, select CloudWatch Logs.
- Next, select the Log group you’d like to stream to LOGIQ.
- Enter a Filter name and optionally add a Filter pattern.
And that’s it! Now all of your selected trails from AWS CloudTrail will be imported into LOGIQ as they are created.
From here, you can easily view, query, visualise and analyse your trails and CloudWatch logs while detecting anomalies in real-time thereby helping you enable governance, risk, compliance, and operational auditing of your AWS account from a single location, while keeping track of important events occurring within your AWS account.
You can also create custom dashboards like this to visualize events occurring across your AWS account.
