Logiq is now patented! This solidifies our position as a leader in the field of modern data engineering.   Know More

Monitoring AWS CloudTrail events using CloudWatch and LOGIQ

Monitoring AWS CloudTrail events using CloudWatch and LOGIQ

If your organization uses AWS services, you’d agree that it can be difficult for AWS administrators to stay on top of all activities and events that occur within your AWS account. If your organization heavily uses AWS services, there’d be a phenomenal amount of events and activities to track, thereby making governance, risk, and compliance audits more complicated than you’d imagine. Keeping this in mind, AWS developed AWS CloudTrail – an AWS service that helps you enable governance, risk, compliance, and operational auditing of your AWS account.

Using AWS CloudTrail logs, you can view and analyze actions that occur within your AWS account along with details of when the action happened, the user or process that kicked off the action, and the resources that were affected by it. You can also enable AWS CloudTrail Insights for trails you create to help you identify and respond to unusual account activity. 

AWS CloudTrail is enabled on your AWS account by default. All events occurring within your AWS account are recorded in CloudTrail events that you can view, search, and download. You can also forward CloudTrail events to Amazon CloudWatch Logs for further analysis. 

Forwarding AWS CloudTrails to CloudWatch

Amazon CloudWatch is an observability and monitoring service that provides actionable insights to monitor your applications, stay on top of performance changes, and optimize resource utilization while providing a centralized view of operational health. CloudWatch collects operational data of your AWS resources, applications, and services running on AWS and on-prem servers in the form of logs, metrics, and events. 

CloudWatch then uses this data to help detect and troubleshoot issues and errors in your environments, visualize logs and metrics, set up and take automated actions, and uncover insights that help keep your applications and deployments running smoothly. 

You can forward your CloudTrail events to CloudWatch, thereby enabling you to search through logs quickly, send alerts on events, and visualize logged events within CloudWatch. To forward CloudTrail events to CloudWatch, you’ll need:

  • A CloudWatch log group and a log stream
  • An IAM role and policy that grants CloudTrail access to CloudWatch and permissions to configure CloudWatch Logs integration

You can also configure CloudTrail forwarding to CloudWatch while creating new trials. To set up CloudTrail forwarding to CloudWatch Logs, first follow the instructions listed on the official AWS documentation site around configuring CloudWatch Logs monitoring with the AWS console

Once you’ve set up and configured the CloudTrail and CloudWatch Logs integration, do the following. 

Set up LOGIQ’s AWS CloudWatch Exporter Lambda function

LOGIQ’s AWS CloudWatch Exporter Lambda function acts as a trigger for a CloudWatch log stream and enables you to export CloudWatch logs to your LOGIQ instance. Whenever CloudWatch receives events from CloudTrail, it triggers the Lambda function to forward the CloudWatch logs to LOGIQ.

You can create the LOGIQ CloudWatch Exporter Lambda Function using the CloudFormation template available at https://logiqcf.s3.amazonaws.com/cloudwatch-exporter/cf.yaml

Note: Alternatively, you can also use the code available in our client integrations Bitbucket repository to create the Lambda function. 

This CloudFormation template creates a Lambda function along with the permissions it needs. Before using this template, you’ll need to configure the following attributes:

APPNAMEA readable application name for LOGIQ to partition logs by.
CLUSTERIDA Cluster ID for LOGIQ to partition logs by.
NAMESPACEA namespace for LOGIQ to partition logs by. 
LOGIQHOSTIP address or hostname of your LOGIQ instance.
INGESTTOKENJWT token to securely ingest logs into LOGIQ

Creating and configuring the CloudWatch trigger

Once you’ve created the AWS Lambda function, it’s time to create and configure the CloudWatch trigger. On your AWS dashboard, do the following:

  1. Navigate to the AWS Lambda function you just created (CloudtrailExporter).
  2. Click Add Trigger.
  3. On the Add Trigger page, select CloudWatch Logs.
  4. Next, select the Log group you’d like to stream to LOGIQ.
  5. Enter a Filter name and optionally add a Filter pattern.

And that’s it! Now all of your selected trails from AWS CloudTrail will be imported into LOGIQ as they are created.

From here, you can easily view, query, visualise and analyse your trails and CloudWatch logs while detecting anomalies in real-time thereby helping you enable governance, risk, compliance, and operational auditing of your AWS account from a single location, while keeping track of important events occurring within your AWS account.

You can also create custom dashboards like this to visualize events occurring across your AWS account.

Ajit Chelat

Ajit Chelat

The LOGIQ blog

Let’s keep this a friendly and inclusive space: A few ground rules: be respectful, stay on topic, and no spam, please.

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.

More insights. More affordable. Less hassle.

Follow Us on LinkedIn
and Twitter!

Before you go, make sure you don’t miss out on our latest updates and insights. Follow us on LinkedIn to stay up-to-date on industry trends, company news, and valuable insights.

Click the “Follow” button below to join our community and stay ahead of the curve. Thank you for visiting our site, and we hope to connect with you soon!