Scheduling AWS Lambdas Using Boto3
AWS Lambda is a serverless compute service that lets you execute code (on the cloud) without having to maintain a server. Lambdas can be extremely powerful, they’re cheap, and they’re relatively easy to set up and maintain.
One of the things that would make AWS Lambdas even more powerful and capable, is the ability to schedule function calls/invocations programmatically and on the fly, using Python’s Boto3 library.
Boto3 is the Amazon Web Services (AWS) SDK for Python. It enables Python developers to create, configure, and manage AWS services
AWS Lambdas can be called/invoked using Boto3 rather easily, and you have the option of either making these function calls synchronously or asynchronously. Scheduling Lambdas however is a little more tricky, and it requires (at least using the approach I will outline in this article) the help of another service, called CloudWatch Events (or Amazon EventBridge).
CloudWatch Events (or Amazon EventBridge) allows you to create “Rules”; these rules can define an “Event Pattern” or a “Schedule”. In this article, we’re interested in the latter; that is, schedule-based rules.
Examples of schedule-based rules can be:
- Run/Trigger [something, such as a Lambda function] every 15 minutes
- Run/Trigger [something, such as a Lambda function] at 8:00am (UTC) every 1st day of the month
- Run/Trigger [something, such as a Lambda function] every 5 minutes Monday through Friday between 8:00am and 5:55pm (UTC)
These schedules can be defined either using a simple fixed-rate expression, or a Cron expression. Here are few examples of fixed-rate expressions:
rate(2 hours) This means run every 2 hours
rate(5 minutes) This means run every 5 minutes
rate(5 days) This means run every 5 daysCron expressions however can be a little more tricky; here are few examples of Cron expressions:
0 */5 * ? * * This means every 5 minutes
0 0 12 ? * SUN,SAT This means every Saturday and Sunday at noon
0 0 12 ? DEC * This means every day at noon in DecemberSo, Why Schedule Lambdas Using Boto3 Instead of Using the AWS Management Console?
Unless you need to, perhaps you shouldn’t. However, in many cases, it might be necessary to be able to dynamically (and quickly) schedule AWS Lambdas based on some external event/trigger. In addition to that, using Python’s Boto3 provides more flexibility and allows for a more dynamic architecture that perhaps otherwise wouldn’t have been possible by only using the AWS Management Console.
How Are “Rules” Relevant to Scheduling AWS Lambdas?
Well, it’s actually quite simple. If you’d like to schedule an AWS Lambda to run at a specific time or at a predefined period, we can do the following:
- Create the rule using ‘put_rule’ method
- Add the Lambda function as a target to the rule
- Add the right permissions to the Lambda function
Let’s start by creating a CloudWatch Event (or EventBridge) rule
To do that, we simply do the following:
# We start by creating an 'events' client
#
event_client = boto3.client('events',
aws_access_key_id="YOUR_ACCESS_KEY",
aws_secret_access_key="YOUR_SECRET_KEY")# Create a rule that's scheduled to run every 5 minutes
#
rslt = event_client.put_rule(Name='DEMO_EVENT',
ScheduleExpression='rate(5 minutes)',
State='ENABLED')# Note here that 'rate(5 minutes)' can be replaced with a
# Cron expression
If you run the code above, and if you check your CloudWatch Events (or EventBridge) console, you should see a new rule created with the name DEMO_EVENT:
If you click on DEMO_EVENT however, you will notice that we don’t have any “Targets”:
What Are “Targets” Anyway?
Targets are the AWS services that will trigger when the rule is executed based on the defined schedule. As of right now, we don’t have any targets; let’s add the existing Lambda function as a target for this rule.
To add a Lambda function as a target to this rule, we do the following:
# Notice here that we'll use the same 'events' client we
# created above
#
rslt = event_client.put_targets(Rule='DEMO_EVENT',
Targets=[
{
'Arn': 'LAMBDA_ARN',
'Id': 'RANDOM_ID',
}
])Keep in mind that a rule’s targets must be in the same region as the rule
Once we execute the code snippet above, we should see that the Lambda function has been added to the list of targets for that rule:
Now you might think that we’re done here, but there’s one very important step that we have to do to make sure that our Lambda function will run/trigger according to the attached rule.
Before we do the last step however, let me show you that we’re not actually done here. If you go to your Lambda function, you will notice that it still doesn’t have any triggers, which means that the rule is somehow not attached to the Lambda function:
To fix this problem, we need to add a permission to our Lambda function:
# Let's start by creating a 'lambda' client
#
lambda_clnt = boto3.client('lambda',
aws_access_key_id="YOUR_ACCESS_KEY",
aws_secret_access_key="YOUR_SECRET_KEY")# Now, we add the permission
#
rslt = lambda_clnt.add_permission(FunctionName='LAMBDA_ARN',
StatementId='RANDOM_ID',
Action='lambda:InvokeFunction',
Principal='events.amazonaws.com',
SourceArn='RULE_ARN')
If we try to view our Lambda function again after executing the code above, we notice that a trigger has been added:
That’s it! Your rule is now ready to trigger your Lambda function based on the specified period (5 minutes in our case).
Also, keep in mind that you can also delete rules or remove targets from rules using Boto3.