Sky Cone

-

With AWS Lambda, web content adminitrators could shorten the waiting time for response. In my previous tuning experience, the duration of each Lambda function execution time has been reduced obviously.



As an illustration, we will host the application with Django framework.


Environment Description

The runtime of the application is Python 3.8. The application web framework uses Django version 3.0.5.

To use Lambda serve web traffic, we could use the Zappa framework to package the Django application, deploy to AWS Serverless products, e.g. API Gateway and Lamdba.

However, using the default configuration of Zappa, the deployment package will be large.

Let's see if we could reduce the Lambda execution duration by slimming the deployment package.

Because when Zappa pack the code, I want to know what the additional code is added and the dependencies packages that it packs into the whole package.

By default, every dependency package will be included in the deployment package for Lambda. This will make the package very large. In this illustration, the package size will be 30 MB approximately. To have a straightforward view of the Python code, the package should be not larger than 3 MB to edit code inline in the Lambda console.

First we move the explicit dependencies to Lambda Layer.


Create a Lambda Layer

After we have visibility of the code in Lambda code inline editor, we are able to figure out the minimum of the dependency codes one step further.

After that, the dependency packages could be migrated from the original package into Lambda Layers.

With Lambda layer, the dependencies and the code of the application could be de-coupled. The Lambda layer could also be version controlled separete from the Django application.

After moving the dependency packages that is injected by Zappa but deemed as irrelevant for the application, the layer is reduced from 29.6 MB to 10.9 MB.


In the Zappa setting, we remove all the dependencies and only let the Zappa package the Django application itself.

After deployed the package to Lambda, we found some additional packages are needed, which is not specified by the requirements.txt file. Below is what the requirements.txt file looks like at this stage:

Django==3.0.5

To fix the dependency issue, I added below dependencies into the code, in addtion to the required packages that are explicitly imported in the code.

The requirements.txt file may similar to below content:

asgiref==3.2.7
pytz==2019.3
jsonpickle==1.4.1
wrapt==1.12.1
sqlparse==0.3.1
future==0.18.2
Werkzeug==0.16.1
wsgi-request-logger==0.4.6
durationpy==0.5
zappa==0.51.0

With these ineviatable packaged added, the Layer size grew back a little bit, from 11 MB to 13 MB. I added the "--no-deps" parameter in the CLI, so that I could clearly know which packages are necessary and which ones aren't.
# python3.8 -m pip install --target /root/python -r requirements.txt --ignore-installed --no-deps

Together with packaging the dependencies, the Python code is also pre-compiled to further improve the Lambda execution efficiency. Consider below CLI command:
# python3.8 -m compileall ./python/

Packege the code with zip, and use -x to remove certain folders and files that we believe unnecessary. This will further reduce the size of the Lambda Layer size. Tailor below command according to your requirements.
# zip -r python_libs_slimmed.zip ./python -x '***'

Finally, deploy the package to Lambda.
# aws lambda publish-layer-version --layer-name [Lambda-Layer-Name] --description [Lambda-Layer-Description] --compatible-runtimes python3.8 --zip-file fileb://python_libs_slimmed.zip --region us-west-2


Deploy code to Lambda function

# python3.8 -m pip install virtualenv

# python3.8 -m pip install zappa

# virtualenv .env

# source .env/bin/activate

(.env) # git init .

Tailor below command to meet your needs.
(.env) # git remote add origin git@github.com:[GitHub-Account-Name]/[Git-Repository-Name].git

(.env) # git pull origin master

Compile files using the compileall module.
(.env) # python3.8 -m compileall .

Modify the Zappa settings file. Specify the files that could be excluded from the Lambda deployment package. Specify the Lambda Layer name and version.

{
    "serverless": {
        ...
        "exclude": [
                "*.gz",
                "*.rar",
                ...
        ],
        "layers": [
                ...
                "arn:aws:lambda:us-west-2:123456789012:layer:[Lambda-Layer-Name]:[Lambda-Version-Number]"
        ],
        ...
    }
}

The deploy package size has been reduced to 87 KB, from 34.9 MB, which is the size of the deployment package generated by Zappa default mode.


Deploy the Serverless application using below command.
(.env) # zappa update serverless

With a few days monitoring, it is observed that the Lambda execution duration has been reduced obviously.



Related Documentation

Best Practices for Working with AWS Lambda Functions


Related Products

AWS Lambda

Zappa