Managing unique AWS credentials on a per project basis

Normally, the AWS CLI (or boto3 library if using python), will look for a file called .aws/credentials or .aws/config in your user's home directory. However, I'm often working on several different clients and therefore need to work with different sets of credentials.

One option is to use the --profile argument with the CLI but boto, requires coding logic to check for that. However, while this is the best solution, it's not always practical, especially when working with someone else's code.

The AWS CLI and BOTO use the environment variable AWS_SHARED_CREDENTIALS_FILE to define the file path containing the credentials, so modifying that will let you create a local set of credentials without needing to specify the --profile argument.

I most often use this alongside SAM projects which use Infrastructure as Code. These will deploy infrastructure and code, usually via lambdas. These projects will typically use a specific repository to hold the components, and so I will create a .aws/credentials file, along with the following bash script awsset in the root folder of the repository

1#! /bin/bash
2export AWS_SHARED_CREDENTIALS_FILE=./.aws/credentials
3
4awsunset() {
5  unset AWS_SHARED_CREDENTIALS_FILE
6}

With the script in place, I can run source awsset to enable the local credentialsß or use awsunset will revert to the standard credentials.

Using with virtualenv

Virtualenv lets you create isolated "projects" where you manage the libraries independently of other projects, even letting different projects use different versions of the same libraries. The libraries in a virtualenv project are installed in the ./lib/python2.7/site-packages directory, making it much simpler to identify the ones you need to package with your code.

Once you have installed virtualenv, you run the command virtualenv _project_ to create a new environment in the directory project. This creates an isolated environment where libraries etc., can be installed and code developed.

To integrate virtualenv and the AWS credential setup described above, I've created a script called virtualaws which combines creating the virtualenv environment with isolated AWS credentials.

1#! /bin/bash
2virtualenv $1
3mkdir $1/.aws
4cat << EOF >> $1/bin/activate
5export AWS_SHARED_CREDENTIALS_FILE=${VIRTUAL_ENV}/.aws/credentials'
6awsunset() {
7  unset AWS_SHARED_CREDENTIALS_FILE
8}
9EOF

Running virtualaws _project_ instead of virtualenv will create the environment including reference to the unique credentials file in _project_/.aws/credentials while running awsunset will revert to the default credentials.