AWS CLI Tricks

When you start using AWS, the CLI becomes a very useful tool.

For the most part, the CLI is just making direct calls to the AWS API. However, there are tricks to help you become a true power user!

Configuration and Profiles (→)

We'll start with some fundamentals.

When you first setup the CLI tool, you're told to run aws configure, but what's this command do for you?

It saves configuration in two locations: ~/.aws/credentials and ~/.aws/config.

The credentials file stores your API access key and secret access key. The config file holds other configurations.

Furthermore, the CLI configuration has the notion of profiles. This lets you run commands under different users and settings:

1# Use the default profile
2aws ec2 describe-availability-zones
3 
4# Use a profile named "foo"
5aws --profile foo ec2 describe-availability-zones

Let's see these two profiles in action.

As stated, the credentials file let's you set the AWS key and secret key:

1[default]
2aws_access_key_id = XXX
3aws_secret_access_key = YYY
4
5[foo]
6aws_access_key_id = XXX
7aws_secret_access_key = YYY

The config file is a bit more interesting. For each profile, you can set some defaults:

1[default]
2output = json
3region = us-east-1
4
5[foo]
6output = text
7region = us-east-2

Here we see the default profile and the foo profile have slightly different settings, including a different default region.

Output Format (→)

You can change what format the CLI uses when returning responses. The format you choose can be personal preference, but some formats may serve you better when creating automations.

For example, returning json might be best for piping output to a tool like jq to do transformations or to reduce the result set to something simpler.

 1aws --output text ec2 describe-availability-zones
 2# Tab-separated text list
 3 
 4aws --output table ec2 describe-availability-zones
 5# ASCII table output
 6 
 7aws --output json ec2 describe-availability-zones | jq '.AvailabilityZones | map(.ZoneId)'
 8 
 9#[
10#  "use2-az1",
11#  "use2-az2",
12#  "use2-az3"
13#]

You might want to use text to do something like count lines of results, or use a type of table to see a nicely formatting table of results.

The formats you can use are:

• json
• yaml (and yaml-stream)
• text
• table

Output Pagination (→)

By default on Mac and Linux, output from commands is sent to less. This lets you paginate through larger result sets, but can sometimes be annoying.

You can adjust this (for example to use more instead of less). You can also stop that behavior altogther and have it output everything to your console.

To adjust this, you can set the AWS_PAGER environment variable:

1# Set the AWS_PAGER environment variable to an empty value
2AWS_PAGER="" aws ec2 describe-availability-zones

Or you can add this as a configuration option in ~/.aws/config:

1[profile foo]
2output = json
3region = us-east-2
4cli_pager =

You can add in any program name here that can read from stdin! For example, to have output sent to a new file in VSCode, use the following:

1# Use `code -` to have VS Code receive input from stdin
2AWS_PAGER="code -" aws ec2 describe-availability-zones

There are pagination-specific options as well, such as setting the results per page.

Query (→)

I use this feature the most.

All AWS CLI commands allow you to use the --query flag. This is similar to the jq command, but uses JMESPath. It can be used in conjunction with jq as well.

It's important to know that --query is not a filter (some commands have a --filter flag).

This --query flag doesn't work on the server-side, but instead on the client side. In other words, a filter takes the results return from the AWS API and will further query/filter/transform them on your machine.

This is especially nice where the jq tool may not be available (when outputting JSON), as it's built into the AWS CLI.

Here are two equivalent commands to demonstrate:

 1# Using jq
 2aws ec2 describe-availability-zones | jq '.AvailabilityZones | map(.ZoneId)'
 3 
 4# Using the --query flag
 5aws ec2 describe-availability-zones --query 'AvailabilityZones[].ZoneId'
 6 
 7#[
 8#  "use2-az1",
 9#  "use2-az2",
10#  "use2-az3"
11#]

You don't need to be using json output for this to work either! For example, this works just as well (outputting tab-separated text output):

1aws ec2 describe-availability-zones \
2    --output text \
3    --query 'AvailabilityZones[].ZoneId'

Auto Prompt (→)

This feature is pretty neat! Version 2 of the AWS CLI has an interactive auto-complete that you can use to help run commands.

You can enable it with the --cli-auto-prompt flag, the aws_cli_auto_prompt environment variable, or the cli_auto_prompt configuration.

Additionally, you can use the value on-partial to have it turn on "conditionally" - with partial/incomplete or invalid commands.

History (→)

The AWS CLI has it's own history feature. This isn't necessarily useful in the same way you might use the Linux history command. Instead, it seems very useful for debugging.

While there IS a --debug flag, it is very verbose. I find the AWS CLI history command to be a bit nicer. It shows you the flags used in a given command, as well as the HTTP API request and responses. This is can be helpful to see why something isn't working as you intended.

You can enable it with aws configure set cli_history enabled, or by editing ~/.aws/config:

1[profile foo]
2output = json
3region = us-east-2
4cli_history = enabled

Then you can run aws --profile foo history list and aws --profile foo history show to view history items per profile.

1# List history
2aws --profile foo history list
3 
4# Show a specific history item
5aws --profile foo history show 5f4d5be7-26bf-4ba1-b7ce-17449fafa9d0

Try it out!