My Journey to the cloud…

This blogpost is in context of recently released update to AWS EC2 instance metadata service (IMDSv2) for improving security and adding an additional defence in depth layer. Recently encoutered a scenariio where customer would like to get the list of EC2 instances that can be safely upgraded to IMDSv2 – without any impact to the application or processes running on that server.

AWS has a useful CloudWatch metric for discovering all the instances that access metadata endpoint without any token. To view this, you need to go Metrics in CloudWatch service where you can query or graph with specific metrics. Under All metrics, select EC2 -> Per-Instance Metrics and from there select MetadataNoToken metric for the instances.

I used this CloudWatch metric to generate a script using Boto3 module to list all the instances which are making IMDSv1 call. This way, customer has the option to carefully filter the instances

1. which cab be easily and safely upgraded to IMDSV2
2. and which ones need further investigation (regd. the application/process making call to IMDSv1)

Script Name – getMetadataNoToken.py

import boto3
import pytz
import logging
from botocore.exceptions import ClientError
import csv
import os
import argparse
from datetime import datetime, timedelta

region_name = 'us-east-1'
DEFAULT_REGION = "us-east-1"
date_time_now = datetime.now().strftime('%Y/%m/%d  %H:%M:%S')

def parse_commandline_arguments():
    global REGION
    global ACCOUNT_ID
    global report_filename

    parser = argparse.ArgumentParser(formatter_class=argparse.ArgumentDefaultsHelpFormatter,
                                     description='Create a CSV Report for listing all EC2 making IMD call using IMDSv1.')
    parser.add_argument("-id", "--accountID", dest="account_id", type=str,required=True,
                        help="The AWS Account Name for which the EC2 info is neeeded")
    parser.add_argument("-r", "--region", dest="region", type=str,
                        default=DEFAULT_REGION, help="Specify the global region to pull the report")
    parser.add_argument("-f", "--report", dest="reportname", type=str,
                        help="Specify the report file Name with path")
    args = parser.parse_args()

    ACCOUNT_ID= args.account_id
    REGION = args.region
    report_filename = args.reportname

def cw_client(region):
    """
    Connects to CloudWatch, returns a connection object
    """
    try:
        session = boto3.Session(region_name=region)
        # Add profile name if it is configured for your env
        #session = boto3.Session(region_name=region, profile_name=profile_name)
        conn = session.client('cloudwatch')
        #conn = boto3.client('rds', region_name=region)

    except Exception as e:
        sys.stderr.write(
            'Could not connect to region: %s. Exception: %s\n' % (region, e))
        conn = None

    return conn

def getmetadanotoken(cloudwatch):
    try:
        # Set timezone to Central Time
        timezone = pytz.timezone('US/Central')

        session = boto3.Session(region_name=region_name)
        #ec2 = session.client('ec2')
        cloudwatch = session.client('cloudwatch')

        # Get current time in Central Time
        now = datetime.now(timezone)

        # Set start time to 3 hours ago
        start_time = now - timedelta(hours=3)

        # Convert start and end times to UTC
        start_time_utc = start_time.astimezone(pytz.utc).strftime('%Y-%m-%dT%H:%M:%SZ')
        end_time_utc = now.astimezone(pytz.utc).strftime('%Y-%m-%dT%H:%M:%SZ')
        # Query expression to get the Sum of MetadataNoToken group by Instance ID
        response = cloudwatch.get_metric_data (
            MetricDataQueries=[
                {
                    'Id': 'm1',
                    'Expression' : "SELECT SUM(MetadataNoToken) FROM SCHEMA(\"AWS/EC2\", InstanceId) GROUP BY InstanceId",
                    'Period' : 600
                }
            ],
            StartTime=start_time_utc,
            EndTime=end_time_utc,
        )

        
        # loop through the dictionary response and get the instance ID
        # if the sum of the metric is > 0
        for instances in response['MetricDataResults']:
            counter = 0
            for key,value in instances.items():
                if key == "Label":
                    instanceID = value
                if key == "Values":
                    listOftokenvalue = value
                    for x in range(len(listOftokenvalue)):
                        if listOftokenvalue[x] > 0:
                            counter += 1
                            
            # CloudWatch metric MetadataNoToken to track the number of calls to IMDSv1. 
            # The zero counts for this metric mean all your software is using IMDSv2
            if counter > 0:
                print(f"Instance ID: {instanceID} - is making IMDSv1 call")
                MakingIMDCall = "Yes"
            else:
                MakingIMDCall = "No"
            print_string = ACCOUNT_ID + "," + REGION + "," + instanceID + "," + MakingIMDCall + "," + date_time_now
            #print(print_string)
            file.write(print_string + "\n")
    except Exception as e:
        logging.error(e)


if __name__ == '__main__':
    try:
        parse_commandline_arguments()
        client = cw_client(REGION)
        if not os.path.isfile(report_filename):
            file = open(report_filename, 'w+')
            print_string_hdr = "AccountID,Region,InstanceID,MakingIMDCall,Reporting_Date_Time\n"
            file.write(print_string_hdr)
        else:
            file = open(report_filename, 'a')
        getmetadanotoken(client)
    except Exception as error:
        print(str(error))

How to run the script:

Make sure you have python3 installed and configured and is in path. Script takes takes 3 arguments

• AWS Account ID against which you need to get the list of EC2
• AWS Region
• & CSV File name with complete path

Usage: python <script_name> -id <AWS Account ID> -r <AWS Region> -f <CSV File name>

e.g. if the script is named as – getMetadataNoToken.py, you can run the script as

$ python getMetadataNoToken.py -id 123456789 -r us-east-1 -f /tmp/getEC2List.csv

You can run this python script against all of your AWS Accounts and all of the region by –

• Invoking this python script from within a shell script with a control file that has all the AWS Accounts and their corresponding regions. Shell script will loop through all the AWS accounts and region and hen generate a consolidated CSV file.

Hope this helps. Keep reading and happy learning !!!

-Anand M