I wanted to share a workaround I came up with for a specific challenge in the AWS PowerShell tools. In a sense, this is an addendum to my Pluralsight course, since it was published before I came across this scenario. I’ll try to keep it brief since you’re probably only here to get the workaround so you can get back to work.
The problem
You need to assume a role. The AWS PowerShell tools have the ability to create a credential profile that assumes a role. It will even take care of renewing the temporary credentials when necessary. Great!
What’s potentially not as great is the fact that “assume role” type credential profiles require you to designate a source profile. In many cases that’s not an issue. But, if you are relying on an EC2 instance profile to authenticate your PowerShell commands, you may not have a profile.
That’s the exact scenario I came across.
The workaround
This is what you’re here for: the code!
# This is the URI for the folder one level above the credentials we need.
$MetadataUri = `
"http://169.254.169.254/latest/meta-data/iam/security-credentials"
# We need to get the contents of the folder to know the name of the subfolder.
# Technically there could be multiple but I haven't seen that happen. We will
# just use the first one.
$CredentialsList = ( `
Invoke-WebRequest -uri $MetadataUri `
).Content.Split()
# Get the credentials and turn the JSON text into an object.
$CredentialsObject = (Invoke-WebRequest `
-uri "$MetadataUri/$($CredentialsList[0])" `
).Content | ConvertFrom-Json
# Create/update a profile using the temporary access key and secret key
# we retrieved from the metadata.
Set-AWSCredential `
-StoreAs InstanceProfile `
-AccessKey $CredentialsObject.AccessKeyId `
-SecretKey $CredentialsObject.SecretAccessKey `
-SessionToken $CredentialsObject.Token
# Create/update the assume role profile.
Set-AWSCredential `
-StoreAs default `
-RoleArn $YourRoleArnHere `
-SourceProfile InstanceProfile
That’s all there is to it, but there are a few things to note about this code.
The temporary credentials found in the metadata do expire, so you’ll need to re-run this periodically to get the latest access key and secret key.
- In the code listed here, the assume role profile is named default. That means the assume role profile is the one that will be used unless you specify another one. You don’t have to do it this way, you can give it an arbitrary name and set it as the session profile using
Set-AWSCredentialor on a cmdlet-by-cmdlet basis using the-ProfileNameparameter. - You will need to specify your own value for the variable
$YourRoleArnHere.
This scenario is a non-issue if you’re using the AWS CLI. I found a feature request on GitHub from 2015. This eventually lead to a change in the botocore module, on top of which both the CLI and the Python SDK are built. The change allows users to create a profile with the type “EC2InstanceMetadata” which can then simply be used as the source profile for the assume role credential profile. This ability is not present in the AWS PowerShell tools.
I would love to see this functionality added to the AWS PowerShell tools eventually. In the meantime, I hope this workaround gets you up and running.
I have a SQL script that runs as job, it’s called by a batch file with the argument for the extension type to send to S3. I have been looking for something that will use the EC2’s role and this seems like the correct idea. If i hardcode my extension variable and test it running the powershell from ISE it properly writes the file to the bucket. This also works with using an IAM user set to the default AWS credential profile. But with the code below the SQL job returns the error “Could not find the file specified”. I’m guessing it is a permission issue somehow but not sure how to remediate
#
# SQL Server Backup to S3
#
# Get the AWS Stuff
Import-Module -Name AWSPowerShell
$YourRoleArnHere = “arn:aws:iam:::role/”
# This is the URI for the folder one level above the credentials we need.
$MetadataUri = `
“http://169.254.169.254/latest/meta-data/iam/security-credentials”
# We need to get the contents of the folder to know the name of the subfolder.
# Technically there could be multiple but I haven’t seen that happen. We will
# just use the first one.
$CredentialsList = ( `
Invoke-WebRequest -uri $MetadataUri `
).Content.Split()
# Get the credentials and turn the JSON text into an object.
$CredentialsObject = (Invoke-WebRequest `
-uri “$MetadataUri/$($CredentialsList[0])” `
).Content | ConvertFrom-Json
# Create/update a profile using the temporary access key and secret key
# we retrieved from the metadata.
Set-AWSCredential `
-StoreAs InstanceProfile `
-AccessKey $CredentialsObject.AccessKeyId `
-SecretKey $CredentialsObject.SecretAccessKey `
-SessionToken $CredentialsObject.Token
# Create/update the assume role profile.
Set-AWSCredential `
-StoreAs default `
-RoleArn $YourRoleArnHere `
-SourceProfile InstanceProfile
# Should be “*.bak” or “*.trn”
if ( $args[0] -eq “*.bak” -or $args[0] -eq “*.trn” ) { $backuptype = $args[0] } else { exit 1 }
# Get credentials from the persisted store
Initialize-AWSDefaults -ProfileName InstanceProfile -Region us-east-1
# Go to base backup location
#Set-Location
# Loop thru the files
Get-ChildItem $backuptype|Sort-Object -Property LastAccessTime| Select-Object -Last 1| Foreach-Object {
# Set the prefix for the S3 key
$keyPrefix = “sql-server-backups/” + $backuptype.Substring(2) +”/” + $_.name
# Copy the file out to Amazon S3 storage
Write-S3Object -BucketName -Key $keyPrefix -File $_.name -ServerSideEncryption AES256
Hi Jason,
A file not found error doesn’t sound like it’s related to the
Write-S3Objectcommand you’re issuing. There are only two other reasons I can think of why you’d get an error like that:1. The process running the job can’t find your script file. Is there anything happening that confirms that the SQL Job is actually running your script?
2. The
Write-S3Objectcommand isn’t able to find the file to back up. Try using$_.FullNameinstead of$_.Nameto ensure you’ve got the full, explicit path to the file.Also, the idea of this script is to set up the profiles to make it easy to work with in an interactive session. For a background job that’s running a script, you can simplify a lot of this. By default, the AWS PowerShell cmdlets will attempt to use the EC2 instance profile automatically, so if the permissions you need are part of the instance profile you don’t even have to think about credentials. If you need to use the instance profile to assume a separate role (I’m assuming
$YourRoleArnHereactually has a real ARN in it and you blanked it out before posting), you can do it this way:# Somewhere up top...
$Creds = (Use-STSRole -RoleArn $YourRoleArnHere -RoleSessionName "MyRoleSessionName").Credentials
.
.
.
# Inside the loop...
Write-S3Object -BucketName -Key $keyPrefix -File $_.name -ServerSideEncryption AES256 -Credential $Creds
(Reference: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-twp.html)
I totally might be misunderstanding your setup, so let me know if that all makes sense and if it helps.