Tech Thoughts

A cloud, PowerShell, server, hardware, technology, and more tech thoughts blog

Manage Cloud with PowerShell

wp-content/uploads/2019/01/learn_powershell-300x206.jpg
Learn PowerShell

Cloud technology has experienced explosive growth over the last several years. Cloud solutions provide rapid provisioning, cost efficiency, enhanced security, and performance. As such, many companies are looking to incorporate cloud offerings into their technology stack. Even if you find yourself managing a more traditional on-premise stack today, you need to be looking to sharpen your cloud skills to remain relevant in the future.

Companies are typically not going “all-in” on cloud. As companies look to take advantage of cloud offerings they do so as new project efforts come up, or they transition low-barrier components. The reality is that most companies operate in a Hybrid Cloud model, with some infrastructure on-site in traditional data centers, and other components in public cloud offerings.

Various teams within your organization may also elect to engage different solutions by the public cloud providers. So, you may find yourself in a multi-cloud scenario where certain components are in AWS, while others are in Azure.

This is where a PowerShell skill-set can really help you shine! PowerShell has the capability to make you successful in all of these scenarios! In this episode of Learn PowerShell we’ll show how you can mange cloud with PowerShell!

PowerShell In The Cloud

Eventually you will be called upon to help manage and solve problems with various cloud-based solutions. Some of those resources will be of a more traditional nature, such as Infrastructure as a Service (IaaS). Others may be more modern solutions such as Software as a Service (SaaS), or Platform as a Service (Paas).

  • SaaS examples:
    • Microsoft Office 365
    • Google Apps (Gmail, Google Docs)
  • PaaS examples:
    • AWS Elastic Beanstalk
    • Azure App Service
    • Google App Engine
  • IaaS examples:
    • AWS EC2
    • Azure Virtual Machine
    • Google Compute Engine (GCE)

Regardless of type, many of these still have traditional authentication, accessibility, inventory, monitoring, and cost challenges to solve. Your investment in learning PowerShell has paid off in a big way, as PowerShell is available to help you manage cloud solutions.

Getting Practice with PowerShell and Cloud

Like anything technology based – it will help to practice some of these tasks in the real world. To do that, you’ll need a cloud account. (Ideally more than one – don’t just go “all in” on one cloud!). It does take a credit card and a valid email address to get an account setup. Even though you’ve provided your credit card each major cloud provider has a free tier that will enable to practice for no cost, or very low cost.

Cloud providers free tier information:

Don’t let cost be your excuse for not diving into cloud learning. Part of your job as a technologist in the modern landscape is to have a basic understanding of cloud costs and billing. You should be comfortable navigating the free tier to learn cloud basics without racking up any type of bill. If you do make a mistake and rack up a $15 bill, that’s money you spent investing in yourself!

Video

If you prefer video format over written documentation, I discuss this topic in the following TechThoughts video:

AWS Tools for PowerShell

Amazon Web Services (AWS) is the largest cloud provider and offers hundreds of cloud based services. Your ability to manage AWS resources using PowerShell will be a great addition to your resume.

What you’ll need to get started:

The AWS Tools for PowerShell let developers and administrators manage their AWS services and resources using PowerShell! AWS Tools is essentially just a module, or collection of modules that enables your existing PowerShell install to mange AWS Cloud! You can find and install the needed module(s) from the PowerShell Gallery using basic commands like Find-Module and Install-Module.

The AWS Tools for PowerShell has evolved over time so lets dive quickly into the differences.

  • AWS Tools for PowerShell – Modular (recommended)
    • AWS.Tools.* – this is the primary modular module, and the one that will receive support moving forward. There is only one “core” small module (AWS.Tools.Common) which contains a few basic AWS commands and allows you to authenticate to your AWS account. Every service offered by AWS is then broken up into a separate module. For example EC2 (Virtual Machine/Compute) cmdlets are found in the AWS.Tools.EC2 module. This version of the module supports Windows PowerShell 5.1+ and PowerShell Core 6+ on Windows, Linux and macOS.
  • AWS Tools for PowerShell
    • AWSPowerShell.NetCore – This is a large monolithic module that contains all the PowerShell cmdlets for AWS in one module. It is quite large and will eventually be deprecated as AWS continues to grow. It supports PowerShell 6+.
  • AWS Tools For PowerShell (Legacy)
    • AWSPowerShell – This is a large monolithic module that contains all the PowerShell cmdlets for AWS in one module. It is quite large and is in the process of being deprecated. It was created to support PowerShell 5.1.

Installing AWS Tools for PowerShell

Because the AWS Tools for PowerShell (modular) is the future, this blog post will focus on this version of the module.

Get started by running the following install command:

# Install AWS.Tools - A Modularized Version of AWS Tools for PowerShell 
Install-Module -Name AWS.Tools.Installer

The AWS.Tools.Installer module provides cmdlets that enable you to install, update, and remove the modules for each of the AWS services. It also automatically install the AWS.Tools.Common module which provides cmdlets for configuration and authentication that are not service specific.

Use the AWS.Tools.Common module not only to authenticate to your AWS account, but also discover what modules are required for each AWS Service. Try running the following:

# Get a list of AWS services supported by the Tools for PowerShell
Import-Module AWS.Tools.Common
Get-AWSService | Select-Object Service,ServiceName,ModuleName | Format-Table -AutoSize

There are a lot of AWS Services, and there is a module to support each one!

AWS operates with the concept of regions. You will likely want to deploy resources to the AWS region that is closest to you. Get a list of available AWS regions by running the following:

# Discover available AWS regions
Get-AWSRegion

Keeping AWS.Tools in Sync

Quick note regarding the modular version of AWS.Tools. As you continue to engage various AWS services using PowerShell you will likely install additional service modules from time to time. As a result, you may find yourself in a state where your module versions are out of sync. For instance, your AWS.Tools.S3 module might be on version 4.0.4.0 and your AWS.Tool.EC2 might be on version 4.0.6.0. This can be easily fixed by simply running Update-AWSToolsModule.

# Keep AWS.Tools versions in sync
Update-AWSToolsModule

Authenticating to AWS using PowerShell

There are several methods for specifying your AWS credentials in PowerShell. In this post I’ll demo one way, which is providing the Access Key and Secret key of a user in your AWS account. If you haven’t done so, go ahead and create a user in your AWS account.

Once you have a user created, navigate to Services ➝ IAM ➝ Users ➝ (Your User) ➝ Security Credentials Tab. From there, you can click the create access key button. Don’t forget to copy your credentials!

AWS Create Access Key

Armed with your user’s access key and secret key, you can now establish a secure connection to your AWS account in your PowerShell session. We will also set the default region that we will be creating and managing resources in.

#___________________________________
# Import the AWS Tools Common module
Import-Module AWS.Tools.Common
# set the credential for the session
Set-AWSCredential -AccessKey $aKey -SecretKey $sKey
# set the default region for the session
Set-DefaultAWSRegion -Region us-west-2
#___________________________________

Managing AWS Cloud with PowerShell

With your PowerShell session authenticated to your AWS account you might be thinking, now what? The answer is the possibilities are nearly limitless. Using PowerShell the entire power of the AWS cloud is at your fingertips.

  • Create a virtual server and establish access
  • Create a fleet of 500 servers pre-configured to your needed specification
  • Retrieve billing information
  • Create monitors and alarms
  • Create serverless solutions

AWS PowerShell Final Example

In this final example we’ll use PowerShell to create a highly secure storage bucket in AWS (S3). Then we’ll upload a file to the bucket, and create a secure pre-signed URL to the file. The URL will be good for 24 hours (configurable). This is a great way to share a file securely worldwide leveraging the power of the cloud.

# AWS FINAL EXAMPLE
# 1 - Create an S3 bucket
# 2 - Make the S3 bucket secure - NO PUBLIC ACCESS
# 3 - Upload a file to the new S3 bucket
# 4 - Create a pre-signed URL to enable people to securely download the file worldwide

# Install the AWS.Tools S3 module to work with Amazon Simple Storage Service (S3)
Install-Module AWS.Tools.S3
Import-Module AWS.Tools.S3

# 1 - This command creates a new private bucket named "techthoughts".
# https://docs.aws.amazon.com/powershell/latest/reference/items/New-S3Bucket.html
New-S3Bucket -BucketName 'techthoughts'
Get-S3PublicAccessBlock -BucketName 'techthoughts' # by default the bucket will not have public access blocked

# 2 - Adjust public access to the bucket to BLOCKED
# https://docs.aws.amazon.com/powershell/latest/reference/items/Add-S3PublicAccessBlock.html
$publicAccessBlockSplat = @{
    PublicAccessBlockConfiguration_BlockPublicAcl       = $true
    PublicAccessBlockConfiguration_IgnorePublicAcl      = $true
    PublicAccessBlockConfiguration_BlockPublicPolicy    = $true
    PublicAccessBlockConfiguration_RestrictPublicBucket = $true
    BucketName                                          = 'techthoughts'
}
Add-S3PublicAccessBlock @publicAccessBlockSplat
Get-S3PublicAccessBlock -BucketName 'techthoughts' # verify the new BLOCKED policy is in place

# 3 - Upload a local file on your drive to the new S3 bucket
$filePath = 'C:\rs-pkgs\techthoughts_text_file.txt'
$key = 'techthoughts_text_file.txt'
$bucketName = 'techthoughts'
# https://docs.aws.amazon.com/powershell/latest/reference/index.html?page=Write-S3Object.html&tocid=Write-S3Object
# http://iwantmyreal.name/s3-download-only-presigned-upload
$writeS3Splat = @{
    BucketName       = $bucketName
    File             = $filePath
    Key              = $key
    HeaderCollection = @{
        'Content-Disposition' = "attachment; filename=""$key"""
    }
}
Write-S3Object @writeS3Splat

# 4 - Create a pre-signed URL to securely allow others to download the file for a set period of time
# https://docs.aws.amazon.com/powershell/latest/reference/items/Get-S3PreSignedURL.html
# This URL will be good for 1 day - 24 hours!
$url = Get-S3PreSignedURL -BucketName $bucketName -Key $key -Expire (Get-Date).AddDays(1)

Additional AWS PowerShell Resources

Azure PowerShell

What you’ll need to get started:

PowerShell for Azure has evolved over time so lets dive quickly into the difference:

  • Az module (recommended) – this is a replacement for AzureRM and AzureRM.Netcore and is now the primary PowerShell module for Azure. It supports PowerShell 5.1 and 6.1+.
  • AzureRM module – officially maintained through December 2020 but will no longer receive new cmdlets or features.

Installing Azure Az PowerShell

Because the Az module is the future, this blog post will focus on this version of the module.

Get started by running the following install command:

# Install the Az module
Install-Module -Name Az

Installing the Az module will install a full list of modules for all services within Azure.

There are many Azure Services, and the Az modules are split up to support each one! You can import all of the Azure cmdlets at once, or you can import just the cmdlets you need. As there are quite a few cmdlets you can save a lot of time by only engaging the Az modules you actually need.

# Import all of the Az cmdlets - time in seconds
(Measure-Command {Import-Module Az}).Seconds
11
# Import just the Sql Az cmdlets - time in seconds
(Measure-Command {Import-Module Az.Sql}).Seconds
1

You can explore which Azure cmdlets are available by running the following:

# Get a list of Azure services supported by the Az modules
Get-Command -Noun Az* | Sort-Object Source
# List all (read only) Get cmdlets that contain VM in the Az.Compute module
Get-Command -Verb Get -Noun AzVM* -Module Az.Compute

Azure operates with the concept of Azure geographies. Each geography contains one or more regions. You will likely want to deploy resources to the geography that is closest to you. Discover the available locations to deploy in Azure by running the following: 

# Get a list of available Azure locations to deploy resources to
Get-AzLocation | Select-Object DisplayName, Location

With Azure you will not set a default region. Instead, you first create a container called a resource group in a region of your choosing. Then, all resources created in the resource group will also be located in that region.

Authenticating to Azure using PowerShell

Azure supports several PowerShell authentication methods. This post will cover one way, the interactive sign in. Use the Connect-AzAccount cmdlet which will prompt you to launch a browser window and authenticate using your Azure account login.

Import-Module Az.Accounts
# Connect to Azure with a browser sign in token
Connect-AzAccount
# If you have more than one subscription associated with your mail account, you can choose the default subscription.
Get-AzSubscription
Select-AzSubscription -Subscription "My Subscription"
# Verify your current and active subscription
Get-AzContext

If your credentials have access to multiple Azure subscriptions you can retrieve a list of them using Get-AzSubscription. If you do have more than one, you can choose the one you want to work with using Select-AzSubscription.

Once you have authenticated via your browser window, you will have now have the ability to run commands against the active Azure subscription. If you are not sure which subscription you are currently under, you can always verify by running Get-AzContext.

Managing Azure Cloud with PowerShell

With your PowerShell session authenticated to your Azure account you can now engage the power of the Azure cloud. Again, the possibilities are nearly limitless. Using PowerShell you can create, manage, delete, monitor, and maintain Azure cloud resources around the world.

Azure PowerShell Final Example

In this final example we’ll use PowerShell to create a highly secure blob storage in Azure. Then we’ll upload a file to the blob, and create a pre-signed URL to the file. The URL will be good for 24 hours (configurable). This is a great way to share a file securely worldwide leveraging the power of the cloud.

# AZURE FINAL EXAMPLE
# 1 - Create a resource group to hold storage account
# 2 - Create a storage account
# 3 - Create a storage container
# 4 - Upload a file to the new storage container
# 5 - Create a pre-signed URL to enable people to securely download the file worldwide

#Resource Group
$resourceGroupName = "techthoughts"
$location = "westus"

# 1 - Create a resource group to hold storage account
# https://docs.microsoft.com/powershell/module/az.resources/new-azresourcegroup?view=azps-4.4.0
$newResourceGroupSplat = @{
    Name     = $resourceGroupName
    Location = $location
}
New-AzResourceGroup @newResourceGroupSplat

# 2 - Create a storage account
# https://docs.microsoft.com/powershell/module/az.storage/new-azstorageaccount?view=azps-4.4.0
$storageAccountName = 'techthoughtsstorage'
$newStorageAccountSplat = @{
    ResourceGroupName = $resourceGroupName
    AccountName       = $storageAccountName
    Location          = $location
    SkuName           = 'Standard_LRS'
    Kind              = 'StorageV2'
}
New-AzStorageAccount @newStorageAccountSplat

# 3 - Create a storage container
# https://docs.microsoft.com/powershell/module/az.storage/get-azstorageaccount?view=azps-4.4.0
$getStorageAccountSplat = @{
    ResourceGroupName = $resourceGroupName
    Name              = $storageAccountName
}
$storageContext = Get-AzStorageAccount @getStorageAccountSplat
# https://docs.microsoft.com/powershell/module/az.storage/new-azstoragecontainer?view=azps-4.4.0
$containerName = 'techthoughtscontainer'
$newStorageContainerSplat = @{
    Context    = $storageContext.Context
    Name       = $containerName
    Permission = 'Off'
}
New-AzStorageContainer @newStorageContainerSplat

# 4 - Upload a file to the new storage container
# https://docs.microsoft.com/powershell/module/az.storage/set-azstorageblobcontent?view=azps-4.4.0
$setAzStorageBlobSplat = @{
    Context   = $storageContext.Context
    Container = $containerName
    File      = 'C:\rs-pkgs\techthoughts_text_file.txt'
    Blob      = 'techthoughts_text_file.txt'
}
Set-AzStorageBlobContent @setAzStorageBlobSplat

# 5 - Create a pre-signed URL to enable people to securely download the file worldwide
# https://docs.microsoft.com/powershell/module/az.storage/new-azstorageblobsastoken?view=azps-4.4.0
$newStorageSASSplat = @{
    Context    = $storageContext.Context
    Container  = $containerName
    Blob       = 'techthoughts_text_file.txt'
    ExpiryTime = (Get-Date).AddDays(1)
    Permission = 'r'
    FullUri    = $true
}
$url = New-AzStorageBlobSASToken @newStorageSASSplat

Additional Azure PowerShell Resources

Google Cloud Tools for PowerShell

Currently the project is in maintenance mode and we do not plan to support adding any new cmdlets. Pull requests are welcomed though if you want to add new cmdlets for this.

Quoc Truong – Dec 5, 2019

As you can see on this issue from the google-cloud-powershell GitHub repo, Google has elected to abandon PowerShell support moving forward.

Google Cloud also has the lowest market share and lowest growth rate of the three major cloud providers. Coincidence? You decide.

The Google Cloud Tools for PowerShell, such as they are, do function and you can leverage them to manage some resources in Google Cloud. As they are end of life though, this post will not dive into details of their use.

Feel free to post your thoughts on Google’s decision on the linked issue above. Hopefully they will reconsider their stance. If you are a PowerShell strong team, you will be better served by steering your business towards a different cloud provider.

Google PowerShell Resources

Series Navigation<< PowerShell FunctionsPowerShell Modules >>

Tags:

Leave a Reply

Your email address will not be published. Required fields are marked *