Automating and scheduling Azure Stack infrastructure backups with Azure Automation

Azure Stack doesn’t currently provide a way to automate or schedule infrastructure backups, so rather than running this manually every day [boring] I wrote some PowerShell to do this. Then instead of having a VM somewhere that runs this PowerShell script as a scheduled task, I put this as a runbook into Azure Automation, which is a much better idea for when Azure Stack is running in a Connected scenario seeing as Azure Stack is publicly accessible – no VM to keep up to date, cheap to run, incredibly easy to use, etc. For running in a Disconnected scenario you could use a hybrid runbook worker. The runbook is easily scheduled from the ‘Schedules’ section in Azure Automation, more information can be found here – https://docs.microsoft.com/en-us/azure/automation/automation-schedules

I split the task into two PowerShell scripts: one to do the backup, and the other to send an email when the status is anything other than ‘Succeeded’. This way for multiple Azure Stack deployments, an email is sent for each deployment and I can reuse the runbook. Note that the backup process waits for the backup to complete, so the runbook takes around 20 minutes to execute. If an email is sent, it looks like this including both the error returned by the backup and the state of the roles (I know it’s not pretty, but it serves a purpose!):

Modules – in your Azure Automation account you will need to add the following modules, all of which can be installed very easily with PowerShell Gallery – https://www.powershellgallery.com/ (correct at the time of writing):

There are four variables to change at the top of the runbook to do the backup:

  1. $location is the region with which you installed Azure Stack, for the Azure Stack Development Kit this is ‘local’
  2. $cloudName is just a friendly name to identify your Azure Stack
  3. $azureStackDomain is the part of the FQDN after the region, e.g. <region>.azurestack.mydomain.com
  4. $credential is for the account to use to perform the Azure Stack infrastructure backup.  In my use I store the user credential securely in Azure Automation and call it here (no passwords embedded) – https://docs.microsoft.com/en-us/azure/automation/automation-credentials. Note that you can login with an Azure AD App instead, but it requires a different approach with Azure Automation. See the end of this article for details.

The rest doesn’t need to be changed apart from the penultimate line where you need to provide the filename of the PowerShell script to send the email if you want (or you can remove this section to not send an email if you just want to automate your infrastructure backups).  Yes I could put all this information into Log Analytics and provide alerts that way as we all already get too many emails, but I just needed something simple to manage this.  Please let me know if you do something better with this!

It’s worth noting that I’m not using the PowerShell cmdlet ‘Start-AzsBackup’ as it sets $ErrorActionPreference = ‘Stop’, which means the email wouldn’t send. Instead I just poke the API to start the backup, which is essentially all ‘Start-AzsBackup’ does anyway.

So, with that here is the Azure Stack infrastructure backup runbook (make sure you publish it!):

… and the email runbook – change the variables at the top for your email (make sure you publish it!):

Logging in with an Azure AD App: I mentioned earlier about logging in with an Azure AD App – In my testing I found that creating an App, giving it permissions to Azure Stack and then using the same ‘Login-AzureRmAccount -EnvironmentName $cloudName -Credential $credential’ command with ‘-TenantID $tenantID -ServicePrincipal’ added (as required to log in using an App) would work locally, but not in Azure Automation. After a bit of searching around, it seems this is a known issue (https://stackoverflow.com/questions/46428031/set-azurermcontext-error-when-executed-within-an-azure-automation-runbook/46567353#46567353).

What can be done to get this to work in this way is to use the Azure Automation RunAs account with the certificate to log in – go to the Azure Automation account, select ‘Run as accounts’ under Account Settings, select the ‘Azure Run As Account’ (not the classic, obviously), then you have all the details you need. The account display name is what you need to add to the Azure Stack Default Provider Subscription, then you’ll also need the Application ID, Thumbprint, and Tenant ID fields (not the subscription ID as this isn’t your Azure Stack subscription ID). Note that the certificate will need to be renewed each year.

Then you can login with this (all of this replaces the previous line 17). Note that this won’t work locally as the certificate won’t be present locally, that is in Azure: