Automating and scheduling Azure Stack infrastructure backups with Azure Automation

Azure Stack doesn’t currently provide a way to automate or schedule infrastructure backups, so rather than running this manually every day [boring] I wrote some PowerShell to do this. Then instead of having a VM somewhere that runs this PowerShell script as a scheduled task, I put this as a runbook into Azure Automation, which is a much better idea for when Azure Stack is running in a Connected scenario seeing as Azure Stack is publicly accessible – no VM to keep up to date, cheap to run, incredibly easy to use, etc. For running in a Disconnected scenario you could use a hybrid runbook worker. The runbook is easily scheduled from the ‘Schedules’ section in Azure Automation, more information can be found here – https://docs.microsoft.com/en-us/azure/automation/automation-schedules

I split the task into two PowerShell scripts: one to do the backup, and the other to send an email when the status is anything other than ‘Succeeded’. This way for multiple Azure Stack deployments, an email is sent for each deployment and I can reuse the runbook. Note that the backup process waits for the backup to complete, so the runbook takes around 20 minutes to execute. If an email is sent, it looks like this including both the error returned by the backup and the state of the roles (I know it’s not pretty, but it serves a purpose!):

Modules – in your Azure Automation account you will need to add the following modules, all of which can be installed very easily with PowerShell Gallery – https://www.powershellgallery.com/ (correct at the time of writing):

AzureRM.profile version 3.4.1 – https://www.powershellgallery.com/packages/AzureRM.profile/3.4.1
AzureRM.Resources version 4.4.1 – https://www.powershellgallery.com/packages/AzureRM.Resources/4.4.1
AzureRM.AzureStackAdmin version 0.11.1 – https://www.powershellgallery.com/packages/AzureRM.AzureStackAdmin/0.11.1
Azs.Backup.Admin version 0.1.0 – https://www.powershellgallery.com/packages/Azs.Backup.Admin/0.1.0

There are four variables to change at the top of the runbook to do the backup:

$location is the region with which you installed Azure Stack, for the Azure Stack Development Kit this is ‘local’
$cloudName is just a friendly name to identify your Azure Stack
$azureStackDomain is the part of the FQDN after the region, e.g. <region>.azurestack.mydomain.com
$credential is for the account to use to perform the Azure Stack infrastructure backup. In my use I store the user credential securely in Azure Automation and call it here (no passwords embedded) – https://docs.microsoft.com/en-us/azure/automation/automation-credentials. Note that you can login with an Azure AD App instead, but it requires a different approach with Azure Automation. See the end of this article for details.

The rest doesn’t need to be changed apart from the penultimate line where you need to provide the filename of the PowerShell script to send the email if you want (or you can remove this section to not send an email if you just want to automate your infrastructure backups). Yes I could put all this information into Log Analytics and provide alerts that way as we all already get too many emails, but I just needed something simple to manage this. Please let me know if you do something better with this!

It’s worth noting that I’m not using the PowerShell cmdlet ‘Start-AzsBackup’ as it sets $ErrorActionPreference = ‘Stop’, which means the email wouldn’t send. Instead I just poke the API to start the backup, which is essentially all ‘Start-AzsBackup’ does anyway.

So, with that here is the Azure Stack infrastructure backup runbook (make sure you publish it!):

# Variables to change
$location = "local"
$cloudName = "My Azure Stack"
$azureStackDomain = "azurestack.mydomain.com"
$credential = Get-AutomationPSCredential -Name 'infrastructurebackupaccount@contoso.com'

# Configure the Azure Stack operator’s PowerShell environment.
$api = "adminmanagement"
$azureStackAdminEndPoint = 'https://{0}.{1}.{2}' -f $api, $location, $azureStackDomain
Write-Output "`n[Info] Adding the Azure Stack environment`n"
Add-AzureRMEnvironment -Name "$cloudName" -ArmEndpoint $azureStackAdminEndPoint -ErrorAction Stop
Write-Output "`n[Info] Setting the Azure Stack environment`n"
Set-AzureRmEnvironment -Name "$cloudName" -GraphAudience "https://graph.windows.net" -ErrorAction Stop

# Sign-in to the operator's console
Write-Output "`n[Info] Logging in to the Azure Stack environment`n"
Login-AzureRmAccount -EnvironmentName "$cloudName" -Credential $credential -ErrorAction Stop

# Start a backup job
Write-Output "`n[Info] Starting the Azure Stack infrastructure backup `n"
$params = @{
    ApiVersion        = "2016-05-01"
    Action            = "createbackup"
    ResourceType      = "Microsoft.Backup.Admin/backupLocations"
    ResourceGroupName = "system.{0}" -f $location
    ResourceName      = "{0}/{1}" -f $location, $location
}
Invoke-AzureRmResourceAction @params -Force -ErrorAction Stop

# Check the state of the latest backup
$pastBackups = Get-AzSBackup -Location $location | Sort-Object CreatedDateTime -Descending
$latestStatus = $pastBackups[0].Status

If ($latestStatus -eq "Succeeded") {
    Write-Output "`n[Info] The Azure Stack infrastructure backup completed successfully"
} elseif ($latestStatus -ne "Succeeded") {
    Write-Output "`n[Warning] The status for this Azure Stack infrastructure backup is $latestStatus"
    $backupError = $Error | select -first 1 | Out-String
    $pastBackups[0].RoleStatus

    # Email a report that the backup was not successful
    $HTMLContent = ($pastBackups[0].RoleStatus | ConvertTo-Html -Fragment | Out-String) -replace 'Succeeded','Succeeded' -replace 'Failed','Failed'
    .\AzureStackInfrastructureBackupEmail.ps1
}

… and the email runbook – change the variables at the top for your email (make sure you publish it!):

# Account details $SMTPServer = "smtp.mydomain.com" $emailCredential = Get-AutomationPSCredential -Name 'myemailaccount@contoso.com' $emailFrom = "myemailaccount@contoso.com" $emailTo = "yayazurestack@contoso.com" # Subject and body $subject = "[Warning] Azure Stack Infrastructure Backup in '$cloudName': $latestStatus" $body = ' Azure Stack Infrastructure Backup Roles

The Azure Stack infrastructure backup for ' + $cloudName + 'has the status ' + $latestStatus + '. Please investigate.

The error reported during this infrastructure backup is as follows:

' + $backupError + '

' + $HTMLContent+ '' # Send the email Send-MailMessage -SmtpServer $SMTPServer -From $emailFrom -To $emailTo -Credential $emailCredential -Subject $subject -Body $body -BodyAsHTML -UseSsl

Logging in with an Azure AD App: I mentioned earlier about logging in with an Azure AD App – In my testing I found that creating an App, giving it permissions to Azure Stack and then using the same ‘Login-AzureRmAccount -EnvironmentName $cloudName -Credential $credential’ command with ‘-TenantID $tenantID -ServicePrincipal’ added (as required to log in using an App) would work locally, but not in Azure Automation. After a bit of searching around, it seems this is a known issue (https://stackoverflow.com/questions/46428031/set-azurermcontext-error-when-executed-within-an-azure-automation-runbook/46567353#46567353).

What can be done to get this to work in this way is to use the Azure Automation RunAs account with the certificate to log in – go to the Azure Automation account, select ‘Run as accounts’ under Account Settings, select the ‘Azure Run As Account’ (not the classic, obviously), then you have all the details you need. The account display name is what you need to add to the Azure Stack Default Provider Subscription, then you’ll also need the Application ID, Thumbprint, and Tenant ID fields (not the subscription ID as this isn’t your Azure Stack subscription ID). Note that the certificate will need to be renewed each year.

Then you can login with this (all of this replaces the previous line 17). Note that this won’t work locally as the certificate won’t be present locally, that is in Azure:

$thumbprint = "certificate thumbprint here"
$appID = "application ID here"
$tenantID = "tenant ID here"

Login-AzureRmAccount `
    -ServicePrincipal `
    -CertificateThumbprint $thumbprint `
    -ApplicationId "$appID " `
    -TenantId "$tenantID " `
    -Environment $cloudName `
    -ErrorAction Stop

Latest Images

Trending Articles

Latest Images