Monday, February 28, 2022

Best Practice of Azure Storage data protection, backup and recovery

 

Table of Content

- Introduction

- Data Backup and Protection

- Non-supporting Storage Recovery

- Supporting Storage Recovery

- Deleted Storage Accounts Recovery from Azure Portal

Introduction 

Microsoft has received customer assistant requests to restore Storage account, or file/data in it, but with the data accessibility Policy (https://www.microsoft.com/en-us/trust-center/privacy) we can only provide very limited help during the process. Azure Storage product team always try their best to help customers to fully control data by improving tools and accessibility, as well as when the data is deleted and needs recovery. The best way to protect data is to take care of backup well and prevent accidental deletion. 

The blog provides customers with options of protecting Azure Storage data from being accidentally deleted, data backup, self-serve recovery scenarios, and Microsoft-assist recovery possibilities. 

 

Data Backup and Protection 

In the Azure Storage documentation, data protection refers to strategies for protecting the storage account and data within it from being deleted or modified, or for restoring data after it has been deleted or modified. This section mainly introduces available data backup & protection options and Azure RBAC choices to avoid accidental account deletion. For more details on this section, you could refer to the data backup & protection options and best practices for Azure RBAC. 

Data Backup Options 

This section describes some possible recovery options after the data protection options have been enabled. 

Scenario 1: Storage Account Recovery 

  • Refers to the section “Deleted storage accounts recovery from Azure Portal” below 

Scenario 2: Blob Container Recovery 

  • Recovery of the soft-deleted container and its contents 

         - Reference link:  https://docs.microsoft.com/en-us/azure/storage/blobs/soft-delete-container-enable?tabs=azure-portal#restore-a- soft-deleted-container 

         - Requirements for recovery: container soft delete is enabled, and the container soft delete retention period has not yet expired. 

  • Recovery from a second storage account 

         - Requirements for recovery: all container and blob operations have been effectively replicated to a second storage account. 

Scenario 3: Blob File Recovery 

  • Recovery of blobs to previous versions via blob versioning 

         - Reference link: https://docs.microsoft.com/en-us/azure/storage/blobs/versioning-enable?tabs=portal 

         - Requirements for recovery: blob versioning is enabled, and the blob has one or more previous versions. 

         - Currently NOT supported for Data Lake Storage workloads. 

         - Recovery procedures: 

           1. Go to the affected blob from Azure portal 

           2. Select “...” for the blob that you would like to recover 

           3. Click on “view previous versions” 

           4. Select the version that is required to restore from 

           5. Click on “make current version” 

versioning.jpg

  • Recovery of blobs via blob soft delete 

         - Reference link: https://docs.microsoft.com/en-us/azure/storage/blobs/soft-delete-blob-manage?tabs=dotnet 

         - Requirements for recovery: blob soft delete is enabled, and the soft delete retention interval has not expired. 

  • Recovery of a set of block blobs via point-in-time 

         - Reference link: https://docs.microsoft.com/en-us/azure/storage/blobs/point-in-time-restore-manage?tabs=portal 

         - Requirements for recovery: point-in-time restore is enabled and the restore point is within the retention interval. The storage account has not been compromised or corrupted. 

  • Recovery of blobs via snapshots 

         - Reference link: https://docs.microsoft.com/en-us/azure/storage/blobs/snapshots-manage-dotnet?tabs=dotnet 

         - Requirements for recovery: the blob has one or more snapshots. 

         - Recovery procedures: 

           1. Go to the affected blob from Azure portal 

           2. Select “...” for the blob that you would like to recover 

           3. Click on “view snapshots” 

           4. Select the snapshot that is required to restore from 

           5. Click on “promote snapshot” 

Qunming_Chen_1-1644197511097.png

 

Data Protection Options 

This section lists some recommended data protection options for storage accounts, containers and blob files. 

Scenario 1: Storage Account Protection 

  • Azure Resource Manager (ARM) lock enabled - to lock all of your storage accounts and prevent deletion of the storage account. 
  • For more details on ARM lock, please refer to the ARM lock link. 
  • Benefits and limitations: 
  • Protect the storage account against deletion or configuration changes. 
  • Does not protect containers or blobs in the account from being deleted or overwritten. 
  • It supports ADLS Gen 2 in Preview. 

Scenario 2: Blob Container Protection 

  • Immutability policy on a container enabled - to protect business critical documents, such as to meet legal or regulatory compliance requirements. 

         - For more details on Immutability policy on a container, please refer to this immutability policy on a container link. 

         - Benefits and limitations: 

           1. Protect a container and its blobs from all deletes and overwrites. 

           2. When a legal hold or a locked time-based retention policy is in effect, the storage account is also protected from deletion. Containers for which no immutability policy has been set are not protected from deletion. 

           3. It supports ADLS Gen 2 in Preview. 

  • Container soft-delete enabled - to restore a deleted container within a specified interval. 

         - For more details on Container soft delete, please refer to the container soft delete link. 

         - Benefits and limitations: 

           1. A deleted container and its contents may be restored within the retention period. And the best practice of minimum retention interval should be of 7 days. 

           2. Only container-level operation like Delete container, can be restored. Container soft delete does not enable you to restore an individual blob in the container if that blob is deleted. 

           3. It supports ADLS Gen 2. 

Scenario 3: Blob File Protection  

  • Immutability policy on a blob version enabled – to prevent a blob version from being deleted for an interval that you control. 

         - For more details on Blob soft delete, please refer to the document link. 

         - Benefits and limitations: 

           1. Protects a blob version from being deleted and its metadata from being overwritten. An overwrite operation creates a new                         version. 

           2. If at least one container has version-level immutability enabled, the storage account is also protected from deletion.  

           3. Container deletion fails if at least one blob exists in the container. 

           4. It’s NOT available for ADLS Gen2. 

  • Blob soft delete enabled - to restore a deleted blob or blob version within a specified interval. 

         - For more details on Blob soft delete, please refer to the blob soft delete link. 

         - Benefits: 

           1. A deleted blob or blob version may be restored within the retention period. And the best practice of minimum retention interval should be of 7 days. 

           2. It supports ADLS Gen 2. 

  • Blob snapshot enabled - to manually save the state of a blob at a given point in time. 

         - For more details on Blob snapshot, please refer to this blob snapshot link. 

         - Benefits and limitations: 

           1. A blob may be restored from a snapshot if the blob is overwritten. However, if the blob is deleted, snapshots are also deleted. 

           2. It supports ADLS Gen 2 in preview. 

  • Blob versioning enabled – to automatically save the state of a blob in a previous version when it is overwritten. 

         - For more details on Blob versioning, please refer to the blob versioning link. 

         - Benefits and limitations: 

            1. Every blob write operation creates a new version. The current version of a blob may be restored from a previous version if the current version is deleted or overwritten. 

            2. It’s NOT available for ADLS Gen2 

  • Point-in-time restore enabled – to restore a set of block blobs to a previous point in time. 

         - For more details on Point-in-time restore, please refer to this particular link. 

         - Benefits and limitations: 

           1. A set of block blobs may be reverted to their state at a specific point in the past. 

           2. Only operations performed on block blobs are reverted 

           3. Any operations performed on containers, page blobs, or append blobs are NOT reverted. 

           4. It’s NOT available for ADLS Gen2. 

  • Copy data to a second account via Azure Storage object replication / tools like Azcopy or Azure Data factory 

          - Benefits and limitations: 

             1. Data can be restored from the second storage account if the primary account is compromised in any way. 

             2. Azcopy and Azure Data Factory are supported. 

             3. However, object replication is not supported. 

Best Practice for Azure RBAC 

Another best practice to avoid accidental account deletion is to limit the number of users who have permissions to delete an account via role-based access control (Azure RBAC). For more information, please visit best practices for Azure RBAC. 

  • Only grant the access users needed 
  • Limit the number of subscription owners 
  • Use Azure AD Privileged Identity Management 
  • Assign roles to groups, NOT users  
  • Assign roles using the unique role ID instead of the role name 

 

Non-supporting Storage Recovery 

This section lists storage recovery scenarios that are NOT supported by Microsoft. 

  • Azure Storage Queue recovery is not supported. 
  • Azure Storage Table entries recovery is not supported, while deleted table recovery is supported (refer to supporting storage recovery section below). 
  • Azure Blob files recovery is not supported but deleted container recovery is supported (refer to supporting storage recovery section below). 

  

Supporting Storage Recovery 

This section describes several scenarios where storage recovery is possible with pre-requisites. 

Microsoft is providing the best effort attempts to recover the data but WITHOUT guarantees on how much data could be restored. 

Scenario 1: Storage account recovery (ARM storage account recovery) 

  • Pre-requisites:  

         - The storage account was deleted within the past 14 days. 

         - The storage account was created with the Azure Resource Manager deployment model. 

         - A new storage account with the same name has not been created since the original account was deleted. 

         - The user who is recovering the storage account must be assigned an Azure RBAC role that provides the Microsoft.Storage/storageAccounts/write permissions. For information about built-in Azure RBAC roles that provide this permission, see Azure built-in roles. 

         - Make sure the resource group that deleted storage account exists. If the resource group was deleted, you must recreate it manually.  

         - [For specific cases only] if the deleted storage account used customer-managed keys with Azure Key Vault and key vault has also been deleted, then you must restore the key vault before you restore the storage account. For more information, see Azure Key Vault recovery overview. 

  • Suggestions: 

          - Storage account recovery from an existing storage account (refer to Deleted storage accounts recovery from Azure Portal section). 

          - Storage account recovery via a support ticket (refer to Deleted storage accounts recovery from Azure Portal section). 

Scenarios 2: Classic Storage Account recovery 

  • Pre-requisites: 

         - A new storage account with the same name has not been created since the original account was deleted. 

         - The storage account was deleted within the past 14 days. 

  • Suggestions:  

        - Seek help from support engineers to evaluate the situation. 

Scenarios 3: Container recovery  

  • Pre-requisites: 

         - The storage account replication was set to GRS, GZRS, RAGZRS or RA-GRS prior to “container” deletion. (Storage account with LRS is not supported to recover deleted container) 

  • Suggestions: 

         - Seek help from support engineers to evaluate the situation. 

Scenario 4: Azure Data Lake Storage (ADLS) Gen 2 Data and File System recovery 

  • Pre-requisites: 

         - Storage account with Hierarchical Namespace (HNS) enabled. 

         - The ADLS Gen2 file or folder was deleted within 3 days. 

  • Suggestions: 

         - Seek help from support engineers to evaluate the situation. 

Scenario 5: Table recovery 

  • Pre-requisites: 

         - The whole Table is deleted by DELETE Table operation without table entry data modified. 

  • Suggestions: 

        - Seek help from support engineers to evaluate the situation. 

Scenario 6: Disk recovery   

The pre-requisites of Disk Recovery vary depending on a few factors. For instance, is soft delete enabled? Or does the recovery disk refer to managed disk or unmanaged disk? etc. It is recommended to seek help from support engineers to evaluate the situation. 

 

Deleted storage accounts recovery from Azure Portal 

This section summarizes the new feature for end users to recover a deleted storage account from Azure portal. In general, there are 2 different ways for users to follow. One is to recover the deleted account from an existing storage account, and the other is to recover the account via a support ticket.   

1. Storage account level recovery from an existing storage account 

To recover a deleted storage account from within another storage account, please follow the steps as follows: 

  • Navigate to the overview page for an existing storage account in the Azure portal. 
  • In the Support + troubleshooting section, select Recover deleted account. 
  • From the dropdown list, select the account to recover, as shown in the following image. If the storage account that you want to recover is not in the dropdown list, then it cannot be recovered. 

Qunming_Chen_2-1644197511099.png

  • Select the Recover button to restore the account. The portal displays a notification that the recovery is in progress. 

Reference: https://docs.microsoft.com/en-us/azure/storage/common/storage-account-recover#recover-a-deleted-account-from-the-azure-portal  

 

2. Storage account recovery via a support ticket 

  • In the Azure portal, navigate to Help + support. 
  • Select Create a support request. 
  • On the Problem description tab, in the Issue type filed, select Technical. 
  • In the Subscription field, select the subscription that contained the deleted storage account. 
  • In the Service field, select Storage Account Management. 
  • In the Resource field, select any storage account resource. The deleted storage account will not appear in the list. 
  • Add a brief summary of the issue. 
  • In the Problem type filed, select Deletion and Recovery. 
  • In the Problem subtype field, select Recover deleted storage account. The following image shows an example of the Problem description tab being filled out: 

Qunming_Chen_3-1644197511102.png

 

 

  • Next, navigate to the Recommended solution tab, and select Customer-Controlled Storage Account Recovery, as shown in the subsequent image: 

Qunming_Chen_4-1644197511105.png

 

 

  • From the dropdown list, select the account to recover, as shown in the following image. If the storage account that you want to recover is not in the dropdown list, then it cannot be recovered. 

Qunming_Chen_5-1644197511108.png

 

 

  • Select the Recover button to restore the account. The portal displays a notification that the recovery is in progress. 

Reference: https://docs.microsoft.com/en-us/azure/storage/common/storage-account-recover#recover-a-deleted-account-via-a-support-ticket 

Posted at https://sl.advdat.com/3vpOqomhttps://sl.advdat.com/3vpOqom