Advanced Data Solutions : Leveraging Azure Data Box Disks to Support your M365 SharePoint Online Migration

Article Problem Statement

The consumption of Software as a Service (SaaS) such as Microsoft 365 (M365) for collaboration and productivity has become an enterprise standard than just a few years ago was viewed with skepticism. The data on the economics, capabilities, and flexibility of leveraging SaaS versus an equivalent on-premises deployment has been overwhelmingly proven in favor for a cloud deployment.

The problem that this article addresses is how to move from legacy on-premises systems to a cloud solution provider (CSP) such as M365 where the egress points to the internet from an organization may not have the required bandwidth capacity or network topology that is conducive to migration of large amounts of data. Such a migration may take far longer than desired. For example, migration of 500TB of data could take more than a year over a 100 Mbps circuit and such an extended migration time frame might not lead to a positive user experience overall. Or, depending on available bandwidth, the migration of all this data over a long period could possibly impact end users by causing Internet access latency. These factors are leading some organizations to seek alternate file share migration strategies such as the one proposed here using Azure Data Box.

Solution Objective

By leveraging the family of Microsoft Azure Data Box solutions, you can rapidly and securely migrate your data on scale from on-premises towards M365 and bypassing your local network infrastructure and leveraging Azure as a point ingress. Azure Data Box allows you to ingest data from Azure into M365 without compromising performance. This migration pattern can be leveraged by a Small to Medium Size Business (SMB) to a large enterprise covering various business verticals including Federal and Department of Defense (DoD) agencies and is supported across all M365 government cloud offerings. The focus of this article will emphasize the use of Azure Data Box Disks and for some customers this may not be the right solution based on overall size of migration.

This article is a living document and is subject to change based on changes dictated by Microsoft Product Groups as well as guidance from M365 service offerings. Please refer to the Azure Data Box online documentation for both product and process updates found here Azure Data Box Documentation.

This article will provide a JSON Template to deploy an Azure virtual network, network security group, storage account, a Bastion Host, and a virtual machine to help orchestrate a migration into M365 once the data has arrived in an Azure storage account and get it into M365. These assets are intended to be deployed once to support the migration project but could be deployed several times in support of various parallel migration efforts. The solution is provided “as is” with the intent of being a solution accelerator that can be discarded once the migration project has completed. The template is at the end of the article. The solution can be peered to an existing virtual network with connectivity to on-premises or be a stand-alone solution in the cloud as this deployment is defined.

The following graphic above is a representation of the solution accelerator provided as part of this article. The solution has been tested in both Microsoft Azure Commercial (MAC) and Microsoft Azure Government (MAG).

Intended Audience

This article is intended for administrators, IT Planners, and managers who are responsible for establishing and reviewing overall deployments, migrations, and oversee operational practices.

Assumptions

The person who is implementing this change has a good understanding of general systems administration and the proper administrative access to execute the tasks listed within this article. It is also assumed that all the requirements listed have been met by the reader of this document. It is also assumed that the person reading this document has the proper administrative access to a Microsoft Azure subscription with the basic knowledge for creating resources.

Scope

The following section defines what is in scope covering the specific instructions and activities that will be covered in this article. There will also be a section called out of scope listing what topics will not be addressed in this document.

In Scope

How to deploy the Azure Resource Manager Template which is the solution accelerator
Instructions on how to order and use the Azure Data Box Disks
Instructions on how to copy data from source to destination mount points within the Azure Data Box Disk
How to properly seal and ship the Azure Data Box Disks to Microsoft
General knowledge sharing on Azure Data Box Disk and Azure storage accounts
Create a network mapping between an Azure Windows virtual machine and Azure storage share hosting the exported data

Out of Scope

The following section will cover the areas that are considered out of scope for this article and may be in scope for the group leading your M365 migration.

M365 migration process. Please visit Microsoft 365 documentation | Microsoft Docs Migrate to Microsoft 365 section
Recommended migration tools to move data from the Azure storage account into M365 destination such as SharePoint Online (SPO) or OneDrive
Validation of data once it is imported into M356

IMPORTANT: After moving your data to Azure via the process outlined in this guide, you can then leverage the Microsoft FastTrack team for M365 which includes a migration benefit for assisting customers with migrating their file share data to M365 SharePoint Online and OneDrive using the Migration Manager tool. You can also engage with a certified partner as well.

Requirements

The following section list out all the requirements an organization will need for a successful migration of data from on-premises with Azure Data Box Disks to an Azure storage account for ingestion into M365.

Have access to an Azure subscription MAC or MAG
Owner or Contributor rights to the identified Azure subscription
The ability to create an Azure storage account with a SMB share for ingestion of migrated data into M365
The ability to order the Azure Data Box Disks
Microsoft Visual Studio Code, Notepad ++or any compatible developer tool supporting ARM Templates
Administrative access to the on-premises information systems hosting the data to a staged location
Universal Serial Bus (USB) ports will have to be accessible on computer systems to connect the Azure Data Box Disks
Access to management system to help orchestrate the copying of data locally from staging storage source to target (Azure Data Box Disks)

IMPORTANT Microsoft will only ship Azure Data Box devices within the continental United States and US Territories.

You must have a supported operating system for the disk unlock and data copy operation: supported operating systems
The following are the required software for Windows clients for the disk unlock and data copy Microsoft Azure Data Box Disk system requirements | Microsoft Docs

Instructions

The following are the instructions listed in the expected sequence to execute on the migration of data towards Azure as a staging point for the M365 Migration Team. The ARM JSON Template is attached at the bottom of this article.

Deploying the Solution Accelerator ARM Template

1	Logon to your Azure subscription with the proper administrative access. For MAC please use https://portal.azure.com and for MAG please use https://portal.azure.us
2	From the Azure portal menu, in the search box, type deploy, and then select Deploy a custom template.
3	Select Build your own template in the editor
4	Delete all of it
5	Download the solution accelerator at the end of this article and open the zip file.
6	Open the *solutionaccelerator.json* with your tool of choice. Microsoft recommends Visual Studio Code with ARM Tool Support
7	Copy all of the contents of the *solutionaccelerator.json* and past it in the Azure template deployment screen in the Azure portal
8	Click Save
9	The Azure Arm Form with load. Please enter the following
10	Reference the image below as a reference for you input. Click Create

Validate the solution has deployed without errors in Azure. Information about Azure Bastion can be found in the references section of this article.

Ordering the Azure Data Box Disks

The following instructions are on how to order the Azure Data Box Disks.

1	You will need to access and authenticate against your Azure Portal to order the Azure Data Box Disks https;/portal.azure.com or https://portal.azure.us if this is Government related order.
2	In the upper left corner of the portal, click “*+ Create a resource”, and search for Azure Data Box, then Click Azure Data Box*.
3	Click Create
4	In the next screen “Get started with Data Box” type in the following Transfer type: Select Import to Azure Subscription: Select the subscription that you have access and where you deployed your storage account in the earlier procedure Resource Group: Select the resource group you created when you created your storage account Source county region: Select the United States Destination Azure Region: Select the Azure Datacenter region where you created your storage account. In this example we will select East US. Click Apply
5	In the next screen select Data Box Disk push the button Select (The maximum capacity of the solution for a single order of 5 disks is 35 TB. You could create multiple orders for larger data sizes.)
6	In the next screen type in a meaningful name in the import order name field: Type in *m365migrationjob1*
7	Next type in the estimated data size in TB and in the toggle button select NO to use a custom key.
8	Click Next Data destination
9	In the data destination drop down field select storage account, then in the following drop-down field select the storage account you created earlier
10	Click Next Contact details

11	Click the + Address and the Add Address blade will appear. Fill it out with your local address where you mail room will be able to receive the delivery from Microsoft. IMPORTANT click on the Validate button
12	Once the address has been validated select add shipping address button
13	Add the email address of the person who should be expecting the Azure Data Box Disks at your office if Microsoft needs to email you an update on the delivery.

14	Next select and Review + Order button, then at the bottom of the page select I acknowledge, then click Order

Unpack, Connect, and unlock Azure Data Box Disk

The following section will provide instructions once you receive your Azure Data Box Disk how to prepare them for your migration job. PLEASE KEEP THE ORIGINAL UPS BOX THE AZURE DATA BOX DISKS DISK CAME IN SO THAT YOU CAN REUSE. You will need a client computer on which you can install the Data Box Disk unlock tool. Your client computer must be a supported operating system identified here as well as required software installed: Microsoft Azure Data Box Disk system requirements | Microsoft Docs . Please refer to this link before you begin.

1	The Data Box Disks are mailed in a small shipping Box. Open the box and remove its contents. Check that the box has 1 to 5 solid-state disks (SSDs) and a USB connecting cable per disk. Inspect the box for any evidence of tampering, or any other obvious damage.
2	If the shipping box is tampered or severely damaged, do not open the box. Contact Microsoft Support to help you assess whether the disks are in good working order and if they need to ship you a replacement.
3	Verify that the box has a clear sleeve containing a shipping label (under the current label) for return shipment. If this label is lost or damaged, you can always download and print a new one from the Azure portal.
4	Save the box and packaging foam for return shipment of the disks.
5	Use the included cable to connect the disk to the client computer running a supported OS as stated in the prerequisites.
6	In the Azure portal, navigate to your Data Box Disk Order. Search for it by navigating to General > All resources, then select your Data Box Disk Order. Use the copy icon to copy the passkey. This passkey will be used to unlock the disks
7	In the Azure portal, navigate to your Data Box Disk Order. Search for it by navigating to General > All resources, then select your Data Box Disk Order
8	Download the Data Box Disk toolset corresponding to the Windows client. This toolset contains 3 tools: Data Box Disk Unlock tool, Data Box Disk Validation tool, and Data Box Disk Split Copy tool.
9	To verify the computer that you are using to unlock the disk meets the operating system requirements, run the system check command. A sample output is shown below (this is being conducted in Windows PowerShell) Windows PowerShell Copyright (C) Microsoft Corporation. All rights reserved. PS C:\Data BoxDiskUnlockTool\DiskUnlock> .\Data BoxDiskUnlock.exe /SystemCheck Successfully verified that the system can run the tool. PS C:\Data BoxDiskUnlockTool\DiskUnlock>
10	In this procedure, you will use only the Data Box Disk Unlock tool. The other two tools will be used later.

11	Click on the following link to download the Data Box toolset for Windows Download Data Box Disk toolset for Windows
12	Extract the toolset on the same computer that you will use to copy the data
13	Open an administrative Windows PowerShell on the same computer
14	Run Data BoxDiskUnlock.exe and supply the passkey you obtained in Connect to disks and get the passkey The drive letter assigned to the disk is displayed. A sample output is shown below: Windows PowerShell Copyright (C) Microsoft Corporation. All rights reserved. PS C:\Data BoxDiskUnlockTool\DiskUnlock> .\Data BoxDiskUnlock.exe /SystemCheck Successfully verified that the system can run the tool. PS C:\Data BoxDiskUnlockTool\DiskUnlock>
15	Run Data BoxDiskUnlock.exe and supply the passkey you obtained in Connect to disks and get the passkey. The drive letter assigned to the disk is displayed.
16	Repeat the unlock steps for any future disk reinserts. Use the help command if you need help with the Data Box Disk unlock tool
17	Once the disk is unlocked, you can view the contents of the disk. If you run into any issues unlocking, please review the following article troubleshoot and unlock issues

Copy Data to Azure Data Box and Verify

The following section will provide instructions on how to copy data to the Azure Data Box Disks. For the M365 migration the proper data upload can be either AzureBlob or AzureFile mount point within the Azure Data Box Disks. Consult with your vendor and migration tool requirements. For SPO migrations leveraging the Migration Manager you will need to use AzureFiles

1	Copy files to a folder within AzureFile folder. A sub-folder within AzureFile folder creates a file share. Files copied directly to AzureFile folder will all be uploaded as block blobs which is not the desired outcome. AzureBlob does not have this requirement.
2	When copying files, ensure that files do not exceed 1 TiB for Azure Files.
3	Robocopy can be used to copy your data. Multiple copy jobs can be initiated using the following Robocopy command: Robocopy <source> <destination> * /MT:64 /E /R:1 /W:1 /NFL /NDL /FFT /Log:c:\RobocopyLog.txt Note: Where <source> is the full path to the source data to be copied, and <destination> is the is the full path to the destination where the data will reside on the Data Box
4	Monitor the log as you copy data from the source to the Azure Data Disk log file located in c:\RobocopyLog.txt
5	Validate the data once it has been copied to Azure Data Box Disk to validate the content has been copied.
6	If you need to split the data copied over various Azure Data Box Disk please review the following Tutorial to copy data to Azure Data Box Disk \| Microsoft Docs in the Split and copy data to disks

Return Azure Data Box Disks to Microsoft

The following procedures are intended to return the Azure Data box Disks back to Microsoft. Please follow them in the proper order as documented to obtain the desired result.

1	Once the data has been copied and validated on the Azure Data Box Disk proceed to remove the cables connecting the devices to the local computer on-premises.
2	Wrap all the disks and the connecting cables with a bubble wrap and place them into the shipping box. Charges may apply if the accessories are missing so, please check before sending back.
3	You can reuse the packing from the original shipment, use a well-secured bubble wrap.
4	Logon to the Azure portal with your administrative credentials.
5	Go to Overview, click Download shipping labels and download a return ship label.
6	Schedule a pickup with UPS
7	Copy and save your tracking number.

Verify Data Upload to Azure

The following instructions are intended to validate the import of the data in Azure via the Azure Data Box Disks.

1	Once the disks are picked up by your carrier, the order status in the portal updates to Picked up. A tracking ID is also displayed.
2	When Microsoft receives and scans the disk, job status is updated to Received.
3	The data automatically gets copied once the disks are connected to a server in the Azure datacenter. Depending upon the data size, the copy operation may take a few hours to days to complete. You can monitor the copy job progress in the portal.
4	Once the copy is complete, order status updates to Completed
5	If there are errors during the copying, please refer to this guide to resolve Troubleshooting Upload Errors
6	Please contact the M365 Team to help ingesting your data into M365

If you required to create an Azure file share on the storage account that was created earlier with the solution accelerator please follow these instructions Create and use an Azure file shares on Windows VMs | Microsoft Docs under Create an Azure File share.

You may now deploy the M365 Migration Manager on the virtual machine that was created earlier and start to move your data into SPO.

References

The following are the technical references that were used in the drafting of this document.

Create a storage account - Azure Storage | Microsoft Docs

Azure Data Box Documentation - Offline Transfer | Microsoft Docs

Tutorial to copy data to Azure Data Box Disk | Microsoft Docs

Azure documentation | Microsoft Docs

Mount SMB Azure file share on Windows | Microsoft Docs

Deploy template - Azure portal - Azure Resource Manager | Microsoft Docs

Azure Bastion | Microsoft Docs

Migration Manager Prerequisites and Endpoints - Migrate to Microsoft 365 | Microsoft Docs

Author