Advanced Data Solutions : Automating Azure SQL DB index and statistics maintenance using Azure Data Factory

Overview: As we know, it's crucial that you run Database maintenance regularly in order to keep your database performance up with the latest statistics and healthy indexes, here we provide another way to schedule this job using Azure Data Factory; this can be done on a scheduled time interval weekly/bi-weekly/monthly.

Solution: This article will show steps to schedule Database maintenance using the Azure Data Factory(ADF). ADF can use as an SQL agent to schedule and automate Azure SQL database tasks. In addition, database maintenance plans not supported in Azure can also be replaced with Azure pipelines.

Note: Using ADF will incur an additional cost.

Use case:

After large data deletion from database/tables or after data purge routine.
Post ETL pipelines, as a final activity step to perform database update stats and rebuild indexes per business needs.
After database migrations or database refresh/restore from production to Dev/Test environments.
Use Azure Data Factory pipeline with store procedure activity replacement of Database maintenance plans, which are unsupported in Azure SQL DB.

This can be done via AzureSQLMaintenance store procedure (SP) to perform complete Database maintenance, Store procedure and blog written by Yochanan Rachamim are listed below.

Blogs:
How to maintain Azure SQL Indexes and Statistics - Microsoft Tech Community

Automating Azure SQL DB index and statistics maintenance using Azure Automation - Microsoft Tech Community

Code:

AzureSQLMaintenance

Main steps:

Use existing or create an Azure Data Factory
Add SQL Authentication login on database
Create "AzureSQLMaintenance" SP on the user databases
Create pipeline
Schedule pipeline

Step by Step walkthrough:

Please create one using the Azure portal if you do not have a data factory.
https://docs.microsoft.com/en-us/azure/data-factory/quickstart-create-data-factory-portal#create-a-data-factory
Add New login to perform maintenance tasks, which will help identify when the maintenance tasks are running on the database. You can use the existing login and user.

-- Run on Master

CREATE LOGIN DBAdmin WITH password='<your-password>';

-- Run on user database

CREATE USER DBAdmin FROM LOGIN DBAdmin;

-- permission

EXEC sp_addrolemember 'db_owner', 'DBAdmin';

3. Download and deploy AzureSQLMaintenance to a user database. Code: AzureSQLMaintenance

4. Open Azure data factory to create a pipeline and name it.

Add store procedure activity.

Press "+ New" and set up linked Serviced by adding a name, type, subscription, Server, database, authentication type, User, and password. Test connection for the connectivity check.

After selecting Linked Service, Select AzureSQLMaintenance store procedure

Import parameter, for operation parameter, provides the value "ALL".
Parameter value represents the database task it will perform. Blog

ALL = Run through all indexes and statistics and take a smart decision about steps taken for each object
STATISTICS = run smart maintenance only for statistics
INDEX = run smart maintenance only for indexes

All other parameters can be removed from the setting tab.