By Naitik Thanki
In this post, we will explore one of the services offered at Microsoft Services related to Performance testing and Optimization.
Specifically, the Performance and Quality Services team (part of Customer Success) offers customers to have an application reviewed by our subject matter experts (SMEs) to identify performance issues and provide recommendations to resolve them.
An Application Optimization offering (also known as an AO) typically requires 6-8 weeks. There are four distinct phases for this service which break down performance testing into digestible time-boxed durations.
The four phases of an Application Optimization (AO) offering are the Plan Phase, the Prepare Phase, the Execution Phase, and the Reporting Phase. Below we will review what each of the phases entails so that you know what to expect in an Application Optimization offering.
PLAN PHASE
During the Plan Phase of the Application Optimization, you can expect the subject matter experts to work with the customer team to understand how the application works. This requires interviewing application stakeholders such as product owners, business analysts, and testers that know the application thoroughly or work on the development or testing of it regularly. These resources help us identify the most common actions or steps an end-user might take while using the application.
By comprehending the common actions or processes used, we can create automation test scripts to model the end users’ actions more accurately. Furthermore, we discuss and estimate throughput, load profiles, and various types of tests we can run during the Execution Phase. Afterward, we deliver a final Test Plan document at the end of the Plan Phase, which will serve as the basis for the performance testing to be run and help measure the success of the Application Optimization offering. The Test Plan provides goals or metrics we aim to meet by the end of the service.
PREPARE PHASE
During the Prepare Phase of the Application Optimization, we focus on scripting out the common workflows found in the Prepare Phase. We work with you and your testing team to create the automation test scripts, define the load test profiles, and create a test harness. The automation scripts are based on the Test Plan surrounding the most common user flow scenarios of the application. We help customers identify the preferred performance testing software to use based on the current application infrastructure.
EXECUTION PHASE
During the Execution Phase of the Application Optimization, we begin running our automation test scripts against an isolated Test environment. This Test environment should mirror the Production environment as close as possible so that the test results are more accurate. After running a few diagnostic tests to make sure the scripts and environment are functioning properly, we work with customers to run a baseline test. After analyzing the baseline test results, the subject matter experts (SMEs) from our team suggest changes.
This is an iterative process yielding many Tests of Records (TORs) with the hope that each change improves the application’s performance. Based on the Test Plan, the types of tests can range from load tests, endurance tests, or max capacity. Generally, we run endurance tests overnight as they take longer durations to complete.
REPORTING PHASE
During the Reporting Phase of the Application Optimization, you can expect us to compile the results of the Execution Phase into an Engagement Report. We put together the report and deliver it to you within a week or two of the Execution Phase ending. The report includes a detailed analysis of the Tests of Records and provides a comprehensive list of changes made during the Execution Phase, which should be applied to the Production environment.
Furthermore, the subject matter experts (SMEs) add explanations for the changes. After delivering the Engagement Report to your team, we schedule a meeting to review the findings with the appropriate stakeholders of the application.
How can an Application Optimization offering help you?
An Application Optimization offering can provide insight into your application’s current performance and open opportunities for potential changes that can drastically improve the performance.
There are multiple common reasons for a customer to need to complete performance testing and want to engage with Microsoft for our Test & Attack app optimization methodology. These are some of them:
What requirements do you need to have completed before starting this type of offering?
To ensure a successful Application Optimization offering, the customer generally needs to provide us:
- An isolated Test environment that models the Production environment as close as possible, which would include among other: hardware specifications, # of nodes, software versions, OS/software configurations, disks rates, and IOs, network bandwidth, etc.
- Dedicated resources fully aligned to this engagement:
- During the Plan Phase, we need business analysts and test leads from your team to identify product owners or subject matter experts because we need to interview these resources on the most common user workflows.
- During the Prepare Phase, we need business analysts, test leads, and automation testers to work with us to create the scripts and provide insight on the throughput, load profiles, and test harness infrastructure.
- During the Execution Phase, we need to work with business analysts, test leads, application developers, automation testers, and application subject matter experts while running the tests and making changes to the application code.
I hope you guys liked this offering/service and had a chance to look at some of the stuff we do at the PQS team in Microsoft. If you or someone you know would like to learn more or are interested in this service, please do not hesitate to leave any questions or comments below. Thanks!
Posted at https://sl.advdat.com/3EdG5Gk
