Microsoft Biztalk Server 2010 Performance Optimization Guide

Microsoft BizTalk Server 2010 Performance Optimization Guide

Microsoft Corporation

Published: February, 2011

Summary

The BizTalk Server 2010 Performance Optimization Guide contains prescriptive guidance for optimizing BizTalk Server performance, based upon hands-on experience of IT professionals who have worked extensively with BizTalk Server. This guide contains four main sections: Finding and Eliminating Bottlenecks, Optimizing Performance, Scaling a Production BizTalk Server Environment, and BizTalk Server Performance Testing Methodology.

Information in this document, including URL and other Internet Web site references, is subject to change without notice. Unless otherwise noted, the companies, organizations, products, domain names, e-mail addresses, logos, people, places, and events depicted in examples herein are fictitious. No association with any real company, organization, product, domain name, e-mail address, logo, person, place, or event is intended or should be inferred. Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Microsoft Corporation.

Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property.

Microsoft, MS-DOS, Windows, Windows Server, Windows Vista, Active Directory, BizTalk, Excel, SharePoint, SQL Server, Visio, Visual C#, and Visual Studioare trademarks of the Microsoft group of companies. All other trademarks are property of their respective owners.

Contents

Finding and Eliminating Bottlenecks

Best Practices for Avoiding Bottlenecks

Investigating Bottlenecks

System-Level Bottlenecks

Bottlenecks in the BizTalk Server Tier

Bottlenecks in the Database Tier

How to Identify Bottlenecks in the MessageBox Database

How to Identify Bottlenecks in the Tracking Database

How to Identify Bottlenecks in the BAM Primary Import Database

How to Avoid Disk Contention

Performance Tools

Optimizing Performance

Optimizing Operating System Performance

General Guidelines for Improving Operating System Performance

Optimizing Network Performance

General Guidelines for Improving Network Performance

Registry Settings that can be Modified to Improve Network Performance

Optimizing IIS Performance

Optimizing Database Performance

Pre-Configuration Database Optimizations

Post-Configuration Database Optimizations

Optimizing Filegroups for the Databases

BizTalk Server MessageBox Database Filegroups SQL Script

Monitoring SQL Server Performance

Optimizing BizTalk Server Performance

General BizTalk Server Optimizations

Low-Latency Scenario Optimizations

Optimizing MQSeries Adapter Performance

Optimizing Business Activity Monitoring (BAM) Performance

Optimizing Business Rule Engine (BRE) Performance

Optimizing BizTalk Server WCF Adapter Performance

Optimizing Orchestration Performance

Optimizing WCF Web Service Performance

Optimizing BizTalk Server Applications

Optimizing Pipeline Performance

Optimizing Memory Usage with Streaming

Message Considerations

Windows PowerShell Scripts

Scaling a Production BizTalk Server Environment

Scenario Overview

Observations and Recommendations

Key Performance Indicators

BizTalk Server Performance Testing Methodology

Establishing Performance Criteria

Phases of a Performance Assessment

Phase 1: Scoping the Assessment

Phase 2: Planning the Assessment

Phase 3: Preparing for the Assessment

Phase 4: Building the Assessment Environment

Phase 5: Executing the Assessment

Implementing Automated Testing

Why It Is Important to Test

Automating the Build Process

Using BizUnit to Facilitate Automated Testing

Using Visual Studio to Facilitate Automated Testing

Unit Testing

Microsoft BizTalk Server 2010 Performance Optimization Guide

Welcome to the Microsoft® BizTalk®Server2010 Performance Optimization Guide. We created this guide to provide in-depth information for optimizing the performance of a BizTalk Server solution. Full end-to-end performance testing is frequently overlooked during enterprise application deployment. Knowing that Microsoft has built a scalable messaging infrastructure, many organizations that use BizTalk Server spend little or no time conducting performance testing of their own applications. BizTalk Server applications consist of many parts, which may include custom-built components as well as those provided by Microsoft. It is impossible for Microsoft to performance test every possible combination of these components. Therefore, fully and properly conducting a performance test of your application is a critical step of any deployment.

The purpose of this guide is to consolidate and provide prescriptive guidance on the best practices and techniques that should be followed to optimize BizTalk Server performance.

To download a copy of this guide in chm, pdf, or docx form, go to

What's in it?

Generally, the performance of a server is determined by the component that has the lowest performance—the bottleneck in the system. The key to improving performance is being able to identify bottlenecks, determine their cause, and apply the appropriate corrective action.

As you plan your BizTalk Server 2010 deployment, use this guide to help design and optimize your environment. The concept of performance is closely related to the concept of scalability. When you have a solid understanding of the factors influencing the performance of system components, you can deploy components in a way that scales to support periods of high demand.

This guide provides guidance for optimizing performance, based upon hands-on experience of IT professionals who have worked extensively with BizTalk Server. Specifically, this guide includes four main sections:

Finding and Eliminating Bottlenecks: The Finding and Eliminating Bottlenecks section describes various types of performance bottlenecks as they relate to BizTalk Server solutions and information about how to resolve the bottlenecks.

Optimizing Performance: The Optimizing Performance section provides guidance for optimizing performance of a BizTalk Server solution. BizTalk Server performance is closely tied to performance of the platform upon which BizTalk Server is installed. This section provides recommendations for optimizing performance of both BizTalk Server and the BizTalk Server platform.

Scaling a Production BizTalk Server Environment: The Scaling a Production BizTalk Server Environment section provides detailed results of BizTalk Server 2010 performance testing completed by the BizTalk product team. These tests focused on scalability and measured the impact of adding BizTalk Server 2010 computers, the impact of adding BizTalk Server MessageBox databases, and the impact of adding both BizTalk Server 2010 computers and BizTalk Server 2010 MessageBox databases to a solution simultaneously.

When increasing the number of BizTalk Server computers in a BizTalk Server group, for these tests only one BizTalk Server MessageBox database was configured for the BizTalk Server group. These tests focused solely on the impact of adding BizTalk Server computers to a BizTalk Server group.

When increasing the number of BizTalk Server MessageBox databases used by the BizTalk Server group. These tests focused solely on the impact of adding BizTalk Server MessageBox databases to a BizTalk Server group.

When increasing the number of both BizTalk Server computers and BizTalk Server MessageBox databases used by the BizTalk Server group. These tests measured the impact of adding both adding BizTalk Server computers and BizTalk Server MessageBox databases to a BizTalk Server group.

BizTalk Server Performance Testing Methodology: The BizTalk Server Performance Testing Methodology section provides detailed information about how to test and optimize BizTalk Server performance. It includes information about which performance criteria to focus on and the fundamental phases that should be applied when assessing BizTalk Server performance.

Additions to this version of the guide

Using Visual Studio to Facilitate Automated Testing – Describes the use of Visual Studio Load testing to evaluate the performance of a BizTalk Server application.

Acknowledgments

We in the BizTalk Server User Education team gratefully acknowledge the outstanding contributions of the following individuals for providing both technical feedback as well as a good deal of content for the BizTalk Server 2010 Performance Optimization Guide:

Authors

Tim Wieman, Microsoft

Paolo Salvatori, Microsoft

Trace Young, Microsoft

Reviewers

Tim Wieman, Microsoft

Paolo Salvatori, Microsoft

Finding and Eliminating Bottlenecks

A successful BizTalk Server performance assessment is largely a matter of discovering the existence of, and then resolving, bottlenecks to accommodate either more throughput or reduced latency. This section describes various types of performance bottlenecks as they relate to BizTalk Server solutions and information about how to resolve the bottlenecks.

In This Section

Best Practices for Avoiding Bottlenecks

Investigating Bottlenecks

System-Level Bottlenecks

Bottlenecks in the BizTalk Server Tier

Bottlenecks in the Database Tier

Performance Tools

Best Practices for Avoiding Bottlenecks

While the default settings in BizTalk Server provide optimal performance for many hardware and software configurations, in some scenarios it may be beneficial to modify the settings or deployment configuration. When configuring BizTalk Server, consider the following performance guidelines:

To prevent resource contention, isolate receiving, orchestration, and sending on separate hosts. To further minimize contention, isolate the tracking service from other hosts.

If CPU processing on the computer running BizTalk Server is the bottleneck, scale up the computer running BizTalk Server by including additional CPUs or upgrading to faster CPUs.

SQL Server Guidelines

Consider the following performance guidelines when configuring Microsoft SQL Server with BizTalk Server:

Whenever possible, use a fast disk subsystem with SQL Server. Use a redundant array of independent disks type 10 (RAID10/0+1) or a storage area network (SAN) with backup power supply.

Isolate each MessageBox database on a separate server from the BizTalk Tracking database (BizTalkDTADb). For smaller deployments if CPU resources are available, it might be sufficient to isolate the MessageBox database on a separate physical disk from the BizTalk Tracking database.

The primary MessageBox database could be the bottleneck due to CPU processor saturation or latency from disk operations (average disk queue length). If CPU processing is the bottleneck, add CPU processors to the primary MessageBox. If not, try to disable publishing on the master MessageBox database. This way the master MessageBox database can more efficiently handle routing of messages to the other MessageBox databases. The option to disable publishing is valid when you are using multiple MessageBox databases.

If disk operations are the bottleneck, move the BizTalk Tracking database to a dedicated SQL Server computer and/or dedicated disk. If CPU processing and disk operations on the primary MessageBox database are not the bottleneck, you can create new MessageBox databases on the same SQL Server computer to leverage your existing hardware.

Follow recommendations in Optimizing Filegroups for the Databases to isolate the transaction and data log files for the MessageBox and BizTalk Tracking databases onto separate physical disks.

Allocate sufficient storage space for the data and log files. Otherwise SQL Server will automatically consume all of the available space on the disks where the log files are kept. The initial size of the log files depends on the specific requirements in your scenario. Estimate the average file size in your deployment based on testing results, and expand the storage space before implementing your solution.

Allocate sufficient storage space for high-disk-use databases, such as the MessageBox, Health and Activity Tracking (HAT), and Business Activity Monitoring (BAM). If your solution uses the BizTalk Framework messaging protocol, allocate sufficient storage space for the BizTalk Configuration database (BizTalkMgmtDb).

Depending on business needs, such as data retention periods, and the volume of data processed in your scenario, configure the "DTA Archive and Purge" SQL Server Agent job on the HAT-Tracking database such that the BizTalk Tracking database does not grow too large. The growth of this database can degrade performance because reaching the full capacity of the database imposes a limit on the rate of data insertion. This is especially true when one BizTalk Tracking database supports multiple MessageBox databases.

Scale up the servers hosting the MessageBox and BizTalk Tracking databases if they are the bottleneck. You can scale up the hardware by adding CPUs, adding memory, upgrading to faster CPUs, and using high-speed dedicated disks.

Splitting the TempDB files across multiple files may resolve performance issues related to I/O operations. As a general guideline, create one file data file per processor and use the same size for all files created.

Change the database auto-grow settings to a fixed value such as 100-150MB. By default the database growth is configured to 10%, which can lead to delays when growing larger databases.

SQL Server memory should be set to a fixed value by setting both Min Server Memory and Max Server Memory to the same value. In general, allocate 75% of physical memory to SQL Server and leave 25% for the rest of the operating system and any applications. If this is a dedicated SQL Server, you can decrease the amount reserved for the operating system to a minimum of 1GB.

See Also

Finding and Eliminating Bottlenecks

Investigating Bottlenecks

This topic describes a recommended process for investigating bottlenecks.

What is the source of the problem?

The source of the bottleneck could be hardware or software related. When resources are underused, it is usually an indication of a bottleneck. Bottlenecks can be caused by hardware limitations, by inefficient software configurations, or by both.

Identifying bottlenecks is an incremental process whereby alleviating one bottleneck can lead to the discovery of the next one. The science of identifying and alleviating these bottlenecks is the objective of this topic. It is possible for a system to perform at peaks for short periods of time. However, for sustainable throughput a system can only process as fast as its slowest performing component.

Using an iterative approach to testing

Bottlenecks can occur at the endpoints (entry/exit) of the system or in the middle (orchestration/database). After the bottleneck has been isolated, use a structured approach to identify the source. After the bottleneck is eased, it is important to measure performance again to ensure that a new bottleneck has not been introduced elsewhere in the system.

The process of identifying and fixing bottlenecks should be done in an iterative manner. Vary only one parameter at a time, repeat exactly the same steps during each test run, and then measure performance to verify the impact of the single modification. Varying more than one parameter at a time could conceal the effect of the change.

For example, changing parameter 1 could improve performance. However, changing parameter 2 in conjunction with changing parameter 1 could have a detrimental effect and negate the benefits of changing parameter 1. This leads to a net zero effect and results in a false negative on the effect of varying parameter 1 and a false positive on the effect of varying parameter 2.

Testing consistency

Measuring performance characteristics after changing settings should be done to validate the effect of the change.

Hardware - Use consistent hardware because varying the hardware can cause inconsistent behavior and produce misleading results. For example, you would not use a laptop to test performance of a BizTalk solution.

Test Run Duration - Measure performance for a fixed minimum period to ensure that the results are sustainable. Running tests for longer periods also ensures the system has gone through the initial warm/ramp up period where all caches are populated, database tables have reached expected counts, and throttling is given sufficient time to regulate throughput once predefined thresholds are hit. This approach will help discover optimal sustainable throughput.

Test Parameters – Do not vary test parameters from test run to test run. For example, varying map complexity and/or document sizes can produce different throughput and latency results.

Clean State - After a test is complete, ensure that the state of the test environment is clean before running the next test. For example, historical data can build up in the database affecting run-time throughput. Recycling the service instances helps to release cached resources like memory, database connections, and threads. In your test environment, you may want to create and execute the bts_CleanupMsgbox stored procedure as described in How to Manually Purge Data from the MessageBox Database in a Test Environment ( This script is intended to return your BizTalk Server test environment to a fresh state with regards to the Message Box between runs. The script deletes all running instances and all information about those instances including state, messages, and subscriptions, but leaves all activation subscriptions so you do not have to re-enlist your orchestrations or send ports. Note that this tool is not supported on production systems.

Performance Testing and Tuning - The goal of this test category is to maximize performance and throughput of your application and find the Maximum Sustainable Throughput (MST) of your system. For more information about planning and measuring Maximum Sustainable Performance see Planning for Sustained Performance ( and What Is Sustainable Performance? (

The MST is the highest load of message traffic that a system can handle indefinitely in a production environment. All BizTalk applications should be tested for performance and throughput before going into production. At a minimum, you should run a representative set of test cases that represent the most common usage scenarios. We recommend that you test against expected loads and peak loads in a separate environment that matches characteristics of the production environment. This environment should have all of the corporate standard services installed and running, such as monitoring agents, and antivirus software.