Single Server Scalability
Comparison of Dell PowerEdge 6650 1.6 GHz Quad Processor and Dell PowerEdge 2650 2.2 GHz Dual Processor Systems with Hyper-Threading
Revision 1.0
10/15/2002
John D’Agati
Citrix Systems, Inc.
Table of Contents
Table of Contents
Overview
Results
Understanding the Test Score
Dell PowerEdge 6650
Dell PowerEdge 2650
Test Methodology
Measurements and Calculations
Time
Bytes
KBps
AppScore
FailScore
IterationScore
Standard Performance Data Log Counters
Overview
The Single Server Scalability test is designed to quantify the maximum number of client sessions that can be connected to a MetaFrame server with acceptable performance.
Server / Dell PowerEdge 6650 / Dell PowerEdge 2650Results
(Simulated Users) / 160 ± 1 / 130 ± 1
Results
The results of the Single Server Salability test conclude that the performance of the Dell PowerEdge 6650 servicing 160 concurrent users is equivalent to the performance of the Dell PowerEdge 2650 servicing 130 concurrent users with the variables defined in this test scenario. Extending the number of concurrent users beyond the recommendation in this test environment would result in decreased performance and impact the end user experience on the MetaFrame XP server. When sizing MetaFrame XP servers the number of actual users per server varies based on the applications deployed. Note: Hyper-Threading was enabled on both servers during the test.
Understanding the Test Score
Viewing the score column in the test results is the easiest way to determine the server’s degradation point. For the Single Server Scalability test a score of 80 has been determined as the optimal load for a server. This means that the server has enough additional CPU and memory resource to handle spikes in performance. When the test iteration score drops below 80, additional users added to the server consume more resources causing lower test scores and slower performance.
Dell PowerEdge 6650
Iterations / Time / Bytes Sent / KBps / ICAMark Score / ErrorsStep 1 / 14:29 / 143908.1 / 0.506667 / 101.5387 / 0
Step 2 / 14:28 / 111319 / 0.392667 / 101.6727 / 0
Step 3 / 14:28 / 100787.8 / 0.354444 / 101.6276 / 0
Step 4 / 14:32 / 97297.13 / 0.343 / 101.6498 / 0
Step 5 / 14:29 / 92450.16 / 0.325333 / 101.6181 / 0
Step 6 / 14:28 / 90024.38 / 0.315444 / 101.5763 / 0
Step 7 / 14:26 / 88919.06 / 0.312571 / 101.6449 / 0
Step 8 / 14:30 / 87757.28 / 0.308167 / 101.6103 / 0
Step 9 / 14:27 / 86296.92 / 0.302889 / 101.5524 / 0
Step 10 / 14:26 / 86434.46 / 0.303733 / 101.5575 / 0
Step 11 / 14:33 / 85301.16 / 0.299091 / 101.482 / 0
Step 12 / 14:30 / 84924.84 / 0.297611 / 101.4601 / 0
Step 13 / 14:33 / 84392.58 / 0.295385 / 101.3811 / 0
Step 14 / 14:29 / 83907.72 / 0.293301 / 100.8809 / 1
Step 15 / 14:32 / 84281.68 / 0.292933 / 100.6812 / 0
Step 16 / 14:51 / 83674.29 / 0.287958 / 99.72829 / 0
Step 17 / 14:38 / 83331.05 / 0.285765 / 99.40906 / 0
Step 18 / 15:12 / 83067.93 / 0.281222 / 98.24378 / 0
Step 19 / 15:10 / 82818.7 / 0.276947 / 97.23137 / 0
Step 20 / 15:03 / 82966.25 / 0.2738 / 96.25767 / 0
Step 21 / 15:23 / 82575.41 / 0.26873 / 95.09181 / 0
Step 22 / 15:35 / 82457.97 / 0.265303 / 94.20085 / 0
Step 23 / 15:59 / 82457.63 / 0.261652 / 92.98913 / 0
Step 24 / 16:20 / 82085.99 / 0.253417 / 91.19544 / 0
Step 25 / 16:08 / 81922.34 / 0.24904 / 90.08792 / 0
Step 26 / 16:40 / 81855.64 / 0.243821 / 88.67903 / 0
Step 27 / 17:12 / 81639.75 / 0.239704 / 87.5438 / 0
Step 28 / 17:35 / 81360.61 / 0.236627 / 86.54049 / 2
Step 29 / 17:11 / 78662.46 / 0.22669 / 85.58214 / 0
Step 30 / 18:59 / 77103.19 / 0.22139 / 84.29615 / 4
Step 31 / 12:19 / 78735.75 / 0.222715 / 81.84556 / 23
Step 32 / 13:15 / 74263.24 / 0.219731 / 80.20949 / 71
Step 33 / 13:36 / 76770.95 / 0.218329 / 76.8943 / 88
Step 34 / 12:17 / 78742.63 / 0.216886 / 73.91071 / 99
Step 35 / 12:41 / 77804.47 / 0.20852 / 71.40398 / 106
Step 36 / 18:57 / 75977.11 / 0.194591 / 69.26769 / 124
Dell PowerEdge 2650
Iterations / Time / Bytes Sent / KBps / ICAMark Score / ErrorsStep 1 / 14:31 / 141644.9 / 0.499 / 101.5113 / 0
Step 2 / 14:31 / 134669.4 / 0.4455 / 101.4886 / 0
Step 3 / 14:32 / 111481.8 / 0.392 / 101.4658 / 0
Step 4 / 14:31 / 129265.4 / 0.4455 / 101.4366 / 0
Step 5 / 14:30 / 101318.5 / 0.355667 / 101.4073 / 0
Step 6 / 14:31 / 120373.6 / 0.397722 / 101.3357 / 0
Step 7 / 14:31 / 96181.67 / 0.336917 / 101.2641 / 0
Step 8 / 14:35 / 108277.6 / 0.367319 / 100.9413 / 0
Step 9 / 14:38 / 92715.07 / 0.3216 / 100.6186 / 0
Step 10 / 14:44 / 100496.4 / 0.34446 / 99.57663 / 0
Step 11 / 14:52 / 90981.59 / 0.308056 / 98.53467 / 0
Step 12 / 15:22 / 95738.97 / 0.326258 / 97.1685 / 0
Step 13 / 15:46 / 89256.81 / 0.292286 / 95.80233 / 0
Step 14 / 15:13 / 92497.89 / 0.309272 / 94.54998 / 0
Step 15 / 15:21 / 88023.49 / 0.27925 / 93.29763 / 0
Step 16 / 16:41 / 90260.69 / 0.294261 / 91.20831 / 0
Step 17 / 16:48 / 87051.18 / 0.261037 / 89.119 / 0
Step 18 / 16:49 / 88655.94 / 0.277649 / 88.3113 / 0
Step 19 / 17:00 / 86336.96 / 0.253367 / 87.5036 / 0
Step 20 / 17:03 / 87496.45 / 0.265508 / 86.33535 / 0
Step 21 / 17:15 / 85655.49 / 0.243576 / 85.16709 / 0
Step 22 / 18:27 / 86575.97 / 0.254542 / 84.02889 / 0
Step 23 / 19:14 / 80699.43 / 0.226873 / 82.89069 / 5
Step 24 / 18:46 / 83637.7 / 0.241664 / 81.52284 / 3
Step 25 / 17:35 / 84183.03 / 0.228246 / 80.8119 / 8
Step 26 / 15:11 / 83910.36 / 0.234955 / 80.155 / 34
Step 27 / 13:45 / 83898.49 / 0.206124 / 72.76551 / 113
Step 28 / 15:01 / 83904.43 / 0.220539 / 65.37603 / 107
Step 29 / 15:55 / 86229.91 / 0.201538 / 63.76063 / 125
Step 30 / 15:46 / 85067.17 / 0.211039 / 62.14524 / 162
Server Hardware Configurations
Dell PowerEdge 6650
Quad Processor - 1.6GHz Xeon with 256 KB L2 and 1 MB L3 Cache
35 GB HDD with Dell PERC 3/DC Raid Controller
3.5 GB RAM
4 GB Page File
Step Size: 5 users
Iterations: 36 Iterations
Total Users in this test: 180 users
Script Version: 1.0
Dell PowerEdge 2650
Dual Processor - 2.2GHz Xeon with 512KB L2 Cache
16 GB HDD with Dell PERC 3/Di Raid Controller
4 GB RAM
4 GB Page File
Step Size: 5 users
Iterations: 30 Iterations
Total Users in this test: 130 users
Script Version: 1.0
Client Hardware Configuration
Dual Pentium P3 667 w/256 kb Cache and 256 MB RAM, 9 GB HDD with Adaptec SCSI Controller
Client Software Configuration
Windows 2000 SP2, Citrix ICA Windows 32-bit Program Neighborhood Client Version 6.30.1050
Server Software Configuration
Windows 2000 Server SP2, MetaFrame XP FR2 SP2, Microsoft Office 97, Excel, PowerPoint, and Access
Test Methodology
A standard test begins by launching a set number of client sessions to the MetaFrame server being tested. Once all of the sessions have successfully connected, each of the sessions begins to run a set of scripts. Each script is staggered by one second so that the sessions are not running in parallel. The scripts are timed, the values are compared to a calibration value, and the iteration’s score is computed. The process then begins again, adding more sessions for each successive iteration.
The calibration value was determined by running the scripts on a calibration machine. This machine is considered to perform at the level we expect an ICA Session to work. Each script was run locally, and the data recorded. For this version, the scores were gathered on NEC PowerMate V2133, which is a Pentium 133 MHz, with 32 Megs of memory. It was running Windows 98 SE, and Office 97
The Single Server Scalability tests are conducted using two internally developed tools – ICAMark and Spawn. ICAMark is responsible for ensuring that all sessions have connected before launching and timing the scripts. Spawn is responsible for launching client sessions on the client machines.
ICAMark uses scripts to simulate typical user actions performed using Microsoft Office. The standard scripts that are used are PowerPoint, Excel, and Access. These scripts have a tendency to fail, usually during heavy load when a window does not appear within a specified amount of time.
All the scripts have been modified to fail gracefully when they cannot find a window. If a script failure occurs, the script sets a registry key that identifies where it failed and what the time of that failure was. The ICAMark console computes an estimated ICAMark score, by dividing the calibration score at this fail point by the script’s score at the recorded point. Delta time, delta bytes and KBps is disregarded for a failed script.
Four pieces of data are collected and stored by ICAMark: Delta Time, Delta Bytes, KBps, and ICAMark Score. All values are recorded on an individual basis for each script, and for the iteration as a whole. The delta time is simply the running time of each script, and the delta bytes are the number of bytes sent over the wire to each session. The KBps is calculated by dividing the delta bytes by the delta time (in seconds). The ICAMark score is calculated by comparing a calibration value of the Delta Time, with the time gathered during this iteration.
ICAMark has been configured to launch the Performance Data Log service in order to capture counter data from the Microsoft Performance Monitor. The Performance Data Log must be configured to capture the counter data to be record.
By using Spawn technology, the ICAMark console actually controls the logging in and starting of an iteration. Each iteration is setup as a specified test in the Spawn Server console. The user supplies the sessions and machines that will run them. A Spawn client must be running on each client machine. The ICAMark console signals the Spawn server console, via a Spawn Agent, to launch the iteration.
Measurements and Calculations
There are many calculations performed by ICAMark. Here is a summary of the more complex equations involved and measurements used in ICAMark.
Time
The time is taken using Ctime objects provided by MFC. A start and stop time are taken, and the difference is calculated using a CTimeSpan object.
Bytes
The byte count is measured by taking the output bytes from a WFAPI call and saving them as the start bytes. Then the script is run, and the stop bytes are taken. The difference is recorded.
KBps
This is taken by dividing the delta bytes by the time. The time is converted to seconds using the CtimeSpan.GetTotalSeconds() function.
AppScore
This is the score a script receives after it runs. The ICAMark client calculates it after a run of the script. Calibration time is the time recorded in the registry for each script. It was generated on the calibration machine. Delta time is the time recorded for the script just ran.
FailScore
This is the recovery score that is recorded when a script fails. It is calculated by dividing the calibration fail score for the script by the fail score.
IterationScore
This is the summary of the ICAMark score per iteration. It is the sum off all the app scores divided by the number of users. This score will be calculated at the end of the test. Data is recorded along the way to allow for this calculation.
n = the number of apps being tested.
U = current user.
Users = the number of users involved in an iteration.
AppScore includes both FailScore and AppScore values listed above.
Standard Performance Data Log Counters
This is the list of the standard Performance Monitor counters that are gathered for each ICAMark test.
Memory\Available KBytes
Memory\Free System Page Table Entries
Memory\Page Reads/Sec
Memory\Pages/Sec
Memory\Pool Nonpaged Bytes
Memory\Pool Paged Bytes
Memory\Pool Paged Resident Bytes
PhysicalDisk(_Total)\% Disk Time
PhysicalDisk(_Total)\Current Disk Queue Length
Process(termsrv)\% Processor Time
Process(termsrv)\Handle Count
Process(termsrv)\Private Bytes
Process(termsrv)\Thread Count
Processor(_Total)\% Processor Time
Server\Context Blocks Queued/sec
System\Context Switches/sec
System\Processor Queue Length
Network\Bytes Received/sec
Network\Bytes Sent/sec
Network\Current Bandwidth