1
COMSLIVE Project: Wonderland Stress Test 1:12/2/10
Aims
Main aim: Find any problems that may arise from multiple users (>10) connecting to a Wonderland system, that are not present when few users are connected (<5).
Find out how many users can connect to a world, and gather data on bottlenecks and problems that would stop more users connecting.
Secondary aims: As this will be the first time the Toshiba laptops are all connected and the first time all the ports in the rooms 351, 353 and 355 are used, report any problems that may occur.
Method
Server setup:
Node1 will be hosting an empty world, OWL version preview 3, with evolver module installed.
Node2 will be hosting a virtual ward with about 60 3D objects, also preview 3.
Clients will be set to cap frame rate at 30fps (default) and the frame rate monitor displayed, ensuring that a headset is used, or audioturned off to avoid echoes/background noise.
We will communicate during the test using both OWL’s chat box, and IRC (freenode.net #bcuwonderlandtest). Freenode has its ownweb chat, so to use it only a browser will be needed.
- First we will set up the laptops in the designated rooms and everyone will log into Node1 with a basic avatar.
- We will then change to evolver avatars and check for any drops in frame rate or any loading issues.
- Then we will log onto Node2, using basic avatars, watching for asset loading issues.
- Check for any drop in frame rate or loading issues when we switch to using evolver avatars.
- We will then attempt to log in all clients simultaneously, watching for loading issues.
Data from clients will be in the form of frame rate drops, and console output.
Servers are monitored, and records can be viewed after the test.
Client Specifications
Toshiba Laptop:
Processor / Intel Core Duo CPU T9400@ 2.53GHz 2.53GHzInstalled RAM / 3.00GB
Sys Type / 32-bit Operating sys
Operating sys / Win 7
Video Card / ATI Mobility Radeon HD 3400 Series
Results
The client’s performance during the test was mostly favourable with a few exceptions.
There was no problem with having multiple(15)users online. Not all of the laptops were available as a few didn’t have enough charge, and the power outlets nearby were not active.
The frame rate was limited to 30fps, and only dropped below that during asset/avatar loading, during which a pronounced stuttering could be observed. This was noticeable when entering Node2, as a large number of assets were required to load. Having more avatars may exacerbate the issue, but it was not a major problem with 15 clients.
In the empty world (Node 1) all evolver avatars loaded correctly and were visible to all. However in the ward area an‘OutOfMemoryError’ exception was thrown with more than 8 evolver avatars, meaning new avatars were not displayed, and occasionally caused a client crash. Tests need to be carried out on the new multimesh evolver avatars to see if the same problem occurs.
Voice appeared to be fine, and of a good quality, however due to many of the laptops transmitting, there was a pronounced background noise. Ensuring that each user was equipped with a headset would resolve this.
X11 was not tested due to problems with X11 on Preview 3, and time constraints. Testing X11 must be a major part of the next test.
The Servers responded very well during the test. Maximum CPU usage for Node1 was just 2.18%, and for Node2 it was just 3.93%.
Figure 1: The CPU graph for world node Hatter (Node1).
Figure 2: The CPU graph for world node WhiteRabbit (Node 2).
Figure 3: The CPU graph for SAS.
Maximum network traffic for Node1 was 4.34Mbs-1 inbound and 17.12 Mbs-1 outbound. For Node2 it was 6.64Mbs-1 and 19.70Mbs-1.
On a 1Gbs-1 connection, that is a utilisation of less than 2%, meaning the platform could be stretched much further.
Figure 4: The bandwidth graph for world node Hatter (Node1).
Figure 5: The bandwidth graph for world node WhiteRabbit (Node 2).
Figure 6: The bandwidth graph for SAS.
Analysis
The fact that the clients ran out of memory when multiple evolver avatars were in world means that the 2048x2048 pixel textures for avatars must be too large. Reducing these texture sizes down to 512x512 will free up a lot of memory (16 512px textures take the same memory as one 2048px texture) and should not deteriorate the look of the avatar.
The low network utilisation seems to suggest that we could have many more users connected; however, there may be bottlenecks elsewhere in the network.
Also, the low server CPU is promising, but implies that any problems we’ve had with getting lots of avatars/assets in world have likely been problems on the client side. The next test will need to include data from client monitor and process explorer/task manager, in order to evaluate client load.
Setting up the laptops took longer than expected, and while the Ethernet ports in the rooms worked fine, not all of the power sockets worked, so a few laptops ran out of battery before or during the test.
Client Bandwidth:
Inbound (upload bandwidth) / Outbound (download bandwidth used)Hatter / 4.34/15=0.2893333Mbs-1(mega bits per second)
*8*60 = 138.87MB/min(Mega Bytesper minute) / 1.1413333 Mbs-1
547.83998 MB/min
WhiteRabbit / 0.4426666 Mbs-1
212.48 MB/min / 1.3133333Mbs-1
630.40MB/min
Alice (SAS) / 1.139333 Mbs-1
546.87 MB/min / 8.0366666 Mbs-1
3857.60 MB/min