WAS Profiling with Purifyplus

WAS Profiling With PurifyPlus

Web Application Servers in general and WebSphere WAS specifically can be profiled with PurifyPlus as can any other Java application. However, the complexity of WAS, its startup sequence and the vast number of classes involved in the WAS infrastructure imply some restrictions and certain ‘know-how’ involved in enabling and configuring PurifyPlus for WAS based applications. This document contains a brief summary of the profiling steps, options and data acquisition techniques applicable to PurifyPlus in the WAS environment. Although this document considers PurifyPlus in the WAS context, the information provided is not limited to WAS (except where WAS is referenced directly) and may be applied to other Java applications.

How to enable profiling

Java profiling in McKinley and Baltic versions of PurifyPlus is built around a JVMPI agent. The PurifyPlus implementation of a JVMPI agent is shipped in the module named PureJVMPI.dll which may be found in the Common folder for IBM Rational products.

If you are not sure where to find Rational Common and PurifyPlus home folders, you can use the utility named ‘PurifyMD.exe’. Start it simply by typing purifymd in the command prompt. In the ‘System Information’ pane you will see these PurifyPlus related folders. For example:

rational.common = C:\Program Files\Rational\common

rational.purifyplus = C:\ Program Files \Rational\PurifyPlus

computer.name = MyWasMachine

It is important to have Rational Common folder in the system PATH environment, or PurifyPlus may fail to initialize.

PurifyPlus for Java supports three types of profiling:

· Memory distribution and leaks (Purify)

· Java byte code performance (Quantify)

· Java code coverage (PureCoverage)

Each of these profiling types can be applied separately to any running Java program. You can not combine two or more types of profiling in a single run.

Every profiling type is associated with a certain type of PurifyPlus Server. PurifyPlus Server is a special process that collects information from the running PureJVMPI agent and reports it in the UI or saves it in the appropriate persistent dataset:

· .PMY file for Purify

· .QFY file for Quantify

· .CFY file for PureCoverage.

You can start the PurifyPlus server by entering a command corresponding to the profiling type (Purify, Quantify or Coverage).

There are three ways to enable WAS profiling:

· Use PurifyPlus command line

· Modify system environment to enable JVMPI

· Modify Java command line options in the server configuration

Each of these options has advantages and shortcomings.

The first option is the easiest to use. You don’t need to undo any changes to switch back to the normal (non-profiling) mode. You can specify additional PurifyPlus options in the same command. However, if you use this option, you will get two profiling sessions running. The first session belongs to the WAS starter process. This process creates a new JVM which will host WAS, it sets up the environment, generates Java command line from the server settings and makes sure the WAS process got initialized properly. After that the starter process terminates. Since you apply PurifyPlus command line to the starter process, both WAS and the starter process will be profiled.

The second option is good in situations where you can’t use the command line, for example, if you want to enable profiling of JVM hosted by a system service. If you specify the environment settings in the SYSTEM environment, then all Java processes that start in your system will run with profiler attached.

The third method is preferred when you target a specific server on a specific WAS node. The best way to apply configuration changes is to use the WAS administrative console. It is possible to edit the server configuration file directly. However, if configuration was cached, these changes may not apply until you restart all active WAS nodes.

As I mentioned with the first approach (using PurifyPlus command line) it is easy to specify additional PurifyPlus options in the same command line. IS it still possible to use additional PurifyPlus options when PurifyPlus command was not used to start WAS profiling? It is. You still can specify PurifyPlus options in the ‘Settings/Default Settings’ dialog. On the ‘Files’ page you will find the ‘Additional options’ box where you can specify same options as you would in the command line:

Let’s review each of these options in more detail:

1. Use PurifyPlus command line (recommended)

*** Something about why this is recommended here, maybe? *****

*** Also this section needs brief description of why you’ll see multiple runs ****

This method requires using the appropriate command line for each type of profiling:

purify /java startserver.bat <server name> Enables Java memory profiling.

quantify /java startserver.bat <server name> Enables Java performance profiling

coverage /java startserver.bat <server name> Enables code coverage profiling

As you may see in all three cases the command line includes the startserver.bat (WAS startup sequence initialization). Therefore all rules that apply to using the startserver.bat command are enforced. You should set the WAS binary folder as the current folder, you should use the existing server name, and you may use other commands and environment variables accepted by the startserver.bat script. Make sure your WAS startup sequence doesn’t engage any other JVMPI or JVMDI related agents, or they may interfere with the required PurifyPlus agent (PureJVMPI.DLL).

There are additional PurifyPlus command line options you may choose to use in your profiling session:

· -save-data=<profile name> This option will start PurifyPlus server without the UI. This mode is also known as the ‘headless’ operation. After completion the appropriate profile file will be created and you can reopen it with PurifyPlus

· -save-data=<profile name>%d A slight variation on the previous option. Creates numbered versions of the profile file without overriding the existing files. This option is very practical in unattended batch operations. For example, when PurifyPlus is used in an automated test system.

Later in this paper I will mention additional options that may help you to reduce memory overhead inflicted by PurifyPlus.

2. Modify the environment before starting WAS

*** I might use this option, why? ****

The Java Virtual Machine recognizes an environment variable for adding options that were not specified in the command line. The name of this variable is vendor specific. The IBM JVM which is commonly used with WAS uses the IBM_JAVA_OPTIONS. The Sun JVM uses _JAVA_OPTIONS variable. You can set the appropriate variable in the environment hosting the WAS JVM. These are the values to be set:

PPLUS_HOME=<PurifyPlus home folder>

This value should be set to the location of PurifyPlus installation folder. If you are not sure about the location, use the value reported by PurifyMD in the ‘rational.purifyplus’ field (see above).

IBM_JAVA_OPTIONS=-XrunPureJVMPI:<Profiling Type>

-Xbootclasspath/a:%PPLUS_HOME%\Rational.jar

Where <Profiling Type> is Purify |Quantify|Coverage

If you start the WAS server using the Startserver.bat script, you may set these variables in the user environment or in the environment of the command shell where you start the script. If you want to apply these settings for a WAS server running as a system service, you have to add these variables to the system environment and reboot your system. Note, if you set these variables in the system environment, then any Java process started in your system will be running with the profiler attached. If you are planning to run JVMs other than one hosting the WAS, consider option 3.

3. Change Java command in the server configuration file

*** Mention of clustering, caches and how details are beyond the scope of this article? ***

When the Java process for hosting WAS gets started, it gets its command line options from the server configuration file. This file exists for every WAS node and every WAS server. The location and name of this file is defined like this:

<WAS root>\profiles\<your active profile>\config\cells\< cell id>\nodes\<node id>\servers\<server name>\server.xml

For example, if your WAS installed in C:\WAS, your profile name is ‘MyProfile’, cell id is ‘MyCellId’, node id is ‘MyNodeId’ and the server name is‘MyServer’, then the configuration file path is defined like this:

C:\WAS\profiles\MyProfile\config\cells\MyCellId\nodes\MyNodeId\servers\MyServer\server.xml

The best way to change the profile is using the WAS administrative console. However, if you do not have access to the console or do not have sufficient rights to edit the server configuration, you may edit the XML file directly. You should locate the jvmEntries element in this file and change the appropriate VM options by adding the JVMPI loading option: -XrunPureJVMPI:<Profiling Type>. For example:

After saving the file or applying these changes from the administrative console you should restart the server process. After the new instance of WAS starts, it will run with the appropriate profiler attached.

To enable PurifyPlus profiling in the WAS administrative console, you should log in as administrator, navigate to the following location:

Application servers >[ server name] > Process Definition > Process Execution > Java Virtual Machine

And in the ‘Generic JVM Arguments’ box add the following string:

–XrunPureJVMPI:<Profiling Type>

Important: If you want to take advantage of the line level profiling, you will have to add this line: -Xbootclasspath/a:$(PPLUS_HOME)\Rational.jar

Of course the PPLUS_HOME value should be properly defined.

Specific notes for WAS running as a system service.

There are two things to remember about the system services:

1. They usually do not have a UI enabled console, also known as WinStation.

2. They run in a security context different from registered user; usually in the “NT AUTHORITY\SYSTEM” context.

If you just enable profiling for a service, it will try to create a PurifyPlus Server process as soon as the JVMPI agent gets loaded. If you do not change the default settings, this process will inherit the ‘headless’ console and the service security context. This may stop PurifyPlus from initializing properly.

The easiest workaround is to start an instance of the PurifyPlus server before you enable profiling and restart the WAS hosting service. If JVMI agent detects that there is already an instance of PurifyPlus server running, it doesn’t create a new one, but rather opens communication channel with the existing server.

If you start PurifyPlus server from a remote terminal session and plan to hook it up with a system service, you should also set the additional environment variable in the SYSTEM environment:

PURE_USE_GLOBAL_NAMESPACE=1

Processes started from remote terminal sessions by default create system objects in the remote session specific namespace. At the same time, the PurifyPlus JVMPI agent loaded into the service will be creating system objects in the ‘BaseNamedObjects’ namespace. This will make the existing PurifyPlus server invisible to the JVMPI agent. The environment variable above helps to overcome this problem by instructing both the agent and the PurifyPlus server process to create named objects in the ‘Global’ namespace.

Reducing the memory and performance overhead

Reduce the amount of tracked classes by applying prefilters

Each class and each method tracked by PurifyPlus allocates a certain amount of memory for the tracking structures. The size of this memory varies for different types of profiling, with Coverage taking the least amount of memory and Purify taking the most. J2EE applications running on WAS tend to load tens of thousands classes and create a very large number of objects. To limit the amount of traceable objects, PurifyPlus offers the ability to prefilter unnecessary classes. The prefiltering dialog can be reached from the main menu in the “Settings:Default Settings” or “Settings:Executable Settings” dialog, “PowerCheck” page. Click the “Configure” button in the “Java/Managed” section, and it will open the “Class Instrumentation” dialog. The better name would be “Class Tracking” as this dialog defines which classes are going to be tracked by PurifyPlus and which are going to be ignored (see the picture below).

The less classes you track, the more memory you save. If you know exactly what classes to track, you should use the “Selected Classes” option and populate the list with definition of classes you want to profile. You may use regular expressions in the class specification.

If you are not sure what classes to profile, it is recommended to start with predefined prefilters in the “All Classes” list. It will exclude most of the system and the WAS internal classes thus leaving more space for the user-defined code.

Important: If you use Purify to profile Java memory, you should set the -memprof-track-filtered-method-objects=OFF. This will reduce the amount of memory allocated for tracking the garbage collected heap. When this option is set to ON, Purify will track all allocated objects, even those allocated by the prefiltered classes.

Trim down the dataset

Another way to reduce PurifyPlus memory overhead is to run it in the ‘Function’ mode. In this mode you can not get annotated source, but other than that you will have access to the same profiling data. It is possible to switch back to the “Line” mode after you collect enough information about your code to adjust prefilters and have more memory available for tracking the line level information. To switch before ‘Line’ and ‘Function’ mode use the ‘Powercheck’ page of the ‘Default Settings’ or ‘Executable Settings’ dialog.

Purify for Java has probably the highest memory overhead out of all PurifyPlus tools. In order to track allocated objects and display object information and object reference graphs, Purify needs to keep tracking allocation for each object allocated by the profiled Java program. The tracking information may be comparable in size or even bigger than the object itself. This means that total virtual memory demand can more than double if you collect the detailed tracking information for all objects. The first step in limiting the memory overhead is using -memprof-track-filtered-method-objects=OFF option in conjunction with managed code prefilters as was shown above. However, there a few more little known Purify options that can help you to reduce the memory footprint:

· memprof-generate-calllgraph=no. This will turn of the call graph generation, so you will not be able to see memory distribution along the cal graph, however this will give some additional space for the object tracking information. You may turn on call graph generation in a separate run, where the object tracking information is disabled. You can later combine information from these two runs to get the complete picture.

· memprof-object-dump=0. This will disable object dump, hence removing all the object information from your results. You can use it to generate just the call graph information. This option can be used complementary to the previous one.

· Memprof-objecg-gcage=no. Do not keep object age in the tracking data. Saves four bytes per object tracking structure.