Siebel File System Cleanup
1 Introduction 2
1.1 The Siebel File System 2
1.2 Using the Siebel File System Cleanup Utility 2
1.3 Using the SFSMKDIR Utility 4
2 Procedure 5
2.1 Stop Siebel Server 5
2.2 Backup 5
2.3 Report 6
2.4 Repeated Cleanup 6
2.5 Dividing 6
2.6 Start Siebel Server 6
1 Introduction
1.1 The Siebel File System
The Siebel File System consists of a shared directory, which is network-accessible to the Siebel Server that contains the physical files used by Siebel clients. To gain access to files, Web clients connect to the appropriate Siebel Server to request file uploads or downloads. The Siebel Server then accesses the Siebel File System using the File System Manager (FSM) component. File System Manager processes these requests through interaction with the Siebel File System directory.
Files stored in the Siebel File System are compressed at the Siebel Server-level and named according to the format <TABLENAME>_<ROW_ID>_<REVISION#>.SAF. (Siebel Attachment File size displayed in the GUI represents the size of the compressed .saf file, not the actual file size.) The Siebel File System storage location of the compressed files is set by the Siebel Enterprise Server parameter Siebel File System (alias FileSystem). The files stored in the Siebel File System are not directly accessible by users and must be decompressed and returned to the user through the Web client.
1.2 Using the Siebel File System Cleanup Utility
Sometimes, for example due to bad synchronizing, an expected file does not exist for a database record. On the other hand the File System can contain files that are not used (anymore) by Siebel, for example files with an old revision number. These files do not correspond with a database record and are called orphan files.
The Siebel File System Cleanup Utility can be used to verify the integrity between database records and files in the Siebel File System, or to remove files from the Siebel File System that no longer have a corresponding database record. It is a command-line utility, named sfsutl.exe in Siebel 6 or sfscleanup.exe in Siebel 7, located in the bin subdirectory within the Siebel Server root directory. The sfsutl.exe utility processes every file in the file attachment directory and performs one of several operations to each file depending on the file type and the parameters that you set. For descriptions of the run-time parameters that you can set when running sfsutl.exe, see Table 1. For descriptions of the file types and the associated operation that will be performed by sfsutl.exe during processing, see Table 2. Run sfsutl.exe after running siebenv.bat (for setting the Siebel environment variables) and use the parameters listed in Table 1 as shown in the following example:
sfscleanup /u sadmin /p password /f \\server\files /x \\server\sfscleanup.log
Tabel 1. sfsutl.exe Parameters
Parameter / Value / Description / Required?/u / Username / Username ID. / Y
/p / Password / Username password. / Y
/c / ODBC data source / Set this value to the ODBC data source. Default value is set to the environment variable, SIEBEL_DATA_SOURCE. / N
/d / Siebel table owner / Set this value to the Siebel table owner. Default value is set to the environment variable, SIEBEL_TABLE_OWNER. / N
/f / Path for file directory / Set this value to the path for the file attachment directory. Do not append att to the file attachment directory path. / Y
/x / P ath for output file / Set this value to the path for the output file. / N
/m / Path for move directory / Set this value to the path for the directory where discarded files will be moved. / N
/n
(since Siebel 7) / Remove old revisions / Determines if old versions of file attachments will be removed. To remove old versions, set this value to Y. Default value is N. / N
/r
(since Siebel 7) / Generate report file only / Set this value to Y to generate only a report file. If set to Y, the report file will contain only the columns File Name and File Type. Default value is N. / N
/g
(since Siebel 7) / Garbage files / Set this value to remove garbage or non-Siebel files. Default value is N. / N
If you specified an output file using the /x parameter, sfsutl.exe generates a log file listing the operations that were performed. The output file is a tab-delimited text file that contains the following columns:
· File Name
This column lists the name of each file that was processed.
· File Type
This column lists the type of each file that was processed. Table 2 lists the possible file types and the associated operation that will be performed by sfsutl.exe during processing.
Tabel 2. File Types and Associated Operation
File Type / Description / OperationCURRENT / The file has a corresponding record in the file attachment database table. / KEPT
NEW / The file is less than one hour old. Sfsutl.exe will not check for the file in the file attachment database table. / KEPT
ORPHAN / The file does not have a corresponding record in the file attachment database table. / DELETED
I NVALID / The file (or directory) is not a file attachment. If sfsutl.exe is attempting to delete a subdirectory that is not empty, the operation will error out. This gives you an opportunity to review the files contained within the directory before deletion. / KEPT
ANCIENT / The file has an associated record in the database with a different revision number. / KEPT
· Operation
This column lists the type of operation that was performed during processing. Table 3 lists the types of operation that sfsutl.exe may have performed during processing.
Tabel 3. Operations
Operation / DescriptionKEPT / The file was kept.
DELETED / The file was deleted.
MOVED / The file was moved to the directory specified by the /m parameter. Files will only be moved if you used the /m parameter.
KEPT_DIR / The item was kept because it was a directory and requires manual processing.
KEPT_ERROR / The file was kept because an error occurred while trying to move or delete the file.
Note that the sfsutl.exe utility removes only a few files at a time, and needs to be re-run several times in order to remove all orphaned files. This behavior is being addressed in Product Defect 12-4OI6JS. (Please, refer to Service Request 38-276688359 on Siebel SupportWeb.)
1.3 Using the SFSMKDIR Utility
If the Siebel File System contains a large number of files, a decrease in performance can be expected. NTFS supports volumes with large numbers of files and folders, but Microsoft warns us to avoid putting a large number of files into a folder if you use programs that create, delete, open, or close files quickly or frequently. They suggest as solution to logically separate the files into folders so that the workload can be distributed on multiple folders at a time. (See http://www.microsoft.com/technet/treeview/default.asp?url=/TechNet/prodtechnol/winxppro/reskit/prkc_fil_punq.asp.) Microsoft Research and third-party experts were contacted and they considered 20,000 files per subdirectory as good middle ground (SupportWeb SR 38-258916951, "Sizing Siebfile Subdirectories".)
The SFSMKDIR utility allows you to create sub-directories in the Siebel File System (SiebFS) in order that you can avoid the Windows limitations on the maximium number of files in a given directory. The subdirectories which are added will also increase the speed of access to files. If your file system currently consists of a single directory, the SFSMKDIR utility will help you create a set number of directories underneath this main file system root directory. For example, if the current file system directory is d:/siebel/FS, running "sfsmkdir d:/siebel/FS 10" will create 10 sub-directories: d:/siebel/FS/A, d:/siebel/FS/B... d:/siebel/FS/J. The attachment files will be added across the subdirectories to keep the sizes balanced.
There is no maximum size for each subdirectory that is set by Siebel - the directories can grow as large as the operating system allows. The customer currently does not have any control over which subdirectory is used for which file. This is strictly controlled by the algorithm generated. The algorithm is stored in a binary file called siebfile.cfg and is placed in the root directory of the Siebel File System. This algorithm governs where files are stored and how they are located, and is solely based on the file names and ROW_IDs contained in the name. Statistically, each subdirectory will contain about the same number of files, but there may be some fluctuations.
Having re-configured your file system in this way you can revert to using the single root directory of the filesystem simply by re-running the sfsmkdir utility but instead of specifying a number of sub-directories, you set the command parameter to 0 instead, i.e., "sfsmkdir d:/siebel/FS 0", which will move all of the files back to the root directory.
2 Procedure
I suggest to use a Siebel File System Cleanup procedure as follows:
2.1 Stop Siebel Server
Before starting the cleanup action it is wise to make the Siebel Application static by stopping te Workflows and the Siebel Server. This way there is no contact with the database, which could interfere the cleanup action. Furthermore, you can note down the Row_ID of some existing Siebel Attachments as test candidates for later testing.
2.2 Backup
You can consider making a backup of your Siebel File System by copying the shared directory. A restore can be done by simply copying the shared directory back. Because this backup step works the same way as the sfsutl.exe with the Move parameter, it seems unnessessary to me if you trust the sfsutl.exe Utility.
2.3 Report
A report step can be considered, but seems to me unnessessary, because you can make a report and a cleanup at the same time if you enable logging of the cleanup. The report step is run as follows:
sfsutl /u sadmin /p password /c Srvr_ent /d siebel /f f:\siebfile
/x sfsutl_report.txt
2.4 Repeated Cleanup
Only the orphan files can be cleaned by moving them to another directory, for example \siebfile_ discards. Thereby, the number of files decreases which could increase System performance. In case of mistakes, a restore can be done by putting back the missing Siebel File. The Cleanup is performed by the following command:
sfsutl /u sadmin /p password /c Srvr_ ent /d siebel /f \siebfile
/m \siebfile_discards /x sfsutl_move_<move#>.txt
The Cleanup is repeated until the number of removed Siebel Files does not increase anymore. This can be monitored in the logs by looking at the number of files with Status "DISCARD" and/or by counting the removed Siebel Files in their directory.
2.5 Dividing
This step is only needed when the number of files is so high, that it decreases System performance. (For details, see 1.3 "Using the SFSMKDIR Utility".) When performed, you can check if the SFSMKDIR Utility did create the file siebfile.cfg in the root of the Siebel File System. I suggest to divide the Siebel File Systems in as many subdirectories, that each subdirectory contain about 20,000 files.
2.6 Start Siebel Server
After checking the logs of the previous steps for errors, the Siebel Server and the Workflows can be restarted. A test that the noted test candidates are still accessible from the Siebel Server is highly recommended, even as is a test that a new Siebel Attachment can be made correctly. The latter can be tested by looking for a new file in the Siebel File System with the format <TABLENAME>_<ROW_ID>_<REVISION#>.SAF, where the <ROW_ID> is equal to the Row_ID of the record of the new Siebel Attachment in the Siebel Client.