Journalling and Commitment Control

Journalling and Commitment Control

Journalling

1 sav03

Introduction

To protect your database from corruption or disk errors it is essential to back it up onto another media - usually tape. The AS/400 provides the commands SAVOBJ, SAVCHGOBJ and SAVLIB to save your files but these may not be suitable. A sensitive database, i.e. one which is changed frequently, may need to be backed up as each change occurs. Every record which has been altered needs to be logged so that it can be recovered in the event of an error. Saving an entire file every time one record changes is clearly impractical.

Journalling is the term given when database record changes are saved separately as they occur. The AS/400 provides 2 objects; a journal and a journal receiver for this purpose.

Receiver

A journal receiver is a special type of file which holds the copy of the changed records. It can be saved to tape or diskette whenever necessary.

Journal

A journal is the object which controls the journalling procedure. It is linked to the database files being changed and to the journal receiver.

Journalling on the AS/400

The AS/400 provides a system-wide journalling facility which enables files to be journalled without reference to the programs or applications which are accessing the files. Each physical file holds a flag to say whether it is being currently journalled and this flag can be set on or off as and when required. Whenever a record is added, changed or deleted in the file, whether by an application program or a utility such as CPYF, the record is written to the journal receiver along with information relating to the file change. This information includes the job, user, time and date stamp, type of operation and program name. Each journal entry comprises 75 bytes of information followed by the record being journalled.

Because the system performs the journalling it is done using far less overhead on resources than if you had written your own journalling procedure.

Threshold

When the receiver is created a threshold size in kilobytes can be specified. When the receiver reaches this threshold a message is sent by the system warning that the receiver is full. (Journalling is not interrupted though - the receiver will continue to grow until a CHGJRN (Change Journal) command is issued to detach the receiver and connect to a new one. This guards against receivers not getting saved regularly.

Twin Receivers

It is possible to journal to twin receivers. Every change is recorded twice - once for each receiver. These receivers can be placed on different disk drives for additional security - see the chapter on Save/Restore.

Before/After Images

You can specify that before- and after-images are journalled. This means for every change of a record the contents before the change and after the change are both stored in the receiver. This is essential if you wish to use the receiver entries to back-out the changes (known as roll back). See the section on Commitment Control.

Other entries

It is possible to journal when the file is opened and closed and also to put your own entries onto the receiver as reference points.

Common Commands

There are a number of AS/400 commands related to journalling. These are the more common ones:

CRTJRNRCVCreate a journal receiver. Receivers have to be created before the journal.

CRTJRNCreate a journal. You must specify receiver(s).

STRJRNPFStart journalling a physical file. The journal and receiver(s) must exist.

STRJRNAPStart journalling access paths. To save time spent by the system rebuilding access paths after a system failure, you can journal the access path changes as well as the records. The system will automatically recreate the access path rather than rebuild, which could prove a heavy overhead if many changes are made, both in storage and CPU.

DSPJRNDisplay a journal. This command can be used to extract selected journal entries and either print them or direct them to a database file.

CHGJRNChange a journal. Mainly to attach new receivers when the current ones are full.

APYJRNCHGApply journal changes. Mainly to bring a restored file up to date when the latest version of the file is lost or damaged.

RMVJRNCHGRemove journal changes (Roll back).

ENDJRNPFEnd journalling a physical file.

ENDJRNAPEnd journalling access paths.

Less common are the following:

RCVJRNEReceive Journal entry. Allows a specified user program to continuously receive journal entries one by one. Useful to save the entries onto another system.

SNDJRNESend journal entry. Allows you to add your own user-created journal entries. NOT the opposite of RCVJRNE!

RTVJRNERetrieve Journal entry. Retrieve a journal entry into a CL program variable.

DSPJRNRCVADisplay journal receiver attributes.

CMPJRNIMGCompare journal images. Compares the before-images and the after-images of record level changes in a member and indicates where differences occur.

WRKJRNWork with journals.

WRKJRNAWork with journal attributes.

DLTJRNDelete journal.

DLTJRNRCVDelete journal receiver.

Setting up Journalling

A typical sequence of events:

•Create a journal receiver on a particular disk drive

•Create another receiver on a different disk drive

•Create a journal which will feed these receivers duplicate information.

•Start journalling a number of physical files through the journal. Specify that both before and after images are to be recorded

•Run the programs which alter those physical files. Each record changed will cause 4 records to be written to the receivers - a before and after image to each receiver

•A message is sent saying the receivers are full. The CHGJRN command is used to attach new receivers

•New versions of the receivers are created with a new generation number on the end of the object name. Otherwise they are identical to their predecessors. The old, full receivers are detached from the journal, the new ones are attached and journalling continues

•The detached receivers are saved to tape and stored offsite for security reasons

•At a particular time, say overnight, the files being updated are saved in their entirety via SAVOBJ or SAVLIB commands. Journalling is maintained during this operation or journalled changes cannot be applied to or removed from the saved version

Types of Disaster

Damaged File

A file becomes damaged and unusable due to a disk problem or internal OS/400 damage. Take the following steps:

•Prevent the users from accessing the database

•Delete the damaged file after deleting dependent logicals

•Restore the latest save of the file (and the dependent logicals)

•Restore, if necessary, the journal receivers covering the time between the save and the corruption

•Apply the journal changes (APYJRNCHG) to bring the save copy of the file up to date

•Restart journalling and release the application to the users

Corrupt File

A program error has been detected which has been corrupting the files from a known date and time.

•Deny access to the file

•Restore, if necessary, the journal receivers covering the time of the corruption

•Back out (reverse) the journal changes for the corrupted data using the RMVJRNCHG command

•Correct the program error

•The files can then be brought up to date by re-entering the changes and/or re-applying selected journal changes

•Restart journalling and release the fileto the users

Machine Failure

The machine crashes and all data is lost.

•The system is restored via IPL, loading SAVSYS and SAVLIB *NONSYS tapes

•The latest SAVLIB/SAVOBJ of the database files are used to bring the database more up to date

•The journal receivers saved since the last SAVLIB/SAVOBJ of the database are restored

•The database files are brought back to date by applying the journal changes held in the journal receivers

•Journalling is restarted and the machine is released to the users

Commitment Control

Commitment control is an extension of the journalling function. It enables you to treat a group of changes, to any number of files, as one transaction. In the event of a job or system failure an automatic roll back is performed by the system on an incomplete transaction.

Such a transaction is defined in the user job between consecutive COMMIT operations. During the transaction all the records used are locked from other users in the normal way. They are released however, only by the next COMMIT (or ROLLBACK) operation. If an abnormal end occurs or the user job performs a ROLLBACK operation then changes made during the latest transaction are backed out of the files.

Setting up Commitment Control

To use commitment control the following conditions are necessary:

•All database files under commitment control must be journalled to the same journal

•Both before and after-images must be journalled. (The system handles this. If only after-images have been specified then when commitment control starts before-images are journalled also. This is to enable roll back to be performed)

•Commitment control must be started via the STRCMTCTL command. This stays in effect until the ENDCMTCTL command is issued. All files opened under commitment control must be closed beforehand. If a COMMIT has not been issued after all the file changes then an automatic ROLLBACK is performed and an escape message is sent

COMMIT and ROLLBACK are CL commands. There are similar operation codes in the High Level Languages.

Record locking

You can specify on the STRCMTCTL command the locking level to be used. The options are:

•*CHG Only records that have been updated retain the record lock until COMMIT or ROLLBACK - the commit boundary. If however a record is read from an update file but is not updated before reading another record from the same file, the lock on the first record is released.

•*ALL Both updated records and records read from input only files retain the record lock until a commit boundary. Further, records read from an update file that have not been updated retain their locks too.

Performance

Journalling creates an overhead on the system. Although it is far more efficient than if you duplicated records to another file when changes occurred, it does reduce performance slightly.

Journal receivers can consume disk space, especially if you journal before and after images to duplicate receivers. Set a reasonable threshold when creating a receiver and save to tape as often as possible.

Journalling access paths can be a boon on large files with few changes. It can be undesirable on small files with frequent changes.

Commitment control adds a further overhead. Apart from the system performing extra functions the record locking can cause other jobs to wait.

The STRCMTCTL and ENDCMTCTL commands affect performance. Avoid ending and restarting unnecessarily in one job.

Job accounting

The IBM-provided job accounting function gathers data so that you can determine who is using your system and what system resources they are using.

Basic statistics are generated automatically by the system and placed in the system history log, a powerful tool accessed using the DSPLOG command, or directly by HLL programs.

It is possible to obtain more information using a special journal object QACGJRN in the QSYS library. There are two optional job accounting functions which place statistics in a journal receiver attached to this journal:

•job resource accounting (CPU time, total running time, database reads/writes, etc.)

•printer file accounting (collects statistics only when spooled files are actually printed)

An accounting code can be placed on a user profile and used for all jobs run under or submitted by this user profile; alternatively, individual jobs may have their own accounting codes. The CHGACGCDE command may also be used within a job in order to, for example, run different menu options under different accounting codes.

To activate these optional functions, the QACGJRN journal must be created, and the system value QACGLVL must be set to *JOB, *PRINT or both.

2 jac01

DO NOT COPY

Journalling and Commitment Control

This page intentionally left blank

DO NOT COPY