File processing

After studying this section you should be able to:

• describe methods of processing data stored in files
• explain when each method is appropriate

File processing options

In any information processing system there will be a master file that will contain the data that are to be kept about the system. In most systems the event that causes the master file to be altered is called a transaction. A transaction might be a customer of a bank withdrawing some money, a shop selling something, an order being placed, and so on. The transactions may be processed as they happen or they may be stored up on a transaction file to be processed later.

Update in place

If the master file is stored as an indexed sequential file or a random file it is possible to process the transactions as they occur, i.e. to update in place (sometimes known as, to update in-situ). This method is preferred when the hit rate (that is the proportion of the master file records that are to be updated) is low.

Image

Update in place is particularly suited to a system where the master file has to represent an up-to-date position, for example an airline booking system. Such systems are called real-time systems. When this processing method is used it is common practice to copy the transactions onto a log file as a backup.

Update in place can occur in other situations but it is essential in realtime applications.

KEY POINT - This would appear to be the ideal processing method but it can be quite slow if a large number of transactions are to be processed, so it is used only when no other method is available.

Sequential update

The other common method is to store the master file on a sequential file. If the transactions are placed on a sequential file in the same sequence as on the master file it is possible to process all the transactions in one pass through the files. The system reads the old master file and writes out a completely new master file. This method is preferred when the hit rate is high.

This process leaves us with an old master file and a new master file. The old master file is normally kept together with the transaction file as a backup. We call the old master file the father file and the new master file the son. This process normally continues until there are four levels of files, giving great grandfather, grandfather, father and son files. Common systems using this process are batch systems that gather together all the transactions for a period onto a transaction file, for example a payroll system. Other systems that use this process are online systems that collect transactions directly during the day and process them as a batch overnight. An example of this is a banking system that is only able to give you your balance at close of trading on the previous day.

Image

KEY POINT - This method is very efficient when a large number of transactions are to be processed.