The FMS Parameter IOTYPE=2 provides the capability of transferring data directly between memory and disk, bypassing the intermediate buffer cache. These transfers can improve performance when the problem size exceeds the amount of physical memory.

By default on most systems, write and read operations first move the data to/from an intermediate buffer. Physical transfers to/from the disk occur as required under the control of the operating system.

If the problem size is less than the amount of physical memory, this intermediate buffer cache can reduce the number of physical transfers to disk. However, in this scenario, the FMS Parameter INCORE provides better performance by using the memory directly as an FMS incore file, which eliminates the moves to/from the buffer cache.

If the amount of data to be moved is larger than the amount of memory, the movement to the intermediate buffer is an unnecessary step, degrading performance. This direct I/O feature performs the write or read operations directly between memory and the physical disk device.

The FMS interface for direct I/O is at the most efficient point in the operating system. Because FMS communicates directly with the disk controller, the following restrictions are placed on the transfers:

  1. The transfer must start on an aligned memory address.
    This is usually a page boundary.
  2. The transfer must start on an aligned disk address.
    This is usually a disk sector, although the effective sector size may be increased when RAID controllers are used to stripe several physical disks.
  3. The transfer length must be an integer multiple of disk units.
    This is usually a disk sector.
    FMS automatically increases the transfer length if required to satisfy this condition.
    You should check for undesirable side effects as discussed below.
  4. The transfer length must not exceed a maximum length.
    FMS automatically breaks up transfers that exceed the maximum length.

When a FMS file is opened for Direct I/O, FMS determines the physical alignment and length requirements for the transfers. If the default sector size LSECTR does not satisfy these requirements, a fatal error is generated and the correct value of LSECTR is printed. This value is dependent on the specific disk configuration of your system and cannot be determined until the file is opened. If you encounter this error, set the correct value of LSECTR in the license file and rerun the job. This value of LSECTR will be valid for all files opened on that file system.

FMS automatically satisfied all the conditions for Direct I/O on transfers performed internally within FMS.

When you perform transfers to/from FMS files using FMSWRT or FMSRED with direct I/O, you must assure that your transfers satisfy the above (4) conditions. The easiest way to satisfy the memory alignment condition is to allocate the memory from the FMS Memory Management system. All allocated memory will automatically start on an aligned memory address. If you use this address as the start of an array, then condition (1) above will be satisfied. In general, if you use an array dimensioned in your application, this condition will NOT be satisfied. You will need to compute an offset into your array where the alignment is satisfied and start the transfers at that address.

The second condition, aligned disk address, will be satisfied if you read/write whole disk records. FMS automatically starts each record on an aligned disk address. If you are reading part of a record, then you probably will need to read/write more than the required data to satisfy this alignment requirement.

The third condition, transfer length, is automatically handled by FMS. By default, FMS automatically increases the length of transfers to satisfy this condition. However, this can have undesirable side effects if your application has not allocated sufficient memory corresponding to the increased length. For FMSWRT operations, you may overwrite useful data on disk if you are transferring part of a record. It is also possible, though unlikely, that the added transfer length will fall outside your address space, causing a segment violation.

For FMSRED operations, you may overwrite data in your application which lies beyond the array you are reading.

The best way to avoid these undesirable side effects is to allocate sufficient memory to hold a copy of the disk image size of the record. The record length for each file type is computed by FMS when the file is opened. This record length has been rounded up from the minimum required to satisfy the length and alignment requirements. The table below lists the file attribute which stores this record length for each file type where direct I/O is available:

File Record Length Computed By FMS
FILE TYPE RECORD LENGTH
Vector File LUX(4)
Matrix File [L] or [U] LUA(5)

Note that relatively small files, including the matrix diagonals, are never transferred using direct I/O.

You may also direct FMS NOT to automatically increase the transfer length by specifying IOWARN=2.

The fourth condition, maximum transfer length, is automatically satisfied by FMS without any undesirable side effects.

If you use FMSROW, FMSCOL or FMSPUT to write data to a FMS matrix file, FMS automatically satisfies the conditions for direct I/O.

If you do not satisfy the conditions, the result is machine dependent. On some systems, a fatal error will occur. On others, the I/O operations on the file will degrade to buffered I/O.

You may use the FMS Parameter IOWARN to check your transfers for direct I/O. To check all the conditions, include 

IOTYPE=2
IOWARN=28

in the license file. All transfers which do not satisfy the conditions will print a warning message containing DIO.

It is normal for LENGTH warning messages to occur when FMS automatically increases the length of the transfer. You should assure that these increases do not producce undesirable side effects as discussed above. By letting FMS round up the transfer lengths, you avoid transferring extra data when buffered I/O is used.

Direct I/O is determined by the value of IOTYPE at the time that the file is opened. While IOTYPE is usually set in the license file to control all appropriate files, you may set it by calling FMSIST within your application before opening an individual file.

This is a high performance option worth including. The code changes required are usually minimal.

EXAMPLE

The following example shows how to write a FMS vector file containing NUMVEC vectors of COMPLEX*16 data. 
C	Initialization for FMS memory management:
	POINTER (CMD_PTR, CMD)
	COMPLEX*16 CMD(0:1)
	CALL FMSIGT ('MEMPTR', CMD_PTR)

C	Open the file:
	CALL FMSOV (NUMEQ, 2, NUMVEC, 'LUX', LUX)
C	Find the record length:
	LENREC_R8  = LUX(4)
	LENREC_C16 = LENREC_R8/2

C	Allocate space for 1 record in memory:
	CALL FMSCMG (CMD, LOCREC, LENREC_C16)
C	Note that CMD(LOCREC) starts on an aligned address:

C	Write the vector file:
	LDISK      = 1
	DO NV = 1,NUMVEC
C	   Fill CMD(LOCREC:LOCREC+LENREC_C16-1) with data:
	   CALL FMSWRT (LUX(1), LDISK, CMD(LOCREC), LENREC_R8)
	   LDISK = LDISK + LENREC_R8
	END DO

C	Return the memory:
	CALL FMSCMR (CMD, LOCREC, LENREC_C16)