Variable Block Database Headers


Topics:

Overview
Constants
Type Definitions
VBD File Header
VBD Block Header
VBD File Lock Header
VBD Record Lock Header


Overview

Type definitions, constants, and data structures needed to create variable block headers.


Constants

// Constants for dynamic data attributes used by the first byte of the
// block header Status member.
const __SBYTE__ vbBadBlock     = 'B'; // Bad Variable Block
const __SBYTE__ vbDeletedBlock = 'D'; // Deleted Variable Block 
const __SBYTE__ vbNormalBlock  = 'N'; // Normal Read/Write attribute 
const __SBYTE__ vbRemovedBlock = 'R'; // Removed Variable Block

// This block was received from a remote device. Added in VBD revision C
const __SBYTE__ vbRemoteDeviceBlock = 'V'; 

// Constants for block control characters used by the second byte of the
// block header Status member. Added in VBD revision C to handle device
// control commands.
const __SBYTE__ vbAddRemoteBlock    = 'A'; // Add a remote block
const __SBYTE__ vbChangeRemoteBlock = 'C'; // Change a remote block
const __SBYTE__ vbDeleteRemoteBlock = 'D'; // Delete a remote block
const __SBYTE__ vbRequestFailed     = 'F'; // The client request failed
const __SBYTE__ vbCloseConnection   = 'L'; // Close Client/Server connection
const __SBYTE__ vbKillServer        = 'K'; // Shutdown Client/Server
const __SBYTE__ vbRequestBlock      = 'R'; // Requesting a block
const __SBYTE__ vbSendBlock         = 'S'; // Sending raw data block
const __SBYTE__ vbAcknowledgeBlock  = 'W'; // Acknowledge a data block

// Constants for file operations and stream position 
const int vbMaxNameLength     = 255;        // Max length of file names
const FAU vbStartOfFile       = 0;          // First byte in the file
const FAU vbCurrAddress       = -1;         // Indicates current location
const vbUINT32 vbCheckWord    = 0x0000fefe; // 32-bit Check Word
const vbINT32 vbFSListCorrupt = -1;         // Free space list is corrupt


Type Definitions

VBD type definitions common to all platforms:
__SBYTE__ - Type definition for 8-bit signed values.
__UBYTE__ - Type definition for 8-bit unsigned values.
__WORD__ - Type definition for native integer types.
__UWORD__ - Type definition for native unsigned integer types.
__LWORD__ - Type definition for native 32-bit signed integer types.
__ULWORD__ - Type definition for native 32-bit unsigned integer types.
__SWORD__ - Type definition for native 16-bit signed integer types.
__USWORD__ - Type definition for native 16-bit unsigned integer types.
__DPFLOAT__ - Type definition for native 64-bit double precision floating points types.

Type definitions used for file operations and stream position:
FAU - File address unit physical file address type.
vbStreamPos - Stream position indicator.

Type definition used to represent CRC-32 checksum values:
vbChecksum - CRC-32 checksum type.


VBD File Header

The vbFileHeader data structure is used to represent the VBD file header in accordance with the VBD File Format.

struct vbFileHeader // VBD file header information (28 bytes total)
{ 
  FAU FreeSpace;          // Address to first block of free heap space
  FAU EndOfFile;          // Address of byte after end of file
  FAU HeapStart;          // Address of the start of the heap space
  FAU HighestVB;          // Highest allocated variable block
  __SBYTE__ Signature[8]; // Signature used for every VBD file 
  vbINT32 Version;        // VBD Version number
};

Free space field:
The "free space" pointer is used by the allocation function to reclaim deleted blocks. The number of deleted blocks is maintained in a non-contiguous list. Each block in the list points to the next block in the list starting at a specified address. The "free space" pointer is used to store the file address of the block where the list starts. In order to prevent VBD files from becoming extremely fragmented due to numerous deletions, blocks marked deleted or removed will be reused by the allocation function. The file manager's allocation function checks the "free space" field before allocating any blocks. If the "free space" field is not empty, the allocation function will walk through the free space list looking for a deleted or removed variable block of the size to be allocated. This includes the size of the block data plus the block header. One of two methods can be used to reclaim deleted or removed blocks, the best-fit method or the first-fit method. The best-fit method works by scanning the entire free s pace list until the best location to reuse a block is found. The first-fit method works by searching the free space list until the first block of the appropriate size is found. Both methods will reuse any unused portion of a reclaimed block. The unused portion is assigned a new block header (marked removed) and placed back on the free space list. NOTE: If the free space list becomes corrupt, the "free space" field will be marked corrupt by the vbFSListCorrupt integer constant and no longer be used.

End of file field:
The "end of file" pointer is used to mark the end of the file and locate where the file can be extended during block allocation. When an existing file is opened the file manager will compare this field to the true end of file. If the values do not match, this pointer will be adjusted if possible.

Start of heap field:
The "start of heap" pointer is used to store the address where the static storage area ends and the dynamic data area begins. The allocation of new blocks will always start at this address. The size of the static storage area is determined when the file is created. Any data stored in the static area will not be affected by any of the allocation routines.

Highest block field:
The "highest block" pointer is used to store the address of the highest allocated variable block. This ensures that the file manager will always know where the first and last blocks are located in the file.

Signature field:
The signature field is used to determine if the file is of the correct type when an existing file is opened. When a new file is created the VBD file manager writes the "VBDFILE" string to this field. The eighth byte is used for all revision changes. Version 1034 sets the revision letter to 'C' and performs a compatibility check to ensure backward compatibility with previous releases. Revision 'A' added a 32-bit CRC checksum routine and reserves space at the end of each block for an application to write a 32-bit check-word. Revision 'B' added a persistent file lock and a persistent record lock and allocates space after the file and block headers to account for the persistent lock headers. Revision 'C' uses the second byte of the block header status field for block control commands.

Version field:
A version number is used to indicate that changes have been made to the file manager class itself. Version numbers are used to conditionally perform certain operations based on its numerical value. This will ensure backward compatibility with previous versions of the file manager class.


VBD Block Header

The vbBlockHeader data structure is used to represent variable block headers in accordance with the VBD File Format and is additionally used for block transfers between local and remote devices. Block headers can be broken down into two specific categories: database blocks and device blocks.

Database Blocks:
A database block header is stored with each variable block allocated by an application and is used by the VBD file manger to keep track of variable blocks stored in the database. Applications can also use database block header information to obtain file locations and status information of its file data. When new blocks are allocated the file manager writes the block header and then reserves enough space to store the number of bytes requested. The application is responsible for writing the actual data in the block starting at the address immediately following the block header.

Device Blocks:
Device block headers contain synchronization and control information. They are used to transfer blocks and to signal events between devices. When an application sends a block the header precedes the block and informs the receiver of the block size and the block status. After reception of a device header the receiver then waits for the block data and processes the data or signals an event according to the status of the block

// Variable Block Header 
struct vbBlockHeader // Marks each variable data block (16 bytes total)
{
  vbUINT32 CkWord;   // Check-word used to mark each block (4 bytes)
  vbUINT32 Length;   // Block Length (4 bytes) 
  vbUINT32 Status;   // First byte  = status of the block's dynamic data 
                     // Second byte = block control commands
                     // Third byte  = reserved for future use
                     // Fourth byte = reserved for future use
  FAU NextDeletedVB; // Pointer to next deleted block (4 bytes) 
};

Check word field:
The "check word" field represents a 32-bit check word. Database blocks use this field for integrity checks and to maintain synchronization between the file manager and the application. Device blocks use this field to maintain synchronization between local and remote devices. The check word field to should always be set to 0x00000FEFE using a vbUINT32 integer type. All VBD revisions use all 32 bits of the block check-word. NOTE: If the check word value is changed, database files and device drivers will no longer be compatible with previous releases.

Length field:
The "length" field stores the total length of the variable block. Block lengths include the size of the object plus the total size of the block overhead determined by the by the VBD revision letter and whether the block is a database block or a device block.

Database Block Lengths:
Revision 0 database blocks account for the length of the object plus the size of the block header. Revision 'A' database blocks account for the length of the object plus the size of the block header and the size of the block's CRC checksum value. Revision 'B' and higher database blocks account for the length of the object plus the size of the block header, record lock header, and the size of the block's CRC checksum value. The VBD file manager uses this field to index the file block by block. The "check-word" field is used to ensure that the next block in sequence is a valid variable block. The length of the object can be calculated by subtracting the total size of the block overhead from the "length" field.

Device Block Lengths:
VBD device blocks are used to send and receive variable blocks to and from various I/O devices. All device blocks will only account for the length of the object. Device blocks used to send and receive control signals will only account for the length of the block header as no additional data will be sent or received other then the block header itself.

Status field:
The "status" field stores the status of the dynamic data contained in the block. Only two bytes of the status field are used. The remaining two bytes of the status field are reserved for future use.

Database Block Status Information:
The status of a database block can be determined by one of four primary byte values stored in the first block of the status field: 'N' for normal (ASCII 78), 'D' for deleted (ASCII 68), 'R' for removed (ASCII 82), or 'B' for bad (ASCII 66.) The VBD file manager uses a signed byte constant named vbNormalVB to mark normal database blocks, vbDeletedVB to mark deleted database blocks, vbRemovedVB to mark database removed blocks, and vbBadBlock to mark a block bad blocks. A block marked 'N' for normal indicates that the database block is in use and cannot be reclaimed by the allocation function. A block marked 'D' for deleted means that the data in the block is still valid, but the block can be overwritten if needed. Marking deleted blocks 'N' for normal can restore deleted database blocks. A block marked 'R' for removed means that the data in the block has been removed and the block can be overwritten. Once a block is removed it can never be restored. A block marked 'B' for bad means that the block is unusable a nd cannot be read or written to.

Device Block Status Infomation:
The status of a device block can be determined by one of several byte values stored in the second block of the status field. Block control characters were added in VBD revision 'C' to handle device control commands. Device control commands allow blocks to added, changed, removed, and requested by local and remote devices. Device control commands are also used to signal events between local and remote devices.

Next deleted block field:
Database blocks use the "next deleted block" field to store a pointer to the next deleted or removed variable block, only if this block has been deleted or removed. Device block headers ignore this field by default. The total number of deleted database blocks is maintained in a non-contiguous list. Each deleted or removed database block in the list points to the next deleted or removed database block in the list starting at the head of the list. The "free space" pointer in the VBD file header is used to store the file address where the head of the free space list is located. When a database block is deleted or removed the next deleted block field is set to the VBD header's "free space" value to become the head of the free space list and the VBD file header's "free space" pointer is set to the address of this deleted or removed database block.


VBD File Lock Header

The VBD persistent lock header revision has not been finalized as of this document's publication. 


VBD Record Lock Header

The VBD persistent lock header revision has not been finalized as of this document's publication.  


End Of Document