TFS
-=-
Summary:

   TFS is a front-end filing system which interfaces between all users and
any filing systems available under RISC-OS. Users do not have to concern
themselves with how TFS manages this other than it creates a number of files
in a RO directory in which the TFS directory tree is resident.

TFS bypasses the FileCore and FileSwitch restrictions placing all its files
inside a single RISC-OS file, and managing the contents of the file by
itself. It is capable of dynamically altering its size, reclaiming free space
automatically, on-the-fly file compression and file encryption.

An optional extra which must be supported is the creation and utilisation of
a seperate 'map' file which allows many operations to be sped up on slower
filing systems with bad seek times especially (eg; CD-ROMs, floppies etc).

Features:

 * Files and directories are arranged in a hierarchical format
 * Files may be up to 2^64 bytes long
 * Filenames and directory names may be of any length and may contain any
ASCII characters except those under 32
 * File data can be compressed on the fly using a variety of compression
methods
 * Entire directories may be archived
 * File and directory data can be encrypted in a number of secure methods
 * A large amount of infomation can be attached to each file eg; who created
the file, comments about the file, when it was created, what kind of data is
in the file etc.
 * Free data within the archive is automatically reclaimed during use and
returned to the host disc and facilities exist for splitting of large files
up into smaller sections.

TFS requires the tornado kernel to be present to work and is written entirely
in assembler and uses fast block i/o for maximum speed. Typical speed losses
should be around 5% for most media, although some may suffer more than this.

Techie
-=-=-=
   I have to admit I debated with myself for a while about implementing this.
It's an awful diversion from what I really want to do, but as it's necessary
for tornado to be even slightly functional at all, it had to be done.
   I daddled with standard formats for a while (zip etc - thanks Karl), but
found that the archive formats do not deal with frequently changing files
well (they require the rebuilding of many CRCs). So I plumped for a very
familiar format which will have dividends for virtual memory (similar code)
later. You'll see it when you do.
   TFS has a small problem in that it can have any alphanumeric character in
its name, and so adfs:$.Niall.File is impossible. Therefore, TFS works from
the command line as follows:

*. tfs:adfs:Niall's interesting directory<9>With loads of funny stuff in the
name !@#$%^&*(). although it isn't a problem<9>Humm, this is a stupid file
name........<9>You can even use *nix slashes:/\/\/\/\\/\/\/\ - what
else?<9>Finally, the file!<0>

Of course, RO's command line limit is a problem here, but that's not my
fault! (use *Set Sys$RCLimit to increase it). Anyway, the use of wildcards
(as yet undefined, but will be control characters) can lessen the length (and
typing time) considerably.

The TFS archive format:

Header:
 +00:TFS + ASCII 0a
 +04:Archive format - this is 0
 +08,+12,+16,+20:16 bytes for the creator of the archive eg; "TFS (nd)"
 +24:version of creator of the archive * 100
 +28:minimum version of reader required * 100
 +32,+36:2 words of date and time of creation
 +40,+44:Date of last updation (written before anything is done)
 +48,+52:Copy of the above written after alteration done
 +56:The start of a tornado heap
 
 +56,+60:"TNHP", heap format (this is 0)
 +64,+68:Total length of the heap
 +72:Flags
    bits 0,1:Heap type
         %00:this heap is fully resident in ordinary memory
         %01:Relocatable blocks in this memory-based heap may be stored
             on disc temporarily
         %10:this heap is resident on a media
         %11:Blocks in this media-based heap may be cached in memory
             temporarily
    bit 2:this heap cannot be altered in any way.

    bit 8:this heap is auto-extending using Tornado_Extblk, ie; it is a heap
within another heap block in another tornado heap
    bit 9:this heap is auto-extending using OS_ChangeDynamicArea on a RPC or
OS_Module in RMA on older machines
    bits 10,11:reserved
 +76:offset to path of the disc-based heap if bits 0,1 AND %10=%10
 +80,+84:offset to heap base (ie; the start of the data)
 +88,+92:offset to list of free blocks
 +96,+100:offset to list of relocatable blocks
 +104,+108:offset to list of non-relocatable blocks
 +112:length of heap pointer chain below
 +116:start of chain on heap pointers in which this heap resides
      eg; if this heap were in a heap in a heap in the system heap:
      +116:0 for the system heap as top heap
      +120:-5 (say) for the block in the system heap containing the next heap
      +124:-2 (say) as the block in the heap in the block above containing
           the next heap
      +128:-114 (say) as the block in the heap in the block above containing
           the next heap
      +132:-56326273 (say) as the block in the heap in the block above
           containing our heap, ie; this heap.
 +xx:The path of the disc-based heap as pointed to by +76
 +xx:Free block list. The first word is the length of the list, with
     subsequent quads of words offsets from the heap base of the free blocks
     and the remaining two words the length of the free block in units of 4
     bytes. If an offset is 0, then it means that this free space entry is
     currently unused.
        Also, the first word of the free block itself is the offset into the
     free block list of that block's entry.
 +xx:Relocatable block list. Same format as for the free block list
 +xx:Non-relocatable block list. Same format as for the free block list

Relocatable and non-relocatable blocks:
 -12:Flags
    bit 0:non-relocatable block
 -08,-04:precise length of the block in bytes. It is negative if there is no
         data in memory at +00 but it is on the disc-based copy of this block
 +00:the block's data

For files:
 +00:"TNFL"
 +12:offset to start of file data
 +16:length of file
 +20:length of file
 +24,+28:filetype:
    +24:%
 +32:Flags
    bits 0,1,2:CRC increment
      %000:byte CRC
      %001:word CRC
      %010:64 byte CRC
      %011:1024 byte CRC
    bits 3,4,5:Encryption
      %000:None
      %001:Simple section word EOR with password
      %010:Tornado encryption (Section word EOR with pseudo-random
           progression based on derivative from password)
    bits 6,7,8:Compression type
      %000:No compression
      %001:LZW quick compression
 +36:CRC with increment as bits 0,1 in flags
 +40,+44:Date of file creation
 +48,+52:Date of last updation
 +56,+60:Date of last viewing (for security)
 +64:Length of filename
 +68:Length of comment
 +72:filename, terminated by null
 +xx:comment, terminated by null (usually used for creator of file)
 
Directories:
 +00:"TNDR"
 +04:the number of files in this directory
 +08:offset to first file block
 +12:offset to the entry after the end of the directory
 +16:offset to the entry after the end of the directory
 +20,+24:filetype
 +28:flags
    bits 3,4,5:Encryption
      %000:None
      %001:Simple section word EOR with password
      %010:Tornado encryption (Section word EOR with pseudo-random
           progression based on derivative from password)
    bits 6,7,8:Compression type
      %000:No compression
      %001:LZW quick compression
    If encryption isn't %000, then all data within the directory is
    encrypted and the offset to the file block is invalid without decryption
    first.
 +32,+36:Date of directory creation
 +40,+44:Date of last updation
 +48,+52:Date of last viewing
 +56:Length of filename
 +60:Length of comment
 +64:filename, terminated by null
 +xx:comment, terminated by null

Note that compression of directories is for archival purposes only, and thus
is very slow (after compression of the directory the data is reinserted and
any free space remaining is removed by shuffling all data after the directory
down to fill the slot)





The map file is laid out as follows:

 +00:"TMAP"
 +04,+08:Date of last updation (written after everything's done)
 +12:Map format - this is 0
 +16:Data - whatever
 
Files:
 +00:"TNFL"
 +04:offset of header into archive
 +08:offset of header into archive
 +12:offset to next entry
 +16:length of file
 +20:length of file
 +24,+28:filetype
 +32:Flags
    bits 3,4,5:Encryption
      %000:None
      %001:Simple section word EOR with password
      %010:Tornado encryption (Section word EOR with pseudo-random
           progression based on derivative from password)
    bits 6,7,8:Compression type
      %000:No compression
      %001:LZW quick compression
 +36:reserved
 +40,+44:Date of file creation
 +48,+52:Date of last updation
 +56,+60:Date of last viewing (for security)
 +64:Length of filename
 +68:Length of comment
 +72:filename, terminated by null
 +xx:comment, terminated by null (usually used for creator of file)

Directories:
 +00:"TNDR"
 +04:the number of files in this directory
 +08:offset to first file block
 +12:offset into archive of the directory header
 +16:offset into archive of the directory header
 +20,+24:filetype
 +28:flags
    bits 3,4,5:Encryption
      %000:None
      %001:Simple section word EOR with password
      %010:Tornado encryption (Section word EOR with pseudo-random
           progression based on derivative from password)
    bits 6,7,8:Compression type
      %000:No compression
      %001:LZW quick compression
    If encryption isn't %000, then all data within the directory is
    encrypted and the offset to the file block is invalid without decryption
    first.
 +32,+36:Date of directory creation
 +40,+44:Date of last updation
 +48,+52:Date of last viewing
 +56:Length of filename
 +60:Length of comment
 +64:filename, terminated by null
 +xx:comment, terminated by null
