(tar) Standard

(tar) Attributes (tar) Formats (tar) Extensions
 
 The Standard Format
 ===================
 
      _(This message will disappear, once this node revised.)_
 
    While an archive may contain many files, the archive itself is a
 single ordinary file.  Like any other file, an archive file can be
 written to a storage device such as a tape or disk, sent through a pipe
 or over a network, saved on the active file system, or even stored in
 another archive.  An archive file is not easy to read or manipulate
 without using the `tar' utility or Tar mode in GNU Emacs.
 
    Physically, an archive consists of a series of file entries
 terminated by an end-of-archive entry, which consists of 512 zero
 bytes.  A file entry usually describes one of the files in the archive
 (an "archive member"), and consists of a file header and the contents
 of the file.  File headers contain file names and statistics, checksum
 information which `tar' uses to detect file corruption, and information
 about file types.
 
    Archives are permitted to have more than one member with the same
 member name.  One way this situation can occur is if more than one
 version of a file has been stored in the archive.  For information
 about adding new versions of a file to an archive, see  update.
 
    In addition to entries describing archive members, an archive may
 contain entries which `tar' itself uses to store information.
  label, for an example of such an archive entry.
 
    A `tar' archive file contains a series of blocks.  Each block
 contains `BLOCKSIZE' bytes.  Although this format may be thought of as
 being on magnetic tape, other media are often used.
 
    Each file archived is represented by a header block which describes
 the file, followed by zero or more blocks which give the contents of
 the file.  At the end of the archive file there may be a block filled
 with binary zeros as an end-of-file marker.  A reasonable system should
 write a block of zeros at the end, but must not assume that such a
 block exists when reading an archive.
 
    The blocks may be "blocked" for physical I/O operations.  Each
 record of N blocks (where N is set by the `--blocking-factor=512-SIZE'
 (`-b 512-SIZE') option to `tar') is written with a single `write ()'
 operation.  On magnetic tapes, the result of such a write is a single
 record.  When writing an archive, the last record of blocks should be
 written at the full size, with blocks after the zero block containing
 all zeros.  When reading an archive, a reasonable system should
 properly handle an archive whose last record is shorter than the rest,
 or which contains garbage records after a zero block.
 
    The header block is defined in C as follows.  In the GNU `tar'
 distribution, this is part of file `src/tar.h':
 
      /* GNU tar Archive Format description.
      
         Copyright (C) 1988, 1989, 1991, 1992, 1993, 1994, 1995, 1996,
         1997, 2000, 2001 Free Software Foundation, Inc.
      
         This program is free software; you can redistribute it and/or modify it
         under the terms of the GNU General Public License as published by the
         Free Software Foundation; either version 2, or (at your option) any later
         version.
      
         This program is distributed in the hope that it will be useful, but
         WITHOUT ANY WARRANTY; without even the implied warranty of
         MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General
         Public License for more details.
      
         You should have received a copy of the GNU General Public License along
         with this program; if not, write to the Free Software Foundation, Inc.,
         59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.  */
      
      /* If OLDGNU_COMPATIBILITY is not zero, tar produces archives which, by
         default, are readable by older versions of GNU tar.  This can be
         overriden by using --posix; in this case, POSIXLY_CORRECT in environment
         may be set for enforcing stricter conformance.  If OLDGNU_COMPATIBILITY
         is zero or undefined, tar will eventually produces archives which, by
         default, POSIX compatible; then either using --posix or defining
         POSIXLY_CORRECT enforces stricter conformance.
      
         This #define will disappear in a few years.  FP, June 1995.  */
      #define OLDGNU_COMPATIBILITY 1
      
      /* tar Header Block, from POSIX 1003.1-1990.  */
      
      /* POSIX header.  */
      
      struct posix_header
      {                               /* byte offset */
        char name[100];               /*   0 */
        char mode[8];                 /* 100 */
        char uid[8];                  /* 108 */
        char gid[8];                  /* 116 */
        char size[12];                /* 124 */
        char mtime[12];               /* 136 */
        char chksum[8];               /* 148 */
        char typeflag;                /* 156 */
        char linkname[100];           /* 157 */
        char magic[6];                /* 257 */
        char version[2];              /* 263 */
        char uname[32];               /* 265 */
        char gname[32];               /* 297 */
        char devmajor[8];             /* 329 */
        char devminor[8];             /* 337 */
        char prefix[155];             /* 345 */
                                      /* 500 */
      };
      
      #define TMAGIC   "ustar"        /* ustar and a null */
      #define TMAGLEN  6
      #define TVERSION "00"           /* 00 and no null */
      #define TVERSLEN 2
      
      /* Values used in typeflag field.  */
      #define REGTYPE  '0'            /* regular file */
      #define AREGTYPE '\0'           /* regular file */
      #define LNKTYPE  '1'            /* link */
      #define SYMTYPE  '2'            /* reserved */
      #define CHRTYPE  '3'            /* character special */
      #define BLKTYPE  '4'            /* block special */
      #define DIRTYPE  '5'            /* directory */
      #define FIFOTYPE '6'            /* FIFO special */
      #define CONTTYPE '7'            /* reserved */
      
      /* Bits used in the mode field, values in octal.  */
      #define TSUID    04000          /* set UID on execution */
      #define TSGID    02000          /* set GID on execution */
      #define TSVTX    01000          /* reserved */
                                      /* file permissions */
      #define TUREAD   00400          /* read by owner */
      #define TUWRITE  00200          /* write by owner */
      #define TUEXEC   00100          /* execute/search by owner */
      #define TGREAD   00040          /* read by group */
      #define TGWRITE  00020          /* write by group */
      #define TGEXEC   00010          /* execute/search by group */
      #define TOREAD   00004          /* read by other */
      #define TOWRITE  00002          /* write by other */
      #define TOEXEC   00001          /* execute/search by other */
      
      /* tar Header Block, GNU extensions.  */
      
      /* In GNU tar, SYMTYPE is for to symbolic links, and CONTTYPE is for
         contiguous files, so maybe disobeying the `reserved' comment in POSIX
         header description.  I suspect these were meant to be used this way, and
         should not have really been `reserved' in the published standards.  */
      
      /* *BEWARE* *BEWARE* *BEWARE* that the following information is still
         boiling, and may change.  Even if the OLDGNU format description should be
         accurate, the so-called GNU format is not yet fully decided.  It is
         surely meant to use only extensions allowed by POSIX, but the sketch
         below repeats some ugliness from the OLDGNU format, which should rather
         go away.  Sparse files should be saved in such a way that they do *not*
         require two passes at archive creation time.  Huge files get some POSIX
         fields to overflow, alternate solutions have to be sought for this.  */
      
      /* Descriptor for a single file hole.  */
      
      struct sparse
      {                               /* byte offset */
        char offset[12];              /*   0 */
        char numbytes[12];            /*  12 */
                                      /*  24 */
      };
      
      /* Sparse files are not supported in POSIX ustar format.  For sparse files
         with a POSIX header, a GNU extra header is provided which holds overall
         sparse information and a few sparse descriptors.  When an old GNU header
         replaces both the POSIX header and the GNU extra header, it holds some
         sparse descriptors too.  Whether POSIX or not, if more sparse descriptors
         are still needed, they are put into as many successive sparse headers as
         necessary.  The following constants tell how many sparse descriptors fit
         in each kind of header able to hold them.  */
      
      #define SPARSES_IN_EXTRA_HEADER  16
      #define SPARSES_IN_OLDGNU_HEADER 4
      #define SPARSES_IN_SPARSE_HEADER 21
      
      /* The GNU extra header contains some information GNU tar needs, but not
         foreseen in POSIX header format.  It is only used after a POSIX header
         (and never with old GNU headers), and immediately follows this POSIX
         header, when typeflag is a letter rather than a digit, so signaling a GNU
         extension.  */
      
      struct extra_header
      {                               /* byte offset */
        char atime[12];               /*   0 */
        char ctime[12];               /*  12 */
        char offset[12];              /*  24 */
        char realsize[12];            /*  36 */
        char longnames[4];            /*  48 */
        char unused_pad1[68];         /*  52 */
        struct sparse sp[SPARSES_IN_EXTRA_HEADER];
                                      /* 120 */
        char isextended;              /* 504 */
                                      /* 505 */
      };
      
      /* Extension header for sparse files, used immediately after the GNU extra
         header, and used only if all sparse information cannot fit into that
         extra header.  There might even be many such extension headers, one after
         the other, until all sparse information has been recorded.  */
      
      struct sparse_header
      {                               /* byte offset */
        struct sparse sp[SPARSES_IN_SPARSE_HEADER];
                                      /*   0 */
        char isextended;              /* 504 */
                                      /* 505 */
      };
      
      /* The old GNU format header conflicts with POSIX format in such a way that
         POSIX archives may fool old GNU tar's, and POSIX tar's might well be
         fooled by old GNU tar archives.  An old GNU format header uses the space
         used by the prefix field in a POSIX header, and cumulates information
         normally found in a GNU extra header.  With an old GNU tar header, we
         never see any POSIX header nor GNU extra header.  Supplementary sparse
         headers are allowed, however.  */
      
      struct oldgnu_header
      {                               /* byte offset */
        char unused_pad1[345];        /*   0 */
        char atime[12];               /* 345 */
        char ctime[12];               /* 357 */
        char offset[12];              /* 369 */
        char longnames[4];            /* 381 */
        char unused_pad2;             /* 385 */
        struct sparse sp[SPARSES_IN_OLDGNU_HEADER];
                                      /* 386 */
        char isextended;              /* 482 */
        char realsize[12];            /* 483 */
                                      /* 495 */
      };
      
      /* OLDGNU_MAGIC uses both magic and version fields, which are contiguous.
         Found in an archive, it indicates an old GNU header format, which will be
         hopefully become obsolescent.  With OLDGNU_MAGIC, uname and gname are
         valid, though the header is not truly POSIX conforming.  */
      #define OLDGNU_MAGIC "ustar  "  /* 7 chars and a null */
      
      /* The standards committee allows only capital A through capital Z for
         user-defined expansion.  */
      
      /* This is a dir entry that contains the names of files that were in the
         dir at the time the dump was made.  */
      #define GNUTYPE_DUMPDIR 'D'
      
      /* Identifies the *next* file on the tape as having a long linkname.  */
      #define GNUTYPE_LONGLINK 'K'
      
      /* Identifies the *next* file on the tape as having a long name.  */
      #define GNUTYPE_LONGNAME 'L'
      
      /* This is the continuation of a file that began on another volume.  */
      #define GNUTYPE_MULTIVOL 'M'
      
      /* For storing filenames that do not fit into the main header.  */
      #define GNUTYPE_NAMES 'N'
      
      /* This is for sparse files.  */
      #define GNUTYPE_SPARSE 'S'
      
      /* This file is a tape/volume header.  Ignore it on extraction.  */
      #define GNUTYPE_VOLHDR 'V'
      
      /* tar Header Block, overall structure.  */
      
      /* tar files are made in basic blocks of this size.  */
      #define BLOCKSIZE 512
      
      enum archive_format
      {
        DEFAULT_FORMAT,               /* format to be decided later */
        V7_FORMAT,                    /* old V7 tar format */
        OLDGNU_FORMAT,                /* GNU format as per before tar 1.12 */
        POSIX_FORMAT,                 /* restricted, pure POSIX format */
        GNU_FORMAT                    /* POSIX format with GNU extensions */
      };
      
      union block
      {
        char buffer[BLOCKSIZE];
        struct posix_header header;
        struct extra_header extra_header;
        struct oldgnu_header oldgnu_header;
        struct sparse_header sparse_header;
      };
      
      /* End of Format description.  */
 
    All characters in header blocks are represented by using 8-bit
 characters in the local variant of ASCII.  Each field within the
 structure is contiguous; that is, there is no padding used within the
 structure.  Each character on the archive medium is stored contiguously.
 
    Bytes representing the contents of files (after the header block of
 each file) are not translated in any way and are not constrained to
 represent characters in any character set.  The `tar' format does not
 distinguish text files from binary files, and no translation of file
 contents is performed.
 
    The `name', `linkname', `magic', `uname', and `gname' are
 null-terminated character strings.  All other fields are zero-filled
 octal numbers in ASCII.  Each numeric field of width W contains W minus
 2 digits, a space, and a null, except `size', and `mtime', which do not
 contain the trailing null.
 
    The `name' field is the file name of the file, with directory names
 (if any) preceding the file name, separated by slashes.
 
    The `mode' field provides nine bits specifying file permissions and
 three bits to specify the Set UID, Set GID, and Save Text ("sticky")
 modes.  Values for these bits are defined above.  When special
 permissions are required to create a file with a given mode, and the
 user restoring files from the archive does not hold such permissions,
 the mode bit(s) specifying those special permissions are ignored.
 Modes which are not supported by the operating system restoring files
 from the archive will be ignored.  Unsupported modes should be faked up
 when creating or updating an archive; e.g. the group permission could
 be copied from the _other_ permission.
 
    The `uid' and `gid' fields are the numeric user and group ID of the
 file owners, respectively.  If the operating system does not support
 numeric user or group IDs, these fields should be ignored.
 
    The `size' field is the size of the file in bytes; linked files are
 archived with this field specified as zero.
 
    The `mtime' field is the modification time of the file at the time
 it was archived.  It is the ASCII representation of the octal value of
 the last time the file was modified, represented as an integer number of
 seconds since January 1, 1970, 00:00 Coordinated Universal Time.
 
    The `chksum' field is the ASCII representation of the octal value of
 the simple sum of all bytes in the header block.  Each 8-bit byte in
 the header is added to an unsigned integer, initialized to zero, the
 precision of which shall be no less than seventeen bits.  When
 calculating the checksum, the `chksum' field is treated as if it were
 all blanks.
 
    The `typeflag' field specifies the type of file archived.  If a
 particular implementation does not recognize or permit the specified
 type, the file will be extracted as if it were a regular file.  As this
 action occurs, `tar' issues a warning to the standard error.
 
    The `atime' and `ctime' fields are used in making incremental
 backups; they store, respectively, the particular file's access time
 and last inode-change time.
 
    The `offset' is used by the `--multi-volume' (`-M') option, when
 making a multi-volume archive.  The offset is number of bytes into the
 file that we need to restart at to continue the file on the next tape,
 i.e., where we store the location that a continued file is continued at.
 
    The following fields were added to deal with sparse files.  A file
 is "sparse" if it takes in unallocated blocks which end up being
 represented as zeros, i.e., no useful data.  A test to see if a file is
 sparse is to look at the number blocks allocated for it versus the
 number of characters in the file; if there are fewer blocks allocated
 for the file than would normally be allocated for a file of that size,
 then the file is sparse.  This is the method `tar' uses to detect a
 sparse file, and once such a file is detected, it is treated
 differently from non-sparse files.
 
    Sparse files are often `dbm' files, or other database-type files
 which have data at some points and emptiness in the greater part of the
 file.  Such files can appear to be very large when an `ls -l' is done
 on them, when in truth, there may be a very small amount of important
 data contained in the file.  It is thus undesirable to have `tar' think
 that it must back up this entire file, as great quantities of room are
 wasted on empty blocks, which can lead to running out of room on a tape
 far earlier than is necessary.  Thus, sparse files are dealt with so
 that these empty blocks are not written to the tape.  Instead, what is
 written to the tape is a description, of sorts, of the sparse file:
 where the holes are, how big the holes are, and how much data is found
 at the end of the hole.  This way, the file takes up potentially far
 less room on the tape, and when the file is extracted later on, it will
 look exactly the way it looked beforehand.  The following is a
 description of the fields used to handle a sparse file:
 
    The `sp' is an array of `struct sparse'.  Each `struct sparse'
 contains two 12-character strings which represent an offset into the
 file and a number of bytes to be written at that offset.  The offset is
 absolute, and not relative to the offset in preceding array element.
 
    The header can hold four of these `struct sparse' at the moment; if
 more are needed, they are not stored in the header.
 
    The `isextended' flag is set when an `extended_header' is needed to
 deal with a file.  Note that this means that this flag can only be set
 when dealing with a sparse file, and it is only set in the event that
 the description of the file will not fit in the allotted room for
 sparse structures in the header.  In other words, an extended_header is
 needed.
 
    The `extended_header' structure is used for sparse files which need
 more sparse structures than can fit in the header.  The header can fit
 4 such structures; if more are needed, the flag `isextended' gets set
 and the next block is an `extended_header'.
 
    Each `extended_header' structure contains an array of 21 sparse
 structures, along with a similar `isextended' flag that the header had.
 There can be an indeterminate number of such `extended_header's to
 describe a sparse file.
 
 `REGTYPE'
 `AREGTYPE'
      These flags represent a regular file.  In order to be compatible
      with older versions of `tar', a `typeflag' value of `AREGTYPE'
      should be silently recognized as a regular file.  New archives
      should be created using `REGTYPE'.  Also, for backward
      compatibility, `tar' treats a regular file whose name ends with a
      slash as a directory.
 
 `LNKTYPE'
      This flag represents a file linked to another file, of any type,
      previously archived.  Such files are identified in Unix by each
      file having the same device and inode number.  The linked-to name
      is specified in the `linkname' field with a trailing null.
 
 `SYMTYPE'
      This represents a symbolic link to another file.  The linked-to
      name is specified in the `linkname' field with a trailing null.
 
 `CHRTYPE'
 `BLKTYPE'
      These represent character special files and block special files
      respectively.  In this case the `devmajor' and `devminor' fields
      will contain the major and minor device numbers respectively.
      Operating systems may map the device specifications to their own
      local specification, or may ignore the entry.
 
 `DIRTYPE'
      This flag specifies a directory or sub-directory.  The directory
      name in the `name' field should end with a slash.  On systems where
      disk allocation is performed on a directory basis, the `size' field
      will contain the maximum number of bytes (which may be rounded to
      the nearest disk block allocation unit) which the directory may
      hold.  A `size' field of zero indicates no such limiting.  Systems
      which do not support limiting in this manner should ignore the
      `size' field.
 
 `FIFOTYPE'
      This specifies a FIFO special file.  Note that the archiving of a
      FIFO file archives the existence of this file and not its contents.
 
 `CONTTYPE'
      This specifies a contiguous file, which is the same as a normal
      file except that, in operating systems which support it, all its
      space is allocated contiguously on the disk.  Operating systems
      which do not allow contiguous allocation should silently treat this
      type as a normal file.
 
 `A' ... `Z'
      These are reserved for custom implementations.  Some of these are
      used in the GNU modified format, as described below.
 
    Other values are reserved for specification in future revisions of
 the P1003 standard, and should not be used by any `tar' program.
 
    The `magic' field indicates that this archive was output in the
 P1003 archive format.  If this field contains `TMAGIC', the `uname' and
 `gname' fields will contain the ASCII representation of the owner and
 group of the file respectively.  If found, the user and group IDs are
 used rather than the values in the `uid' and `gid' fields.
 
    For references, see ISO/IEC 9945-1:1990 or IEEE Std 1003.1-1990,
 pages 169-173 (section 10.1) for `Archive/Interchange File Format'; and
 IEEE Std 1003.2-1992, pages 380-388 (section 4.48) and pages 936-940
 (section E.4.48) for `pax - Portable archive interchange'.
 
(tar) Attributes (tar) Formats (tar) Extensions
automatically generated by info2html