[Cache from http://www.loc.gov/standards/mets/METSOverview.html; please use this canonical URL/source if possible.]

METS: An Overview & Tutorial

Introduction

Maintaining a library of digital objects of necessity requires maintaining metadata about those objects. The metadata necessary for successful management and use of digital objects is both more extensive than and different from the metadata used for managing collections of printed works and other physical materials. While a library may record descriptive metadata regarding a book in its collection, the book will not dissolve into a series of unconnected pages if the library fails to record structural metadata regarding the book's organization, nor will scholars be unable to evaluate the book's worth if the library fails to note that the book was produced using a Ryobi offset press. The same cannot be said for a digital version of the same book. Without structural metadata, the page image or text files comprising the digital work are of little use, and without technical metadata regarding the digitization process, scholars may be unsure of how accurate a reflection of the original the digital version provides. For internal management purposes, a library must have access to appropriate technical metadata in order to periodically refresh and migrate the data, ensuring the durability of valuable resources.

The Making of America II project (MOA2) attempted to address these issues in part by providing an encoding format for descriptive, administrative, and structural metadata for textual and image-based works. METS, a Digital Library Federation initiative, attempts to build upon the work of MOA2 and provide an XML document format for encoding metadata necessary for both management of digital library objects within a repository and exchange of such objects between repositories (or between repositories and their users). Depending on its use, a METS document could be used in the role of Submission Information Package (SIP), Archival Information Package (AIP), or Dissemination Information Package (DIP) within the Open Archival Information System (OAIS) Reference Model.

A METS document consists of four major sections:

Descriptive Metadata - The descriptive metadata section may point to descriptive metadata external to the METS document (e.g., a MARC record in an OPAC or an EAD finding aid maintained on a WWW server), or contain internally embedded descriptive metadata, or both. Multiple instances of both external and internal descriptive metadata may be included in the descriptive metadata section.
Administrative Metadata - The administrative metadata section provides information regarding how the files were created and stored, intellectual property rights, metadata regarding the original source object from which the digital library object derives, and information regarding the provenance of the files comprising the digital library object (i.e., master/derivative file relationships, and migration/transformation information). As with descriptive metadata, administrative metadata may be either external to the METS document, or encoded internally.
File Groups - The file group section lists all files comprising all electronic versions of the digital object. File group elements may nest, to provide for subdividing the files by object version.
Structural Map - The structural map is the heart of a METS document. It outlines a hierarchical structure for the digital library object, and links the elements of that structure to content files and metadata that pertain to each element.

A more detailed explanation of each section and their inter-relations follows.

Descriptive Metadata

The descriptive metadata section of a METS document consists of one or more <dmdSec> (Descriptive Metadata Section) elements. Each <dmdSec> element may contain a pointer to external metadata (an <mdRef> element), internally embedded metadata (within an <mdWrap> element), or both.

External Descriptive Metadata (mdRef): an mdRef element provides a URI which may be used in retrieving the external metadata. For example, the following metadata reference points to the finding aid for a particular digital library object:

<dmdSec ID="dmd001">
<mdRef LOCTYPE="URN" MIMETYPE="application/xml" MDTYPE="EAD"
LABEL="Berol Collection Finding Aid">urn:x-nyu:fales1735</mdRef>
</dmdSec>

The <mdRef> element of this <dmdSec> contains four attributes. The LOCTYPE attribute specifies the type of locator contained in body of the element; valid values for LOCTYPE include 'URN,' 'URL,' 'PURL,' 'HANDLE,' 'DOI,' and 'OTHER.' The MIMETYPE attribute allows you to specify the MIME type for the external descriptive metadata, and the MDTYPE allows you to indicate what form of metadata is being referenced. Valid values for the MDTYPE element include MARC, EAD, VRA (VRA Core), DC (Dublin Core), NISOIMG (NISO Technical Metadata for Digital Still Images), LC-AV (Library of Congress Audiovisual Metadata) , TEIHDR (TEI Header), DDI (Data Documentation Initiative), FGDC (Federal Geographic Data Committee Metadata Standard [FGDC-STD-001-1998] ), and OTHER. LABEL provides a mechanism for describing this metadata to those viewing a METS document, in a 'Table of Contents' display of the METS document, for example.

Internal Descriptive Metadata (mdWrap): An mdWrap element provides a wrapper around metadata embedded within a METS document. Such metadata can be in one of two forms: 1. XML-encoded metadata, with the XML-encoding identifying itself as belonging to a namespace other than the METS document namespace, or 2. any arbitrary binary or textual form, PROVIDED that the metadata is Base64 encoded and wrapped in a <binData> element within the mdWrap element. The following examples demonstrate the use of the mdWrap element:

<dmdSec ID="dmd002">
    <mdWrap MIMETYPE="text/xml" MDTYPE="DC" LABEL="Dublin Core Metadata">
        <dc:title>Alice's Adventures in Wonderland</dc:title>
        <dc:creator>Lewis Carroll</dc:creator>
        <dc:date>between 1872 and 1890</dc:date>
        <dc:publisher>McCloughlin Brothers</dc:publisher>
        <dc:type>text</dc:type>
    </mdWrap>
</dmdSec>

<dmdSec ID="dmd003">
    <mdWrap MIMETYPE="application/marc" MDTYPE="MARC" LABEL="OPAC Record">
   <binData>MDI0ODdjam0gIDIyMDA1ODkgYSA0NU0wMDAxMDA...(etc.)
        </binData>
    </mdWrap>
</dmdSec>

Note that all <dmdSec> elements must possess an ID attribute. This attribute provides a unique, internal name for each <dmdSec> element which can be used in the structural map to link a particular division of the document hierarchy to a particular <dmdSec> element. This allows specific sections of descriptive metadata to be linked to specific parts of the digital object.

Administrative Metadata

<amdSec> elements contain the administrative metadata pertaining to the files comprising a digital library object, as well as that pertaining to the original source material used to create the object. There are four main forms of administrative metadata provided for in a METS document: 1. Technical Metadata (information regarding files' creation, format, and use characteristics), 2. Intellectual Property Rights Metadata (copyright and license information), 3. Source Metadata (descriptive and administrative metadata regarding the analog source from which a digital library object derives), and 4. Digital Provenance Metadata (information regarding source/destination relationships between files, including master/derivative relationships between files and information regarding migrations/transformations employed on files between original digitization of an artifact and its current incarnation as a digital library object).

The <amdSec> element employs the same content model as <dmdSec>: it may contain an <mdRef> element to point to external administrative metadata, an <mdWrap> element to use when embedding administrative metadata within a METS document, or both. Multiple <amdSec> elements may occur within a METS document, and all of them must carry an ID attribute so that other elements within the METS document (such as divisions within the structural map or <file> elements) may be linked to the <amdSec> elements which describe them. One might, for example, have an <amdSec> element which includes technical metadata regarding a file's preparation:

<amdSec ID="AMD001">
    <mdWrap MIMETYPE="text/xml" MDTYPE="NISOIMG" LABEL="NISO Img. Data">
        <niso:MIMEtype>image/tiff</niso:MIMEtype>
        <niso:Compression>LZW</niso:Compression>
        <niso:PhotometricInterpretation>8</niso:PhotometricInterpretation>
        <niso:Orientation>1</niso:Orientation>
        <niso:ScanningAgency>NYU Press</niso:ScanningAgency>
   </mdWrap>
</amdSec>

A <file> element within a <fileGrp> might then identify this administrative metadata as pertaining to the file it identifies by using an ADMID attribute to point to this <amdSec> element:

<file ID="FILE001" ADMID="AMD001">
<FLocat LOCTYPE="URL">http://dlib.nyu.edu/press/testimg.tif</FLocat>
</file>

File Groups

The file group section consists of one or more <fileGrp> elements used to group together related files. A <fileGrp> lists all of the files which comprise a single electronic version of the digital library object. For example, there might be separate <fileGrp> elements for the thumbnails, the master archival images, the pdf versions, the TEI encoded text versions, etc.

Consider the following example of file groups from a digital library object for an oral history which has three different versions: a TEI-encoded transcript, a master audio file in WAV format, and a derivative audio file in MP3 format:

<fileGrp>
    <fileGrp ID="VERS1">
        <file ID="FILE001" MIMETYPE="application/xml" SIZE="257537"
        CREATED="2001-06-10">
            <FLocat LOCTYPE="URL">
                http://dlib.nyu.edu/tamwag/beame.xml
            </FLocat>
        </file>
    </fileGrp>
    <fileGrp ID="VERS2">
        <file ID="FILE002" MIMETYPE="audio/wav" SIZE="64232836"
        CREATED="2001-05-17" GROUPID="AUDIO1">
            <FLocat LOCTYPE="URL">
                http://dlib.nyu.edu/tamwag/beame.wav
            </FLocat>
        </file>
    </fileGrp>
    <fileGrp ID="VERS3" VERSDATE="2001-05-18">
        <file ID="FILE003" MIMETYPE="audio/mpeg" SIZE="8238866"
        CREATED="2001-05-18" GROUPID="AUDIO1">
            <FLocat LOCTYPE="URL">
                http://dlib.nyu.edu/tamwag/beame.mp3
            </FLocat>
        </file>
    </fileGrp>
</fileGrp>

The outermost <fileGrp> element exists only to separate the <fileGrp> section from the surrounding portions of the METS document. It contains; three subsidiary <fileGrp> elements, one for each different version; of the object. The first is an XML-encoded transcription file, the second is a master audio file in WAV format, and the third is a derivative audio file in MP3 format. While such a basic example does not really seem to need the <fileGrp> elements to distinguish the different versions of the object, <fileGrp> becomes much more useful for objects consisting of large numbers of scanned page images, or indeed any case where a single version of the object consists of a large number of files. In those cases, being able to separate <file> elements into <fileGrp>s makes identifying the files belonging to a particular version of the document a simple task.

You may note the presence of the GROUPID attributes with identical values on the two audio <file> elements; these indicate that the two files, while belonging to different versions of the object, contain the same basic information (you can use the GROUPID for the same purpose to indicate equivalent page image files in digital library objects containing many scanned page images).

You should also note that all of the <file> elements have a unique ID attribute. This attribute provides a unique, internal name for this file which can be referenced by other portions of the document. You’ll see this type of referencing in action when we look at the Structural Map Section.

It should be mentioned that <file> elements may possess an <FContent> element rather than an <FLocat> element. <FContent> elements are used to embed the actual contents of the file within the METS document; if this is done, the file contents must be Base64-encoded. The <FContent> element has an attribute of CHECKSUM which may be used to include an MD5 checksum value for the embedded file. While embedding files is not something one would typically do when preparing a METS document for use in displaying a digital library objects to users, it can be a valuable feature for exchanging digital library objects between repositories, or for archiving versions of digital library objects for off-site storage.

Structural Map

The structural map section of a METS document defines a hierarchical structure which can be presented to users of the digital library object to allow them to navigate through it. The <structMap> element encodes this hierarchy as a nested series of <div> elements. Each <div> carries attribute information specifying what kind of division it is, and also may contain multiple METS pointer (<mptr>) and file pointer (<fptr>) elements. METS pointers specify separate METS documents as containing the relevant file information for the <div> containing them. This can be useful when encoding large collections of material (e.g., an entire journal run) to keep the size of each METS file in the set relatively small. File pointers specify files (or locations within files) within the current METS document's <fileGrp> section that correspond to the portion in the hierarchy represented by the current <div>.

The following provides an example of an extremely simple structural map:

This structural map shows that we have an oral history (with Mayor Abraham Beame of New York City) that includes three subsections: an opening introduction by the interviewer, some family history from Mayor Beame, and a discussion of how he came to be involved with the teachers' union in New York. Each of these subsections/divisions is linked to three files (taken from our earlier example of file groups): an XML transcription, and a master and derivative audio file. A subsidiary <area> element is used in each <fptr> to indicate that this division corresponds with only a portion of the linked file, and to identify the exact portion of each linked file. For example, the first division (the interviewer introduction) is linked to a portion of the XML transcription file (FILE001) which is found between the two tags in the transcription file with ID attribute values of "INTVWBG" and "INTVWND." It is also linked to the two different audio files; in these cases, rather than specifying ID attribute values within the linked files, the begin and end points of the linked material within the files is indicated by a simple time code value of the form HH:MM:SS. So, the interviewer introduction can be found in both audio files in the segment beginning at time 00:00:00 in the file and extending through time 00:01:47.

Conclusion

The METS schema provides a flexible mechanism for encoding descriptive, administrative, and structural metadata for a digital library object, and for expressing the complex links between these various forms of metadata. It can therefore provide a useful standard for the exchange of digital library objects between repositories. The above discussion highlights the major features of the schema, but a thorough examination of the schema and its included documentation is necessary to understand the full range of its capabilities.

The Library of Congress

Comments: lcweb@loc.gov (06/14/01)