LOC recommends formats for preservation of software, data sets
Connecting state and local government leaders
The Library of Congress released a set of recommended formats for long-term preservation of a range of works, both for the Library and for library communities, creators, vendors and archivists.
The Library of Congress released a set of recommended formats for long-term preservation of a range of works, both for submission to the Library and for library communities, creators, vendors and archivists.
The annual update covered a variety of media: textual work and musical compositions, still images, audio, moving images, software and electronic gaming and learning as well as data sets/databases. The specifications do not replace or supersede its Best Edition Statement, LOC said, nor will they result in the exclusion of other formats from consideration for the Library’s permanent collections
“The Library’s mission is not simply to collect the extraordinary and diverse creative content of the American people and from around the world, but to make sure the collections are available and accessible for many generations to come," said Roberta Shaffer, association librarian for Library Services in a statement. "The goal of these recommendations and the ongoing process of reviewing and updating them annually is both to inform our internal acquisitions teams and also to share with external audiences what we are observing works well from a preservation perspective."
The Library’s preservation recommendations for software and datasets/databases includes guidance on content, delivery method, metadata and technology measures:
Software, which includes desktop, mobile/handheld and gaming and learning system software, should be archived so as to include:
1. All documentation and accompanying material, such as instructions and read me files) as well as a copy of the program. If sold as part of a book, a copy of the book should be included.
2. A copy of the source code, either on physical or file-based media.
3. If the operating system is not commonly available, a copy of the OS. If there are different versions (such as Mac and Windows), a copy of each.
4. If the system requires a stand-alone or proprietary platform (e.g. a gaming system or child’s toy), then that should be included.
5. Metadata that specifies which compiler was used to create the final code for commercial release..
When delivering the content to an archive via a mass storage device, the information may be distinct files or grouped together using a packaging format such as BagIt or AXF object. It can be on a hard drive with USB, CD-ROM disc, flash drive with USB interface or DVD disk. If it is a direct file submission, it should be grouped in a submission package such as BagIt, tar or AXF object.
Metadata should include the title, creator, creation date, place of publication, publisher/ producer/ distributor and contact information. If available, archivists should include the language of the work, other relevant identifiers, subject descriptors, abstracts – as well as a key or reference to each data field. It should also include production metadata such as credit, rights, and files which are available at the time of production.
Data set formats (not including geospatial data) should be:
1. Platform independent, character-based formats, as long as data is complete and has full detail and precision. Preferred formats are widely adopted marketplace standards, including XML, JSON, CSV, TSV and Excel.
2. Supplemental materials should be included and delivery method should be the same as for software listed above.
Metadata should include all applicable metadata, data dictionaries, XML schemas and technical specifications. Discipline-specific metadata standards should be used whenever possible. Other pertinent information to include: title, creator, creation date, place of publication, publisher/ producer/ distributor, contact information..
Geospatial data should include all layers and appendices, even if proprietary. The data should be in a format compatible with GIS, such as ArcGIS or compatible with geospatial open source and open data communities.
Databases should include a complete set of the content in the database, conforming to preferred specifications. Access should be via a web interface, with comprehensive and user-friendly search and discovery, and include counter compliant usage statistics; or be delivered via a preferred preservation content submission.
Electronic files must contain no measures that control access to or use of the digital work, such as digital rights management or encryption.
The Library of Congress is nation’s oldest federal cultural institution and the largest library in the world. It holds more than 158 million items in various languages, disciplines and formats. More information on the Library of Congress’ digital preservation initiatives and best practices is available at http://www.digitalpreservation.gov/.
NEXT STORY: Data lakes: Don’t dive in just yet