World Library  
Flag as Inappropriate
Email this Article

Open Archives Initiative Protocol for Metadata Harvesting

Article Id: WHEBN0000905790
Reproduction Date:

Title: Open Archives Initiative Protocol for Metadata Harvesting  
Author: World Heritage Encyclopedia
Language: English
Subject: Metadata, Open archive, ALTO (XML), BASE (search engine), Herbert Van de Sompel
Collection: Archival Science, Digital Libraries, Internet Protocols, Metadata, Open Access (Publishing), Open Access Projects
Publisher: World Heritage Encyclopedia
Publication
Date:
 

Open Archives Initiative Protocol for Metadata Harvesting

The Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH) is a protocol developed by the Open Archives Initiative. It is used to harvest (or collect) the metadata descriptions of the records in an archive so that services can be built using metadata from many archives. An implementation of OAI-PMH must support representing metadata in Dublin Core, but may also support additional representations.[1]

The protocol is usually just referred to as the OAI Protocol.

OAI-PMH uses XML over HTTP. Version 2.0 of the protocol was released in 2002, the document was last updated in 2015. It has a Creative Commons license BY-SA.

Contents

  • History 1
  • Registries 2
  • Uses 3
  • Software 4
  • Archives 5
  • Workshops 6
  • See also 7
  • Notes 8
  • References 9
  • External links 10

History

In the late 1990s, Herbert Van de Sompel (Ghent University) was working with researchers and librarians at Los Alamos National Laboratory (US) and called a meeting to address difficulties related to interoperability issues of e-print servers and digital repositories. The meeting was held in Santa Fe, New Mexico, in October 1999. A key development from the meeting was the definition of an interface that permitted e-print servers to expose metadata for the papers it held in a structured fashion so other repositories could identify and copy papers of interest with each other. This interface/protocol was named the "Santa Fe Convention".[1]

Several workshops were held in 2000 at the ACM Digital Libraries conference[2] and elsewhere to share the ideas from the Santa Fe Convention. It was discovered at the workshops that the problems faced by the e-print community were also shared by libraries, museums, journal publishers, and others who needed to share distributed resources. To address these needs, the Coalition for Networked Information[3] and the Digital Library Federation[4] provided funding to establish an Open Archives Initiative (OAI) secretariat managed by Herbert Van de Sompel and Carl Lagoze. The OAI held a meeting at Cornell University (Ithaca, New York) in September 2000 to improve the interface developed at the Santa Fe Convention. The specifications were refined over e-mail.

OAI-PMH version 1.0 was introduced to the public in January 2001 at a workshop in Washington D.C., and another in February in Berlin, Germany. Subsequent modifications to the XML standard by the W3C required making minor modifications to OAI-PMH resulting in version 1.1. The current version, 2.0, was released in June 2002. It contained several technical changes and enhancements and is not backward compatible.

Registries

The OAI Protocol was adopted by many digital libraries, institutional repositories, and digital archives. Although registration is not mandatory, it is encouraged.

There are several large registries of OAI-compliant repositories:

  1. The Open Archives list of registered OAI repositories
  2. The OAI registry at University of Illinois at Urbana-Champaign
  3. The Celestial OAI registry
  4. Eprint’s Institutional Archives Registry
  5. Openarchives.eu The European Guide to OAI-PMH compliant repositories in the world
  6. ScientificCommons.org A worldwide service and registry

Uses

Commercial search engines have started using OAI-PMH to acquire more resources. Google is using OAI-PMH to harvest information from the National Library of Australia Digital Object Repository.[5] In 2004, Yahoo! acquired content from OAIster (University of Michigan) that was obtained through metadata harvesting with OAI-PMH. Google did accept OAI-PMH as part of their Sitemap Protocol, though decided to stop doing so in 2008.[6] Wikimedia uses an OAI-PMH repository to provide feeds of WorldHeritage and related site updates for search engines and other bulk analysis/republishing endeavors.[7] Especially when dealing with thousands of files being harvested every day, OAI-PMH can help in reducing the network traffic and other resource usage by doing incremental harvesting.[8] NASA's Mercury: Metadata Search System uses OAI-PMH to index thousands of metadata records from Global Change Master Directory (GCMD) every day.[9]

The mod_oai project is using OAI-PMH to expose content to web crawlers that is accessible from Apache Web servers.

Software

OAI-PMH is based on a client–server architecture, in which "harvesters" request information on updated records from "repositories". Requests for data can be based on a datestamp range, and can be restricted to named sets defined by the provider. Data providers are required to provide XML metadata in Dublin Core format, and may also provide it in other XML formats.

A number of software systems support the OAI-PMH, including Fedora, GNU EPrints from the University of Southampton, Open Journal Systems from the Public Knowledge Project, Desire2Learn, DSpace from MIT, HyperJournal from the University of Pisa, Digibib from Digibis, Primo, DigiTool, Rosetta and MetaLib from Ex Libris, ArchivalWare from PTFS, DOOR [10] from the eLab[11] in Lugano, Switzerland, panFMP from the PANGAEA (data library),[12] SimpleDL from Roaring Development, and jOAI.[13]

Archives

A number of large archives support the protocol including arXiv and the CERN Document Server.

Workshops

A dedicated workshop has been held at CERN in Geneva on a regular basis since 2001. It is now co-organised by University of Geneva and CERN every two years in June. OAI8 was held in June 2013 and OAI9 is going to be held in 2015.

See also

Notes

  1. ^ a b Marshall Breeding (September 2002). "Understanding the Protocol for Metadata Harvesting of the Open Archives Initiative". Computers in Libraries 8 (24): 24–29. Retrieved October 11, 2013. 
  2. ^ ACM Digital Libraries conference
  3. ^ Coalition for Networked Information
  4. ^ Digital Library Federation
  5. ^ National Library of Australia Digital Object Repository
  6. ^ Google Webmaster blog
  7. ^ "Wikimedia update feed service". Wikimedia Meta-Wiki. Retrieved 14 July 2013. 
  8. ^ incremental harvesting
  9. ^ R. Devarakonda, G. Palanisamy, J. Green and B. Wilson (2010). "Data sharing and retrieval uses OAI-PMH". Earth Science Informatics (Springer Berlin / Heidelberg) 4 (1): 1–5.  
  10. ^ DOOR
  11. ^ eLab
  12. ^ panFMP
  13. ^ jOAI

References

  • Carl Lagoze and Herbert Van de Sompel (2001). "The Open Archives Initiative: Building a Low-Barrier Interoperability Framework". Proceedings of the first ACM/IEEE-CS  
  • Lynch, Clifford A. (2001). "Metadata harvesting and the open archives initiative". ARL Bimonthly Report 217.
  • Frank McCown, Xiaoming Liu, Michael L. Nelson and Mohammed Zubair (March–April 2006). "Search Engine Coverage of the OAI-PMH Corpus" (PDF). IEEE Internet Computing 10 (2): 66–73.  
  • Herbert Van de Sompel and Carl Lagoze (2000). "The Santa Fe Convention of the Open Archives Initiative". D-Lib Magazine 6 (2).  
  • Herbert Van de Sompel, Jeffrey A. Young, and Thomas B. Hickey (2003). "Using the OAI-PMH ... Differently". D-Lib Magazine 9 (7/8).  
  • Ranjeet Devarakonda, Giri Palanisamy, Jim Green and Bruce Wilson (2010). "Data sharing and retrieval uses OAI-PMH". Earth Science Informatics (Springer Berlin / Heidelberg) 4 (1): 1–5.  

External links

  • Suleyman Demirel University Open Archives Harvester
  • Protocol specification
  • National Library of Congress, Digital Collections and Programs
  • Library of Congress, National Digital Information Infrastructure and Preservation Program
  • Library of Congress, Web Capture
This article was sourced from Creative Commons Attribution-ShareAlike License; additional terms may apply. World Heritage Encyclopedia content is assembled from numerous content providers, Open Access Publishing, and in compliance with The Fair Access to Science and Technology Research Act (FASTR), Wikimedia Foundation, Inc., Public Library of Science, The Encyclopedia of Life, Open Book Publishers (OBP), PubMed, U.S. National Library of Medicine, National Center for Biotechnology Information, U.S. National Library of Medicine, National Institutes of Health (NIH), U.S. Department of Health & Human Services, and USA.gov, which sources content from all federal, state, local, tribal, and territorial government publication portals (.gov, .mil, .edu). Funding for USA.gov and content contributors is made possible from the U.S. Congress, E-Government Act of 2002.
 
Crowd sourced content that is contributed to World Heritage Encyclopedia is peer reviewed and edited by our editorial staff to ensure quality scholarly research articles.
 
By using this site, you agree to the Terms of Use and Privacy Policy. World Heritage Encyclopedia™ is a registered trademark of the World Public Library Association, a non-profit organization.
 


Copyright © World Library Foundation. All rights reserved. eBooks from Project Gutenberg are sponsored by the World Library Foundation,
a 501c(4) Member's Support Non-Profit Organization, and is NOT affiliated with any governmental agency or department.