World Library  
Flag as Inappropriate
Email this Article

Metalink

 

Metalink

Metalink
Metalink logo
Filename extension .meta4, .metalink
Internet media type application/metalink4+xml,
application/metalink+xml
Type of format File distribution
Extended from XML, HTTP
Standard RFC 5854, RFC 6249

Metalink is an extensible metadata file format that describes one or more computer files available for download. It specifies files appropriate for the user's language and operating system; facilitates file verification and recovery from data corruption; and lists alternate download sources (mirror URIs).

The metadata is encoded in HTTP header fields and/or in an XML file with extension .meta4 or .metalink. The duplicate download locations provide reliability in case one method fails. Some clients also achieve faster download speeds by allowing different chunks/segments of each file to be downloaded from multiple resources at the same time (segmented downloading).

Metalink supports listing multiple partial and full file hashes along with PGP signatures. Most clients only support verifying MD5, SHA-1, & SHA-256, however. Besides FTP and HTTP mirror locations and rsync, it also supports listing the P2P methods BitTorrent, ed2k, magnet link or any other that uses a URI.

Contents

  • Development History 1
  • Client programs 2
  • Client Libraries 3
  • In use 4
  • Metalink client feature comparison 5
  • Metalink generation 6
  • Example Metalink 4.0 .meta4 file 7
  • Example Metalink/HTTP header fields 8
  • Example Metalink 3.0 .metalink file 9
  • See also 10
  • References 11
  • External links 12

Development History

Metalink 3.0 was publicly released in 2005. It was designed to aid in downloading Linux Linux distributions. A community developed around it, more download programs supported it (including proprietary ones) and it saw commercial adoption. In 2008, the community took their work to the Internet Engineering Task Force which resulted in Metalink 4.0 in 2010, described in a Standards Track RFC. Metalink 3.0 (with the extension .metalink) and Metalink 4.0 (with the extension .meta4) are incompatible because they have a slightly different format. In 2011, another Standards Track RFC described Metalink in HTTP header fields.

Client programs

  • cURL is a computer software project providing a library and command-line tool for transferring data using various protocols.
  • DownThemAll (GPL), a Firefox extension and download manager that has basic Metalink support.
  • FlashGot (GPL), a Firefox extension for integrating with many Download Managers, supports Metalink.
  • Free Download Manager (GPL), a Windows Download Manager, supports Metalink (HTTP/FTP) in version 2.5.
  • GetRight 6, a Windows Download Manager, supports Metalink (HTTP/FTP/BitTorrent).
  • jDownloader is an open source download manager written in Java.
  • KGet (GPL), KDE's native download manager, supports Metalink in KDE 4.
  • Orbit Downloader, a Windows Download Manager, supports Metalink (HTTP/FTP).
  • Phex (GPL), a Mac/Unix/Windows gnutella client, supports Metalink downloads along with exporting a library to Metalink for sharing between users.
  • SmartFTP, a Windows GUI FTP client, supports Metalink for adding files to an FTP transfer queue and hash verification.
  • VeryCD's easyMule (version 1 is GPLed) is a version of eMule with Metalink support.
  • yum, an open source command line package management utility for RPM-compatible Linux operating systems, uses Metalink internally, invisible to the user.
  • ZYpp, the package management engine that powers YaST2 and Zypper, supports Metalink for downloading. In the beginning it used aria2 internally, meanwhile (since Jul. 2010) its own libcurl based implementation

Client Libraries

  • libmetalink (MIT License) is a Metalink library written in C. It provides the parsing of Metalink XML files to programs written in C and uses Expat (library) or libxml2. It does not handle the actual downloading of files. It is used by cURL and other client programs.
  • metalink-checker (GPL) is a command line downloader written in Python that supports multi-source downloads and chunk checksums, as well as checking mirrors. It can also be used as a Python library.

In use

Mandriva Linux has integrated Metalink into package management with urpmi and aria2. Fedora has integrated Metalink into package management with yum. openSUSE has integrated Metalink into package management with ZYpp and aria2.

Wubi, the Windows-based Ubuntu installer, uses Metadl (LGPL) to download Ubuntu ISO images and takes advantage of Metalink's features of higher availability and increased reliability. If there are errors in the download, they are repaired, instead of restarting the large download.

Appupdater (GPL) for Windows "is similar to apt-get or yum on Linux. It automates the process of installing and maintaining up to date versions of programs."

Currently, cURL offers Metalinks of their downloads. UniProt Consortium also provides a Metalink for their large protein database. Dofus, a Flash MMORPG uses Metalinks for downloads and so does Sage, the open-source mathematical software.

The software hosting and collaboration platform Origo generates Metalinks for all hosted releases.

The following Linux distributions use Metalink for ISO image distribution: 64 Studio, Adios, Bayanihan Linux, BeleniX, Berry Linux, BLAG Linux and GNU, Bluewhite64 Linux, Damn Small Linux, Fedora, GoboLinux, Granular Linux, KateOS, Linux Mint, openSUSE, Pardus Linux, PCLinuxOS, PuppyLinux, Sabayon Linux, StartCom Enterprise Linux, SUSE Linux, Ubuntu.

The following FreeBSD based distributions use Metalink for ISO image distribution: DesktopBSD, MidnightBSD, PC-BSD and TrueBSD.

File based search engine findthatfile provides search for Metalinks.

Metalink client feature comparison

A guide to which clients support which Metalink features:

cURL DownThemAll Free Download Manager GetRight KGet Orbit Downloader Phex SmartFTP
Free Software (GNU General Public License) Free Software (MIT/X derivate license) Free Software (GNU General Public License) Free Software (GNU General Public License) Proprietary / $29.95 for Standard, $49.95 for Pro Free Software (GNU General Public License) Proprietary / Freeware Free Software (GNU General Public License) Proprietary / Freeware for personal, $36.95 commercial
Interface CLI GUI GUI GUI GUI GUI GUI GUI
Metalink 4.0 Yes Yes No No Yes No No Yes
Runs on Linux Yes Yes No No Yes Yes Yes No
Runs on Windows Yes Yes Yes Yes No Yes Yes Yes
Runs on Mac OS X Yes Yes No No No No Yes No
Resume Downloads Yes Yes Yes Yes Yes Yes Yes Yes
Segmented Downloads ? Yes Yes Yes Yes Yes Yes No
Multiple File support ? Yes Yes Yes Yes Yes Yes Yes
"location" ? Yes No No No No No No
OS ? Yes Yes No No No No No
Language ? Yes No No No Yes No No
HTTP with Transparent Negotiation ? Yes No No No No No No
Metalink/HTTP ? No No No No No No No
HTTP Yes Yes Yes Yes Yes Yes Yes Yes
FTP Yes Yes Yes Yes Yes Yes Yes Yes
BitTorrent Yes No Yes Yes Yes No No No
magnet ? No No No No Yes Yes No
MD5 Hashes Yes Yes Yes Yes Yes Yes ? Yes
SHA-1 Hashes Yes Yes Yes Yes Yes Yes Yes Yes
SHA-256 Hashes Yes Yes Yes No No No No Yes
Piece Hashes ? No No No No No No No
PGP Signatures ? No No No No No No No

No clients support rsync at this time.

Metalink generation

Metalink is formatted as XML so it can be either hand-crafted or created with an XML library, useful for automation. These tools are specifically for Metalink.

  • Bouncer is "database driven mirror management app that has three main components - a bounce script, a sentry script to check mirrors and files for integrity and a set of web-based administration tools." Bouncer supports Metalink via a patch that has not yet been integrated upstream.
  • Geo McFly (GPL) is an on the fly generator of metalinks based on the downloader's geographical location, written in Python.
  • Metalink Editor (GPL) is a Cross Platform GUI Metalink editor written in Python which allows you to create and edit Metalinks.
  • Metalink Library (GPL) is a Cross Platform Metalink library for Python and PHP based on Metalink Editor with many automatic features and an easy command-line interface to generate Metalink and Torrent files.
  • Metalink Generator can also be used to create Metalinks by filling out a Web form.
  • Metalink tools (GPL) Unix/Windows command line tools for generating Metalinks with hashes and mirror/p2p lists.
  • Dynamic online metalinks based on metadata. Automatically created metalinks based on metadata published by each mirror.
  • MirrorManager (MIT X11 license) is used by the Fedora Project for dynamically listing mirrors.
  • MirrorBrain (OpenOffice.org.
  • Origo is a new and easy to use software development and hosting platform for open and closed source projects. Among many features, it has easy release management and mirroring with automatic metalink generation.

Example Metalink 4.0 .meta4 file

Metalink 4.0 files have the extension .meta4 and are XML text files. They are served with the application/metalink4+xml Internet media type.

 
 
   2009-05-15T12:23:23Z
   
     14471447
     Example
     1.0
     en
     
     A description of the example file for download.
     
     3d6fece8033d146d8611eab4f032df738c8c1283620fd02a1f2bfec6e27d590d
     ftp://ftp.example.com/example.ext
     http://example.com/example.ext
     http://example.com/example.ext.torrent
   
 

Example Metalink/HTTP header fields

Metalink in HTTP header fields makes use of existing standard HTTP header fields such as ETags, Link header fields (for mirrors and P2P), and Instance Digests (for hashes).

Etag: "thvDyvhfIqlvFe+A9MYgxAfm1q5="
Link: ; rel=duplicate
Link: ; rel=duplicate
Link: ; rel=describedby; type="application/x-bittorrent"
Link: ; rel=describedby; type="application/metalink4+xml"
Link: ; rel=describedby; type="application/pgp-signature"
Digest: SHA-256=MWVkMWQxYTRiMzk5MDQ0MzI3NGU5NDEyZTk5OWY1ZGFmNzgyZTJlODYzYjRjYzFhOTlmNTQwYzI2M2QwM2U2MQ==

Example Metalink 3.0 .metalink file

Metalink 3.0 files have the extension .metalink and are XML text files.

 
 
   
     
       
         example-md5-hash
         example-sha1-hash
         
       
       
         ftp://ftp.example.com/example.ext
         ftp://ftp.example.net/example.ext
         http://example.com/example.ext 
         http://example.net/example.ext
         http://example.org/example.ext.torrent
         
         
         
       
     
   
 

See also

References

External links

  • Official website
This article was sourced from Creative Commons Attribution-ShareAlike License; additional terms may apply. World Heritage Encyclopedia content is assembled from numerous content providers, Open Access Publishing, and in compliance with The Fair Access to Science and Technology Research Act (FASTR), Wikimedia Foundation, Inc., Public Library of Science, The Encyclopedia of Life, Open Book Publishers (OBP), PubMed, U.S. National Library of Medicine, National Center for Biotechnology Information, U.S. National Library of Medicine, National Institutes of Health (NIH), U.S. Department of Health & Human Services, and USA.gov, which sources content from all federal, state, local, tribal, and territorial government publication portals (.gov, .mil, .edu). Funding for USA.gov and content contributors is made possible from the U.S. Congress, E-Government Act of 2002.
 
Crowd sourced content that is contributed to World Heritage Encyclopedia is peer reviewed and edited by our editorial staff to ensure quality scholarly research articles.
 
By using this site, you agree to the Terms of Use and Privacy Policy. World Heritage Encyclopedia™ is a registered trademark of the World Public Library Association, a non-profit organization.
 


Copyright © World Library Foundation. All rights reserved. eBooks from Project Gutenberg are sponsored by the World Library Foundation,
a 501c(4) Member's Support Non-Profit Organization, and is NOT affiliated with any governmental agency or department.