Document toolboxDocument toolbox

DOI (Digital Object Identifier)

Owned by Eric Wu (Unlicensed)
Last updated: Apr 16, 2013Version comment

Background

Refer to /wiki/spaces/SYNCOM/pages/33914913 for the context driving the project.

DOI Basics

A DOI is a persistent identifier that can locate resources on the internet.  Examples, doi:10.1038/nature.2012.10240, doi:10.1126/science.1100533, doi:10.6084/m9.figshare.153827, 10.5240/1489-49A2-3956-4B2D-FE16-5.  The first two are Nature and Science publications respectively.  The third is a figshare data set.  The last one is a movie.  Note that, instead of an HTML web page, the movie DOI points to an XML document about the movie.  Given a DOI, locating its digital content on the internet (i.e. mapping a URI to a URL) is the primary use case of DOI.

Refer to the handbook for the restrictions on DOI format.  But basically the length is unlimited, any printable unicode character is allowed, granularity is open.  Need to pay attention to URL encoding but nothing too surprising.

Design

Format

See examples of DOI above.  Within the registered name space, we are free to choose the DOIs.  But we want the DOI to have all the necessary information but no more.

doi:10.7303/syn<id>.<ver>

As shown, DOIs are published as permanent links.  The DOIs better be optimized for humans and search engines.  The above format is chosen per the result of a survey:

doi:10.7303/synapse.144362         8.3%        1
doi:10.7303/sage.synapse.144362    0.0%        0
doi:10.7303/syn144362              58.3%       7
doi:10.7303/sage.syn144362         8.3%        1
doi:10.7303/synapse.syn144362      25.0%       3

Real examples:

DOIDOI Resolver LinkSynapse Target URL

doi:10.7303/syn1720822

http://dx.doi.org/10.7303/syn1720822https://www.synapse.org/#!Synapse:syn1720822
doi:10.7303/syn1720822.1http://dx.doi.org/10.7303/syn1720822.1https://www.synapse.org/#!Synapse:syn1720822/version/1

 

Metadata

The meta data will include URLs to the corresponding Synapse entity.  More than one URL can be used but all should point to the same entity.  One question is to whether to point the DOI to the Synapse web client or to the backend repo service.  Here are the possibilities:

  1. Point the DOI to the web client URL. This type of DOI is good for human beings as the users.  But it prevents the R and Python clients from using the DOI.
  2. Alternatively, we can point the DOI to the repo URL.  The entity is fetched as an JSON object.  This type of DOI favors the analytical clients.  But the human users of the Synapse website will find it less appealing.
  3. Provide two separate sets of DOIs (doing both 1 and 2).  Different client will pick the proper DOI to use.

If we choose the web URLs as the DOI redirect, the R/Python clients need to rewrite the URLs to repo service URLs.  Now imagine the R/Python client calls dx.doi.org to resolve a DOI.  The servers returns a 3XX response with a Synapse web URL (e.g. http://synapse.sagebase.org/#Synapse:syn12345.2).  The R/Python client then rewrites the web client URL with the corresponding repo service URL (e.g. http://repo.sagebase.org/entity/syn12345/version/2) and proceeds to call the repo service to get the entity.  The rewrite is the additional logic for the R/Python clients to use the Synapse DOIs.  If we combine the idea with projects SWC-422 and SWC-423, the web and repo URLs will assume the same format, only the server part needs to swapped for the rewrite.

Chris, "As an alternative, the client could get redirected from the DOI service to the synapse web app. The clients specify an 'application/json' accept header. Based on that, the web app could redirect to the repo service. Sounds more cumbersome, but maybe there's some advantage?"

Another question about metadata is what else to include in the metadata.  The EZID demo lists the author (who), the title (what), and the timestamp (when).  As I was testing around, several fields were indeed required for generating DataCite DOIs.  Here is the DataCite Schemas.  The most recent schema requires Creator, Title, Publisher, PublicationYear besides DOI.  Again, we need pay attention to the escaping, encoding specs here when supplying the data.  Here is the proposed mapping of the required fields.

Creator: Entity CREATED_BY

Title: Entity NAME

Publisher: Sage Bionetworks

PublicationYear: The timestamp this DOI is being created

Granularity

Ideally, we let users explicitly ask to create DOIs (vs. we auto-generating a DOI for each entity version).  Creating a DOI for a version of a leaf entity also means 1) the entity of the particular version is immutable/undeletable and 2) the entity of the particular version is public and cannot be changed to private.  This is in a sense publishing the entity.  As of now, the publishing aspect of it is still unclear.  For possible approaches, see this page.  We are still looking for a reasonable implementation that is natural to users and at the same time not too intrusive to the current architecture.

The current model is to let users assume the ultimate responsibility of DOIs.  This is to say, users call Synapse to create DOIs for versions of entities, but Synapse does not enforce that the entities are public and immutable.  It is the users' responsibility to make sure that being the case.  In the future, when the specs of publishing are clear, we may add an additional API that does both publishing the entities and creating the DOIs in one step.  It's worth mentioning that figshare automatically mints DOIs for every object.  There is a separate figshare action to publish something.

Asynchronous Client to the DOI Service

According to Joan, "it takes something less than 5 seconds to register a DOI" and 5 minutes for the DOI to propagate through the Handle system.  My test against the mint-DOI operation shows latencies on the client side between 3 and 4 seconds.  The average is 3.4 seconds.  There isn't noticeable difference between calling from my local machine and calling from an east-1 EC2 instance.

DataCite Metadata Store also has DOI APIs.  I suspect it is what EZID ultimately calls.  The servers of mds.datacite.org are EC2 instances in the EU WEST region.  A ping to the servers takes about 200 ms. 

 How do we handle failures?  Record failed DOIs in the database?

Data Migration

Involve of avoid?

DOI

EZID

DataCite