Page Comparison

...

• CACHE_ROOT is user configurable and defaults to ~/.synapseCache
• [Intermediate Folder] is the [File Handle ID] mod 1000.  This extra level is to reduce fan-out when the number of downloaded files is much greater than 1000.
• [File Handle ID] is the S3 file ID used to upload/download the file
• [File Name] is the file name given by the file handle.  If there is a collision and "ifcollision" is "keepBoth", then the name is modified by appending a number (i.e. file.txt may become file(1).txt)

For older types of Synapse Entities, namely Locationables (i.e. Data and Code objects), the cache folder is:

CACHE_ROOT/[Synapse ID]/[File Name]

where:

•  CACHE_ROOT and [File Name] are as defined above
• [Synapse ID] is the "syn"-prefixed ID of the object

Cache Map Design

There is a file for each Synapse FileHandle ID that has been downloaded or uploaded.  The file is located in the cache folder (above) at the same level as [File Name].

The file contains the location and last-modified time stamp of each downloaded or uploaded file.  The data is stored in a JSON map whose keys are file paths and whose values are timestamps, e.g.

{
 "/path/to/file.txt": "2013-03-14T15:09:26.000Z",
 "/alt/folder/file.txt":  "2013-04-06T15:36:41.000Z"
 }

...

Note: the timestamp is in ISO8601 format ("%Y-%m-%dT%H:%M:%S.000Z"), in UTC (aka "Zulu") time zone.

Note:   the file separator is "/" regardless of the platform (that means 'native' path strings on Windows must be cross-platform normalized before being written into a cache-map file.)

File locking: A client must "lock" a .cacheMap file before accessing it.  Opening a file with write access does not lock it on all platforms, therefore we use the following convention, which all clients must follow:  To lock a file <filename> a client must successfully create an (empty) folder in the file's parent folder and named <filename>.lock.  (Folder creation is atomic and exclusive across platforms.  This lock folder acts as a mutex object.)  If the client is successful, it has exclusive access for ten (10) seconds from the time the lock folder is created.  When the creation time stamp is older than ten seconds, any client may delete the lock folder and the client which has created the folder is obligated to refrain from accessing <filename> unless it locks the file again.  The time limit prevents stale locks from blocking other clients.  The client creating the lock should delete the lock folder when it's done accessing <filename>.

...