Versions Compared

Old Version 8

changes.mady.by.user John Hill

Saved on

compared with

New Version 9

changes.mady.by.user John Hill

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Table of Contents

Synapse Status and Maintenance Modes

A synapse stack can be in the following modes:

Mode

Accepted (200)

Unavailable (503)

Description

READ_WRITE

GET, PUT, POST, & DELETE

none

This is the standard operating mode for Synapse. While in this mode all HTTP methods will be accepted (200s).

READ_ONLY

GET

PUT, POST, & DELETE

While in this mode Synapse is read-only. While in this mode PUT, POST, and DELETE will all result in a 503 (Service Unavailable) response.

DOWN

none

GET, PUT, POST, & DELETE

While in this mode Synapse will return 503 (Service Unavailable) for any non-administration call

Note: Any 'reop/v1/admin/' will be accepted (200s) for any mode including READ_ONLY and DOWN. *

Getting the Current Stack Status

The following example shows how to get the current status. Note: This call does not require an authentication token.
Request

Code Block

curl -i -k -H Accept:application/json -H Content-Type:application/json http://localhost:8080/services-repository-0.8-SNAPSHOT/repo/v1/admin/synapse/status

Response

Code Block

HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Content-Type: application/json
Transfer-Encoding: chunked
Date: Sat, 10 Dec 2011 23:35:07 GMT
{
    "status":"READ_WRITE",
    "pendingMaintenanceMessage":"Synapse will be down for maintenance Saturday 12/10 from 11:58 to ll:59.",
    "currentMessage":"Synapse is currently up and running."
}

There are two messages associated with the current status.

  • currentMessage - This is the massage that is returned with any 503 (Service Unavailable) response. It is used to inform the caller of the current state. The administrator should set the message whenever the status changes.
  • pendingMaintenanceMessage - This message shows pending maintenance information. This can be shown to end users to let them know of upcoming down time.

Updating the Current Status

Only an administrator can change the current status of Synapse. The administrator can set all three fields (status, currentMessage, and pendingMaintenanceMessage) with a single call:
Request

Code Block

curl -i -k -H sessionToken:<your admin token> -H Accept:application/json -H Content-Type:application/json -X PUT -d '{
  "status": "DOWN",
  "currentMessage": "Synapse is currently DOWN for maintenance.  It will be back up in few minute.",
  "pendingMaintenanceMessage":null,
}' http://localhost:8080/services-repository-0.8-SNAPSHOT/repo/v1/admin/synapse/status

Response

Code Block

HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Content-Type: application/json
Transfer-Encoding: chunked
Date: Sat, 10 Dec 2011 23:47:18 GMT
{
    "status":"DOWN",
    "currentMessage":"Synapse is currently DOWN for maintenance.  It will be back up in few minute."
}

Creating a Repository Backup Snapshot

...

  1. First, log-in as an administrator (use the platform@sagebase.org account) to get an administrator's session token:
    Code Block
    curl -i -k -H sessionToken:YourSessionToken -H Accept:application/json -H Content-Type:application/json -d '{
  "email": "<admin username>",
  "password": "<admin password>"
}' https://staging-auth.elasticbeanstalk.com/auth/v1/session
  2. Once you have the admin token the backup daemon can be started with the following command:
    Request
    Code Block
    curl -i -k -H sessionToken:<your admin token> -H Accept:application/json -H Content-Type:application/json -d '{
}' https://staging-reposervice.elasticbeanstalk.com/repo/v1/startBackupDaemonadmin/daemon/backup
    Response
    Code Block
    HTTP/1.1 201 Created
Content-Type: application/json
Date: Thu, 18 Aug 2011 22:38:56 GMT
Server: Apache-Coyote/1.1
Content-Length: 256
Connection: keep-alive
{
	"id":"6695",
	"type":"BACKUP",
	"status":"STARTED",
	"errorMessage":null,
	"progresssMessage":"Starting...",
	"progresssCurrent":0,
	"progresssTotal":0,
	"errorDetails":null,
	"backupUrl":null,
	"totalTimeMS":0,
	"startedBy":"platform@sagebase.org",
	"startedOn":1313707136615
}
  3. Once the daemon is started, its progress can be monitored using its 'id' returned from the 'admin/startBackupDaemondaemon' call:
    Request
    Code Block
    curl -i -k -H sessionToken:<your admin token> -H Accept:application/json -H Content-Type:application/json  https://staging-reposervice.elasticbeanstalk.com/repo/v1/daemonStatusadmin/daemon/6695
    Response
    Code Block
    HTTP/1.1 200 OK
Content-Type: application/json
Date: Thu, 18 Aug 2011 22:46:06 GMT
Server: Apache-Coyote/1.1
Content-Length: 1114
Connection: keep-alive

{
	"id":"6695",
	"type":"BACKUP",
	"status":"FAILED",
	"errorMessage":"Access Denied",
	"progresssMessage":"Starting to upload temp file: /opt/tomcat7/temp/BackupDaemonJob6695-762955157468269943.zip to S3...",
	"progresssCurrent":785,
	"progresssTotal":863,
	"errorDetails":"Status Code: 403, AWS Request ID: CF8A8149099FE5F9, AWS Error Code: AccessDenied,...",
	"backupUrl":null,
	"totalTimeMS":26145,
	"startedBy":"platform@sagebase.org",
	"startedOn":1313707136615
}
    In this example, we are showing that the backup failed. When there is a failure, the status will show as 'FAILED', and 'errorMessage' and 'errorDetails' should show the message and stack trace of the failure. In this example, the service IAM user did not have permission to write the backup file to S3. After fixing the AWS permissions we can try running the backup again:
    Response
    Code Block
    HTTP/1.1 200 OK
Content-Type: application/json
Date: Thu, 18 Aug 2011 23:00:10 GMT
Server: Apache-Coyote/1.1
transfer-encoding: chunked
Connection: keep-alive

{
	"id":"6696",
	"type":"BACKUP",
	"status":"COMPLETED",
	"errorMessage":null,
	"progresssMessage":"Finished: BACKUP",
	"progresssCurrent":863,
	"progresssTotal":863,
	"errorDetails":null,
	"backupUrl":"https://s3.amazonaws.com/stagingdata.sagebase.org/BackupDaemonJob6696-911306061719227050.zip",
	"totalTimeMS":24880,
	"startedBy":"platform@sagebase.org",
	"startedOn":1313708374613
}
    This time we can see that the backup 'status'='COMPLETED', and that the 'backupUrl' is no longer null and that the entire backup ~25 seconds to complete. We can now use the file found at the 'backupUrl' to restore synapse.

...

  1. Fist make sure the backup file that you want to restore is in the S3 bucket that belongs to that service. A service can only download files from its own bucket.
  2. Once the backup in place in S3 we are ready to authenticate and get an administrator's session token:
    Code Block
    curl -i -k -H sessionToken:YourSessionToken -H Accept:application/json -H Content-Type:application/json -d '{
  "email": "<admin username>",
  "password": "<admin password>"
}' http://localhost:8080/services-authentication-0.6-SNAPSHOT/auth/v1/session
  3. Now use the administrator's token to start the restore daemon. You must provide the file name of the backup file found on S3 to the daemon:
    Request
    Code Block
    curl -i -k -H sessionToken:YourSessionToken -H Accept:application/json -H Content-Type:application/json -d '{
  "url": "BackupDaemonJob6696-911306061719227050.zip"
}' http://localhost:8080/services-repository-0.6-SNAPSHOT/repo/v1/startRestoreDaemonadmin/daemon/restore
    Response
    Code Block
    HTTP/1.1 201 Created
Server: Apache-Coyote/1.1
Content-Type: application/json
Transfer-Encoding: chunked
Date: Thu, 18 Aug 2011 23:31:28 GMT

{
	"id":"4",
	"type":"RESTORE",
	"status":"STARTED",
	"progresssMessage":"Starting...",
	"progresssCurrent":0,
	"progresssTotal":0,
	"errorMessage":null,
	"errorDetails":null,
	"backupUrl":null,
	"totalTimeMS":0,
	"startedBy":"platform@sagebase.org",
	"startedOn":1313710288153
}
  4. Once the daemon is started its progress can be monitored in the same way as we monitored the backup daemon, using the 'id' provided by the /admin/startRestoreDaemondaemon:
    Request
    Code Block
    curl -i -k -H sessionToken:YourSessionToken -H Accept:application/json -H Content-Type:application/json  http://localhost:8080/services-repository-0.6-SNAPSHOT/repo/v1/admin/daemonStatusdaemon/4
    Response
    Code Block
    HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Content-Type: application/json
Transfer-Encoding: chunked
Date: Thu, 18 Aug 2011 23:37:16 GMT

{
	"id":"4",
	"type":"RESTORE",
	"status":"FAILED",
	"progresssMessage":"Starting to download the file from S3...",
	"progresssCurrent":0,
	"progresssTotal":0,
	"errorMessage":"Access Denied",
	"errorDetails":"Status Code: 403, AWS Request ID: 5A6F56E7416E1203, AWS Error Code: AccessDenied, AWS Error Message: Access Denied, S3 Extended Request ID: ...",
	"totalTimeMS":781,
	"startedBy":"platform@sagebase.org",
	"startedOn":1313710288153
}
    Just like in our backup example, the restore failed as indicated by the 'status'='FAILED'. Again it looks like the service did not have permission to download the S3 file. Upon closer inspection we find that we have a cut and paste error with the file name. Once our error is identified, we can start another restore daemon with the following results:
    Response
    Code Block
    HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Content-Type: application/json
Transfer-Encoding: chunked
Date: Thu, 18 Aug 2011 23:59:13 GMT

{
	"id":"5",
	"type":"RESTORE",
	"status":"COMPLETED",
	"progresssMessage":"Finished: RESTORE",
	"progresssCurrent":1164611,
	"progresssTotal":1164611,
	"errorMessage":null,
	"errorDetails":null,
	"backupUrl":"https://s3.amazonaws.com/devdata.sagebase.org/BackupDaemonJob6696-5911306061719227050.zip",
	"totalTimeMS":46800,
	"startedBy":"platform@sagebase.org",
	"startedOn":1313711855784
}
    This time we can see that the restore 'status'='COMPLETED', and that the entire restore took ~47 seconds to complete. We have successfully migrated all data from the staging to our local repository service.