Overview

Phase 1: Bridge Export to Google Cloud Instead of Synapse (18 story points, 2.6 dev weeks)

Phase 2: Bridge Running on Google Cloud Instead of AWS (38 story points, 7.6 dev weeks)

Phase 3: Bridge Stores Data in Google Cloud Instead of AWS (Stretch Goal) (69 story points, 13.8 dev weeks)

Estimates assume a velocity of 5 story points per dev week.

Phase 1: Bridge Export to Google Cloud Instead of Synapse

Subtasks:

• Refactor SynapseHelper to be a generic Export Adapter with pluggable implementations for Synapse and Google Cloud (5 story points)
• Define Google Cloud storage format (1 story points)
• Define Google Cloud permissions model (1 story points)
• Implement Google Cloud Export Adapter (5 story points)
• Define and implement Bridge Study config to select between Synapse and Google Cloud (3 story points)
• Implement selector in Exporter to select between Synapse and Google Cloud (3 story points)

Total: 18 story points (2.6 dev weeks)

Phase 2: Bridge Running on Google Cloud Instead of AWS

Total: 38 story points (7.6 dev weeks)

Phase 2A: Bridge Server on Google Cloud

Subtasks:

• Launch Spring MVC Service on Google Cloud (8 story points)
• Set up DNS and SSL (2 story points)
• Set up secure credentials (1 story point)
• Log aggregation (1 story point)
• Dashboards, monitoring, and alarms (1 story point)
• Grant access to AWS Elasticache and RDS from Google Cloud OR partition and create a Redis and SQL instance in Google Cloud (3 story points)

Total: 16 story points (3.2 dev weeks)

Phase 2B: Bridge Exporter on Google Cloud

Subtasks:

• Launch Spring Boot Worker on Google Cloud (8 story points)
• Set up secure credentials (1 story point)
• Log aggregation (1 story point)
• Dashboards, monitoring, and alarms (1 story point)

Total: 11 story points (2.2 dev weeks)

Phase 2C: Bridge Worker on Google Cloud

Subtasks:

• Launch Spring Boot Worker on Google Cloud (8 story points)
• Set up secure credentials (1 story point)
• Log aggregation (1 story point)
• Dashboards, monitoring, and alarms (1 story point)

Total: 11 story points (2.2 dev weeks)

Phase 3: Bridge Stores Data in Google Cloud Instead of AWS (Stretch Goal)

Relevant docs:

• https://cloud.google.com/docs/compare/aws/
• https://cloud.google.com/free/docs/map-aws-google-cloud-platform

AWS Services that need to be replaced:

• DynamoDB → BigTable? Firestore? (53 story points)
• Lambda (Cron scheduler) → Cloud Scheduler (3 story points)
• RDS (MySQL) → Cloud SQL (1 story point)
• S3 → Cloud Storage (5 story points)
• S3 Presigned URLs → Cloud Storage Signed URLs (3 story points)
• S3 Public Webhosting → Cloud Storage? (1 story points)
• SES (Simple Email Service) → NO EQUIVALENT
• SNS Push Notifications → Firebase Cloud Message MAY REQUIRE CLIENT APP CHANGES
• SNS SMS → NO EQUIVALENT
• SQS → PubSub (3 story points)

Total: 69 story points (13.8 dev weeks)

DynamoDB Migration

Subtasks (53 story points)

• Bridge Server (40 story points)
  
  • 25 object/dao pairs (+ 2 survey element classes) (30 story points)
  • DynamoInitializer or equivalent (3 story points)
  • 9 data type marshallers (5 story points)
  • 3 backfill classes, to be ported or removed (2 story points)
• Exporter accesses the following tables directly (5 story points)
  
  • ExportTime
  • HealthDataAttachment
  • HealthDataRecord
  • Study
  • SynapseMetaTables
  • SynapseTables
• Worker accesses the following tables directly (5 story points)
  
  • FitBitTables
  • NotificationConfig
  • NotificationLog
  • Study
  • SynapseSurveyTables
  • SynapseTables
  • UploadSchema
  • WorkerLog
• Master Scheduler (3 story points)
  
  • Config
  • Status

Note: Spring Migration required us to migrate ~56 classes, which took about 3 weeks (15 days), which suggests a velocity of ~3.75 classes per day. However, these were mostly trivial copy-pastes with minimal transformation. Re-writing all the DynamoDB classes would be signficantly more work. We'll base our estimates around 1.8 classes per day.