Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

At a high level, a VPC is a named container for virtual network.  For example, the Synapse production VPC was assigned a CIDR of 10.2220.0.0/16.  This means Synapse networks contains 65,536 IP address between 10.2220.0.0 and 10.2220.255.255.  The VPC is then further divided into subnets which can be declared as either Private or Public.  Instances contained in Public subnets can be assigned public IP addresses and therefore can be seen on the public internet.  Conversely, instances deployed in private subnets do not have public IP address and cannot be seen by the public internet.  Instance in private subnets are only visible to machines deployed within the containing VPC.  We will cover how internal developers can access to machines in private subnets in a later section.Image RemovedImage Added

Figure 1. Synapse VPC

The Synapse VPC is divided into two six public and six twenty-four private subnets that span two six Availability Zone (AZ): us-east-1a and , us-east-b, us-east-c, us-east-d, us-east-e, us-east-f (see Figure 1).   The deployment of all instances (EC2s and Databases) across two six zones ensure redundancy should an outage occur in a single zone (such as the event in April 2011), and broadens the availablity of instance types. For details on each subnet see Table 1.

...

Subnet NameTypeCIDRFirstLastTotal
PublicUsEast1aPublic10.2220.0.0/202110.2220.0.010.2220.157.25540962048
PublicUsEast1bPublic10.2220.168.0/202110.2220.168.010.2220.3115.25540962048
RedPrivateUsEast1aPublicUsEast1cPrivate10.2220.3216.0/2110.2220.3216.010.2220.3923.2552048
RedPrivateUsEast1bPublicUsEast1dPrivate10.2220.4024.0/2110.2220.4024.010.2220.4731.2552048
BluePrivateUsEast1aPublicUsEast1ePrivate10.2220.4832.0/2110.2220.48.010.2220.5539.2552048
BluePrivateUsEast1bPublicUsEast1fPrivate10.2220.5640.0/2110.20.40.010.20.47.2552048
RedPrivateUsEast1aPrivate10.22.56.0.20.48.0/2410.20.48.010.20.48.255256
RedPrivateUsEast1bPrivate10.20.49.0/2410.22.63.2552048
GreenPrivateUsEast1aPrivate10.22.64.0/2110.22.64.010.22.71.2552048
GreenPrivateUsEast1bPrivate10.22.72.0/2110.22.72.010.22.79.255204820.49.010.20.49.255256
RedPrivateUsEast1cPrivate10.20.50.0/2410.20.50.010.20.50.255256
RedPrivateUsEast1dPrivate10.20.51.0/2410.20.51.010.20.51.255256
RedPrivateUsEast1ePrivate10.20.52.0/2410.20.52.010.20.52.255256
RedPrivateUsEast1fPrivate10.20.53.0/2410.20.53.010.20.53.255256
BluePrivateUsEast1aPrivate10.20.56.0/2410.20.56.010.20.56.255256
BluePrivateUsEast1bPrivate10.20.57.0/2410.20.57.010.20.57.255256
BluePrivateUsEast1cPrivate10.20.58.0/2410.20.58.010.20.58.255256
BluePrivateUsEast1dPrivate10.20.59.0/2410.20.59.010.20.59.255256
BluePrivateUsEast1ePrivate10.20.60.0/2410.20.60.010.20.60.255256
BluePrivateUsEast1fPrivate10.20.61.0/2410.20.61.010.20.61.255256
GreenPrivateUsEast1aPrivate10.20.64.0/2410.20.64.010.20.64.255256
GreenPrivateUsEast1bPrivate10.20.65.0/2410.20.65.010.20.65.255256
GreenPrivateUsEast1cPrivate10.20.66.0/2410.20.66.010.20.66.255256
GreenPrivateUsEast1dPrivate10.20.67.0/2410.20.67.010.20.67.255256
GreenPrivateUsEast1ePrivate10.20.68.0/2410.20.68.010.20.68.255256
GreenPrivateUsEast1fPrivate10.20.69.0/2410.20.69.010.20.69.255256
OrangePrivateUsEast1aPrivate10.20.72.0/2410.20.72.010.20.72.255256
OrangePrivateUsEast1bPrivate10.20.73.0/2410.20.73.010.20.73.255256
OrangePrivateUsEast1cPrivate10.20.74.0/2410.20.74.010.20.74.255256
OrangePrivateUsEast1dPrivate10.20.75.0/2410.20.75.010.20.75.255256
OrangePrivateUsEast1ePrivate10.20.76.0/2410.20.76.010.20.76.255256
OrangePrivateUsEast1fPrivate10.20.77.0/2410.20.77.010.20.77.255256

Table 1.  Synapse VPC 10.22.0.0/16 subnets

...

Given that subnet address cannot overlap and must be allocated from a fix range of IP addresses (defined by the VPC), it would be awkward to dynamical allocate new subnets for each new stack.  Instead, we decided to create fixed/permanent subnets to deploy new stacks into each week, thus returning to the blue green naming scheme.  Since we occasionally need to have three production-capable stacks running at a time, we included a red subnets.  We will continue to give each new stack a numeric designation, but we will also assign a color to each new stack.  Each color will be assigned in a round-robin manner.  For example, stack 226 will be deployed to Red, 227 to Blue, 228 to Green, 229 to Red etc.. The Orange subnets are reserved for shared resources such as the ID generator database.


Public subnets

Any machine that must be publicly visible must be deployed to a public subnet.  Since it is not possible to isolate public subnets from each other, there was little value in creating a public subnet for each color.  Instead, one public subnet per availability zone was created.  Each public subnet will contain the Elastic Beanstalk loadbalancers for each environment (portal, repo, works) of each stack.  There is also a NAT Gateway deployed to each public subnet (need one NAT per AZ).  We will cover the details of the NAT Gateways in a later section.

...

DestinationTargetDescription
10.2220.0.0/16localThe default entry that identifies all address within this VPC (10.2220.0.0/16) can be found within this VPC (local)
10.1.0.0/16VPN Peering ConnectionIf the destination IP address in the VPN VPC (10.1.0.0/16), then use the VPC peering connection that connects the two VPCs
0.0.0.0/0Internet GatewayIf the destination is any other IP address (0.0.0.0/0), then use the VPC's Internet Gateway.

...

Color GroupCIDRFirstLastTotal
Red10.2220.3248.0/202110.2220.3248.010.2220.4753.25540962048
Blue10.2220.4856.0/202110.2220.4856.010.22.63.25540962048
Green10.2220.64.0/202110.2220.64.010.22.7969.25540962048

Table 4. CIDR for each Color Group

...

TypeCIDR/SGPort RangDescription
HTTP10.2220.3248.0/202180Allow machines within either Red private subnet to connect with HTTP
SSH10.1.0.0/1622Allows traffic from the Sage VPN (10.1.0.0/16) to connect with SSH
HTTPLoadbalancer Security Group80Allow the loadbalancers from this stack to connect with HTTP

...

TypeCIDRPort RangeDescription
MySQL10.2220.3248.0/20213306Allows machines within either Red private subnet to connect on port 3306
MySQL10.1.0.0/163306Allows traffic from the Sage VPN (10.1.0.0/16) to connect on port 3306

...

DestinationTargetDescription
10.2220.0.0/16localThe default entry that identifies all address within this VPC (10.2220.0.0/16) can be found within this VPC (local)
10.1.0.0/16VPN Peering ConnectionIf the destination IP address in the VPN VPC (10.1.0.0/16), then use the VPC peering connection that connects the two VPCs
0.0.0.0/0NAT Gateway us-east-1aIf the destination is any other IP address (0.0.0.0/0), then use the NAT Gateway that is within the same availability zone.

...

Earlier we stated that instances deployed to private subnets are only visible to machines within the same VPC.  This is a true statement.  So how does a developer connect to an instances deployed in a private subnet?  The short answer: the developer must tunnel into the target VPC.  Figure 2. shows an example of an of how a developer connects to the MySQL database at 10.2220.47.107 contained within a private subnet.

...

When a developer makes a VPN connection to vpn.sagebase.org, the VPN client will add a route for 10.2220.0.0/16 on the developer's machine.  This route will direct all network traffic with a destination within 10.2220.0.0/16 to Sage-Admin-Central firewall.  Once connected to the VPN, an attempt to connect to 10.2220.47.107 will proceed as follows:

  1. The call to 10.2220.47.107 will be directed to the Sage Firewall
  2. If the VPN users belongs to the 'Synapse Developers' group the request will be forwarded to the SNAT
  3. The SNAT (a type of NAT) will replace the caller's real address with its own address: 10.1.33.107 and forward the request
  4. A route table entry directs all traffic with a destination of 10.2220.0.0/16 to the Synapse VPC Peering connection.
  5. The peering connection has been granted access to allow all traffic between the two VPC (10.1.0.0/16 and 10.2220.0.0/16) so the request can tunnel to the Synapse VPC
  6. Once in the Synapse VPC, requestor's address will be 10.1.33.107.  Since the database security group allows 10.1.0.0/16 to connect on port 3306, the connection is allowed if the correct database username/password is provided.

...

The resources of the Synapse stack are manged as five separate cloud formation stacks templates shown in Table 9..  All template JSON files are created by merging actual templates with input parameters using Apache's Velocity.

Stack NameFrequencyDescriptionExample
synapse-prod-vpcone timeA one-time stack to create the Synapse VPC and associated subnets, route tables, network ACLs, security groups, and VPC peering
View file
namesynapse-prod-vpc.json
height250
prod-<instance>-shared-resourcesonce per weekCreated once each week (i.e. prod-226-shared-resources) to build all of the of the shared resources of a given stack including MySQL databases and the Elastic Beanstalk Application.
View file
nameprod-226-shared-resources.json
height250
repo-prod-<instance>-<number>one per version per weekCreated for each version of the repository-services deployed per week.  For example, if we needed two separate instances of repository services for prod-226, then the first would be repo-prod-226-0 and the second would be repo-prod-226-1.  Creates the Elastic Beanstalk Environment and CNAME for the repository-services.
View file
nameworkers-prod-226-0.json
height250
workers-prod-<instance>-<number>one per version per weekCreated for each version of the workers deployed per week.  Similar to repo-prod.  Creates the Elastic Beanstalk Environment and CNAME for the workers.
View file
nameworkers-prod-226-0.json
height250
portal-prod-<instance>-<number>one per version per weekCreated for each version of the portal deployed per week.  Similar to repo-prod.  Creates the Elastic Beanstalk Environment and CNAME for the portal.
View file
nameportal-prod-226-0.json
height250

Table 9.  The cloudformation templates of Synapse

...