The following provides instructions on how to log on to the Sage Scientific Compute workspace using your Synapse credentials, and how to use the products provided in the AWS Service Catalog to setup or modify EC2 instances and S3 buckets.

...

Note: You can add additional custom tags when provisioning resources however there are 3 reserved tags that you should avoid adding: Department, Project, CostCenter and OwnerEmail. The owner email tag is automatically set to <Synapse Username>@synapse.org

Notifications

Please skip the Notifications pane. SNS notifications are not operational at this time.

...

The AWS SSM allows direct access to private instances from your own computer terminal. To setup access with the AWS SSM we need to create a special Synapse personal access token (PAT) that will work with the Sage Service Catalog. This is special PAT that can only be created using this workflow, creating a PAT from the Synapse personal token manager web page will NOT work.

1. Request a Synapse PAT by visiting https://sc.sageit.org/personalaccesstoken , for Sage employees, or https://ad.strides.sc.sageit.org/personalaccesstoken for AMP-AD members. (You may need to login to Synapse.) If you have already created a PAT through this mechanism and are repeating the process you must first visit the token management page in Synapse and delete the existing one with the same name.

2. After logging into Synapse a file containing the PAT, which is a long character string (i.e. eyJ0eXAiOiJ...Z8t9Eg), is returned to you. Save the file to your local machine and note the location where you saved it to then close the browser session.

Note: At this point you can verify that the PAT for the Service Catalog was successfully created by viewing the Synapse token management page. When the PAT expires you will need to repeat these steps to create a new PAT. The PAT should look something like this

...

1. Setup profile for SSM access

2. Run an application on the EC2 (i.e. docker run -p 80:80 httpd)

   Code Block

   [ec2-user@ip-10-49-26-50 ~]$ docker run -p 80:80 httpd Unable to find image 'httpd:latest' locally latest: Pulling from library/httpd 33847f680f63: Pull complete d74938eee980: Pull complete 963cfdce5a0c: Pull complete 8d5a3cca778c: Pull complete e06a573b193b: Pull complete Digest: sha256:71a3a8e0572f18a6ce71b9bac7298d07e151e4a1b562d399779b86fef7cf580c Status: Downloaded newer image for httpd:latest AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 172.17.0.2. Set the 'ServerName' directive globally to suppress this message AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 172.17.0.2. Set the 'ServerName' directive globally to suppress this message [Thu Jul 22 23:54:12.106344 2021] [mpm_event:notice] [pid 1:tid 140706544895104] AH00489: Apache/2.4.48 (Unix) configured -- resuming normal operations [Thu Jul 22 23:54:12.107307 2021] [core:notice] [pid 1:tid 140706544895104] AH00094: Command line: 'httpd -D FOREGROUND'

3. To provide access to that app, an SC user can use the port forwarding feature to gain access to the app by running the AWS SSM CLI command:

   Code Block
   aws ssm start-session --profile service-catalog \ --target i-0fd5c9ff0ef675ceb \ --document-name AWS-StartPortForwardingSession \ --parameters '{"portNumber":["80"],"localPortNumber":["9090"]}'

4. To provide access to that app in the Windows Command Prompt use this syntax:

   Code Block
   aws ssm start-session --profile service-catalog \ --target i-0fd5c9ff0ef675ceb \ --document-name AWS-StartPortForwardingSession \ --parameters "{\"portNumber\":[\"80\"],\"localPortNumber\":[\"9090\"]}"

5. Now you should be able to access that app on your local machine at http://localhost:9090.

...

Using the update action allows you to change parameters or update to a new version of the product. WARNING: changes to configuration parameters usually result in a recreation (“replacement”) of the instance, any data saved on the instance will be lost, and the nature of the update by Amazon is difficult to predict. We recommend that you save any important data to S3, provision a new instance and terminate the original.

Terminate

The terminate action deletes the instance permanently.

...

Scheduled jobs are essentially cron jobs that we’ve setup to run in AWS batch. This product allows you to run an arbitrary task on the cloud to process your workload. The task must be run using a docker image. The task can be manually triggered or it can be setup to run on a schedule.

Creating Scheduled Job Products

To create a scheduled job, select the “Products List” from the navigation panel on the left. Next, select “Scheduled Jobs” from the list. On the product page, click the orange “LAUNCH PRODUCT” button under the product description, then fill out the wizard. Most of the parameters contain helpful information describing the valid inputs.

Note: The “Environment” parameters are required fields. You can replace the default values, however please do not leave these fields empty. Also pay special attention to the formatting that’s required for the values. The deployment will fail if the formatting isn’t correct.

...

Manually Run Scheduled Jobs

Once provisioning is complete your Scheduled Jobs product will appear in the “Provisioned Products” list, showing status Available. Select “Provisioned Product Details” from the navigation panel on the left, and click on your product. A product that has a “Succeeded” event will have outputs that include a “SubmitJobApi” link. That is a REST request to trigger the job and should return something like this..

...

View Scheduled Job Status

Click on the “Jobs” link to view the batch job status. Once triggered the job should transition from STARTING → RUNNING → SUCCEEDED.

...

Access Scheduled Job Logs

The job will send logs to AWS cloudwatch. To access the logs click on the “Logs” link in PROVISION_PRODUCT outputs. Below is an example of a log for a job that ran the command “printenv”.

...

Scheduled Job Secrets

Secrets are stored in the AWS secrets manager and exposed to the job as environment variables. The logs above print out the environment variables from the job. Take note of the “SCHEDULED_JOB_SECRETS” parameter in the logs. The secrets that are passed into this product are exposed as environment variables in the logs by the “printenv” command. Please make sure to never expose secrets in this way. DO NOT PRINT ENVIRONMENT VARIABLES.

Accessing Scheduled Job Secrets

Job secrets can be access a number of different ways. The first way is simply to get it from the docker container environment variable SCHEDULED_JOB_SECRETS.

Environment variable example:

...

# Use this code snippet in your app.
# If you need more information about configurations or implementing the sample code, visit the AWS docs:   
# https://aws.amazon.com/developers/getting-started/python/

import boto3
import base64
from botocore.exceptions import ClientError


def get_secret():

    secret_name = "arn:aws:secretsmanager:us-east-1:465877038949:secret:JobSecrets-rEx1eKL9pokj-h7hCGO"
    region_name = "us-east-1"

    # Create a Secrets Manager client
    session = boto3.session.Session()
    client = session.client(
        service_name='secretsmanager',
        region_name=region_name
    )

    # In this sample we only handle the specific exceptions for the 'GetSecretValue' API.
    # See https://docs.aws.amazon.com/secretsmanager/latest/apireference/API_GetSecretValue.html
    # We rethrow the exception by default.

    try:
        get_secret_value_response = client.get_secret_value(
            SecretId=secret_name
        )
    except ClientError as e:
        if e.response['Error']['Code'] == 'DecryptionFailureException':
            # Secrets Manager can't decrypt the protected secret text using the provided KMS key.
            # Deal with the exception here, and/or rethrow at your discretion.
            raise e
        elif e.response['Error']['Code'] == 'InternalServiceErrorException':
            # An error occurred on the server side.
            # Deal with the exception here, and/or rethrow at your discretion.
            raise e
        elif e.response['Error']['Code'] == 'InvalidParameterException':
            # You provided an invalid value for a parameter.
            # Deal with the exception here, and/or rethrow at your discretion.
            raise e
        elif e.response['Error']['Code'] == 'InvalidRequestException':
            # You provided a parameter value that is not valid for the current state of the resource.
            # Deal with the exception here, and/or rethrow at your discretion.
            raise e
        elif e.response['Error']['Code'] == 'ResourceNotFoundException':
            # We can't find the resource that you asked for.
            # Deal with the exception here, and/or rethrow at your discretion.
            raise e
    else:
        # Decrypts secret using the associated KMS key.
        # Depending on whether the secret is a string or binary, one of these fields will be populated.
        if 'SecretString' in get_secret_value_response:
            secret = get_secret_value_response['SecretString']
        else:
            decoded_binary_secret = base64.b64decode(get_secret_value_response['SecretBinary'])
            
    # Your code goes here. 

Tag options, Notifications, and Review

The final three screens of the wizard are the same as in Creating an EC2 Instance above.

Best Practices

When to Use the Service Catalog

...