Versions Compared

Old Version 157

changes.mady.by.user Bruce Hoff

Saved on

compared with

New Version Current

changes.mady.by.user Joni Harker

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Sage Scientific Compute is a secure compute workspace for Sage Bionetworks' workers and their collaborators. The offerings are:

...

On the left is a navigation sidebar. If you do not see, it, look for a hamburger icon in the upper left, and click on it to expand the navigation sidebar. The options in the navigation bar are “Products” and “Provisioned Products”. After you provision a product selected from “Products”, you will be able to see your products under “Provisioned Products”.

...

Compute Instances

...

Compute Instance Products

We currently offer three varieties of virtual machines, each preconfigured for a given purpose, described below.

...

This product is a Microsoft Windows instance.

Creating

...

Compute Products

To create an instance, select “Products List” from the navigation panel on the left. Next, select from the list one of the EC2 compute products described above. On the product page, click the orange “LAUNCH PRODUCT” button under the product description, then fill out the wizard as follows:

...

  • Name: this names the product and the instance. You’ll use this to manage the product later. Please include your name in the product, e.g. if your name is Jane Doe and you are provisioning a Linux instance for your project Foo, you could name it jdoe-linux-foo.

  • Version: choose a version of the product to provision.

Parameters

...

  • EC2 Instance Type: there are many instance types to choose from. To learn about their details, including CPU and memory, see https://aws.amazon.com/ec2/instance-types/. To learn about their costs, see https://aws.amazon.com/ec2/pricing/on-demand/ or use the AWS pricing calculator.Linux Distribution: (EC2 Linux product only) the variety of Linux OS that will be installed.

  • Disk Size: the amount of local storage, in gigabytes.  Please treat the disk as temporary storage.  Long term storage of data should be in a bucket (see below).

...

...

Note: The owner email tag is automatically set to <Synapse Username>@synapse.org

Notifications

Please skip the Notifications pane. SNS notifications are not operational at this time.

...

  • ConnectionURI: if your product has a ConnectionURI link, this will open a shell prompt in a browser tab. When you are done with your session click “Terminate” in the upper right corner.

  • NotebookConnectionURI: Notebook products contain a NotebookConnectionURI link, which will open a notebook in the browser.ConnectionInstructions: For Windows products, click on the ConnectionInstructions link and follow the steps provided therein the browser.

The following instructions guide you to set up command line (“shell”) access via AWS SSM. Windows users can add remote desk top on top of SSM access, and details instructions are below. Sage Bionetworks workers can skip the AWS SSM set-up and instead request that Sage IT provide access via the Sage VPN. Instructions for doing this are here.

Create a Synapse personal access token

The AWS SSM allows direct access to private instances from your own computer terminal. To setup access with the AWS SSM we need to create a special Synapse personal access token (PAT) that will work with the Sage Service Catalog. This is special PAT that can only be created using this workflow, creating a PAT from the Synapse personal token manager web page will NOT work.

  1. Request a Synapse PAT by visiting https://sc.sageit.org/personalaccesstoken , for Sage employees, or https://ad.strides.sc.sageit.org/personalaccesstoken for AMP-AD members. (You may need to login to Synapse.) If you have already created a PAT through this mechanism and are repeating the process you must first visit the token management page in Synapse and delete the existing one with the same name.

  2. After logging into Synapse a file containing the PAT, which is a long character string (i.e. eyJ0eXAiOiJ...Z8t9Eg), is returned to you. Save the file to your local machine and note the location where you saved it to then close the browser session.

Note: At this point you can verify that the PAT for the Service Catalog was successfully created by viewing the Synapse token management page. When the PAT expires you will need to repeat these steps to create a new PAT. The PAT should look something like this

...

8. If you plan to use Docker with your instance(for example, with Rstudio or Jupyter notebooks), complete the instructions in SSM access to applications.

...

  1. Setup profile for SSM access

  2. Run an application on the EC2 (i.e. docker run -p 80:80 httpd)

    Code Block
    [ec2-user@ip-10-49-26-50 ~]$ docker run -p 80:80 httpd
Unable to find image 'httpd:latest' locally
latest: Pulling from library/httpd
33847f680f63: Pull complete
d74938eee980: Pull complete
963cfdce5a0c: Pull complete
8d5a3cca778c: Pull complete
e06a573b193b: Pull complete
Digest: sha256:71a3a8e0572f18a6ce71b9bac7298d07e151e4a1b562d399779b86fef7cf580c
Status: Downloaded newer image for httpd:latest
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 172.17.0.2. Set the 'ServerName' directive globally to suppress this message
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 172.17.0.2. Set the 'ServerName' directive globally to suppress this message
[Thu Jul 22 23:54:12.106344 2021] [mpm_event:notice] [pid 1:tid 140706544895104] AH00489: Apache/2.4.48 (Unix) configured -- resuming normal operations
[Thu Jul 22 23:54:12.107307 2021] [core:notice] [pid 1:tid 140706544895104] AH00094: Command line: 'httpd -D FOREGROUND'

  3. To provide access to that app, an SC user can use the port forwarding feature to gain access to the app by running the AWS SSM CLI command:

    Code Block
    aws ssm start-session --profile service-catalog \
                      --target i-0fd5c9ff0ef675ceb \
                      --document-name AWS-StartPortForwardingSession \
                      --parameters '{"portNumber":["80"],"localPortNumber":["9090"]}'

  4. To provide access to that app in the Windows Command Prompt use this syntax:

    Code Block
    aws ssm start-session --profile service-catalog \
                      --target i-0fd5c9ff0ef675ceb \
                      --document-name AWS-StartPortForwardingSession \
                      --parameters "{\"portNumber\":[\"80\"],\"localPortNumber\":[\"9090\"]}"

  5. Now you should be able to access that app on your local machine at http://localhost:9090.

...

  1. Connect to the Windows shell.

  2. Create a new user and and it to the “Administrators” group

    Code Block
    $Password = ConvertTo-SecureString "P@ssW0rD!" -AsPlainText -Force
New-LocalUser "admin" -Password $Password -PasswordNeverExpires
Add-LocalGroupMember -Group "Administrators" -Member "admin"

  3. Follow the SSM access to applications instructions to setup port forwarding to windows RDP

    Code Block
    aws ssm start-session --profile service-catalog \
                      --target i-0fd5c9ff0ef675ceb \
                      --document-name AWS-StartPortForwardingSession \
                      --parameters '{"portNumber":["3389"],"localPortNumber":["3389"]}'

  4. Install the Microsoft Remote Desktop client on your computer.

    1. Click “+” to add a new PC. In the “PC Name” field, enter “localhost”. 

  5. Log in with username “admin” and password "P@ssW0rD!"

...

  1. Once an instance is provisioned locate its instance id (i.e. i-06531e8f977ca20ea)

  2. Create a Jira IT issue and make a request to associate your jumpcloud user with that instance id

  3. Once Sage IT will make the association you can login to the VPN and use remote desktop to login to the instance with using its IP address (seen in the Outputs tab of the Service Catalog provisioned product) as the PC name and your Jumpcloud credentials as user/password.

Provisioning and Using a Notebook

...

Using the update action allows you to change parameters or update to a new version of the product. WARNING: changes to configuration parameters usually result in a recreation (“replacement”) of the instance, any data saved on the instance will be lost, and the nature of the update by Amazon is difficult to predict. We recommend that you save any important data to S3, provision a new instance and terminate the original.

Terminate

The terminate action deletes the instance permanently.

...

  • The “Environment” parameters are required fields. You can replace the default values, however please do not leave these fields empty. Also pay special attention to the formatting that’s required for the values. The deployment will fail if the formatting isn’t correct.

  • There is an AWS bug that prevents disabling the scheduled job after it has been enabled. The workaround is to either (1) Terminate the job and create a new one or (2) Set the rate to some distant time in the future (i.e. 3650 days).

...

Secrets are stored in the AWS secrets manager and exposed to the job as environment variables. The logs above print out the environment variables from the job. Take note of the “SCHEDULED_JOB_SECRETS” parameter in the logs. The secrets that are passed into this product are exposed as environment variables in the logs by the “printenv” command. Please make sure to never expose secrets in this way. DO NOT PRINT ENVIRONMENT VARIABLES.

Accessing Scheduled Job Secrets

Job secrets can be access a number of different ways. The first way is simply to get it from the docker container environment variable SCHEDULED_JOB_SECRETS.

Environment variable example:

...