Security Monitoring and Remediation

Security Monitoring and Remediation

This article explains Sage’s approach for monitoring Synapse infrastructure in support of security compliance and the process for Synapse engineers to follow.

 

Sage has engaged with Stack Armor as an external monitor of our infrastructure. Their role is to scan our infrastructure and report gaps in security that they identify, which Sage must then remediate. The standard we are held to is FISMA Moderate. The findings from Stack Armor are provided both to the Synapse Engineering team and to Sage’s Information Security Officer, who in turn shares with our sponsor, NIH.

Stack Armor performs four kinds of scans:

  • Vulnerability scans of the Synapse servers, using tools from Tenable based on the open source Nessus scanner. The findings are libraries seen on the servers having known vulnerabilities.

  • Center for Internet Security (CIS) Compliance scans of the Synapse servers, using tools from Tenable based on the open source Nessus scanner. The findings are generally operating system settings which are considered insecure. The scans are based in the CIS Level 1 benchmark.

  • AWS Infrastructure scans using Microsoft Defender. (MS Defender appears to closely mirror AWS Config.) The findings are AWS service configurations which are considered insecure (e.g., an S3 bucket which is publicly readable).

  • Web application scans, using tools from Tenable, targeting the Synapse staging server.

The Tenable service depends on an “agent” that Sage installs on the Synapse production EC2 instances. Microsoft Defender accesses Sage’s infrastructure using an AWS IAM Role that Sage provides Stack Armor.

The output from all four scans is provided in two forms: On a weekly basis Stack Armor sends a report (an emailed .zip file). The findings are also provided as issues in a GitLab project. While the emailed report is useful for review by the ISO and sharing with our sponsor, the GitLab issues are more useful for the engineering team, as they can be folded into the team’s work backlog and tracked through to completion. IT meets with StackArmor weekly to discuss the scanning activity and output. The minutes from our (weekly) discussions are kept here. Sage uses automation from Unito.io to mirror the GitLab issues into Jira, as a one-way sync'. The issues can be seen in the StackArmor GitLab (SAGL) Jira project. The Synapse Engineering team will consider the GitLab issues to be the ground truth of the current findings. If the emailed report or Jira issues are different, the GitLab issues are to be taken as authoritative and the other sources will be brought into sync'. Each GitLab issue has a severity (Critical, High, Medium, Low) and a Due Date, which is based on the FISMA remediation timeline. The tags on the GitLab issues can also be used to categorize the issue according to its source:

  • Vulnerability findings are tagged with integration::threatalert::tenable.

  • CIS Compliance findings are tagged with integration::threatalert::tenable and either Compliance::AutomaticCheck or Compliance::ManualCheckmatic

  • Microsoft Defender findings are tagged with integration::threatalert::msdefender

  • Web application scan findings are tagged with integration::threatalert::tenablewas

To address findings from StackArmor, the Synapse Engineering team can either (1) update our servers or infrastructure to address the finding, or (2) document that the finding won’t be addressed. When a finding has been addressed, StackArmor's scans and automation will detect the fix, and the issue will automatically close out in GitLab. Similarly Sage’s automation (using Unito.io) will mirror the change to Jira, closing it. Engineers should not manually close these issues. Alternatively, the team can decide not to address an issue if the perceived risk is low and the current behavior is needed for our system to work as intended. (An example might be the public user profiles Synapse provides or the public S3 bucket hosting our RAN.) Such a decision is more likely to be made for low severity issues. Another reason not to address a finding is that a finding is simply incorrect, that the reported security issue does not exist.

After making such an assessment, the following steps are to be taken:

  1. Add to the GitLab issue the tag, Deviation Request: Required

  2. Add a second tag, either Operational Requirement or False Positive.

  3. Add a comment, explaining the team’s rationale for the assessment. Also log the same comment in this sheet, to ensure that Sage maintains a copy.

StackArmor will submit the ‘deviation request’ to their workflow, which will ultimately result in the issue being closed.

 

A note on CIS Compliance Findings

The “CIS Compliance” level 1 checks generate over 70 security findings for the Amazon Linux 2023 servers that Elastic Beanstalk uses by default. To address this, Sage has adopted an Amazon Machine Image which is a variant of the Amazon Linux 2023 image maintained by CIS (purchased through the AWS marketplace). The image has to be changed slightly to work with Elastic Beanstalk. Sage created an AWS Image Builder Pipeline to create the modified CIS image used by Synapse. This also provides a mechanism for making further customizations to the image, as required. Changes to the image pipeline should be done here, in the aws-infra repo', and deployed to the ‘aws-ami-monorepo’ as explained here.