Open issues

ROBUST-MIS challenge participants experience very slow data download
PLFM-5683
R-based app' encounters limit appending data to table
PLFM-5665
Make Synapse down time predictable
PLFM-5660
Update Final files in GDC projects that have been created between last manifest and changing the storage location
PLFM-5621
Changing password successfully shoud reset unsuccesfuly login attempt count
PLFM-5607
User locked out of login for 30 hours?
PLFM-5600
allow querying on multiple values for each annotation name in a file view
PLFM-5550
Upload file view .csv as table "cannot access files" error
PLFM-5435
UX recommendations for password policy
PLFM-5385
misc. EC2 instances must use encrypted disks and encrypt traffic from their load balancers
PLFM-5245
Synapse Password Policy
PLFM-5210
AWS BAA for HIPAA/PHI Requirements
PLFM-5197
Synapse recovery procedure
PLFM-5062
set WebACL thresholds/denial rules
PLFM-5043
Rerunnable Workflows
PLFM-4897
Select star query and ORDER BY toggling fails if a column name contains double quotes.
PLFM-4892
Support for Table/View Versions
PLFM-4698
Backlog of @Ignored tests
PLFM-3628
CommonMind token
PLFM-5686
The current schema is retured for tables when GET /entity/{id}/version/{versionNumber} on an old version
PLFM-5685
Allow requests to /repo/v1/status even during downtime
PLFM-5680
Docs for everything Google Cloud related
PLFM-5677
Add tags to resources deployed for Synapse
PLFM-5675
Refactor Backend code to centralize file downloads
PLFM-5673
X-Frame-Options header is set to SAMEORIGIN for portal
PLFM-5672
Notification of updates from API calls
PLFM-5669
Suggested VersionInfo statistics to surface for Table/View changes (from version to version)
PLFM-5668
Add documentation in Authentication Services REST document about how to use sessionTokens and API key
PLFM-5661
Decomission challenge Jenkins server
PLFM-5658
Script to set up challenge form user folders
PLFM-5657
Automatic JIRA ticket collector on Synapse
PLFM-5655
Include md5 in file view default columns
PLFM-5651
Implement multivalue annotations and add support for them in Entity Views
PLFM-5649
Rewrite DOI v2 integration tests
PLFM-5648
Fields and Content for Drug Nomination Tool
PLFM-5645
Integration tests that runs every synapse python client function before stack update
PLFM-5640
Should only be able to get forum metadata of a Synapse Project
PLFM-5631
Drug Nomination Tool MVP pass
PLFM-5629
unexpected behavior when SELECTing column from a table
PLFM-5628
typo in error message
PLFM-5624
Documentation on Synapse limits
PLFM-5622
Add support for copy to a private S3 bucket
PLFM-5620
Provide Designs for Portals Basic Search
PLFM-5618
Policy for providing access to a user locked out from their account.
PLFM-5615
don't allow malformed md5s when creating file handles
PLFM-5613
Search Kinesis error in worker logs
PLFM-5611
MD5 issue when downloading files from remote S3 bucket
PLFM-5609
Help users recover account when they can't access registered email address
PLFM-5605
ability to create an external storage location using a google compute bucket
PLFM-5604
implement stable/dependable search index
PLFM-5598
issue 1 of 394

ROBUST-MIS challenge participants experience very slow data download

Description

Here's the initial issue statement, received via email:

Lena asked me to provide you additional information regarding the download issue.

We provide our data on two ways (https://www.synapse.org/#!Synapse:syn18779624/files/):

  • As single files that can be downloaded via a script

  • As zip file

In both cases we receive emails from user that they can't download our data or that the download is really really slow. Overall, the size of our dataset is >400GB

What can we do to enable our participants a faster download?

Here are responses to my clarifying questions:

When you say "from user" do you mean one user mentioned the issue or more than one? If more, how many? How many total users are downloading the data? Do you yourself find that the data download is slow?

​Currently we are having 30 registered participants of our challenge and all of them need our data.
I got 8 messages that they have problems to download the data. All of them are talking about a slow download rate. They also mention that the download stops very often.
Yes, I also find the download with <10Mbps slow.

When you say "really really slow", can you quantify that? How fast do you expect the download to be? When you say "can't download our data" can you explain what the symptom is, for example is there an error that is seen? Can you see the error (and, perhaps, send us a screen shot)?

At my institute we are having a download speed capacity of ~740.39Mbps​ but downloading your data just goes with a speed of 10 Mbps. So currently it would take me ~4 days to download the data.
For me I would be happy with a download time of 8h (~100Mbps) for this data size.
I didn't got a network out of time error so far or a cancellation of my connection, but I can ask the users for screenshots.

Environment

None

Status

Assignee

Bruce Hoff

Reporter

Bruce Hoff

Labels

None

Validator

Bruce Hoff

Release Version History

None

Sprint

Priority

Critical