We're updating the issue view to help you get more done. 

Single thread uploading fails in Lambda python3.6 environment

Description

If I try to `syn.store` a file object within a Lambda function, I get the following traceback:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 START RequestId: 2a8d1f38-ce4f-11e8-a50f-ff7f7551687c Version: $LATEST Welcome, Phil Snyder! [INFO] 2018-10-12T18:46:46.648Z 2a8d1f38-ce4f-11e8-a50f-ff7f7551687c Welcome, Phil Snyder! ################################################## Uploading file to Synapse storage ################################################## [INFO] 2018-10-12T18:46:46.736Z 2a8d1f38-ce4f-11e8-a50f-ff7f7551687c ################################################## Uploading file to Synapse storage ################################################## [Errno 38] Function not implemented: OSError Traceback (most recent call last): File "/var/task/main.py", line 10, in handler syn.store(f) File "/var/task/synapseclient/client.py", line 970, in store mimetype=local_state_fh.get('contentType')) File "/var/task/synapseclient/upload_functions.py", line 67, in upload_file_handle return upload_synapse_s3(syn, expanded_upload_path, location['storageLocationId'], mimetype=mimetype) File "/var/task/synapseclient/upload_functions.py", line 126, in upload_synapse_s3 file_handle_id = multipart_upload(syn, file_path, contentType=mimetype, storageLocationId=storageLocationId) File "/var/task/synapseclient/multipart_upload.py", line 222, in multipart_upload **kwargs) File "/var/task/synapseclient/multipart_upload.py", line 360, in _multipart_upload completed = Value('d', min(completedParts * partSize, fileSize)) File "/var/lang/lib/python3.6/multiprocessing/context.py", line 135, in Value ctx=self.get_context()) File "/var/lang/lib/python3.6/multiprocessing/sharedctypes.py", line 78, in Value lock = ctx.RLock() File "/var/lang/lib/python3.6/multiprocessing/context.py", line 72, in RLock return RLock(ctx=self.get_context()) File "/var/lang/lib/python3.6/multiprocessing/synchronize.py", line 188, in __init__ SemLock.__init__(self, RECURSIVE_MUTEX, 1, 1, ctx=ctx) File "/var/lang/lib/python3.6/multiprocessing/synchronize.py", line 60, in __init__ unlink_now) OSError: [Errno 38] Function not implemented END RequestId: 2a8d1f38-ce4f-11e8-a50f-ff7f7551687c REPORT RequestId: 2a8d1f38-ce4f-11e8-a50f-ff7f7551687c Duration: 1468.27 ms Billed Duration: 1500 ms Memory Size: 128 MB Max Memory Used: 36 MB

I've created a repo that you can clone to reproduce the problem. To reproduce:

1. Create an EC2 instance using this AMI (the same AMI which Lambda uses)
2. Assign the instance an IAM role with a Lambda access policy
3. Create a Lambda function with the name `test_single_thread`. Change the handler to `main.handler`, and give it environment variables `synapseUsername` and `synapsePassword`.
4. Clone this repo onto the instance and run `main_builder.sh`.
4. Test the function with an arbitrary test event.

Environment

Status

Assignee

Kimyen Truong

Reporter

Phil Snyder

Labels

Validator

Phil Snyder

Sprint

None

Fix versions

Affects versions

py-1.9.0

Priority

Major