We're updating the issue view to help you get more done. 

Single thread uploading fails in Lambda python3.6 environment

Description

If I try to `syn.store` a file object within a Lambda function, I get the following traceback:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 START RequestId: 2a8d1f38-ce4f-11e8-a50f-ff7f7551687c Version: $LATEST Welcome, Phil Snyder! [INFO] 2018-10-12T18:46:46.648Z 2a8d1f38-ce4f-11e8-a50f-ff7f7551687c Welcome, Phil Snyder! ################################################## Uploading file to Synapse storage ################################################## [INFO] 2018-10-12T18:46:46.736Z 2a8d1f38-ce4f-11e8-a50f-ff7f7551687c ################################################## Uploading file to Synapse storage ################################################## [Errno 38] Function not implemented: OSError Traceback (most recent call last): File "/var/task/main.py", line 10, in handler syn.store(f) File "/var/task/synapseclient/client.py", line 970, in store mimetype=local_state_fh.get('contentType')) File "/var/task/synapseclient/upload_functions.py", line 67, in upload_file_handle return upload_synapse_s3(syn, expanded_upload_path, location['storageLocationId'], mimetype=mimetype) File "/var/task/synapseclient/upload_functions.py", line 126, in upload_synapse_s3 file_handle_id = multipart_upload(syn, file_path, contentType=mimetype, storageLocationId=storageLocationId) File "/var/task/synapseclient/multipart_upload.py", line 222, in multipart_upload **kwargs) File "/var/task/synapseclient/multipart_upload.py", line 360, in _multipart_upload completed = Value('d', min(completedParts * partSize, fileSize)) File "/var/lang/lib/python3.6/multiprocessing/context.py", line 135, in Value ctx=self.get_context()) File "/var/lang/lib/python3.6/multiprocessing/sharedctypes.py", line 78, in Value lock = ctx.RLock() File "/var/lang/lib/python3.6/multiprocessing/context.py", line 72, in RLock return RLock(ctx=self.get_context()) File "/var/lang/lib/python3.6/multiprocessing/synchronize.py", line 188, in __init__ SemLock.__init__(self, RECURSIVE_MUTEX, 1, 1, ctx=ctx) File "/var/lang/lib/python3.6/multiprocessing/synchronize.py", line 60, in __init__ unlink_now) OSError: [Errno 38] Function not implemented END RequestId: 2a8d1f38-ce4f-11e8-a50f-ff7f7551687c REPORT RequestId: 2a8d1f38-ce4f-11e8-a50f-ff7f7551687c Duration: 1468.27 ms Billed Duration: 1500 ms Memory Size: 128 MB Max Memory Used: 36 MB

I've created a repo that you can clone to reproduce the problem. To reproduce:

1. Create an EC2 instance using this AMI (the same AMI which Lambda uses)
2. Assign the instance an IAM role with a Lambda access policy
3. Create a Lambda function with the name `test_single_thread`. Change the handler to `main.handler`, and give it environment variables `synapseUsername` and `synapsePassword`.
4. Clone this repo onto the instance and run `main_builder.sh`.
4. Test the function with an arbitrary test event.

Environment

None

Status

Assignee

Kimyen Truong

Reporter

Phil Snyder

Labels

Validator

Phil Snyder

Release Version History

None

Sprint

None

Fix versions

Affects versions

py-1.9.0

Priority

Major