We're updating the issue view to help you get more done. 

Missing dtypes in table.DTYPE_2_TABLETYPE

Description

DTYPE_2_TABLETYPE holds a map of numpy data types to Synapse table column types. It's missing some:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 In [128]: for k, v in np.sctypeDict.iteritems(): .....: if isinstance(k,str) and len(k)==1: .....: print '{0:14s} : {1:40s}'.format(str(k), v) .....: D : <type 'numpy.complex128'> H : <type 'numpy.uint16'> L : <type 'numpy.uint64'> P : <type 'numpy.uint64'> d : <type 'numpy.float64'> h : <type 'numpy.int16'> l : <type 'numpy.int64'> p : <type 'numpy.int64'> ? : <type 'numpy.bool_'> G : <type 'numpy.complex256'> O : <type 'numpy.object_'> S : <type 'numpy.string_'> g : <type 'numpy.float128'> B : <type 'numpy.uint8'> F : <type 'numpy.complex64'> V : <type 'numpy.void'> b : <type 'numpy.int8'> f : <type 'numpy.float32'> q : <type 'numpy.int64'> I : <type 'numpy.uint32'> M : <type 'numpy.datetime64'> Q : <type 'numpy.uint64'> U : <type 'numpy.unicode_'> a : <type 'numpy.string_'> e : <type 'numpy.float16'> i : <type 'numpy.int32'> m : <type 'numpy.timedelta64'>

We should handle those that we can (for example 'M' = datetime64) and maybe fall back to strings where it's not obvious what to do to avoid errors like the following:

1 2 3 4 5 6 7 8 file_map = syn.downloadTableColumns(Table(schema, df_to_download), args.data_columns) File "/home/ubuntu/install/miniconda2/lib/python2.7/site-packages/synapseclient/table.py", line 769, in Table return CsvFileTable.from_data_frame(schema, values, **kwargs) File "/home/ubuntu/install/miniconda2/lib/python2.7/site-packages/synapseclient/table.py", line 1048, in from_data_frame cols = as_table_columns(df) File "/home/ubuntu/install/miniconda2/lib/python2.7/site-packages/synapseclient/table.py", line 352, in as_table_columns columnType = DTYPE_2_TABLETYPE[df[col].dtype.char] KeyError: 'M'

Environment

None

Status

Assignee

Ziming Dong

Reporter

Chris Bare

Labels

None

Validator

Larsson Omberg

Release Version History

None

Components

Fix versions

Affects versions

py-1.5

Priority

Minor