Table columns should be able to accept either python null type as an empty value

Description

If I have a Synapse table

user

is_awake

phil

true

Where the column type of user is String and the column type of is_awake is Boolean, I should be able to store the python list

or

as a new row to my table without being rejected for having the wrong data type.

This problem arises for me when I am applying incremental updates to tables, but the incremental update has all null values in a Boolean column, and pandas interprets this as float64 rather than object.

Environment

None

Activity

Show:
Phil Snyder
October 25, 2018, 4:37 PM

Well I could fetch the columns, iterate through each, checking it's type, and convert any nan in my dataframe to None if the Synapse column type isn't numerical... but that seems much less elegant than Synapse being smart about python null types in non-numeric columns.

I would argue that pandas specific problems are python specific problems. If anyone is going to pull down some scientific data using the python client, do some data munging, and push the result back up – they aren't going to be working with RowSets or the csv module.

Kimyen Truong
October 25, 2018, 4:18 AM

This seems like a Pandas specific problem. Table feature in the Python client allows users to interact with table in many different ways besides Pandas data frame (like RowSet, csv, ...) , do you have a work around for this problem?

Assignee

Unassigned

Reporter

Phil Snyder

Labels

Validator

Kenneth Daily

Development Area

None

Release Version History

None

Slack Channel

None

Epic Link

Components

Priority

Major