This document is an attempt to capture how special values like None/NULL/NA/NaN/Inf/-Inf are handled in PLFM and across different clients as of today.
...
| Client | Empty value | NULL | None | NA | NaN | Inf | -Inf | Date | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| curl (repo-201) |
Empty list |
|
|
|
|
"Infinity" or "+Infinity" |
"-Infinity" | 10/19/2017 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| web client (201) |
|
|
|
|
|
"Infinity" or "+Infinity" |
"-Infinity" | 10/20/2017 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| python (synapseclient_1.7.2) |
|
|
Becomes string |
|
|
|
| 10/20/2017 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| old r client |
|
|
|
|
|
|
| 10/23/2017 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| synapser |
|
|
|
Becomes "None" |
|
|
| 10/23/2017 |
Submission Annotations
API
At the API level, an annotation on a submission has the following types:
- StringAnnotation
- DoubleAnnotation
- LongAnnotation
Client Behaviors
The below experiements are done on DoubleAnnotation on the following services:
...
Status colour Green title PUT
Status colour Green title GET
'value' is not set
...
Status colour Green title PUT
Status colour Green title GET
...
Related Jira Tickets
Jira Legacy server System JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId ba6fb084-9827-3160-8067-8ac7470f78b2 key SWC-3860 Jira Legacy server System JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId ba6fb084-9827-3160-8067-8ac7470f78b2 key PLFM-4576
Submission Annotations
API
At the API level, an annotation on a submission has the following types:
- StringAnnotation
- DoubleAnnotation
- LongAnnotation
Client Behaviors
The below experiements are done on DoubleAnnotation on the following services:
| Client | Empty value | NULL | None | NA | NaN | Inf | -Inf | Date | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| curl (repo-201) |
'value' is not set |
| Yellow
| Yellow
| "Infinity" or "+Infinity"
|
|
"-Infinity" | 10/19/2017 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| web client (201) | Not supported | 10/20/2017 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| python (synapseclient_1.7.2) | | Status | | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| colour | Green | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Status | ||||||
|---|---|---|---|---|---|---|
|
Status colour
Yellow title GET
"Infinity" or "+Infinity"
Status colour Yellow title PUT
Status colour Yellow title GET
Becomes empty
"-Infinity"
Status colour Green title PUT
Status colour Green title GET
| Status | |||
|---|---|---|---|
|
|
|
"Infinity" or "+Infinity"Status colour Yellow title GET PUT
Status colour Yellow title GETPUT
Becomes empty
| Status | |||
|---|---|---|---|
|
|
"-Infinity"
|
Status colour
Green title PUT
Status colour
Green title
GET
Status colour
Yellow title
PUT
Status colour
Yellow title
GET
"Infinity" or "+Infinity"
Status colour
Yellow title PUT
Status colour
Yellow title
GET
"-Infinity"
| Status | ||||||
|---|---|---|---|---|---|---|
|
| Status | |||
|---|---|---|---|
|
|
| Status | |||
|---|---|---|---|
|
|
|
| Status | ||||||
|---|---|---|---|---|---|---|
|
| Status | ||||||
|---|---|---|---|---|---|---|
|
| Status | |||
|---|---|---|---|
|
|
| Status | |||
|---|---|---|---|
|
|
|
Status colour Green title PUT
Status colour Green title GET
| Status | |||
|---|---|---|---|
|
|
| Status | |||
|---|---|---|---|
|
|
"Infinity" or "+Infinity"Status GET colour Yellow title PUT
Status colour Yellow title GETPUT
Becomes empty
Status colour YellowGreen title GET
"-Infinity"
Table
API
At the API level, a table column has the following types:
- STRING
- DOUBLE
- INTEGER
- BOOLEAN
- DATE
- FILEHANDLEID
- ENTITYID
- LINK
- LARGETEXT
- USERID
Client Behaviors
The below experiments are done on DOUBLE on the following services:
- POST /entity/{id}/table/append/async/start
- GET /entity/{id}/table/append/async/get/{asyncToken}
- POST /entity/{id}/table/query/async/start
- GET /entity/{id}/table/query/async/get/{asyncToken}
| Client | Empty value | NULL | None | NA | NaN | Inf | -Inf | Date |
|---|---|---|---|---|---|---|---|---|
| curl (repo-201) | ||||||||
| Status | ||||||||
| colour | Yellow | |||||||
| title | update | |||||||
| Status | ||||||||
| colour | Yellow | |||||||
| title | query | |||||||
| Status | ||||||||
| colour | Green | |||||||
| title | update | |||||||
| Status | ||||||||
| colour | Green | |||||||
| title | query | |||||||
| Status | colour |
|
"Infinity" or "+Infinity" |
"-Infinity" | 10/23/2017 |
Table
API
At the API level, a table column has the following types:
- STRING
- DOUBLE
- INTEGER
- BOOLEAN
- DATE
- FILEHANDLEID
- ENTITYID
- LINK
- LARGETEXT
- USERID
Client Behaviors
The below experiments are done on DOUBLE on the following services:
- POST /entity/{id}/table/append/async/start
- GET /entity/{id}/table/append/async/get/{asyncToken}
- POST /entity/{id}/table/query/async/start
- GET /entity/{id}/table/query/async/get/{asyncToken}
| Client | Empty value | NULL | None | NA | NaN | Inf | -Inf | Date | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| curl (repo-201) |
"" becomes NULL |
|
|
|
|
Becomes "Infinity" or "+Infinity" |
Becomes "-Infinity" | 10/23/2017 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| web client (201) |
|
|
|
|
|
"Infinity" or "+Infinity" |
"-Infinity" | 10/20/2017 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| python (synapseclient_1.7.2) |
|
|
|
|
|
|
| synapser | | Status | | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| colour | Grey | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| title | Not applicable | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Status | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| colour | Grey | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| title | Not applicable | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Status | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| colour | Grey | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| title | Not applicable |
| Status | ||||||
|---|---|---|---|---|---|---|
|
colour YellowGreen title query
| Status | ||||||
|---|---|---|---|---|---|---|
|
| Status | ||||||
|---|---|---|---|---|---|---|
|
| Status | ||||||
|---|---|---|---|---|---|---|
|
Status colour Yellow title update
Status colour Yellow title query
Becomes NULL
Status colour Green title update
Status colour Green title query
Status colour Yellow title update
Status colour Yellow title query
Becomes "Infinity" or "+Infinity"
Status colour Yellow title update
Status colour Yellow title query
Becomes "-Infinity"
The below experiments are done on DOUBLE on the following services:
- POST /entity/{id}/table/upload/csv/async/start
- GET /entity/{id}/table/upload/csv/async/get/{asyncToken}
- POST /entity/{id}/table/download/csv/async/start
- GET /entity/{id}/table/download/csv/async/get/{asyncToken}
| Client | Empty Value | NULL | None | NA | NaN | Inf | -Inf | Date | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| curl (repo-201) | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| web client (201) |
Becomes NULLIgnore empty rows |
|
| Green
| query
|
| Yellow
| update
|
|
| query | 10/23/2017 |
The below experiments are done on DOUBLE on the following services:
- POST /entity/{id}/table/upload/csv/async/start
- GET /entity/{id}/table/upload/csv/async/get/{asyncToken}
- POST /entity/{id}/table/download/csv/async/start
- GET /entity/{id}/table/download/csv/async/get/{asyncToken}
| Client | Empty Value | NULL | None | NA | NaN | Inf | -Inf | Date | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| curl (repo-201) | web client (201) | | Status | | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| colour | Green | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| title | update | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Status | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| colour | Green | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| title | query |
|
Status colour Yellow title query
Status colour Yellow title update
Status colour Yellow title query
Status colour Red title update
Can be uploaded from csv, cannot be read from Pandas
| Status | ||||||
|---|---|---|---|---|---|---|
|
Status colour Red title update
Lost in Pandas for row with only None, and becomes NaN in float column
| Status | ||||||
|---|---|---|---|---|---|---|
|
Status colour
Yellow title
update
Status colour
Yellow title
query
Status colour Yellow title update
Status colour Yellow title query
float inf turns into "Infinity"
Status colour Yellow title update
Status colour Yellow title query
float --inf turns into "-Infinity"
| Status | |||
|---|---|---|---|
|
|
|
| Status | ||||||
|---|---|---|---|---|---|---|
|
| Status | ||||||
|---|---|---|---|---|---|---|
|
Status colour
Green title update
Lost in Pandas for row with only None, and becomes NaN in float column
Status colour
Green title
query
Status colour
Green title update
Status colour
Green title query
Status colour YellowGreen title update
float inf turns into "Infinity"Status colour YellowGreen title query
Status colour
Green title update
Status colour
Green title query
float --inf turns into "-Infinity"
| Status | ||||||
|---|---|---|---|---|---|---|
|
| Status | ||||||
|---|---|---|---|---|---|---|
|
| Status | ||||||
|---|---|---|---|---|---|---|
|
Status colour GreenYellow title update
Status colour Green title query
Cannot save a csv that contains 'NA'. However, 'NA' in r data.frame made the round trip.
Status colour Green title update
Status colour Green title query
Status colour Green title update
Status colour Green title query
Status colour Green title update
Status colour Green title query
Related Jira Tickets
...
Jira Legacy
...
server
...
When read into data.frame, each unique value becomes a column.
...
System JIRA
...
Written to csv as "NA"
...
When read into data.frame, each unique value becomes a column.
columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId ba6fb084-9827-3160-8067-8ac7470f78b2 key SYNPY-576 Jira Legacy server System JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId ba6fb084-9827-3160-8067-8ac7470f78b2 key SYNR-1150 Jira Legacy server System JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId ba6fb084-9827-3160-8067-8ac7470f78b2 key SYNR-1140