Hi Kim, there currently isn't any detailed documentation about dates in the docs site. There's an open Github issue to address this gap, but it's pretty low on the list of priorities. We can bump if it up if it'll help clear up some of the confusion for users though.
Thanks ! I think I need the decision on the solution more than the docs itself. So that I know how to address this issue. For example, we either go with:
A. Automatically convert timestamp to Date type (and back) when users use asDataFrame() in Python or as.data.frame() in R + document that user will get timestamp if they download csv and work with csv using a third party application.
B. Do not automatically convert timestamp to Date type, but provide helper function/ option that allow users to do so + document on how to use the helper function/option + document how data is stored for DATE column type.
C. any other suggestion?
My 2 cents: where there is a logical mapping between Schema Column types and Data Frame classes, we should absolutely handle these conversions. These strongly typed columns are in fact why Tables are so useful.
After looking into this a bit and having a few discussions, I want to make a note to point out that the issue is not loss of precision but conversion.
So no data is lost in dividing by 1000 and converting the value to POSIXct format if and only if the Date is converted back to a number and multiplied by 1000 before calling synStore().
Whether or not this is appropriate in the client is a different discussion 😉.
To validate, rerunning the steps provided in the description should work and give the expected result