There are a series of table operations that we would like to support that involve moving potentially large amounts of data between clients and Synapse. For example, create a new table entity from a local CSV file. Or for a particularly large query result, download the entire result set as a CSV file. This document outlines the proposed services that will support these types of features.
Create or Update a Table from CSV
- The client must first create a ColumnModel for each column in the CSV.
- Create or update the TableEntity with the desired column model IDs from step one.
- Upload the CSV as a file handle (see: File Services).
- Start a table job passing the FileHandle id from step 3. and the TableEntity ID from step 2. This call will return a job ID. A background process will be launched to create a RowSet from the data in the CSV files.
- Monitor the status of the job using the job ID provide from step 2. Once the job completes all of the data from the CSV will be applied to the table. If the job fails, the job status will provide details on the failure.