For use cases see: Bulk File/Table Download via Web Client
Introduction
The bulk file download via the web client is a new feature that will allow a user to browse for files they wish to download, add the files to a user's download list and then download the entire list of files as a single zip file. The user can add files to their download list using the following operations:
- Add all files from a folder. Note: This operation is not recursive, so files within sub-folders will not be added.
- Add a single file from a folder.
- Add all of the files listed in a view query. We do not plan to support adding individual files from a view, instead the user is expected to refine their query to sub-select files from views. Note: This operation will add all files from the view query, not just the single page of files shown in the UI.
The user will be able to view and refine their download list prior to downloading the files. Download list refinement operation include the follow:
- Remove selected files from the list
- Request access to files with unmet access restrictions
- Name/rename the zip file that to be downloaded.
- Clear the entire list.
Limitations
Managing file selection across a paginated list of results creates an awkward user experience. Therefore, the entire download list must be presented to the user without pagination (scrollbars are allowed). This means there must be a limit on the number of files allowed in the download list. The download list must be small enough to be fetched as a single web-service request. A download list will have a limit of 100 files.
There is also a limit to how long a users will wait for their file downloads to be created. Currently, it can take 10 minutes to prepare a 2 GB zip file for download. A download list will have a maximum size of 2 GB (sum unzipped files must be less than 2 GB).
File sizes
One of the implied requirements from Ljubomir's design is the availability of the total size of all files in both view query results and folder navigation. The file sizes will be used to estimate the download time based on the user's current network speeds. The sizes will also be used to help the user keep their download list under the maximum size.
View Query Results
Since the user will only have the option to add all of the files from a give view result, and not just the currently shown page, the file size results will need to include the size of all files for a given query. This is similar to the query count already available to in table query results. The proposal is to add a new mask to the existing QueryBundleRequest.partMask call 'fileSizes' with a value of 0x20. When the 'fileSize' mask is include the resulting QueryResultBundle will include a numeric value called 'sizeOfAllFilesMB'.
Folder Navigation
Unlike View query results, users will have the option to add one file at a time from the folder navigation. This implies that we will need to show the size of each individual file in the folder navigation. We should be able to use the existing POST/fileHandle/batch to get the file handles for a single page of files shown in the folder navigation.
To support adding all of the files in a folder (non-recursive) (use case 1a) we will need to return the total number of files in a folder and the total size of all files in the folder from POST/entity/children. The proposal is to add a 'partMask' (similar to QueryBundleRequest) with 0x01=count and 0x02=totalFileSizeMB.
Download List
A user's download list represent the current list of file the a user has added to their list. The download list does not contain any information about how the files were added to the list.
Figure 1. DownloadList class diagram