Data Origin
The data comes from two sources:
- Synapse Repository Services (possibly via logs, but possibly through other means)
- Synapse Crowd Server(s) (via the logs)
These logs are scanned on a daily basis (could be more often if wanted).
Data Collection/Aggregation
The scanning process will involve calling two Simple
Workflow workers:
- scan log data and update the RDS window (labeled Agg)
- take data from the RDS window and update the RRD(labeled Upd)
Web UI
This will be written in GWT/GXT3. This exposes and does pretty things with the data, which is available from two sources:
- An interface to a temporary high-resolution data window, backed by any of several various implementations
- Long-term RRD (implemented with rrd4j and backed by any of a variety of storage mechanisms).
Diagrams
Below is a diagram of the basic data flow for the application. I've endeavored to show the relationship's of all the moving parts, and specifically called out what needs to be built as part of this project.
RRD Considerations
See the page on RRD Size Estimates for information about what the storage needs are. (Hint: they're small).