...
where do we store this data? ideas: (1) log file, (2) dynamoDB
noteNote: dynamoDB has ability to archive content to S3
might need tools that would digest the captured information to answer common questions like ("When did this object change?")
Note: Might be sufficient just to capture the web request, without capturing the database level activity.
What information should be captured and what is the mechanism?
Info should be captured to logs (local disk files) which are processed and moved to other system(s) (e.g. DynamoDb) for indexing.
Per Eric: Could capture each request in an S3 file (one file per request), putting time stamp, entity id or other key information in file name. Use MapReduce to efficiently process files to answer queries.
John adds: The S3 files created during migration could be retained as a "log" of migrator activity. (Bruce says, the files would be used with the migration requests to create the actual history.)
While PLFM logs incoming requests, Portal could log the requests it thinks it made.
Next Steps:
1) Capture individual requests (and responses) to the repo-svcs as S3 files.
2) Capture Catalina logs (e.g. to get stack traces associated with 500 status codes).