Reply to post:

Mine all the data, they said. It will be worth your while, they said

Coofer Cat

For my money, a TSDB is good for 'telemetry' and ELK or similar good for log aggregation (more like a SIEM). For all that to work out though, you need to have some idea of what's important. Most application log files I've ever seen are loads of "just in case" information because the developer knew that at some point in the future (s)he would have to debug a production problem using nothing but those logs. IMHO, there's justification for most apps to have a general log for all the usual chatter (which you don't bother to index), and an 'audit' log which contains just the things that have happened (eg. 'user logged in', 'user requested statement', 'user made a payment' etc). The audit log wants to have no personally identifiable information in it though - anonymised data only (and even then only sparingly).

Sadly, non-functional features generally get pretty short-shrift in most commercially driven organisations. I'd imagine the beauty and simplicity of the log files would be pretty close to the bottom of the non-functional priority list. If it's not there in day 1, you're never going to get around to doing it.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2021