We just surveyed our customers and related customers using log data for SIEM/IT Ops/Compliance and got a lot of interesting insight into what features people are using. We're hoping to get some cool stories as a part of ongoing research, hopefully we'll get some we can share.
Here are some that come to mind while people get participating:
1. Company has a situation where downtime directly costs them money, but does not invoke any regulatory compliance issues. A virus, an outage, etc, means people are literally not spending money with them, and seconds tick by fast. A security issue causes at least an hour's worth of downtime and could put them out for the entire day. For all these reasons, they have service accounts that they have to share the passwords to (think whiteboard with passwords that have admin access to a set of servers) so that a set of operators can fix issues quickly. Their IT team had thought they restricted usage of these accounts via GPO, since they were highly privileged. Not so - we were able to audit usage of these accounts, find people logging on to them, and making unexpected changes to their own systems (like adding themselves to local admins, installing software, etc).
2. We used an example in the first SolarWinds Lab episode of a customer whose firewall kept going down, down, down, regardless of what they did. Their connections were being used up so quickly they thought there was a bug in their firmware. Interface utilization was off the charts. We were able to figure out it was actually a worm - almost every machine in their infrastructure was infected. They were a healthcare org so it wasn't business crippling entirely, but all of their remote sites/clinics were isolated (connected back via VPN - which couldn't connect or maintain) which affected patient care, access to records, etc. We were able to resolve it by identifying infected machines, cleaning them up, then continuing to filter and monitor for new infections.
3. When I managed IT for TriGeo before acquisition, I ran into all kinds of stuff that would have taken me forever without a system aggregating logs. However, the most amusing thing was everyone knowing we used it, and coming to me when issues happened before I really knew they were a problem - because they assumed that in a "big brother" sort of way I already knew (Sometimes I did, sometimes I didn't - yet.) I could probably drag out a bunch of stories of how logs saved my bacon or really sped up my job, I'm pretty sure if we didn't we'd have had to hire "real" IT people other than myself and a couple of people who helped on both helpdesk level support and our hardware burn-in/imprint process.
Next...