When a system is broken into, the management often has a choice to make: should they take some time to try to figure out who was behind the break-in, should they bring in the police, or should they just clean up the mess that they find and move on. This is the choice that the City of Norfolk faced when a time bomb clobbered 784 systems, according to this blog. Debugging and understanding how a break-in occurred is a bit of a black art unto itself, requiring a substantial amount of expertise that focuses on the innards of Windows, and it requires time for the experts to track back what they think the source of the problem is, and even then the ability to do a trace may not be possible. For one, it depends on what sort of forensic evidence can be found within logs, whether those logs themselves have been tampered with, and what sort of backups were taken of the systems involved.
Here’s the problem with not trying to trace back: the miscreant who screwed you the first time can do the same thing again, using the precise same attack vector. At the very least it helps to have relationships with your security vendor to be able to report the problem, but as defenses get more complex, our continuing game of Cat and Mouse demands that so do the attacks. An initial attack vector might itself lead to the use of secondary means to attack. For instance, probing attacks work very poorly against a walled off Intranet, and in fact can be a means to alert The Guys In White Hats that the probing system has been broken into. However, the likelihood of that happening from within the Intranet is smaller. What’s more, as white collar criminal investigators know, one cannot rule out the possibility that someone on the inside will in fact have gotten things going.
This supports the whole notion of what Cisco calls Borderless Networking. That’s a marketing mouthful for a concept that Steve Bellovin articulated many many years ago, which says that bottleneck firewalls are going to need to give way to more sophisticated forms of defense on devices themselves.
A combination of good backups and logging to secure systems might have helped. Logs give some notion as to who did what when, assuming that you are logging the right things. Backups provide you a means to preserve state. This works in three dimensions: you can, perhaps even incrementally, look back into the history of a system for forensic purposes, you can preserve a crime scene through a very low level backup, and you can get back to a known good state.