Ed, thanks for the story. Since I don't get into designing and maintaining mission critical systems, I can only relate through other people's stories.

Big lesson I've always heard is to make sure you don't have a single point of failure. Looks like the memory chip was that point in your situation. I've read about error correcting memory. Would that have made a difference?

About the backups that didn't happen. Again, backups are only as good as the execution. A regular plan for testing backups might have caught the lack of effort, but somehow, I doubt that would get done regularly either!

Do you have a system of rotating backups out of the sequence so you have some older history? In this case, you could have had corrupted data for some period of time before it was noticed. Just having a backup from the past week or so might have still ended up with the same result.

Personal story. I just discovered the other day that the backup program on my home computer was not picking up my personal email. Not a big deal for me as I don't have much going on in that account and no data was lost anyway, but it goes to show how easily an error can be made. I discovered it when doing a random check of the files/folders that are mapped by the program.