-
This can only happen to HA
Posted on April 8th, 2009 No commentsTalk about rotten luck, I had a drive go out in a raid 5 array Thursday, the hot spare kicked in like it was supposed to, life is good or so I thought.
I had one of the guys on site (who had done this before) put in a new spare for me while I walked him through the process over the phone, did I mention this happened in California and I am in North Carolina. Well we brought the server back up and told it there was a new drive available for the spare pool and it still looked like life was good.Fast forward to this morning were there were a number of errors in the logs and after some troubleshooting it turned out there was some corruption on the C Drive. Not too tragic, the OS is on D, nothing on C but logs and boot files. Nothing to worry about, had this happen before just run chkdsk right? Ever seen the message mft is corrupt and chkdsk is aborting? Not the way to start the day. A little googling and it seems the /r switch will usually fix that, we will give it a try. It fails same message, but now explorer says unrecognized media, insert good media in drive C. Now we have gone from annoying problem to tragic situation, the server is still running I don;t dare reboot with no C Drive, but the SQL logs and Exchange logs all live on C: so not much is working on the server.
Did I mention that we just tore up three bedroom to get ready for the drywall guy who is starting Monday and we are now living in the living-room, literally.
Well anyways, time to make some plane reservations and try to find a room, because you guessed it, this is the peak season in Silicon Valley, rooms are scarce and expensive. Well got over that hurdle, have my flights booked although some annoyingly long layovers, still I managed to find semi-reasonable fares, got a room locked up and a car. Time to do some last minute errands and pack for the trip tomorrow, I go to my local bank and notice the screen looks funny, then it hits me, does this CR@P happen to anyone besides me? Click the picture for a larger view.
Leave a reply