Like a fox?
So as many of you have emailed, there are still a few things outstanding with the site. We will be finishing up work tomorrow to get things back to normal.
Most of the badges are still turned off while we work on the database. We will restore them as soon as we can. Don't worry, you haven't lost them... they will be back.
While attempted to run a Repair/Optimize on one of our tables, the command never completed, leaving that database table in a crashed state. I am in the process of restoring a copy from back up and sending it over to the front end database server. This is a HUGE table and therefore this is taking forever. I will most likely go to bed and complete it in the morning. This means replication is down for the time being until I can get that squared away. When replication is down... strange things happen... so don't get to freaked out if you see something weird going on.
As if that wasn't enough I had to do some software upgrades to fix some bugs on the server side and we are building a redundant data capture process on one of our other servers so that if something ever happens to our primary job server in the future, we have a back up collection of the nightly data for emergency purposes. It is a mega-pain to set this up... but something that we have learned these past few days will be worth it. The best we can hope for is we go through all this effort and we never have another crash and therefore never need it! :)
Comments