We are currently seeing a loss of SQL Cluster. We are working to resolve this and have also begun DR tasks to restore to a 2nd cluster if required.
Update 0630 - We are still working to restore the failed cluster and a restore of data from replication slave is also being performed. Next update at 0700
Update 0730 - We are still working to restore. The new cluster is up and data is being transfered to it. Next update at 0800
update 0800 - This has now been restored. We are monitoring the DB
Update 0830 - We have identified further corruption on 1 db in the cluster. We are working to resolve this. Next update at 0900
Update 0900 - Sadlt this remaining corruption is taking time to resolve. THis only effects 1 instance but does keep taking down the cluster. We are looking at remioving that 1 instance Next update at 0930
Update 0930 - The issue is still oingoing... We are now having to recreate the entire cluster .. However this time we are removing the corruption before this is restored. Next update at 1000
Update 1000 - Restore of new New cluster is underway. Next update at 1030
Update 1030 - Restore of new New Cluster from Replication Server is still oingoing this is sadly a slow process. Next Update is at 1130
Update 1130 - Restore of new New Cluster from Replication Server is still oingoing this is sadly a slow process. At 25% Done Next Update is at 1230
Update 1230 - Restore of new New Cluster from Replication Server is still oingoing this is sadly a slow process. At 35% Done Next Update is at 1330
Update 1330 - Restore of new New Cluster from Replication Server is still oingoing this is sadly a slow process. At 45% Done Next Update is at 1430 - We expect to be starting instances in the next hour as each finihses their restore
Update 1430 - Continuing Restore. At 65% Done Next Update is at 1530 - 50% of Effected Instances (Was around 5% of all Instances initially now only 2%) have been restored and are available . Instances will continue to be started as each finishes its restore
Update 1530 - Continuing Restore. At 75% Done Next Update is at 1730 - 70% of Effected Instances (Was around 5% of all Instances initially now only 2%) have been restored and are available . Instances will continue to be started as each finishes its restore
Update 1730 - Continuing Restore. At 85% Done Next Update is at 1830 - 85% of Effected Instances (Currently 1% of instances effected) have been restored and are available . Instances will continue to be started as each finishes its restore
Update 1830 - Continuing Restore. At 90% Done Next Update is at 1930 - 90% of Effected Instances (Currently 1% of instances effected) have been restored and are available . Instances will continue to be started as each finishes its restore
Update 1930 - Continuing Restore. At 95% Done Next Update is at 2030 - 95% of Effected Instances (Currently less than 1% of instances effected) have been restored and are available . Instances will continue to be started as each finishes its restore
2130 All Restored . All Instances Available.