Page 1 of 2

Server down for a bit

Posted: Tue Jan 09, 2007 10:03 am
by terryrayc
Okayw e are goign through some strange power issue here in lakeland...my ups's are going crazy..I'm going to have to shutdown the servers for a bit

Posted: Tue Jan 09, 2007 1:37 pm
by terryrayc
all that jumping back and forth last night seems to have had an import on the servers powersupply..I'm going to work on getting the backup server going until I can get a new powersupply tonight

Posted: Tue Jan 09, 2007 2:12 pm
by eclipse
The backup server is not fully set I can log in the server with one of my local vault char, um :) a good way to test the arkaz server with a uber buffed local build

Posted: Tue Jan 09, 2007 3:29 pm
by Alphonse
Please note.

In order to avoid potential issues with 'accidental dropping' of uber gear, the database will be rolled back to before the server was set to local vault.

Posted: Tue Jan 09, 2007 5:44 pm
by terryrayc
crap, I forgot to check to make sure there was no local vault....argh.

Okay I did pick up a new power supply.

I'll be headed home from the office at 4pm est...30 in 3 hours. I'll have the main server up shortly after that...also I'll only roll back the containers tables. So please do not store anything in your pchests. Also I'll be doing a vault restore from midnight last night....I'm sorry about that but it's better safe then sorry.

Posted: Tue Jan 09, 2007 5:57 pm
by terryrayc
I think I told the IRC folks this but the vaultster is down...I didn't have a copy of the correct version on hand when I was finishing up the backup server this morning and therefore didn't want to risk turning it on..so the portal to Hala is down. Again I expect everything to be up by 5pm EST(10pm GMT) tonight.

Posted: Tue Jan 09, 2007 6:46 pm
by circesmoon
So this means that nobody should log on to the server until everything is fixed?*runs around looking for something else to do*

Posted: Tue Jan 09, 2007 6:57 pm
by terryrayc
no, you can log in. I'm not going to restore the crafting db so if you want to do some crafting it'll count ;)

Posted: Tue Jan 09, 2007 8:58 pm
by terryrayc
okay server is back up.

The Servervault is current as of when the server went down...around 4:30am. The DB was restored from the 0030 backup.

I'm sorry if that caused any problems but it was the best we could do.

If anyone has any issues please let me know.

Posted: Tue Jan 09, 2007 9:11 pm
by terryrayc
testing is complete and everything seems to be fine now.

I'll be addressing this in the future.

Seems the local power company was installing new main feeds in my area and for about 2 hours he were hit with non-stop spikes, brownouts, serges and the like...I'm just luck all I lost was a powersupply.

It was bad timing though, I was in the process of rebuilding both my backup server and test server so I had to do a rush job this morning before work...which was why there were problems....normalyl here is what should have happened.

the DB server backs up every night..(I'm going to make it twice a day shortly), The vault backs up 4 times a day. (Those backups are offloaded to tape and CD)

In case of a failure, I'd bring up the backup server, restore the last DB backup and last vaultbackup(If needed)

repoint the router and off we go...this outage caught me with my pants down. In the future I'll make sure the backup system is always ready to go.