Server Reset Request Thread

[mhogar]

revived them again

[Dull Fred]

Down again.

[mhogar]

Down again.

sorry .. cant revive it this time

[terryrayc]

bringing the system down to replace the power supply, I hope this solves the issue long term.

[terryrayc]

ok will be a bit, for some reason one of the drives isn't being detected so I need to take a look. It's bath time here so I'm working on getting the kids ready for and in bed, I'll be back shortly.

[terryrayc]

sigh,

3 of the hard drives are not working now.

I've tried pretty much everything.

Sigh I'll have to replace them, problem is one of them is the hosts the db server and the other the netbackup guest. When I have a backup copy of the vault and DB I might end up having to replace the drives and rebuild.

I'll let you know.

[mhogar]

eeks

[Zerub]

*gulps* Hope you can get a backup for sure.

Thanks.

Scott

[terryrayc]

I'm guessing what happened was a bad port on the power supply.

These new supplies are modular. If you've not seen them what that means is the cables that come out and hook up and not hardwired. You have ports on the back of the power supply which allows you to switch out the cables for different connector types. These 3 drives were all on the same port. The issue is I have 7 drives in that server

2 SSDs (Game servers)
5 SATA in a raid 10 for DB and backup server

These 3 drives were part of the raid 10 set. That raid group cannot handle 3 drive failures....2 would have been okay, 3 not so much.

Where is how I back up the data.

On the ESX server is a linux guest that runs NetBackup. I backup everything to a puredisk volume that allows dedupped backups (Saves a lot of disk space). Everyday there is a script that runs that also moves a copy of the files to another computer for an offsite solution. The issue is I got lazy and didn't watch it closely and it seems that copy job has worked in 2 months. so if I have to rebuild it'll be with data from 2 months ago. That's an issue.

I've ordered new controller boards for the drives that failed, they'll be here in a few days. I'm going to replace those in the hopes that the damage was only to them. If so then I'll bring then systems up copy off all the data and replace the drives. If they will still not boot after that then we'll be forces to roll back to the last good copy in August.

Sigh

[Defending_angel]

Thanks for all the time and effort you are putting into solving it Terry, you rock!