Author Topic: Server Issues Megathread  (Read 1513 times)

Conan

  • Postcount killed Trogdor
  • *****
  • Posts: 847
  • E-points: +44/-12
  • \(_o)/
    • View Profile
Server Issues Megathread
« on: January 01, 2016, 07:45:45 pm »
Just a few hours into 2016 and we already have one server failure.

https://twitter.com/furaffinity/status/683127881220894720
Quote
One of our older systems suffered a power and motherboard failure. We had to swap systems to get them online. Still working.

What are the chances that the box they "swapped" to was bought six months ago with the intent of replacing the box that died?

mahadri

  • Posts: 10
  • E-points: +3/-0
  • Uninitiated Rube
    • View Profile
Re: Server Issues Megathread
« Reply #1 on: January 01, 2016, 11:57:52 pm »
What are the chances that the box they "swapped" to was bought six months ago with the intent of replacing the box that died?

"The system we pulled parts from had an identical build to the server that failed, so we pulled everything over knowing that the pats were in working order." https://www.furaffinity.net/journal/7284824#cid:48464653

Just a few hours into 2016 and we already have one server failure.

t.facdn.net was down for ~25 minutes between 2015-12-31T03:30Z and 2015-12-31T04:00Z, a full 38 hours since this outage beginning 2016-01-01T17:48Z. 500 Internal Server Error, 502 Bad Gateway, 524 Origin Timeout.

Conan

  • Postcount killed Trogdor
  • *****
  • Posts: 847
  • E-points: +44/-12
  • \(_o)/
    • View Profile
Re: Server Issues Megathread
« Reply #2 on: January 02, 2016, 01:10:54 am »
What are the chances that the box they "swapped" to was bought six months ago with the intent of replacing the box that died?

"The system we pulled parts from had an identical build to the server that failed, so we pulled everything over knowing that the pats were in working order." https://www.furaffinity.net/journal/7284824#cid:48464653

The last major issue involved the same server. It held up for a few months after it dropped, but it was likely a pre-cursor to what happened tonight.

That sounds like the outage in September which was blamed on someone exploiting the user icon system.

Piche seems to have his dates mixed up, because IO was installed in 2011. Unless they bought an old server, which doesn't seem like them.

They installed a 48TB file storage server called "Smaug" back in May but apparently aren't using it for files. Piche tweeted a photo of the server that failed, and the handles on the front don't match the ones Smaug has.

I think Smaug may have been one of the final servers acquired via donations, which would mean once again they're letting their user's good will rot away because they're too lazy to move things to the new boxes they have.

Conan

  • Postcount killed Trogdor
  • *****
  • Posts: 847
  • E-points: +44/-12
  • \(_o)/
    • View Profile
Re: Server Issues Megathread
« Reply #3 on: March 03, 2016, 12:00:32 am »
FA has been down for a few hours as of this writing. Who knows what's wrong. Yak is reportedly pissing all over the code in an attempt to get it running again.

EDIT: 6+ hours after the site went down, someone finally explained what is going on:

https://twitter.com/furaffinity/status/705356411765653504
Quote
We ran into an issue while doing a server migration, moving data to a new system. We're in the process of upgrading FA's main storage server

So it took them 6 hours to figure out that they fucked up yet another server migration.
« Last Edit: March 03, 2016, 03:27:51 pm by Conan »