Subscribe to ListverseImportant Update
- Published January 30, 2008 - 21 Comments
I am pretty sure you have all noticed the outages we are currently suffering. The cause of this two front pages yesterday – one on i-am-bored and one on foxnews.com. The site has been limping along for the most part of today but it went down again about an hour and a half ago. I am expecting that this is how we will continue for the rest of the day as we weather the storm.
New Blog Always Up
I have created a NEW list universe blog which I will be using to keep you informed of outages etc. This is hosted on an independent website, so when this site goes down, the blog will not. The new blog is a http://blog.listverse.com – it is an easy address to remember.
Where to from here?
I am in talks with the site hosts to work out a solution to this long term – at the moment (cost depending) it looks like some kind of clustered system with a load balancer will be the best bet. I will update the new blog when I have more information.
The catastrophe in a graph
For those who are curious, this is what the site traffic graph looked like last night when we went down:
Click it for a larger view. The very big spikes were when we hit the front page of Fox.
Afterword
If you are having difficulties with not appearing to be logged in, refresh the page. The sysadmins installed Squid yesterday to help with the overload and it seems to be causing some temporary problems with the logins.
All I can say is that this downtime is regretful and I promise I am doing all in my power to resolve it – please stick with us – it is going to be resolved one way or the other!
January 30th, 2008 at 8:32 am
You rock Jamie. Thanks for being so dedicated.
January 30th, 2008 at 8:35 am
Mystern: thanks
The downtime at least gave me time to finish another list – which I have just published!
January 30th, 2008 at 8:48 am
Fucking hell, 489 mb/s! Hopefully people’ll come back & have a look at the page again now it’s back up.
January 30th, 2008 at 9:14 am
Hey JF, i thought u had a long long list of lists waiting to be published…:-)
January 30th, 2008 at 9:20 am
heavybison: you mean that stuff that got depleted over Christmas? I wish I did have a long list
January 30th, 2008 at 9:23 am
JF: I’ll submit a list today. At least it will be today in MST. Might not be today GMT.
January 30th, 2008 at 9:26 am
Let me try this again, I apologize if this is a dupe comment. JFrater, are you still looking for the link at Foxnews.com? I found it at http://www.foxnews.com/story/0,2933,326474,00.html
January 30th, 2008 at 9:35 am
Mystern: great – thanks
I look forward to it!
January 30th, 2008 at 9:41 am
Thanks Rocky
January 30th, 2008 at 9:43 am
JF: One question about submitting lists. Should I just submit the text already in HTML format? If so, how do code a specific quote?
January 30th, 2008 at 10:01 am
Mystern: I use the blockquote tag for quotes – but you can send as text if you prefer.
January 30th, 2008 at 10:24 am
what were yee on fox for?
January 30th, 2008 at 10:35 am
Im so happy for the sight, tons a business and publicity
January 30th, 2008 at 9:53 pm
Slow down on the lists. I am still trying to catch up. Everyone posted while I was working and then I get home and the site is down. So I try in the morning. I read for about 15 minutes and the site is down again. Uggg. I am just now catching up. Would someone please, tell my kids’ schools to stop having evening band concerts and talent shows while am getting addicted to a site and trying to catch up on posts?
January 31st, 2008 at 3:16 pm
Have you tried a reverse proxy configuration such as Squid or Varnish yet?
I know you were discussing it before, I don’t know what the outcome was. Load Balancing is an option, but it tends to be hardware heavy, in that you’ll end up with a cluster of servers all generating separate output to individual requests, all managed through the Load Balancer. In other words, if I request “Page A” and someone else requests “Page A”, the page will be completely generated from whichever server is tasked with that particular page for each request.
On the other hand, a reverse proxy can cache the output of the most commonly accessed pages, relieving the backend server(s) tasked with generating new pages relatively unloaded as the request never gets that far for many of the requests made. If I request “Page A” and someone else requests “Page A” within the next few minutes, only the cached page will be sent to the second (third, fourth…) requester, without having to completely regenerate the new page.
You still really need a minimum of two servers (technically, I believe that you can do it with one, but it is more complicated, and nowhere near as efficient) but it doesn’t require as many servers as load balancing does.
January 31st, 2008 at 3:17 pm
oops, missed the “squid” in your update, maybe this will work. It will probably require some tuning however…
January 31st, 2008 at 3:40 pm
Jongleur: we did try with Squid – it caused some strange behaviour and is now gone – but it may still be an option.
January 31st, 2008 at 4:13 pm
I wouldn’t throw it out. It almost always requires some tuning, even from people experienced from prior installations, as the environment is almost always different. Someone who hasn’t already been through the process before may take a bit longer, but if they’re patient they should see results in fairly short order.
I’m not at all surprised about the login problems, as security is often the link that causes the most problems when trying to hand off between multiple servers. As you pointed out, simply doing a fresh login would probably clear up most of those.
February 1st, 2008 at 1:13 am
(off topic)
i’m so glad that this month is finally over. horrible month…!
February 1st, 2008 at 1:21 am
jrafter:
new topic (if you decide it’s worthwhile)…saw that you were low on list topics…
top 10 demographic locations of listverse registered users.
(not sure if you keep track of this)
I’m in Seattle… Yay!
sorry, i like data… =/
February 1st, 2008 at 10:57 pm
This is just insane, I clicked “notify me” for a few lists. BAMM, just nex day my inbox has 673 new mail!!!