?

Log in

No account? Create an account
Paid user "express lane" - Paid Members [entries|archive|friends|userinfo]
Paid Members

[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

Paid user "express lane" [Jul. 8th, 2004|03:22 pm]
Paid Members

paidmembers

[bradfitz]
New paid user feature!

If you're logged in as a paid user, you "cut in line" of other web requests that we've received but not yet generated a response for.

So our load balancer retrieves all incoming web requests immediately after your browser sends them, then sorts those requests into two lines: free users and paid users.

Whenever a real server (not the load balancer) is ready to do more work, it always fulfills a request from the paid user line, and only does free user requests once no paid users are waiting.

We've had this done for a couple weeks, but just put it live today. We were going to wait to announce it until we had the cool part done, but thought it best you all know what's up. The cool part is: we're going to make the load balancer tell the real servers how many seconds/milliseconds you saved by cutting in line and how many people you cut in front of, and then the real servers will add that to the response, so you can do "View Source" in your browser and at the top it'll show you those stats, so you can verify it's working.

This is all different from what we did a few years ago, which was to have "free servers" and "paid servers". Instead all the servers do both free and paid, but access to the servers is prioritized by your account type.

Here are some stats of it in action:
queues
web_proxy-normal.age 0
web_proxy-normal.count 3
web_proxy-highpri.age 0
web_proxy-highpri.count 0
.
queues
web_proxy-normal.age 0
web_proxy-normal.count 0
web_proxy-highpri.age 0
web_proxy-highpri.count 0
.
queues
web_proxy-normal.age 0
web_proxy-normal.count 4
web_proxy-highpri.age 0
web_proxy-highpri.count 0
.
queues
web_proxy-normal.age 1
web_proxy-normal.count 4
web_proxy-highpri.age 0
web_proxy-highpri.count 0
.
queues
web_proxy-normal.age 0
web_proxy-normal.count 3
web_proxy-highpri.age 0
web_proxy-highpri.count 0
.
queues
web_proxy-normal.age 0
web_proxy-normal.count 5
web_proxy-highpri.age 0
web_proxy-highpri.count 0
.


At the top it showed there were 3 free users (waiting for less than a second --- age 0) waiting to get responses, and 0 paid users. Then it was all clear, then 4 new free users were waiting, then waiting up to a second, then those cleared, and 3 new free users were waiting under a second. Then 5 later, still under a second.

But throughout, no paid users were ever waiting.

Obviously the above isn't an interesting demo of this feature, but next time we hit a hiccup and the site gets slow, the paid users will be affected the least, since you won't have to wait in line long for a page to be generated. (just have to wait for paid users that came before you)
LinkReply

Comments:
Page 1 of 9
<<[1] [2] [3] [4] [5] [6] [7] [8] [9] >>
[User Picture]From: cynica
2004-07-08 03:23 pm (UTC)
is there any way to SWITCH which server we're on if we're paid users?
(Reply) (Thread)
[User Picture]From: bradfitz
2004-07-08 03:29 pm (UTC)
Nope. And it wouldn't matter, anyway.

When the site's slow for you, it's not because your database server is slow. It's because all of our web servers are locked up, serving up friends pages for other people, and their friends are spread out all over all the database servers, and if just one of the database servers is really slow and we don't catch it, the whole site gets slow. The reason then it's slow for you is no web server is available to process your request, even if your request is not for a friends page and the database you need to hit isn't being slow.

So currently we monitor all databases, and if one gets slow, we move stuff around. But generally the process of moving stuff around makes other stuff slow in the process.

We're aware of this general problem and we're working to make it all automatic while keeping the site fast, but it's a bitch of a problem, and we don't want to do it quickly while not being careful.
(Reply) (Parent) (Thread) (Expand)
[User Picture]From: honchi
2004-07-08 03:24 pm (UTC)
Wow! :D Thanks for all of your hard work!
(Reply) (Thread)
From: duffey
2004-07-08 03:24 pm (UTC)
Oh hey, this is almost like hot sex with females while in a tub of chocolate! <333
(Reply) (Thread)
[User Picture]From: subbes
2004-07-08 03:46 pm (UTC)
if by "almost" you mean "not at all."
(Reply) (Parent) (Thread) (Expand)
[User Picture]From: xskaboyx
2004-07-08 03:25 pm (UTC)
this rocks my socks.
(Reply) (Thread)
(Deleted comment)
[User Picture]From: mishkin
2004-07-08 03:25 pm (UTC)
hahah woot.

awesome :)
(Reply) (Thread)
[User Picture]From: dougs
2004-07-08 03:25 pm (UTC)
Well done boys and girls.
(Reply) (Thread)
From: littleyaoichick
2004-07-08 03:26 pm (UTC)
Thanks for letting us know! ^__^
(Reply) (Thread)
[User Picture]From: angelswings
2004-07-08 04:16 pm (UTC)
<3 Icon love!
(Reply) (Parent) (Thread) (Expand)
[User Picture]From: theldara
2004-07-08 03:26 pm (UTC)
Extremely geeky, but very nifty at the same time. Pretty cool.
(Reply) (Thread)
From: gumbyavatar
2004-09-01 02:51 pm (UTC)
Grrr, love the icon
(Reply) (Parent) (Thread)
[User Picture]From: sorrows_crow
2004-07-08 03:26 pm (UTC)
I think that's a very nice improvement.
(Reply) (Thread)
[User Picture]From: subbes
2004-07-08 03:27 pm (UTC)
So it gives a similar effect to the end-user of the old "fast server" distinction between free and paid?
(Reply) (Thread)
[User Picture]From: subbes
2004-07-08 03:28 pm (UTC)
Ignore the "?' there.
(Reply) (Parent) (Thread)
(Deleted comment)
[User Picture]From: honig
2004-07-08 03:27 pm (UTC)
Sweet!
Best wishes and keep up the good work!
-Honig
(Reply) (Thread)
[User Picture]From: chrishaas
2004-07-08 03:27 pm (UTC)
I appreciate the front of the line service, Brad.
(Reply) (Thread)
[User Picture]From: aeire
2004-07-08 03:28 pm (UTC)
Ohhhhh thank you.
(Reply) (Thread)
From: emissivity
2004-07-08 03:30 pm (UTC)
Neat feature! It's about time. Thanks so much! :D
(Reply) (Thread)
[User Picture]From: maully
2004-07-08 03:30 pm (UTC)
Good. Let's hope this will solve the speed problems..
(Reply) (Thread)
[User Picture]From: bradfitz
2004-07-08 03:32 pm (UTC)
See my reply above:

http://www.livejournal.com/community/paidmembers/13583.html?thread=1412623#t1412623

While this new feature will mitigate the speed problems, it won't entirely fix it. It'll just hide the problem a bit for paid users.

We're working on redoing the troublesome part of our database setup so web nodes never get blocked up doing nothing.
(Reply) (Parent) (Thread) (Expand)
Page 1 of 9
<<[1] [2] [3] [4] [5] [6] [7] [8] [9] >>