?

Log in

Status Update - Paid Members [entries|archive|friends|userinfo]
Paid Members

[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

Status Update [Jan. 12th, 2002|10:10 pm]
Paid Members

paidmembers

[bradfitz]
It's been awhile since my last post here so I thought I'd let everybody know what's up...

For all of you, it probably seems that not much is up, actually. There hasn't been anything new in awhile, the site keeps getting slower, and all the new features we keep promising "soon" never show up.

Tell me about it. *sigh*

But here's the basic problem.... everytime I'm just about done with a project, be it S2, the directory rewrite, a good documentation system etc., the site breaks.

So earlier this month I thought: Screw this, I'm going to make redesigning the site's architecture into something that can scale beautifully a personal challenge and I'll work on nothing else but that until it's done. After that, I could get back to all the other projects.

Because really, the site working and working damn fast takes priority over everything else. Who cares if we have a super easy & powerful style system or a directory if you can't load your friends page?

So, I've been working on the new architecture the last couple weeks. Mostly, the challenge has been converting the old database format into the new one. After some test runs on a dev server and some math, I found it'd take several days to convert all of LJ. Several days of downtime would suck, even if we had read-only access enabled during that time.

The other day I came up with my 3rd plan of attack for conversion. The necessary downtime now is only a minute and 30 seconds, about.

That code is all written now and tested on a copy of the real data.

The current project is converting all the affected code on the site to understand the new db schema, which isn't too tough. A pre-req for that project is a tool to convert somebody from the old schema to the new schema on some other cluster. I wrote that tonight and have tested moving my own journal (well, the copy on the test server). That's all working. So now that I have data in both the old & new format, I can proceed with making the code on the site understand both types.

Then, for a week or more, we'll slowly be converting users one at a time to the new format. While your journal is being converted you won't be able to post new things and people won't be able to comment in your journal (you will, however, be able to comment in other people's). The downtime per-user won't be more than a few minutes and you'll probably not even notice it. If you do, you'll get a message that says your journal is in read-only mode.

Anyway, this is getting longer than I'd intended.

In summary:
  • our architecture sucks now
  • it won't suck shortly
  • after that, fun stuff.
Sorry for the less than perfect service as of late. I hope you understand that I'm working as hard as I can to make this site permanently rock.
LinkReply

Comments:
Page 1 of 3
<<[1] [2] [3] >>
[User Picture]From: resilient
2002-01-12 10:08 pm (UTC)

That certainly sounds like the intelligent way to approach things. Thank you for all of the effort you and the other people on the LJ team have put into making this such a great site.
(Reply) (Thread)
[User Picture]From: skreidle
2002-01-12 10:10 pm (UTC)
We understand, and we appreciate it.. the site pretty much rocks all the time as it is, in spite of occasional breakage, thanks to you. :)
(Reply) (Thread)
[User Picture]From: katayamma
2002-01-12 10:10 pm (UTC)

Excelent progress! I had no doubt that y'all have been working to make things better. Will you be posting the format for the new database like you did the old one? Just curious as I'd lile to look it over.

Cheers
(Reply) (Thread)
[User Picture]From: bradfitz
2002-01-12 10:14 pm (UTC)
Of course. All our code will continue to be Open Source.

When we update the main site to the new code, the schema page will automatically update to show the new layout.
(Reply) (Parent) (Thread)
[User Picture]From: hamsterhuey
2002-01-12 10:13 pm (UTC)
The site rocks now....
(Reply) (Thread)
From: deadbodygrl182
2002-01-12 10:13 pm (UTC)
thanks for all your hard work, it's greatly appreciated.
(Reply) (Thread)
From: ex_evil
2002-01-12 10:14 pm (UTC)
hey as far as I'm concerned all you guys are doing an awesome job! :0) keep up the good work.
(Reply) (Thread)
[User Picture]From: halkeye
2002-01-12 10:15 pm (UTC)
could the conversion script/proccess slow down (quite temporary) the site for the other users?
and how soon before we start to see major speeds up?
(Reply) (Thread)
[User Picture]From: nyxie
2002-01-12 10:16 pm (UTC)
As always... thank you. :)
(Reply) (Thread)
[User Picture]From: kyrielle
2002-01-12 10:16 pm (UTC)
Yay! Thank you!
(Reply) (Thread)
[User Picture]From: timotheophany
2002-01-12 10:17 pm (UTC)

when's the wedding?

seeing new comments appear almost real-time is fun :)

whee... it was 3, then 7, then 8, hehehhe what number will I be?
(Reply) (Thread)
[User Picture]From: hamsterhuey
2002-01-12 10:19 pm (UTC)

Re: when's the wedding?

gotta love the icon.
(Reply) (Parent) (Thread) (Expand)
[User Picture]From: maliwane
2002-01-12 10:17 pm (UTC)

Jumping on the me too wagon. What they said!!
(Reply) (Thread)
[User Picture]From: pointedview
2002-01-12 10:17 pm (UTC)
A really good, straight-up post. Appreciate the communication and the work. Still keeping my fingers crossed for the directory. :)
(Reply) (Thread)
From: awesometc
2002-01-12 10:22 pm (UTC)
Thanks for the update!
I can't wait for LJ to permanently rock.
(Reply) (Thread)
From: ex_candle321
2002-01-12 10:24 pm (UTC)

I got my

Eye on you .
(Reply) (Thread)
From: ladycinderella
2002-01-12 10:35 pm (UTC)

Sounds like a plan! :)
Good work so far!
(Reply) (Thread)
From: scientaestubiqu
2002-01-12 10:38 pm (UTC)

thanks guys

we appreciate your hard work...
(Reply) (Thread)
[User Picture]From: aerojad
2002-01-12 10:41 pm (UTC)
i just wanted to say that the site already is a perma-rocking site.. just becasue it exists in the first place. you guys could of long since by now sold this out and well.. the site addy could be http://livejournal.msn.com

in which case.. people would have to die.. quickly

thanks for all the hard work and.. yeah.. dont give up... this site is already a highly addictive drug for at least a few ten-thousand people... and a casual drug for the other couple hundred thousand. we'd be lost without or lj fixes and stuff...


~aerojad
(Reply) (Thread)
[User Picture]From: crayolaab
2002-01-13 02:45 pm (UTC)
I second all of that..
and add my thanks for all that you do
(Reply) (Parent) (Thread)
[User Picture]From: lindsaybits
2002-01-12 10:47 pm (UTC)

yay!

you guys rock. 'nuff said. :D
(Reply) (Thread)
[User Picture]From: eieye
2002-01-12 11:14 pm (UTC)
You so own me. You rock. Thanks a much.
(Reply) (Thread)
[User Picture]From: ccnuggie
2002-01-12 11:17 pm (UTC)
Dude when that directory gets back up I'm gonna dance a jig.
(Reply) (Thread)
[User Picture]From: meester_gone
2002-01-12 11:23 pm (UTC)
Understand, Brad. sounds like a workable (awesome, in fact) plan of attack! Keep up the good work. :)
(Reply) (Thread)
[User Picture]From: tobysionann
2002-01-12 11:28 pm (UTC)
Thanks for this awesome site, man.
I appreciate the hard work.
(Reply) (Thread)
From: q
2002-01-12 11:49 pm (UTC)
Sounds like a plan, and an excellent plan at that!

Question: Wouldn't generating the friends list involve hitting a lot (all) of the clusters? Each hit would be more likely to be in the cache... it seems like a bottleneck could develop eventually. Is there a present though or fix, like keeping a journal of each user's friends page on their cluster? Same with comments?
(Reply) (Thread)
From: evan
2002-01-13 12:28 am (UTC)

A bottleneck will develop eventually, but we'll deal with it once we encounter it.
(Reply) (Parent) (Thread) (Expand)
[User Picture]From: ironhead1957
2002-01-13 12:06 am (UTC)

Thank you!

I just got a cable modem this week so now I understand better what you are dealing with.

I hope everyone has patience since you just beautifully explained that you are working on speeding the system up.

LJ's growth has to be hard to keep up with. Though things don't always work right, I've been around long enough to know that when it doesn't it bothers you personally and will do all you can do make it right.

Thank you Brad for all that you and the LJ team does! :-)
(Reply) (Thread)
[User Picture]From: chaizzilla
2002-01-13 12:07 am (UTC)
i know it's a drag but i highly reccommend changing the text on the status page every hour or so just to keep the angst level down. it does wonders when something's funky & does antiwonders when it's not updated. thanks for the hard work & i was thrilled with my so far first & only experience using your support system, i don't see them that snappy very often.
(Reply) (Thread)
Page 1 of 3
<<[1] [2] [3] >>