?

Log in

No account? Create an account
Status Update - Paid Members [entries|archive|friends|userinfo]
Paid Members

[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

Status Update [Jan. 12th, 2002|10:10 pm]
Paid Members

paidmembers

[bradfitz]
It's been awhile since my last post here so I thought I'd let everybody know what's up...

For all of you, it probably seems that not much is up, actually. There hasn't been anything new in awhile, the site keeps getting slower, and all the new features we keep promising "soon" never show up.

Tell me about it. *sigh*

But here's the basic problem.... everytime I'm just about done with a project, be it S2, the directory rewrite, a good documentation system etc., the site breaks.

So earlier this month I thought: Screw this, I'm going to make redesigning the site's architecture into something that can scale beautifully a personal challenge and I'll work on nothing else but that until it's done. After that, I could get back to all the other projects.

Because really, the site working and working damn fast takes priority over everything else. Who cares if we have a super easy & powerful style system or a directory if you can't load your friends page?

So, I've been working on the new architecture the last couple weeks. Mostly, the challenge has been converting the old database format into the new one. After some test runs on a dev server and some math, I found it'd take several days to convert all of LJ. Several days of downtime would suck, even if we had read-only access enabled during that time.

The other day I came up with my 3rd plan of attack for conversion. The necessary downtime now is only a minute and 30 seconds, about.

That code is all written now and tested on a copy of the real data.

The current project is converting all the affected code on the site to understand the new db schema, which isn't too tough. A pre-req for that project is a tool to convert somebody from the old schema to the new schema on some other cluster. I wrote that tonight and have tested moving my own journal (well, the copy on the test server). That's all working. So now that I have data in both the old & new format, I can proceed with making the code on the site understand both types.

Then, for a week or more, we'll slowly be converting users one at a time to the new format. While your journal is being converted you won't be able to post new things and people won't be able to comment in your journal (you will, however, be able to comment in other people's). The downtime per-user won't be more than a few minutes and you'll probably not even notice it. If you do, you'll get a message that says your journal is in read-only mode.

Anyway, this is getting longer than I'd intended.

In summary:
  • our architecture sucks now
  • it won't suck shortly
  • after that, fun stuff.
Sorry for the less than perfect service as of late. I hope you understand that I'm working as hard as I can to make this site permanently rock.
LinkReply

Comments:
Page 1 of 4
<<[1] [2] [3] [4] >>
[User Picture]From: resilient
2002-01-12 10:08 pm (UTC)

That certainly sounds like the intelligent way to approach things. Thank you for all of the effort you and the other people on the LJ team have put into making this such a great site.
(Reply) (Thread)
[User Picture]From: skreidle
2002-01-12 10:10 pm (UTC)
We understand, and we appreciate it.. the site pretty much rocks all the time as it is, in spite of occasional breakage, thanks to you. :)
(Reply) (Thread)
[User Picture]From: katayamma
2002-01-12 10:10 pm (UTC)

Excelent progress! I had no doubt that y'all have been working to make things better. Will you be posting the format for the new database like you did the old one? Just curious as I'd lile to look it over.

Cheers
(Reply) (Thread)
[User Picture]From: bradfitz
2002-01-12 10:14 pm (UTC)
Of course. All our code will continue to be Open Source.

When we update the main site to the new code, the schema page will automatically update to show the new layout.
(Reply) (Parent) (Thread)
[User Picture]From: hamsterhuey
2002-01-12 10:13 pm (UTC)
The site rocks now....
(Reply) (Thread)
From: deadbodygrl182
2002-01-12 10:13 pm (UTC)
thanks for all your hard work, it's greatly appreciated.
(Reply) (Thread)
From: ex_evil
2002-01-12 10:14 pm (UTC)
hey as far as I'm concerned all you guys are doing an awesome job! :0) keep up the good work.
(Reply) (Thread)
[User Picture]From: halkeye
2002-01-12 10:15 pm (UTC)
could the conversion script/proccess slow down (quite temporary) the site for the other users?
and how soon before we start to see major speeds up?
(Reply) (Thread)
[User Picture]From: nyxie
2002-01-12 10:16 pm (UTC)
As always... thank you. :)
(Reply) (Thread)
[User Picture]From: kyrielle
2002-01-12 10:16 pm (UTC)
Yay! Thank you!
(Reply) (Thread)
[User Picture]From: timotheophany
2002-01-12 10:17 pm (UTC)

when's the wedding?

seeing new comments appear almost real-time is fun :)

whee... it was 3, then 7, then 8, hehehhe what number will I be?
(Reply) (Thread)
[User Picture]From: hamsterhuey
2002-01-12 10:19 pm (UTC)

Re: when's the wedding?

gotta love the icon.
(Reply) (Parent) (Thread) (Expand)
[User Picture]From: maliwane
2002-01-12 10:17 pm (UTC)

Jumping on the me too wagon. What they said!!
(Reply) (Thread)
[User Picture]From: pointedview
2002-01-12 10:17 pm (UTC)
A really good, straight-up post. Appreciate the communication and the work. Still keeping my fingers crossed for the directory. :)
(Reply) (Thread)
From: awesometc
2002-01-12 10:22 pm (UTC)
Thanks for the update!
I can't wait for LJ to permanently rock.
(Reply) (Thread)
From: ex_candle321
2002-01-12 10:24 pm (UTC)

I got my

Eye on you .
(Reply) (Thread)
From: ladycinderella
2002-01-12 10:35 pm (UTC)

Sounds like a plan! :)
Good work so far!
(Reply) (Thread)
Page 1 of 4
<<[1] [2] [3] [4] >>