Twitter News Feed (Timeline)
Home timeline: fan-out write
Search: fan-out read
Home Timeline
Each write is fanned out 3 times for redundancy purpose.
We query the Social Graph Service to fan out the write to all the followers.
Every write is consist of:
tweet ID: the sent tweet
user ID: the sender's ID
bits: miscellaneous product related bits.
max limit: 800 tweets in your timeline
If you are not a active user (logged in in the last 30 days), we don't fan out write into your timeline.
If your cache is empty, we will reconstruct your timeline.
Search
Earlybird is a modified version of Lucene.
Twitter mainly rank the contents based on:
Re-tweet
Like
Reply
The home timeline requires O(n) write and O(1) read
The search timeline requires O(1) write and O(n) read
Improve fanout write
Race condition:
Lady gaga sends out tweet
I retweet or reply
My fanout write is done before Lady gaga's is done.
400 million tweets a day, 4600 tweets a second
Merge home timeline and search timeline.
Don't fanout write for users with large number of followers. Only do fan out for users with small number of followers.
Stats
Reference
Timelines at Scale (APR 03, 2013): https://www.infoq.com/presentations/Twitter-Timeline-Scalability/
Last updated
