Design Instagram: Photo Sharing at Scale

1
Monthly active users: 3 billion
2
Daily active users: 500M+
3
Photos uploaded daily: 95M-100M (conservative estimate)
4

5
Upload traffic:
6
- 100M uploads/day = ~1,150 uploads/second average
7
- Peak: 3x average = ~3,500 uploads/second
8
- Average image size: 2MB (after client compression)
9
- Daily upload ingestion: ~200TB/day
10

11
Storage per image:
12
- Original: 2MB
13
- Resolutions: 1080p, 640p, 320p, 150p (thumbnail) × 2 (square + original aspect)
14
- Total variants: ~8 files
15
- Average total per image: ~5MB (compressed variants)
16
- Daily storage growth: ~500TB/day
17

18
Feed reads:
19
- 500M DAU × 20 feed opens/day = 10B feed reads/day
20
- = ~115K feed reads/second average
21
- Peak: 350K+ reads/second
22

23
Social graph:
24
- Average followers per user: ~150
25
- Celebrity accounts (>1M followers): ~50,000
26
- Graph edges: billions

1
Without CDN: All reads from origin
2
With 95% cache hit rate (power-law distribution):
3
- 5% cache misses = 17.5K requests/second to origin
4
- Popular content served entirely from edge

1
Filter = Color LUT + Adjustments + Blending
2
- LUT (Look-Up Table): 3D color mapping
3
- Adjustments: Brightness, contrast, saturation, warmth
4
- Blending: Vignette, frame overlays

1
s3://instagram-media/
2
  /{user_id}/
3
    /{media_id}/
4
      original.jpg          # Raw upload
5
      1080.jpg             # Full resolution
6
      640.jpg              # Medium
7
      320.jpg              # Small
8
      150.jpg              # Thumbnail
9
      metadata.json        # EXIF, dimensions, filter applied

1
Hot tier (SSD): Last 7 days of uploads, frequently accessed
2
Warm tier (HDD): 7 days - 1 year, moderate access
3
Cold tier (archive): > 1 year, rare access
4

5
Migration policy:
6
- Content accessed > 10x/day stays hot
7
- Content accessed < 1x/week moves to warm
8
- Content not accessed in 90 days moves to cold

1
def get_feed(user_id, cursor=None, limit=20):
2
    # 1. Get cached timeline posts
3
    cached_posts = redis.zrevrange(
4
        f"timeline:{user_id}",
5
        start=cursor or 0,
6
        end=(cursor or 0) + limit * 2  # Fetch extra for ranking
7
    )
8

9
    # 2. Get followed celebrities
10
    celebrities = get_followed_celebrities(user_id)
11

12
    # 3. Fetch recent celebrity posts (last 24h)
13
    celebrity_posts = []
14
    for celeb_id in celebrities:
15
        posts = redis.zrevrangebyscore(
16
            f"celebrity:{celeb_id}:posts",
17
            max=now(),
18
            min=now() - 86400,  # 24 hours
19
            limit=5
20
        )
21
        celebrity_posts.extend(posts)
22

23
    # 4. Merge and rank
24
    all_posts = cached_posts + celebrity_posts
25
    ranked_posts = ranking_service.rank(user_id, all_posts)
26

27
    return ranked_posts[:limit]

1
Input: User embeddings + Post embeddings + Context features
2
  ↓
3
Feature extraction (100K+ dense features)
4
  ↓
5
Multi-task neural network
6
  ↓
7
Outputs:
8
  - P(like)
9
  - P(comment)
10
  - P(save)
11
  - P(share)
12
  - P(time_spent > 10s)
13
  ↓
14
Weighted combination → Final score

1
// Request
2
GET /feed?cursor=eyJ0cyI6MTY0...&limit=20
3

4
// Response
5
{
6
  "posts": [...],
7
  "next_cursor": "eyJ0cyI6MTY0...",
8
  "has_more": true
9
}
10

11
// Cursor structure (base64-encoded)
12
{
13
  "ts": 1640000000,  // Timestamp of last item
14
  "pid": "abc123",   // Post ID (for tie-breaking)
15
  "v": 2             // Cursor version (for migrations)
16
}

1
On app open:
2
1. Fetch story ring ordering (lightweight API call)
3
2. Prefetch first 3 story authors' media (background)
4
3. As user views stories, prefetch next 2 authors ahead
5

6
On story view:
7
1. Preload all segments of current story
8
2. Preload first segment of next story
9
3. Mark current story as viewed (async)

1
# User's DM inbox (subscribe on connect)
2
/u/{user_id}/inbox
3

4
# Thread-specific updates
5
/t/{thread_id}/messages
6

7
# Typing indicators
8
/t/{thread_id}/typing

1
interface QueuedMessage {
2
  localId: string // Client-generated UUID
3
  threadId: string
4
  content: string
5
  timestamp: number
6
  status: "pending" | "sent" | "failed"
7
  retryCount: number
8
}
9

10
// Persisted to IndexedDB/SQLite
11
// Survives app restarts

1
-- Thread metadata
2
CREATE TABLE threads (
3
    thread_id UUID PRIMARY KEY,
4
    participant_ids SET<UUID>,
5
    created_at TIMESTAMP,
6
    last_message_at TIMESTAMP,
7
    last_message_preview TEXT
8
);
9

10
-- Messages partitioned by thread
11
CREATE TABLE messages (
12
    thread_id UUID,
13
    message_id TIMEUUID,
14
    sender_id UUID,
15
    content TEXT,
16
    media_url TEXT,
17
    created_at TIMESTAMP,
18
    PRIMARY KEY (thread_id, message_id)
19
) WITH CLUSTERING ORDER BY (message_id DESC);
20

21
-- User's inbox (materialized view for fast inbox loading)
22
CREATE TABLE user_inbox (
23
    user_id UUID,
24
    thread_id UUID,
25
    last_message_at TIMESTAMP,
26
    unread_count INT,
27
    PRIMARY KEY (user_id, last_message_at)
28
) WITH CLUSTERING ORDER BY (last_message_at DESC);

1
User Tower:                     Item Tower:
2
[User features]                 [Item features]
3
      ↓                              ↓
4
   Dense layers                  Dense layers
5
      ↓                              ↓
6
User embedding (128d)           Item embedding (128d)
7
      ↓                              ↓
8
      └────── Dot product ──────────┘
9
                   ↓
10
           Similarity score

1
Outputs:
2
- P(like)        weight: 1.0
3
- P(comment)     weight: 2.0  (higher engagement)
4
- P(save)        weight: 3.0  (strong intent signal)
5
- P(share)       weight: 3.0
6
- P(follow)      weight: 5.0  (acquisition metric)
7
- P(hide)        weight: -10.0 (negative signal)
8

9
Final score = Σ(weight × probability)

1
POST /api/v1/media/upload
2
Content-Type: multipart/form-data
3

4
Request:
5
- file: <binary>
6
- media_type: "image" | "video"
7
- filter_id: "clarendon" | "gingham" | ... (optional)
8

9
Response (200 OK):
10
{
11
  "media_id": "abc123",
12
  "urls": {
13
    "1080": "https://cdn.instagram.com/abc123/1080.jpg",
14
    "640": "https://cdn.instagram.com/abc123/640.jpg",
15
    "320": "https://cdn.instagram.com/abc123/320.jpg",
16
    "150": "https://cdn.instagram.com/abc123/150.jpg"
17
  },
18
  "expires_at": "2024-01-02T00:00:00Z"  // Media must be posted within 24h
19
}

1
POST /api/v1/posts
2

3
Request:
4
{
5
  "media_ids": ["abc123", "def456"],  // Up to 10 for carousel
6
  "caption": "Summer vibes 🌴",
7
  "location_id": "loc_789",           // Optional
8
  "tagged_users": ["user_111"],       // Optional
9
  "alt_text": "Beach sunset"          // Accessibility
10
}
11

12
Response (201 Created):
13
{
14
  "post_id": "post_xyz",
15
  "permalink": "https://instagram.com/p/xyz",
16
  "created_at": "2024-01-01T12:00:00Z"
17
}
18

19
Errors:
20
- 400: Invalid media_id (expired or not found)
21
- 400: Caption too long (> 2200 characters)
22
- 403: Tagged user has blocked you
23
- 429: Rate limited (> 25 posts/day)

1
GET /api/v1/feed?cursor={cursor}&limit=20
2

3
Response (200 OK):
4
{
5
  "posts": [
6
    {
7
      "post_id": "post_xyz",
8
      "author": {
9
        "user_id": "user_123",
10
        "username": "photographer",
11
        "profile_pic_url": "https://...",
12
        "is_verified": true
13
      },
14
      "media": [
15
        {
16
          "type": "image",
17
          "url": "https://cdn.instagram.com/...",
18
          "width": 1080,
19
          "height": 1080,
20
          "alt_text": "Beach sunset"
21
        }
22
      ],
23
      "caption": "Summer vibes 🌴",
24
      "like_count": 1234,
25
      "comment_count": 56,
26
      "created_at": "2024-01-01T12:00:00Z",
27
      "viewer_has_liked": false,
28
      "viewer_has_saved": false
29
    }
30
  ],
31
  "next_cursor": "eyJ0cyI6MTY0...",
32
  "has_more": true
33
}

1
GET /api/v1/stories/feed
2

3
Response (200 OK):
4
{
5
  "story_ring": [
6
    {
7
      "user_id": "user_123",
8
      "username": "friend1",
9
      "profile_pic_url": "https://...",
10
      "has_unseen": true,
11
      "latest_story_ts": "2024-01-01T11:00:00Z"
12
    }
13
  ],
14
  "stories": {
15
    "user_123": [
16
      {
17
        "story_id": "story_abc",
18
        "media_url": "https://...",
19
        "media_type": "image",
20
        "created_at": "2024-01-01T11:00:00Z",
21
        "expires_at": "2024-01-02T11:00:00Z",
22
        "seen": false,
23
        "reply_enabled": true
24
      }
25
    ]
26
  }
27
}

1
POST /api/v1/direct/threads/{thread_id}/messages
2

3
Request:
4
{
5
  "text": "Hey, nice photo!",
6
  "reply_to_story_id": "story_abc"  // Optional
7
}
8

9
Response (201 Created):
10
{
11
  "message_id": "msg_xyz",
12
  "thread_id": "thread_123",
13
  "created_at": "2024-01-01T12:00:00Z",
14
  "status": "sent"
15
}

1
-- Users
2
CREATE TABLE users (
3
    id BIGINT PRIMARY KEY,
4
    username VARCHAR(30) UNIQUE NOT NULL,
5
    email VARCHAR(255) UNIQUE,
6
    phone VARCHAR(20) UNIQUE,
7
    full_name VARCHAR(100),
8
    bio TEXT,
9
    profile_pic_url TEXT,
10
    is_private BOOLEAN DEFAULT false,
11
    is_verified BOOLEAN DEFAULT false,
12
    follower_count INT DEFAULT 0,
13
    following_count INT DEFAULT 0,
14
    post_count INT DEFAULT 0,
15
    created_at TIMESTAMPTZ DEFAULT NOW(),
16
    updated_at TIMESTAMPTZ DEFAULT NOW()
17
);
18

19
CREATE INDEX idx_users_username ON users(username);
20

21
-- Posts
22
CREATE TABLE posts (
23
    id BIGINT PRIMARY KEY,
24
    author_id BIGINT NOT NULL REFERENCES users(id),
25
    caption TEXT,
26
    location_id BIGINT REFERENCES locations(id),
27
    like_count INT DEFAULT 0,
28
    comment_count INT DEFAULT 0,
29
    is_archived BOOLEAN DEFAULT false,
30
    created_at TIMESTAMPTZ DEFAULT NOW(),
31
    deleted_at TIMESTAMPTZ
32
);
33

34
CREATE INDEX idx_posts_author ON posts(author_id, created_at DESC);
35

36
-- Post Media (supports carousel)
37
CREATE TABLE post_media (
38
    id BIGINT PRIMARY KEY,
39
    post_id BIGINT NOT NULL REFERENCES posts(id),
40
    media_type VARCHAR(10) NOT NULL,  -- 'image', 'video'
41
    url TEXT NOT NULL,
42
    width INT,
43
    height INT,
44
    alt_text TEXT,
45
    position SMALLINT DEFAULT 0,
46
    created_at TIMESTAMPTZ DEFAULT NOW()
47
);
48

49
CREATE INDEX idx_post_media_post ON post_media(post_id);
50

51
-- Comments
52
CREATE TABLE comments (
53
    id BIGINT PRIMARY KEY,
54
    post_id BIGINT NOT NULL REFERENCES posts(id),
55
    author_id BIGINT NOT NULL REFERENCES users(id),
56
    parent_id BIGINT REFERENCES comments(id),  -- For replies
57
    content TEXT NOT NULL,
58
    like_count INT DEFAULT 0,
59
    created_at TIMESTAMPTZ DEFAULT NOW(),
60
    deleted_at TIMESTAMPTZ
61
);
62

63
CREATE INDEX idx_comments_post ON comments(post_id, created_at DESC);
64

65
-- Likes (polymorphic)
66
CREATE TABLE likes (
67
    id BIGINT PRIMARY KEY,
68
    user_id BIGINT NOT NULL REFERENCES users(id),
69
    target_type VARCHAR(10) NOT NULL,  -- 'post', 'comment', 'story'
70
    target_id BIGINT NOT NULL,
71
    created_at TIMESTAMPTZ DEFAULT NOW(),
72
    UNIQUE(user_id, target_type, target_id)
73
);
74

75
CREATE INDEX idx_likes_target ON likes(target_type, target_id);

1
-- Follows (partitioned by follower for "who do I follow" queries)
2
CREATE TABLE follows (
3
    follower_id UUID,
4
    following_id UUID,
5
    created_at TIMESTAMP,
6
    PRIMARY KEY (follower_id, following_id)
7
);
8

9
-- Followers (partitioned by following for "who follows me" queries)
10
CREATE TABLE followers (
11
    following_id UUID,
12
    follower_id UUID,
13
    created_at TIMESTAMP,
14
    PRIMARY KEY (following_id, follower_id)
15
);
16

17
-- Activity feed (for "activity" tab)
18
CREATE TABLE activity (
19
    user_id UUID,
20
    activity_id TIMEUUID,
21
    actor_id UUID,
22
    activity_type TEXT,  -- 'like', 'comment', 'follow', 'mention'
23
    target_type TEXT,
24
    target_id UUID,
25
    created_at TIMESTAMP,
26
    PRIMARY KEY (user_id, activity_id)
27
) WITH CLUSTERING ORDER BY (activity_id DESC);

1
-- PL/pgSQL function for globally unique, time-sorted IDs
2
CREATE OR REPLACE FUNCTION instagram_id() RETURNS BIGINT AS $$
3
DECLARE
4
    epoch BIGINT := 1314220021721;  -- Custom epoch (Sep 2011)
5
    seq_id BIGINT;
6
    now_millis BIGINT;
7
    shard_id INT := 1;  -- Set per logical shard
8
    result BIGINT;
9
BEGIN
10
    SELECT nextval('instagram_id_seq') % 1024 INTO seq_id;
11
    SELECT FLOOR(EXTRACT(EPOCH FROM NOW()) * 1000) INTO now_millis;
12

13
    result := (now_millis - epoch) << 23;  -- 41 bits for timestamp
14
    result := result | (shard_id << 10);    -- 13 bits for shard
15
    result := result | (seq_id);            -- 10 bits for sequence
16

17
    RETURN result;
18
END;
19
$$ LANGUAGE PLPGSQL;

1
// Virtualized list configuration
2
const FeedList = () => {
3
  return (
4
    <VirtualizedList
5
      data={posts}
6
      renderItem={({ item }) => <PostCard post={item} />}
7
      estimatedItemSize={600}  // Average post height
8
      overscanCount={3}        // Render 3 items above/below viewport
9
      onEndReached={loadMore}
10
      onEndReachedThreshold={0.5}
11
    />
12
  );
13
};

1
// Progressive image loading
2
const PostImage = ({ post }) => {
3
  const [loaded, setLoaded] = useState(false);
4

5
  return (
6
    <div className="post-image">
7
      {/* Blur placeholder (tiny, inline) */}
8
      <img
9
        src={post.thumbnail_blur}  // 10x10 base64
10
        className={loaded ? 'hidden' : 'blur'}
11
      />
12

13
      {/* Full image (lazy loaded) */}
14
      <img
15
        src={post.media_url}
16
        loading="lazy"
17
        onLoad={() => setLoaded(true)}
18
        className={loaded ? 'visible' : 'hidden'}
19
      />
20
    </div>
21
  );
22
};

1
// Horizontal scroll with snap points
2
const StoriesRing = ({ stories }) => {
3
  return (
4
    <div className="stories-ring" style={{
5
      display: 'flex',
6
      overflowX: 'scroll',
7
      scrollSnapType: 'x mandatory',
8
      WebkitOverflowScrolling: 'touch'  // Smooth iOS scroll
9
    }}>
10
      {stories.map(story => (
11
        <div
12
          key={story.id}
13
          style={{ scrollSnapAlign: 'start' }}
14
        >
15
          <StoryAvatar story={story} />
16
        </div>
17
      ))}
18
    </div>
19
  );
20
};

1
// Like button with optimistic UI
2
const LikeButton = ({ post }) => {
3
  const [optimisticLiked, setOptimisticLiked] = useState(post.viewer_has_liked);
4
  const [optimisticCount, setOptimisticCount] = useState(post.like_count);
5

6
  const handleLike = async () => {
7
    // Optimistic update (immediate feedback)
8
    setOptimisticLiked(!optimisticLiked);
9
    setOptimisticCount(prev => optimisticLiked ? prev - 1 : prev + 1);
10

11
    try {
12
      await api.toggleLike(post.id);
13
    } catch (error) {
14
      // Rollback on failure
15
      setOptimisticLiked(post.viewer_has_liked);
16
      setOptimisticCount(post.like_count);
17
      showError('Failed to like post');
18
    }
19
  };
20

21
  return (
22
    <button onClick={handleLike}>
23
      <HeartIcon filled={optimisticLiked} />
24
      <span>{formatCount(optimisticCount)}</span>
25
    </button>
26
  );
27
};