Design a YouTube-Style Video Platform

A video platform at YouTube scale absorbs hundreds of hours of upload per minute, fans every accepted upload out into dozens of resolution × codec × bitrate variants, and serves segments to a global audience through a deeply tiered CDN. This article designs that system end to end — the upload protocol, the chunk-parallel and per-shot transcoding pipeline, CMAF-packaged HLS/DASH delivery, the hybrid ABR algorithm in the player, multi-tier caching with origin shielding, the Vitess-sharded metadata layer, view counting at billions of events per day, the two-stage watch-time recommender, the Content ID and moderation pipeline, and a brief contrast with the live (LL-HLS / LL-DASH) path. The goal is the strongest possible mental model for a senior engineer designing or interviewing on a VOD platform.

Mental model

A VOD pipeline is shaped by four constraints that pull in different directions:

1. Encoding is computationally expensive and embarrassingly parallel. A single 10-minute upload produces tens of output renditions (resolution × codec × rung). Total wall-clock time scales with chunk count over worker count, not video length.
2. Latency budgets are asymmetric. Uploads tolerate seconds to minutes; playback must start in under two seconds and rebuffer rarely. That asymmetry funds aggressive CDN caching, prefetching, and origin shielding.
3. Demand follows extreme power laws. A small fraction of videos drive the bulk of plays, so storage tiering and shield-layer consolidation pay disproportionate dividends.
4. Codec / device matrices are narrow per device, wide per catalog. Each playback session needs only one codec variant; the catalog needs many. Selective AV1 / HEVC encoding amortizes the high encode cost over expected plays.

Five mechanisms together resolve those constraints:

• Resumable chunked uploads keep multi-GB transfers alive over flaky networks. The de-facto protocol is tus 1.0, now being standardized at the IETF as draft-ietf-httpbis-resumable-upload.
• Chunk-parallel encoding decouples wall time from video length. The split is on GOP / IDR boundaries so chunks can be re-assembled losslessly.
• Per-shot encoding allocates bits to scenes that actually need them — Netflix’s “Dynamic Optimizer” reports roughly 30% bitrate savings and 65% fewer rebuffers on top of per-title.
• CMAF (ISO/IEC 23000-19) lets a single set of fragmented MP4 segments serve both HLS and MPEG-DASH clients — Apple added fMP4 to HLS at WWDC 2016 specifically to enable this.
• Origin shielding consolidates the long tail of edge misses through a small set of regional caches before any request reaches origin storage. Real-world workloads see significant origin offload, though the precise reduction depends heavily on the request distribution.

Requirements

Functional requirements

Non-functional requirements

Scale estimation

Public figures place YouTube at roughly 2.7 billion monthly active users with 500+ hours of video uploaded per minute and over a billion hours watched per day.^{1 2 3} We’ll size the system from those numbers.

1500 hours/min × 60 × 24 = 720,000 hours/day uploaded2Average raw bitrate: ~16 Mbps (mixed 1080p / 4K / mobile)3Daily upload bytes:  720,000 × 3600 × 16e6 / 8 ≈ 5.2 PB/day raw45Storage growth:6  Encoded variants: ~5× original (8 rungs × 3 codecs, with per-shot bit budgets)7  Daily encoded growth: ~25 PB/day8  Annual encoded growth: ~9 EB/year (before lifecycle deletes)

11 B watch hours/day × 3600 s × ~3 Mbps avg / 8 ≈ 1.35 EB/day egress2Peak concurrent: ~250 M players (estimated, asymmetric across time zones)34CDN with 95% byte-level offload from origin:5  ~67 PB/day from origin instead of 1.35 EB/day

Design paths

Path A — single-pass centralized transcoding

A small platform encodes each upload as a single ffmpeg-style job per rung on a few large boxes. Job duration scales with video length × ladder size.

• Pros: trivially simple, easy to operate, low fixed overhead.
• Cons: a 4-hour upload at a deep ladder takes hours of wall clock; failures restart the whole encode; no per-shot bit budgeting.
• Where it fits: < ~10 K uploads/day, content where time-to-publish isn’t critical.

Path B — chunk-parallel per-shot encoding (modern VOD)

Split each upload at IDR boundaries, encode chunks in parallel across a worker pool, optionally analyze each shot to set its own bit budget, then assemble.

• Pros: encode time decouples from video length; failed chunks retry in isolation; per-shot analysis cuts bandwidth without touching perceived quality; the chunk-level structure aligns with ABR segmentation downstream.
• Cons: complex orchestration (job graph, dedup, retries); chunk-boundary artifacts must be handled (overlap or constrained-bitrate at boundaries); scheduling and shuffle dominate the wall-clock at small scale.
• Where it fits: anything beyond a few thousand uploads/day, especially user-generated content with bursty viral demand.

The rest of this article assumes Path B. Three real-world data points back the choice:

• YouTube’s custom Argos VCU ASIC reports 20–33× compute efficiency over optimized software encoding at warehouse scale, with each chip carrying 10 encoder cores capable of real-time 2160p60.⁴
• Netflix’s microservice pipeline (“Cosmos”) fans out roughly 140 video encodes and 552 audio encodes per hour-long episode, generating ~27,000 microservice calls and ~1 M tracing spans for that single episode — a chunk-parallel job graph at industrial scale.⁵
• Netflix’s Dynamic Optimizer (per-shot encoding) shipped to 4K streams in 2020 with a reported ~30% additional bitrate saving over per-title and a >65% rebuffer reduction.⁶

Component overview

The system has three loosely coupled subsystems:

1. Ingest + processing — upload service, chunk manager, job queue, transcoder pool, QC, packager.
2. Storage + delivery — raw object store, encoded segment store, origin shield, CDN edge, player.
3. Metadata + discovery — metadata DB, hot read cache, search index, recommendation pipeline, view counter.

Each subsystem owns its own write path and is rate-limited / scaled independently.

Upload service

Resumable upload protocol

The de-facto standard is tus 1.0, an HTTP-based protocol with Upload-Offset, Upload-Length, and Tus-Resumable headers. The IETF HTTP working group adopted a tus-derived draft, draft-ietf-httpbis-resumable-upload, and that draft is on track to subsume the core tus protocol; tus 1.x will likely remain the working spec for the next few years and a future tus 2.0 will hold extensions (Expiration, Concatenation) that the IETF draft omits.

Chunk size trades resume granularity against per-chunk overhead:

Upload validation and processing

Validation is a hard gate before anything queues:

• Container: MP4 / MOV / MKV / WebM / AVI accepted; reject obscure formats.
• Duration: ≤ 12 hours by default (channel-level overrides).
• Resolution: up to 8K (7680×4320) for the catalog.
• File size: capped at the upload size limit (YouTube currently allows up to 256 GB per file).⁷
• Audio tracks: capped at a small fixed number (e.g. 8) to bound per-rung packaging cost.

Once validated, the upload service writes the raw file to the durable object store, probes container metadata (resolution, codec, FPS, audio layout, color primaries, HDR metadata), and emits a job to the segmentation queue. The video is marked discoverable as soon as a thumbnail is ready — the encoded ladder fills in behind it.

Thumbnails

Thumbnail generation runs on the raw file before encoding finishes:

1. Extract candidate frames at fixed offsets (25, 50, 75% of duration) plus shot-detection peaks.
2. Score each candidate (sharpness, face detection, composition, no near-black/near-white frames).
3. Encode the chosen thumbnail at multiple sizes.
4. Build a sprite sheet (one tile per ~10 s) for the player’s scrub-bar preview.

Transcoding pipeline

Per-chunk × per-rung × per-codec

The job graph for a single video is:

1. Demux raw container into elementary streams (video, audio, subtitles).
2. Segment the video stream at IDR boundaries into 2–4 s chunks. Boundaries align to keyframes so each chunk is independently decodable.
3. Analyze each shot (scene-change detection + motion / texture complexity) to set a per-shot bit budget.
4. Encode the cross-product of (chunk × codec × rung) in parallel. Each chunk is short enough that even AV1’s 5–10× higher encode cost over H.264 fits a single-worker budget.
5. QC each encoded chunk against VMAF; re-encode at a higher rung if quality drops below threshold.
6. Package chunks as CMAF fragmented MP4, then write HLS .m3u8 and DASH .mpd manifests pointing at the same byte streams.

Codec selection

A pragmatic ladder strategy:

1. Always encode H.264 — universal fallback for older devices and embeds.
2. Encode VP9 by default for non-Apple modern browsers; well-supported, relatively cheap.
3. Encode HEVC for Apple devices that don’t support VP9.
4. Encode AV1 selectively: trending content, popular catalog titles, mobile-cellular rungs. The 30%+ bitrate saving pays back the encode cost only after a non-trivial number of plays. YouTube has been progressively expanding AV1 to more videos as decoder availability grows.

Bitrate ladder

A reasonable VP9 ladder for general-purpose UGC:

The ladder is the upper bound. Per-title and per-shot encoding push the actual bitrates lower without losing perceptual quality.

Per-title and per-shot encoding

Netflix’s original 2015 per-title encoding tunes the bitrate ladder per title by sweeping bitrate / quality and picking the convex hull. For example, Orange Is the New Black hit visually-equivalent quality at ~20% lower bitrate than a fixed ladder.

Per-shot (“Dynamic Optimizer”) encoding goes further: each shot gets its own bit budget based on how hard it is to encode (motion, texture, grain). Netflix reports roughly 30% bitrate savings on top of per-title and >65% fewer rebuffers for 4K streams.

1Fixed ladder:        5 Mbps for everything at 1080p2Per-title (doc):     2.5 Mbps achieves VMAF 95     → 50% saving3Per-title (action):  5 Mbps needed for VMAF 95     → ~0% saving4Per-shot (action):   3.8 Mbps avg, 8 Mbps in5                     high-motion shots, 1.5 Mbps in6                     dialog shots                  → ~25% saving

Chunk parallelism math

1Chunks at 2s: 300 / 2 = 150 chunks2Variants:     8 × 3 = 24 outputs per chunk3Total tasks:  150 × 24 = 3,600 encode tasks45Serial encode (single worker, software, 1× realtime):6  ~10 min × 24 variants = ~240 min wall clock78Chunk-parallel (150 workers):9  Per-chunk wall clock = max(per-variant encode time)10  At ~2× realtime for AV1 software: ~4 s per chunk × 24 variants11  Wall clock ≈ 4 s × 24 = ~96 s + assembly + manifest overhead12  Speedup: ~150×

Boundary handling is the subtlety: the encoder needs reference frames near the chunk edges, so workers either get a small overlap (1–2 GOP) of context that’s trimmed during assembly, or the encoder is constrained to closed-GOP encoding that doesn’t reference frames outside the chunk. Closed GOPs cost a small amount of efficiency; the operational simplicity is worth it at scale.

Quality control with VMAF

VMAF is Netflix’s open-source perceptual quality metric, scoring 0–100 with a target of ~93+ for “visually transparent” output relative to source.⁹

The QC stage scores every encoded chunk against the source. A chunk whose 1%-low VMAF dips below threshold is re-encoded at a higher rung and the manifest is patched. Aggregating only on the mean hides quality cliffs in motion-heavy frames, which is why X (formerly Twitter) and Netflix both publish percentile-based VMAF tracking.

Adaptive bitrate streaming

HLS, DASH, and CMAF

HLS is documented in the informational RFC 8216 (2017); the active spec is now draft-pantos-hls-rfc8216bis (“HLS 2nd edition”), where Low-Latency HLS lives. DASH is ISO/IEC 23009-1. At the bytes-on-the-wire level, both protocols can carry the same fragmented MP4 segments, thanks to CMAF (ISO/IEC 23000-19) — Apple added fMP4 segments to HLS at WWDC 2016 explicitly to enable a single library to serve both ecosystems.

Encoding once and packaging twice is the deployment pattern that wins:

Manifest examples

1#EXTM3U2#EXT-X-VERSION:73#EXT-X-INDEPENDENT-SEGMENTS45#EXT-X-STREAM-INF:BANDWIDTH=5000000,RESOLUTION=1920x1080,CODECS="avc1.640028,mp4a.40.2"61080p/playlist.m3u878#EXT-X-STREAM-INF:BANDWIDTH=2500000,RESOLUTION=1280x720,CODECS="avc1.64001f,mp4a.40.2"9720p/playlist.m3u81011#EXT-X-STREAM-INF:BANDWIDTH=1000000,RESOLUTION=854x480,CODECS="avc1.64001e,mp4a.40.2"12480p/playlist.m3u8

1#EXTM3U2#EXT-X-VERSION:73#EXT-X-TARGETDURATION:44#EXT-X-MEDIA-SEQUENCE:056#EXTINF:4.000,7segment_0001.m4s8#EXTINF:4.000,9segment_0002.m4s10#EXTINF:4.000,11segment_0003.m4s12#EXT-X-ENDLIST

ABR algorithm

Three families of ABR algorithm are worth knowing:

1. Throughput-based. Estimate bandwidth from recent segment downloads and pick the highest rung whose bitrate fits, with a safety margin.

   TXTText

   1est_bw = bytes_downloaded / download_time   # EWMA across recent segments2safe   = est_bw × 0.73pick   = max(rung) where bitrate(rung) < safe

2. Buffer-based (BOLA). BOLA (Spiteri, Urgaonkar, Sitaraman, INFOCOM 2016) frames bitrate selection as a Lyapunov-optimization problem on buffer occupancy, with theoretical near-optimality bounds and no need to predict bandwidth. It is the reference buffer-based rule in dash.js.
3. Hybrid (production default). Combine throughput and buffer signals — throughput drives the upper bound, buffer level decides how aggressively to step toward it. dash.js’s default Dynamic rule switches between throughput and BOLA based on buffer level.

Practical constraints layered on top of the algorithm:

• Startup: open at a conservative rung (often 720p or below), prefetch 2–3 segments before pressing play.
• Minimum dwell time at the chosen rung (e.g. 10 s) to prevent flapping.
• Maximum drop per switch (e.g. 2 rungs) to avoid jarring quality dives.
• Buffer emergency: if buffer < 5 s, drop to the lowest rung immediately and prioritize survival over quality.

Segment duration trade-offs

YouTube and most VOD providers settle in the 2–4 s range; Netflix tends toward 4–6 s. Live / low-latency workloads use 2 s with chunked-transfer-encoded delivery.

CDN and delivery

Multi-tier caching

Three layers between the player and the bytes:

Why origin shielding wins

Without a shield layer, every edge PoP that misses sends an independent fetch to origin. With a shield, a single regional cache absorbs many edge misses and forwards at most one origin request per object. The benefit grows with PoP fan-out and with object size — the larger the object, the more dramatic the byte savings even from a small reduction in origin request count.

1Without shield:2  N edge PoPs × p_miss = N × p_miss origin fetches per object burst34With shield:5  N edge PoPs → 1 regional shield → ≤ 1 origin fetch per object,6  while the shield serves the rest of the regional misses from cache

AWS’s multi-CDN Origin Shield case study reports production users seeing as much as 57% origin load reduction; Fastly’s origin offload analysis shows a 4× origin-traffic spike when shielding was disabled in a production case.

Cache key design

1/{video_id}/{rung}/{codec}/{segment_number}.m4s2example: /abc123/1080p/vp9/segment_0042.m4s

What stays out of the cache key:

• Session tokens, signed URL parameters (validate, then strip).
• Per-user identifiers (otherwise every user becomes a cache miss).
• Cache-buster timestamps (segments are immutable; use Cache-Control: public, max-age=31536000, immutable).
• Analytics query parameters.

Multi-CDN

For redundancy, geographic optimization, and price arbitrage, large platforms run more than one CDN.

Storage

Tiering and lifecycle

A small fraction of videos drives most plays, so storage is tiered hot → warm → cold → archive and lifecycle-managed.

1Upload                       → Hot tier2After 30 days                → Warm if views/day < N, else stay Hot3After 90 days low traffic    → Cold; archive originals4After 365 days zero traffic  → Lifecycle delete (cold copy only;5                                  archive copy of originals retained)6On traffic spike             → Promote back to Hot on miss

Per-video storage estimate

1Original raw (H.264, 1080p, ~6 Mbps): ~450 MB2Encoded outputs:3  H.264 ladder (8 rungs):  ~800 MB4  VP9 ladder (8 rungs):    ~500 MB5  AV1 (top 3 rungs only):  ~150 MB6  HEVC ladder (top 4):     ~250 MB (Apple-only fallback)7  Thumbnails + sprite + manifest: ~10 MB8Total per-video footprint: ~2.2 GB (~5× original)

Multi-region replication is then applied selectively:

Metadata and search

Sharded relational store

YouTube’s metadata layer started life as a single MySQL primary and evolved into a horizontally sharded fleet behind Vitess, the project Google open-sourced in 2012 to absorb its sharding, connection-pooling, and online-resharding logic. Two pieces matter for the design:

• VTGate is a stateless SQL proxy. Application code talks to it as if it were a single MySQL endpoint; VTGate consults the topology service, parses the query, applies the vindex (sharding function) to the keyspace ID column, and routes to the right shard.¹⁰
• VTTablet is a sidecar in front of each MySQL instance that pools connections, kills runaway queries, rewrites unsafe statements, and drives the online-resharding workflow (split / merge a shard live with minimal write downtime).¹¹

For a YouTube-shaped workload, the practical implications are: keep the schema MySQL-compatible, partition by channel_id or video_id at the keyspace level, and lean on Vitess to make resharding a runbook event rather than a migration project. The same shape — a thin proxy hiding shard fan-out from the app, with online-resharding as a first-class operation — shows up in PlanetScale, Slack’s Vitess deployment, and the GitHub Vitess migration.

Schema sketch

1CREATE TABLE videos (2    video_id            UUID         PRIMARY KEY,3    channel_id          UUID         NOT NULL REFERENCES channels(id),4    title               VARCHAR(100) NOT NULL,5    description         TEXT,6    duration_seconds    INTEGER      NOT NULL,7    upload_timestamp    TIMESTAMPTZ  NOT NULL,8    publish_timestamp   TIMESTAMPTZ,9    status              VARCHAR(20)  NOT NULL DEFAULT 'processing',10    -- denormalized counters (eventually consistent)11    view_count          BIGINT       DEFAULT 0,12    like_count          BIGINT       DEFAULT 0,13    comment_count       INTEGER      DEFAULT 0,14    -- content signals15    category_id         INTEGER,16    language            VARCHAR(10),17    age_restricted      BOOLEAN      DEFAULT false,18    CONSTRAINT valid_status CHECK (status IN ('processing','ready','failed','deleted'))19);2021CREATE INDEX idx_videos_channel22    ON videos (channel_id, publish_timestamp DESC);23CREATE INDEX idx_videos_category24    ON videos (category_id, publish_timestamp DESC);25CREATE INDEX idx_videos_trending26    ON videos (view_count DESC)27    WHERE status = 'ready'28      AND publish_timestamp > NOW() - INTERVAL '7 days';

Search index

A typical Elasticsearch / OpenSearch mapping:

1{2  "mappings": {3    "properties": {4      "video_id": { "type": "keyword" },5      "title": {6        "type": "text",7        "analyzer": "standard",8        "fields": {9          "exact": { "type": "keyword" },10          "autocomplete": { "type": "search_as_you_type" }11        }12      },13      "description": { "type": "text" },14      "channel_name": {15        "type": "text",16        "fields": { "exact": { "type": "keyword" } }17      },18      "tags": { "type": "keyword" },19      "category": { "type": "keyword" },20      "duration_seconds": { "type": "integer" },21      "view_count": { "type": "long" },22      "publish_date": { "type": "date" },23      "language": { "type": "keyword" },24      "transcript": { "type": "text", "analyzer": "standard" }25    }26  }27}

1{2  "query": {3    "bool": {4      "must": [{5        "multi_match": {6          "query": "kubernetes tutorial",7          "fields": ["title^3", "description", "tags^2", "transcript"]8        }9      }],10      "filter": [11        { "term":  { "language": "en" } },12        { "range": { "duration_seconds": { "gte": 300, "lte": 1800 } } }13      ]14    }15  },16  "sort": [{ "_score": "desc" }, { "view_count": "desc" }]17}

View counting

The view counter must be near real-time, deduplicated, fraud-resistant, and monotonic. The pipeline:

Counting rules in production:

• A view counts only after a minimum watch-time threshold (industry standard ≈ 30 s, shorter for Shorts-style content).
• Per-video bloom filter on hash(video_id, user_id, ip, ua) over a rolling window suppresses replays — accept ~1% false-positive rate (slight under-count) in exchange for O(1) memory.
• Same-IP / same-cookie bursts are rate-limited and cross-checked against ML fraud signals before being added to the public count.
• Public counts are computed at lower precision than internal counters (rounded for high-traffic videos) to disincentivize gaming.

Recommendations

Recommendations drive a striking share of platform watch time — at CES 2018, YouTube’s then-CPO Neal Mohan publicly pegged it at >70% of watch time on the platform.¹²

The reference architecture from Covington, Adams, and Sargin’s RecSys 2016 paper, “Deep Neural Networks for YouTube Recommendations,”¹³ is still the spine that most production recommender stacks copy: an offline-trained candidate generator narrows millions of videos to ~hundreds, and a heavier ranker scores those candidates with rich per-impression features. The same two stages, often re-skinned with newer architectures (transformers, two-tower retrieval, multi-task learning), show up at TikTok, Spotify, and Netflix.

Candidate generation

The retrieval model is an extreme multiclass classifier: predict “the next video the user will watch” out of a corpus of millions. The training-time loss is sampled softmax — full softmax over millions of classes is infeasible, so each training step samples a few thousand negatives and treats the loss as a logistic approximation.¹³ At serving time the trained network produces a user embedding , and candidate retrieval reduces to a maximum-inner-product search:

That MIPS reduction is what makes ANN libraries — HNSW, ScaNN, FAISS — the right substrate. A HNSW or ScaNN index over hundreds of millions of video embeddings returns a few thousand candidates in single-digit milliseconds, on a single replica, with recall well above 95%.

Inputs that feed the user-embedding tower in the original paper:

• Bag-of-watches (embedded video IDs from recent history, average-pooled).
• Bag-of-search-tokens (user search queries, similarly embedded).
• Demographics and geography.
• An example age feature — wall-clock seconds since the training example was logged — that lets the model un-bias toward videos popular at training time and surface fresh content at serving time.¹³

Ranking

The ranker scores the few thousand candidates with deep per-impression features — display position, time since last watch of the same channel, language match, the candidate’s own engagement priors. Two design choices dominate:

• Optimize for expected watch time, not click probability. Click-through-rate ranking rewards clickbait. Watch-time ranking rewards videos people actually finish.
• Learn the watch-time objective via weighted logistic regression. Positive impressions (clicks) are weighted by their observed watch time; negative impressions get unit weight. The learned odds then approximate , which for the small click probabilities seen in production is approximately the expected watch time itself.¹³ At serving time the model emits , which the system uses directly as the watch-time score.

Signals layered on top of the base model in production:

Content moderation and Content ID

Moderation runs as a fan-out at upload time, before the encoded ladder is published to the catalog. Three pipelines run in parallel; any one of them can hold a video in a pending_review state.

Safety hash matching

For categories with zero tolerance — CSAM, terrorist content, known violent-extremist media — the industry standard is perceptual hash matching against an industry-shared blocklist. Microsoft’s PhotoDNA and the GIFCT shared hash database are the canonical references for images and short video clips; the hash function is robust to resizing, recompression, color shifts, and minor crops. A match is treated as ground truth: the upload is blocked, the hash is logged, and (for CSAM) the report flows to NCMEC by statutory requirement.

ML classifiers for unknown harms

Hash matching only catches what’s already in the database. A second tier of classifiers — nudity, graphic violence, spam, hate speech, dangerous misinformation — runs over thumbnails, sampled frames, the audio track, and (when available) the transcript. The output is a bank of category scores; thresholds determine whether the video auto-publishes, queues for human review, or holds pending escalation.

Content ID

YouTube’s Content ID is a rights-management layer, not a safety layer. Rights holders upload reference assets, the system extracts compact perceptual fingerprints over both video (scene-chunked spatio-temporal features) and audio (spectrogram-derived), and every upload is matched against the reference database.¹⁴ When a match fires, the rights holder’s pre-set policy decides:

Three engineering properties are non-obvious:

• Match on chunks, not files. The fingerprint matcher operates on short overlapping windows, so a clip embedded inside a longer original still matches.
• Robust to common adversarial transforms. The fingerprint survives re-encoding, resolution changes, frame-rate conversion, mirror-flip, and modest crop / overlay. New adversarial attacks (audio pitch-shift, time-stretch, image overlays) trigger re-tuning of the fingerprint extractor.
• Disputes are first-class. Creators can dispute a claim; revenue is held in escrow during the dispute window. The system has to keep the public-facing video available during low-confidence claims, because false positives at this scale carry creator-trust costs that exceed the cost of brief unauthorized monetization.

Live streaming vs VOD

The article above is a VOD design. A live-streaming pipeline is shaped by hard real-time constraints that change most stages of the pipeline.

A practical design rule: a platform that does both should share CMAF, HLS, and DASH packaging but not the encoder fleet, the manifest server, or the CDN configuration. The live path needs a separate ingest tier (RTMP, SRT, or WebRTC), a low-latency transcoder pool sized for peaks, and a manifest server that supports blocking playlist reloads and availabilityTimeOffset semantics.

Frontend and player

Video player responsibilities

1. Manifest parsing — HLS / DASH (and CMAF where supported).
2. ABR algorithm — usually the platform’s hybrid implementation; falls back to the platform default in <video> for native HLS on Safari.
3. Buffer management — segment prefetch, throttling under <video> policies (e.g. autoplay restrictions, low-power modes).
4. Codec negotiation — pick the best codec/container the device supports.
5. DRM — Widevine / FairPlay / PlayReady license acquisition, key rotation.
6. QoE telemetry — startup time, rebuffer events, quality switches, buffer underflows, fatal errors.

Buffer strategy

1Target buffer:                30 s2Minimum to start playback:    5 s3Quality-down threshold:       10 s   (drop a rung if buffer drains below this)4Quality-up threshold:         25 s   (consider stepping up if above)5Max prefetch:                 60 s   (cap so we don't waste bytes on aborts)

Playback start

1<link rel="dns-prefetch" href="//cdn.example.com">2<link rel="preconnect"   href="https://cdn.example.com">3<link rel="preload"      href="/video/abc/manifest.m3u8" as="fetch" crossorigin>

Mobile considerations

Infrastructure

Cloud-agnostic component map

AWS reference deployment

When to self-host

Failure modes and operational implications

A short tour of what breaks, how it shows up, and how to mitigate:

Practical takeaways

• Encode once, package twice. CMAF + HLS + DASH lets a single segment library serve every device.
• Chunk parallelism is what makes wall time tractable. Fan out per chunk × per rung × per codec; close GOPs at chunk boundaries to keep assembly trivial.
• Spend encode CPU where plays will land. Per-title for everything; per-shot and AV1 only on the head and on bandwidth-sensitive rungs.
• Cap the player ABR aggression with dwell time and max-drop limits. A jittering player feels worse than a slightly lower steady rung.
• Measure CDN cost in bytes offloaded, not in cache-hit rate. Shield-layer hits are misses to standard CHR but huge wins for origin egress.
• View counts are a stream-processing problem. Bloom-filter dedup, watch-time gates, and ML fraud signals are non-negotiable above a certain scale.
• Treat thumbnails as latency-critical. They land before the encoded ladder does and drive search / discovery impressions.
• Storage tiering exploits the power-law. A small fraction of titles drives most playback; let the cold tier subsidize the hot tier.
• Two-stage recommenders are the default. Cheap high-recall retrieval over the full catalog, expensive high-precision ranking on a few thousand candidates; train the ranker against expected watch time, not click probability.
• Moderation is three pipelines, not one. Hash-match the known-bad, classify the unknown, fingerprint-match the rights-protected; each writes a separate gate on the publish path.
• Live shares packaging with VOD, but nothing else. CMAF / HLS / DASH carry over; the encoder fleet, manifest server, and CDN policy do not.

Appendix

Prerequisites

• Video encoding fundamentals: codecs, containers, GOP / IDR / B-frames, bitrate vs. quality.
• Streaming protocols: HLS, DASH, CMAF.
• CDN basics: edge caching, origin shielding, cache key design.
• Distributed systems: message queues, eventual consistency, sharded counters.

Terminology

References

• tus 1.0 protocol — Resumable Uploads
• IETF draft — draft-ietf-httpbis-resumable-upload
• RFC 8216 — HTTP Live Streaming
• draft-pantos-hls-rfc8216bis — HLS 2nd Edition
• ISO/IEC 23009-1 — DASH
• ISO/IEC 23000-19 — CMAF
• Apple HLS authoring spec
• BOLA — Near-Optimal Bitrate Adaptation for Online Videos (Spiteri et al.)
• dash.js ABR settings
• Netflix per-title encoding
• Netflix per-shot (“Dynamic Optimizer”) encoding for 4K
• Netflix VMAF: the journey continues
• Netflix microservice video pipeline (Cosmos)
• YouTube Argos VCU — Reimagining video infrastructure
• AWS CloudFront Origin Shield
• Fastly — Origin offload as a CDN-efficiency metric
• Google Media CDN overview
• Meta Engineering — AV1 beats x264 and libvpx-vp9 in practical use
• Inside Facebook’s video delivery system
• Vitess — Scaling MySQL at YouTube (USENIX LISA ‘12)
• Vitess docs — Sharding
• Covington, Adams, Sargin — Deep Neural Networks for YouTube Recommendations (RecSys 2016)
• YouTube Help — How Content ID works
• Apple — Enabling Low-Latency HTTP Live Streaming
• DASH-IF — Low-Latency Live Streaming Guidelines