Historically, HLS has favored stream reliability over latency, however back in 2019 during WWDC, Apple announced their own extension to the protocol: LLHLS, which can bring stream latency down to 1-2 seconds.
Regular HLS has an average global latency of 20-30 seconds, which is extremely poor by today’s standards and definitely not realtime, which is what we strive to achieve in our entire product line.
Low Latency HLS can bring this latency down to as low as 1-2 seconds. Apple achieved this by focussing on 4 major components:
Reducing Publishing Latency
In the regular HLS protocol, encoded stream segments are meant to be about 6 seconds long before getting added to the HLS playlist. This already adds a base latency of 6 seconds - plus, clients could request the HLS playlist ~5.9 seconds after the previous segment publish (worst case), which would mean they would be delayed from the live broadcast by about 12 seconds.
LLHLS solves this by publishing small parts (~250ms each) of the main segment to the playlist before the main segment itself is ready.
HLS works through HTTP polling - clients send a HTTP request to fetch the HLS
playlist, which contains information about the latest published segments, from
the server. In regular HLS, this is one URL: playlist.m3u8 (for example) -
because of this, extra latency is added due to edge CDNs caching the playlist
contents for the
TTL (time to live). Also, clients could request segments at
the “worst-case” segment window: just before a new HLS segment is generated.
LLHLS allows clients to discover segments more quickly in 2 ways:
Pre-empted Playlist Requests
Clients can request playlists from the server before the actual segment is generated. The server will then hold on to the request and then, when the playlist updates, it responds back with the immediate latest segment. The client will now be aware of the latest segments in half the round-trip time.
Unique Playlist URLs
To prevent CDNs from caching old playlist entities, LLHLS assigns different playlist URLs per update, which prevents CDNs from caching stale entities and allows for more efficient content delivery.
Eliminating Segment Round Trip
In LLHLS, when clients make a pre-empted playlist request, they also send a PUSH tag - this tells the server to, when returning the HLS playlist, push the latest segment to the client at the same time, so the client doesn’t have to make a second round-trip.
Delta Playlist Updates
In regular HLS, playlists are always returned in full - it returns segments that the client is already aware of. This makes sense for the client’s first playlist request, but not for subsequent requests.
In LLHLS, the full playlist is only returned for the client’s first request. For subsequent requests, only the delta between the latest playlist and their previous request is sent back - this usually fits into a single packet (MTU) of data, which increases network forwarding efficiency.
Hop’s LLHLS Implementation
There’s no doubt that LLHLS is the gold standard of today’s livestreaming protocols; for Pipe, we implement the raw protocol and serve it across our edge network.
However, thanks to our Leap protocol, which is a WebSocket that all frontend SDKs use to signal with Pipe (and Channels!), we’re able to gain greater insight across all realtime viewers from a latency and network environment standpoint. Leap keeps a running heartbeat with clients, which we use to calculate round-trip-time (RTT) between our edge and the client.
Please note that the following feature is experimental and heavily subject to change
While LLHLS is great, users on slow networks may receive older playlists due to network latency, and therefore be de-synced with other viewers. This isn’t acceptable for certain applications, such as watch parties, which expect users to react to the same content.
By default, Pipe will optimize your LLHLS stream for “low latency”, and won’t
take sync into account. However, you can use the
llhls_config object when
creating a Room to fine-tune your playout options.
To achieve better playout sync between users, you can use the
parameter in your
llhls_config. This will now optimize your stream for
sync rather than low-latency, and while still miles better than legacy
HLS, adds a base latency of 5 seconds to your delay from live. This latency can
increase from 5 seconds up to your “wcl_delay” parameter, regardless of your
users’ internet conditions. Users with network conditions unable to meet the
“wcl_delay” parameter will be desynced from other users.
wcl_delay parameter is basically a way to tell Pipe how much you’re
willing to delay your live broadcast in favor of providing a more synced
playback for users with bad network conditions.
wcl_delay stands for “Worst
Case Latency Delay”.
On frontend clients, Leap sends periodic telemetry data about the users’ current playback environment, including currently synced HLS playlist. If we detect that more than 5% of your room is behind the current playlist, we do 2 things:
- Start sending older playlist data (adding latency up to the specified wcl_delay), and
- Adjust the playback rate of other clients to eventually let them sync with the new active delay
LLHLS Configuration Object
|wcl_delay||?number||specify a “worst case” delay; this will optimize delivery for playout sync rather than latency; number must be between 5 and 20 seconds|
|artificial_delay||?number||artificially add latency from your live broadcast; this will take priority over wcl_delay, so only set one|
|max_playout_bitrate_preset||?string||limit your playout bitrate|
Playout Birate Presets (enum)
|preset||video bitrate (kbps)||audio bitrate (kbps)||resolution|