Setting up High-Availability Ingest and Clients

The Phenix platform is built with high availability (HA) in mind for all system components from encoding and ingest through to client connections to the platform. The autonomous operation architecture ensures that faults are detected and corrected without any administrative or end-user impacts.

HA Ingest is the ability to send data into multiple redundant data centers over redundant networking connections effectively addresses risks of origin and first mile failures. Thus providing a deployment with complete end-to-end redundancy.

High availability (HA) ingest leverages the independent and fault tolerant architecture of the Phenix platform to provide streams that can withstand the failure of individual data centers or components therein that would interfere with the health of a stream.

High-Availability logic is only available for content published to Channels, not for content published to Rooms.

Possible single points of failure include:

1. Capture hardware that is running the publisher software

2. The ISP that the publisher is using to access the internet

3. The Phenix data center to which publisher is publishing

4. The stream within the channel

To be fully redundant, a publisher would have separate capture hardware, encoder and/or RTMP hardware, ISPs, data centers, and streams within channels (screenNames).

Setting up HA does not guarantee that the streams will end up on separate egress infrastructure.

This page shows how to set up ingest for high availability, as well as how to set up clients to take advantage of high-availability streams. There are two levels of redundancy discussed in this document: physically separate data centers and redundant streams within a channel; redundant capture hardware, encoder and/or RTMP hardware, and ISPs are out of the scope of this document.

Using alternate ingests is less efficient in terms of internet routing. This is a necessary compromise to ingest into a different data center.

Any screenName ending with the string "primary" is considered primary, and any screenName without the string "primary" is considered an alternate stream having the same content as the primary stream. The string is not case-sensitive.

Should a failure event occur when using a channel, clients that have been set up to take advantage of high availability will automatically fail over from the primary to the alternate stream.

When there is more than one primary stream, the data centers spread the load across primary streams. If no primary streams are available, the load is spread across all alternate streams.

When both stream paths are equal, we recommend using using "primary" in the screen name for both streams. If one path is preferred to the other (e.g., one path uses an ISP that has higher quality of service than the other), the screen name of the stream corresponding to the preferred path should end with "primary".

When using Phenix encoders, set up data centers and screen names as described in this section.

To use multiple data centers for Phenix encoding, use the following hosts for ingest:

• pcast-primary.phenixrts.com

• pcast-secondary.phenixrts.com

To create screen names, use the CLI option: --screen-name=screenName

For example, to create a primary stream, use --screen-name=myPrimary.

When using RTMP ingest, set up data centers and screen names as described in this section.

Set up the ingest such that one stream is ingested into two or more independent, physically separate data centers via RTMP. This is done by specifying the different data center destinations in the stream URI. For example:

• rtmp://ingest.phenixrts.com:80/ingest/streamkey;capabilities=hd, multi-bitrate

• rtmp://ingest-secondary.phenixrts.com:80/ingest/streamkey;capabilities=hd, multi-bitrate

When one stream path is preferred to another's, the preferred stream's screen name should end with the string "primary".

• rtmp://ingest.phenixrts.com:80/ingest/streamkey;screenName=myFeed-Primary;capabilities=hd

• rtmp://ingest-secondary.phenixrts.com:80/ingest/streamkey;screenName=myFeed;capabilities=hd

It's possible to have multiple primaries and alternates within a data center by providing unique screen names, for example:

• rtmp://ingest.phenixrts.com:80/ingest/streamkey;screenName=primary;capabilities=hd

• rtmp://ingest.phenixrts.com:80/ingest/streamkey;screenName=secondary;capabilities=hd

• You can use any capabilities that you normally would use.

• For informational purposes only, you can verify the latency from your site with ping ingest.phenixrts.com and compare it against ping ingest-secondary.phenixrts.com

The client side integration of high-availability logic is simple. In order to leverage the redundant ingest streams, client SDKs need to be configured to treat them as equivalent with the high-availability subscription option.

Clients connect to whichever data center is closest. If the closest data center is unavailable, the client will connect to the next closest data center.

By default, clients will use the most recent stream which will lead to frequent switching between the alternate ingests as they update their tokens periodically.

When building the join-channel options, ensure that the High Availability stream selection strategy is set as shown in this section for the client platform.

let joinChannelOptions =
  PhenixChannelExpressFactory.createJoinChannelOptionsBuilder()
    .withStreamSelectionStrategy(.highAvailability)
    ...
    .buildJoinChannelOptions()
 
channelExpress.joinChannel(
  joinChannelOptions,
  {(requestStatus: PhenixRequestStatus, roomService: PhenixRoomService?) in ...

Further details can be found here.

final JoinChannelOptions joinChannelOptions =
  ChannelExpressFactory.createJoinChannelOptionsBuilder()
    .withStreamSelectionStrategy(StreamSelectionStrategy.HIGH_AVAILABILITY)
    ...
    .buildJoinChannelOptions();

channelExpress.joinChannel(
  joinChannelOptions,
  (RequestStatus status, RoomService roomService) -> { ...

Further details can be found here.

WebSDKv2 does not require any special configuration for high-availability.

When using WebSDKv1, add the streamSelectionStrategy to the joinChannel options:

var options = {
  ...
  streamSelectionStrategy: ‘High-Availability’,
  ...
};
channelExpress.joinChannel(options, ...

Further details can be found here.