Documentation

Today's service meshes generally only support TCP-based applications (and in fact are optimised for HTTP-based web applications). Any support for UDP that is added to service meshes is likely to be focussed on enabling QUIC (since HTTP/3 runs over QUIC).

Real-time applications generally run over UDP rather than TCP. Media Streaming applications typically rely on RTP (the Real-time Transport Protocol) - which runs on top of UDP, and hence RTP will be the initial focus of Media Streaming Mesh. RTP enables measurement of loss and jitter as it carries sequence numbers and timestamps in the packet header and we will monitor these in Media Streaming Mesh.

One challenge with RTP is that it often runs on ephemeral UDP ports which are assigned by a TCP-based control channel such as SIP or RTSP. This prevents kube-proxy from being able to successfully implement the ClusterIP NAT for these protocols. Proxying these TCP-based control plane protocols will enable us both to implement URL/URI-based routing and to create the appropriate RTP proxy rules for the data plane traffic.

Another challenge is that many RTP-based applications rely on IP multicast. Kubernetes networking doesn't generally support IP multicast. Media Streaming Mesh's per-node RTP proxy will enable us to distribute RTP streams from one sender to multiple receivers over regular IP unicast, and will also enable us to convert from multicast to unicast and vice-versa.

Our current implementation consists of the following components:

• per-cluster RTSP control plane proxy written in Golang and deployed as a Kubernetes service

• per-node RTP data plane proxy written in Golang and deployed as a Kubernetes DaemonSet

• per-pod RTSP stub written in asynchronous Rust and deployed as a Kubernetes pod sidecar

• mutating webhook which injects the RTSP stub into pods, written in Golang and implemented as a Kubernetes service

• chained micro-CNI which adds iptables rules to direct traffic into the RTSP stub, written in Golang and implemented as a Kubernetes DaemonSet

The MSM RTSP Stub is largely reponsible for sending control plane messages to the per-cluster control plane using gRPC. Because the stub shares fate with the media app in the same pod we avoid any issues around needing to mirror TCP session state to achieve control plane resilience.

The MSM RTSP Stub also provides an interworking function between RTSP interleaved mode (where RTP and RTCP payloads are sent over the RTSP TCP control channel) and the standard RTP/RTCP over UDP mode.

For inter and extra-cluster traffic the per-node RTP proxies act as data-plane gateways, and MSM RTSP stubs co-located with the RTP proxies act as control-plane gateways.

Longer term our expectation is to implement:

• SPIFFE/SPIRE for pod to pod authentication​

• A new per-node RTP data plane written in asynchronous Rust and supporting WASM plugins​

• A refactored Golang control plane consisting of: ​
  1. pluggable control plane pod supporting multiple protocols
  2. network controller pod which maps logical steams onto the physical network

• An enhanced MSM RTSP stub that also supports RTP multicast to RTSP unicast interworking

• A stripped-down MSM stub that only supports control plane traffic

With that baseline we hope to empower the community to implement multiple control plane protocols (such as SIP, RIST, SMTPE 2110, WebRTC etc.) and to implement various data-plane plugins enabling features such as FEC (Forward Error Correction), NAK-based error correction, congestion control etc.

In order to keep footprint light one key will be to deploy only the required control plane and data plane components for the service being implemented.

We're looking for potential users of Media Streaming Mesh to help us define the solution, and for developers to help us create it!

Please do join our Slack Channel.