srs-unity

Video Streaming and WebRTC Samples for Unity.

Unity supports WebRTC, see com.unity.webrtc or github. However, the demos only work in P2P mode, not with remote SFU or SRS.

To work with SFU or WebRTC server, the best practice is to use WHIP for Unity to publish to SFU, such as SRS. Actually, you're also able to play stream by WHIP.

The most common use scenaio for publishing stream, is to covert video game as live streaming. However you can use OBS to capture the window and audio, but it enable you to capture from Unity inside, cool! It works like this:

(Scenario 1)
Unity Game ---WebRTC----> SRS --+-----RTMP--> YouTube/FFmpeg
                                +---WebRTC--> H5/Unity

And the playing stream can be used for streaming consuming, it's a new feature enabled by Unity WebRTC. It works like this:

                                        (Scenario 2)
OBS/FFmpeg ----RTMP----+--> SRS --WebRTC--> Unity Game
H5/Unity -----WebRTC-----+

It also allows multiple Unity Games to communicate by WebRTC. And there should be a set of new use scenarios for Unity+WebRTC+SRS, that you can finger out and please let us know

Environments

We have tested on:

• Unity editor 2020.3.48f1 LTS, other latest LTS versions should work well also.
• WebRTC 2.4+, because CaptureStream API changed, see #2 for detail.

The supported versions:

The latest version should work well also, please file an issue if not.

Setting up project

First, please setup you Unity Project. If you're stuck, please get help from Discord.

Step 1: Download and setup Unity Hub.

1. Open Unity Hub.
2. Click Installs > Install Editor, please select one release to install.
3. Check modules for Unity Editor, (Windows: make sure Visual Studio installed).

Step 2: Create a Unity project.

1. Open Unity Hub.
2. Click Projects > New Project.
3. Select 3D Core or 3D URP template.
4. Set the Project Name to My project.
5. Click Create project, and an Unity Editor will be opened.

Step 3: Install dependency package com.unity.webrtc.

1. Click Window > Package Manager.
2. Click + > Add package from git URL.
3. Input com.unity.webrtc then click Add.

Step 3.1 Case of Unity 2020.3 or 2021.3:

1. Click Window > Package Manager.
2. Click + > Advanced button and enable Show preview packages
3. Search webrtc and install the package.

Step 3.2 Case of Unity 2019.4:

1. Click Window > Package Manager.
2. Click + > Add package from git URL.
3. Input com.unity.webrtc then click Add.

Step 4: Install srs-unity package.

1. Download package SRS.WebRTC.Samples.unitypackage.
2. Click Asserts > Import Package > Custom Package, select the file SRS.WebRTC.Samples.unitypackage, then click Import.
3. From Project panel, open Asserts > io.ossrs > Samples, where you got all samples there.

Step 5: Start SRS WebRTC media server:

CANDIDATE="192.168.1.10"
docker run --rm -it -p 1935:1935 -p 1985:1985 -p 8080:8080 \
    --env CANDIDATE=$CANDIDATE -p 8000:8000/udp \
    ossrs/srs:5 ./objs/srs -c conf/docker.conf

Note: Make sure your SRS is v4.0.264+ or v5.0.62+. Please read this guide to setup SRS.

Note: Please remember to replace the CANDIDATE to your server IP, please read link for details.

Note: For online service, you might need authentication and other features, please read How to Setup a Video Streaming Service by 1-Click or How to Setup a Video Streaming Service with aaPanel to build a service by SRS.

Bellow is detail guide for different use scenarios.

Usage: Publisher

To publish your WebCamera and Microphone using WebRTC. If you're stuck, please get help from Discord.

Please follow Setting up project, then work with Publisher sample.

1. From Project panel, open Asserts > io.ossrs > Samples > Publisher, then open the Scene.
2. Click Edit > Play to play Unity scene, which publish WebRTC stream to SRS.
3. Open Main camera object in the editor and under the SRS publisher script, change the ip address and stream key to your e.g. http://localhost:1985/rtc/v1/whip/?app=live&stream=livestream
4. Play the WebRTC stream by H5.

Note: Note that the Publisher scene require the WebCamera and Microphone permission, you can try other sample if no device.

The stream flows like this:

Unity ---WebRTC---> SRS --WebRTC--> H5/Chrome
(WebCamera and Microphone)

Note: You could use other WebRTC media server and client to replace SRS and Chrome.

https://user-images.githubusercontent.com/2777660/189331198-e045aeec-ad75-447b-8143-3b6d779e8dde.mp4

Note: The latency is extremely low, which allows you to communicate with another Unity App. For example, you can use this mode to build a metting application by Unity.

Usage: Streamer

To publish your game camera and voice using WebRTC. If you're stuck, please get help from Discord.

Please follow Setting up project, then work with Streamer sample.

1. From Project panel, open Asserts > io.ossrs > Samples > Streamer, then open the Scene.
2. Click Edit > Play to play Unity scene, which publish WebRTC stream to SRS.
3. Open Main camera object in the editor and under the SRS streamer script, change the ip address and stream key to your e.g. http://localhost:1985/rtc/v1/whip/?app=live&stream=livestream
4. Play the WebRTC stream by H5.

Note: Note that the Streamer scene grab an extra camera in game and voice of Main Camera, and we add an example audio clip from hls.

The stream flows like this:

Unity ---WebRTC---> SRS --WebRTC--> H5/Chrome
(Game camera and voice)

Note: You could use other WebRTC media server and client to replace SRS and Chrome.

https://user-images.githubusercontent.com/2777660/189329527-f7569003-f047-449e-8f48-ec59ddee60e4.mp4

Note: A rotating cube is in the video stream, which is also the demostration of WebRTC.

https://user-images.githubusercontent.com/2777660/189353556-047025b2-2019-4c5a-a813-7a4c888840eb.mp4

Note: Please note that we also capture the audio of game, so when we mute the H5 player there is no audio.

Usage: Player

To play stream using WebRTC. If you're stuck, please get help from Discord.

Note: The stream might be published by another WebRTC client, or live stream like OBS or FFmpeg

Please follow Setting up project, then work with Player sample.

1. Publish the WebRTC stream by H5.
2. From Project panel, open Asserts > io.ossrs > Samples > Player, then open the Scene.
3. Open Main camera object in the editor and under the SRS player script, change the ip address and stream key to your e.g. http://localhost:1985/rtc/v1/whip-play/?app=live&stream=livestream
4. Click Edit > Play to play Unity scene, which publish WebRTC stream to SRS.

Note: You're also able to publish a live stream and play it in Unity using WebRTC, see WebRTC for Live Streaming.

The stream flows like this:

H5/Chrome ---WebRTC---> SRS --WebRTC--> Unity

Note: You could use other WebRTC media server and client to replace SRS and Chrome.

https://user-images.githubusercontent.com/2777660/189331098-bd7163cf-d6b9-480c-a204-ad8b721491b6.mp4

Note: By converting live streaming to WebRTC, you're able to play normal stream from OBS or exists stream to Unity. For example, you can view a live sport or music by Unity.