Blog/ Android SurfaceViewRenderer Explained
October 21, 20225 min read
The mighty SurfaceViewRenderer! It's the very end of a long WebRTC chain that gets video from other people to your screen.
Learn how to build Twitter Spaces with 100ms iOS SDK
It can also be a bit difficult to use if it's your first time around. Especially in a dynamic android app with lots of views flicking in and out.
Are you a Flutter developer? check out our comprehensive guide on Flutter WebRTC
Let's take a tour through the many ways it can go wrong and what you should know to keep your video conference calls running smoothly.
Here's the lifecycle of a SurfaceViewRenderer:
Once released the SurfaceViewRenderer can be inited again and the cycle renews. Now let's look at what happens when any one of these is missed or not called with the right parameters.
This one is actually mentioned in the source code of SurfaceViewRenderer, when you call SurfaceViewRenderer.init(context) it must only be with a shared EGL context.
What does that mean? That you call org.webrtc.EglBase.create() once in something like a singleton and then always initialize all the SurfaceViewRenderers with that saved context's eglBaseContext.
If you call create() for each SurfaceView, you'd end up with no errors and no video playing at all. It's easy to get stuck here and wonder why SurfaceViewRenderer is not showing any video when seemingly nothing went wrong.
If you don't release after you've removed a video sink, you're going to get the same result as if you initialized twice.
java.lang.IllegalStateException: videoSurfaceViewAlready initialized at org.webrtc.EglRenderer.init(EglRenderer.java:212)
So you've just got to release at the right time.
The view just never shows up ¯\_(ツ)_/¯
Behind the scenes a render thread was never created and the loop to draw wasn't kicked off so, of course, nothing would show up.
Calling addSink on the same SurfaceViewRenderer is a pretty fun bug. You won't realize anything's wrong if you just added all the videos for people twice until the order of the videos is changed.
Then you'll see flicker as two different video sources try to play onto the same surface. Both video sources will take race condition turns rendering.
Oh, and you'll keep downloading the video from the source peer if you only called removeSink once when disposing of the view.
Here's an image of what the two video streams look like separately and what happens when addSink for both was called on the same SurfaceViewRenderer.
Without initializing them, you can create a heck of a lot. But there's a limit to how many different initialized SurfaceViewRenderers you can prepare at a time.
The limit varies from device to device, when I tried on one modestly powered Nokia 3.4 phone it got up to 30 contexts initialized until it failed with:
E/AndroidRuntime: FATAL EXCEPTION: main process: com.example.myvideocallapp, PID: 532java.lang.RuntimeException: java.lang.RuntimeException: Failed to create EGL context: 0x3003at org.webrtc.EglBase14Impl.createEglContext(EglBase14Impl.java:282)at org.webrtc.EglBase14Impl.(EglBase14Impl.java:78)at org.webrtc.EglBase.createEgl14(EglBase.java:215)at org.webrtc.EglBase.create(EglBase.java:158)
On others, like an Oppo F19 it gets beyond 150 and keep going. Considering that that's the number of videos you're displaying simultaneously, you may not need to get anywhere near that much.
Now let's look at something even tricker than managing a single or a handful of a static number of SurfaceViewRenderers.
As a refresher, the RecyclerView is an essential View on Android that lets you avoid creating expensive view objects when scrolling through a large list.
What's even more expensive than a typical view object? The SurfaceViewRenderer of course.
Let's think about what a RecyclerView does in android.
How does that compare to the lifecycle of a SurfaceView?
This poses a natural problem because there are multiple things that could cause the view to be changed.
To further complicate matters, someone's video may not have been on at first, so you may just show the person's initials, but later they do turn it on, so now the SurfaceViewRender has to be initialized.
If you're to Android, rebinding views is straightforward, you'd just do that in the AdapterView's bind method but a slightly unconventional approach is required here because anyone's video can go on or off at any time.
So the big question becomes, when do you initialize the video?
You'd quickly see a 'bind' method is insufficient.
What we need here are some lesser-known adapter methods:
Called when a new view is added to the window or a cached one is restored.
Called when a view is removed, we don't know if it's going into cache or out permanently.
Take a look at the PeerViewHolder and the PeerAdapter to see how all the disparate cases are handled. This is the simplest example of how you could manage the tricky SurfaceViewRenderer in an Android RecyclerView.
This is part of the "hello world" demonstration of how the 100ms SDK can be used.
To look into more advanced use cases, take a look at our fully featured sample app.
See all articles
March 17, 202210 min read
A Comprehensive Guide to Flutter-WebRTC
This article will demonstrate how to use WebRTC and imp...
April 22, 202211 min read
Agora vs Twilio vs Jitsi vs Zoom vs 100ms: An In-depth Comparison
A quick, unbiased comparison between 5 popular audio-vi...
May 24, 202218 min read
Virtual Classroom - The Definitive Guide for 2022
Thinking of livestreaming classes online? Or are you th...