Screen Sharing

iOS SDK provides support for sharing the entire screen of the device in the room or you can share just your App's Screen.

Share Just Your App's Screen

You can use the hmsSDK instance to start sharing your app screen like below:

    hmsSDK.startAppScreenCapture() { error in
        // handle any errors
    }

You can stop stop sharing your screen like below:

    hmsSDK.stopAppScreenCapture() { error in
        // handle any errors
    }

How to check if user is sharing their screen

There are 2 approaches that we suggest for checking if user is sharing screen:

1. Check if the screen is added as a local track for the user like shown below

if localPeer?.auxiliaryTracks?.first(where: { $0.source == HMSCommonTrackSource.screen }) != nil {
    // Screen is getting shared
}

2. Maintain a state variable when screen track is added or removed like shown below

    var isSharingScreen = false
    func on(track: HMSTrack, update: HMSTrackUpdate, for peer: HMSPeer) {

        if update == .trackAdded {

            if peer.isLocal && track.source == HMSCommonTrackSource.screen {
                isSharingScreen = true
            }
        }

        if update == .trackRemoved {

            if peer.isLocal && track.source == HMSCommonTrackSource.screen {
                isSharingScreen = false
            }
        }
    }

Share The Entire Device Screen

To share entire device screen the SDK uses Replaykit framework from Apple. To understand the documentation better consider familiarising yourself with live screen broadcast in iOS: https://developer.apple.com/videos/play/wwdc2018/601

Please note that for a peer to share their screen, their role must have screenshare enabled in the dashboard. Also, select the appropriate resolution that you desire the screenshare track should be of.

How to start screenshare from the app

You create an iOS broadcast upload extension. It uses Apple's ReplayKit framework to record the device screen and delivers frame samples to your broadcast extension. You can share not only your own app but also the entire device screen including other apps on the device.

HMSBroadcastExtensionSDK is made specifically to be used in the iOS broadcast upload extension target. After importing the SDK, you use HMSScreenRenderer class to process the sample buffer coming from RPBroadcastSampleHandler. These samples are then presented in your main meeting app as a screen recording track.

Video walk-through of implementation

Example project shown in the walkthrough is available here.

How to import

Using Swift Package Manager

You can use Swift Package Manager (use https://github.com/100mslive/100ms-ios-broadcast-sdk.git as the package source)

Add HMSBroadcastExtensionSDK to your main app target from Swift Package Manager. Then add HMSBroadcastExtensionSDK to your iOS broadcast upload extension target.

Using CocoaPods

Get the HMSBroadcastExtensionSDK via CocoaPods. Add the pod 'HMSBroadcastExtensionSDK' to your broadcast upload extension target in Podfile as follows:

# Podfile

platform :ios, '12.0'

target 'MainApp' do
  pod 'HMSSDK'
end

target 'HMSScreenShare' do
  pod 'HMSBroadcastExtensionSDK'
end

How to use HMSBroadcastExtensionSDK to share iOS screen with other participants in the room?

iOS broadcast extension provides you with CMSampleBuffers that you can process using HMSBroadcastExtensionSDK. This enabled you to share the screen with other participants in the room in your main app that uses HMSSDK.

How to create Replaykit Broadcast extension?

1. Create a Broadcast Upload Extension target that is embedded in your app project target.
2. You create a subclass of RPBroadcastSampleHandler and override the processSampleBuffer function. This gives you access to the CMSampleBuffer provided by the Replaykit framework.

How to configure your main app and extension to enable screen sharing

1. Configure an App Group for your main app. Choose the same App Group for your broadcast extension target.
2. In your main app set the app group string to the hmssdk instance like below:

    hmsSDK = HMSSDK.build { sdk in
            sdk.appGroup = "group.live.100ms.videoapp"
            ...
        }

3. In your broadcast extension import HMSBroadcastExtensionSDK

    import HMSBroadcastExtensionSDK

4. Create an instance of HMSScreenRenderer passing the app group string like below:

  class SampleHandler: RPBroadcastSampleHandler {

  let screenRenderer = HMSScreenRenderer(appGroup: "group.live.100ms.videoapp")

  ...

HMSScreenRenderer uses this app group string to talk to your main app.

How to share the screen in meeting

In your Broadcast Extension, you create a subclass of RPBroadcastSampleHandler class. This class handles screen frame samples produced by Replaykit. Replaykit delivers these frames as CMSampleBuffer to processSampleBuffer function.

In your processSampleBuffer function call process function on your screenRenderer instance passing the CMSampleBuffer like below:

  override func processSampleBuffer(_ sampleBuffer: CMSampleBuffer, with sampleBufferType: RPSampleBufferType) {
      switch sampleBufferType {
      case RPSampleBufferType.video:
          // Handle video sample buffer
          screenRenderer.process(sampleBuffer)
      ...
  }

How to invalidate the connection between main app and extension when the user stops screen recording

When the user stops screen sharing broadcastFinished function is called on your RPBroadcastSampleHandler class. At that point, call invalidate on your screenRenderer instance to stop screen sharing.

    override func broadcastFinished() {
        // User has requested to finish the broadcast.
        self.screenRenderer.invalidate()
    }

Stop screen sharing when the user leaves the room

When a user leaves the meeting room from the main app, you will receive an error HMSScreenRendererErrorCode.noActiveMeeting when calling process on screenRenderer instance. You can handle this error to stop the broadcast extension like below:

    override func processSampleBuffer(_ sampleBuffer: CMSampleBuffer, with sampleBufferType: RPSampleBufferType) {
        switch sampleBufferType {
        case RPSampleBufferType.video:
            // Handle video sample buffer
            if let error = screenRenderer.process(sampleBuffer) {
                if error.code == .noActiveMeeting {
                    finishBroadcastWithError(NSError(domain: "ScreenShare", code: error.code.rawValue, userInfo: [NSLocalizedFailureReasonErrorKey : "You are not in a meeting."]))
                }
            }
        ...
    }

How to check if user is sharing their screen

There are 3 approaches that we suggest for checking if user is sharing screen:

1. Check if the screen is added as a local track for the user like shown below

if localPeer?.auxiliaryTracks?.first(where: { $0.source == HMSCommonTrackSource.screen }) != nil {
    // Screen is getting shared
}

2. Maintain a state variable when screen track is added or removed like shown below

    var isSharingScreen = false
    func on(track: HMSTrack, update: HMSTrackUpdate, for peer: HMSPeer) {

        if update == .trackAdded {

            if peer.isLocal && track.source == HMSCommonTrackSource.screen {
                isSharingScreen = true
            }
        }

        if update == .trackRemoved {

            if peer.isLocal && track.source == HMSCommonTrackSource.screen {
                isSharingScreen = false
            }
        }
    }

3. Use the app group user-defaults to store the state of screen sharing from broadcast extension (using broadcastStarted and broadcastFinished functions)

Debugging issues in your screen-share implementation

1. If you use CocoaPods with M1 mac and compile for simulator then you might run into an error like this: "error framework not found Pods_hmsscreenshare". This issue is specific to CocoaPods method of integrating the 100ms SDK. If you use swift package manager, you wouldn't run into this error. As a workaround please use a snippet similar to at the end of this example Podfile to exclude arm64 arch for simulator from your pod screen share target: Workaround for M1 simulator issue https://github.com/100mslive/100ms-ios-sdk/blob/main/Example/Podfile Note: Pods-HMSScreenShare in the snippet is your CocoaPods framework target for broadcast extension (it's generally Pods-[Your Broadcast Extension target name])

2. If you are getting a crash while running with screen share feature, please make sure that the app group identifier is not misspelled in both the main app and the broadcast extension. Also make sure your entitlement file has the correct app group name.

Frequently asked questions

1. How can I show only my screen share extension to the user when they want to share the screen using RPSystemBroadcastPickerView?

You can configure your RPSystemBroadcastPickerView instance to display only your extension by assigning a preferredExtension value to it:

    let systemBroadcastPicker = RPSystemBroadcastPickerView(frame: CGRect(x: 0, y:0, width: 44, height: 44))
    systemBroadcastPicker.preferredExtension = "live.100ms.videoapp.screenshare"

Replace "live.100ms.videoapp.screenshare" with the bundle ID of your own broadcast extension. This ensures that only your screen share extension is displayed to the user when selecting the option to share their screen.

Example implementation

👀 To see an Example iOS broadcast upload extension implementation for screen sharing using 100ms Broadcast extension SDK, checkout our Example project.

📲 Download the 100ms fully featured Sample iOS app here: https://testflight.apple.com/join/dhUSE7N8