100ms Logo

100ms

Docs

Search docs
/

Screen Share

Flutter SDK provides support for sharing the entire screen of the device to the room.

Please note that for a peer to share their screen, their role must have Screenshare enabled in the dashboard. Also select the appropriate resolution for the Screen share quality. 1080p is recommended for better text readability.

Prerequisites

Let's first do some setup required for both the platforms

Android Setup

You also need to pass the intent from android native side to HMS SDK in the following way :

In your app's MainActivity add -

import live.hms.hmssdk_flutter.HmssdkFlutterPlugin import android.app.Activity import android.content.Intent import live.hms.hmssdk_flutter.Constants override fun onActivityResult(requestCode: Int, resultCode: Int, data: Intent?) { super.onActivityResult(requestCode, resultCode, data) if (requestCode == Constants.SCREEN_SHARE_INTENT_REQUEST_CODE && resultCode == Activity.RESULT_OK){ HmssdkFlutterPlugin.hmssdkFlutterPlugin?.requestScreenShare(data) } }

DONOT forget to add the permission for foreground service in AndroidManifest.xml

<uses-permission android:name="android.permission.FOREGROUND_SERVICE" />

iOS Setup

You need to create an iOS broadcast upload extension. It uses Apple's ReplayKit framework to record the device screen and delivers frame samples to your broadcast extension. You can share not only your own app but also the entire device sceeen including other apps on the device.

Step 1 - Open project

Open your iOS Xcode project, such as ios/Runner.xcworkspace for full-Flutter apps.

Step 2 - Add Broadcast Upload Extension

Click on your project in the Project Navigator to show the project settings. Press + at the bottom of the target list to add a new target.

AddExtension

Select the Broadcast Upload Extension type for your new target.

SelectExtension

Enter your new target detail in the dialog. Uncheck Include UI Extension option.

DetailExtension

In the following dialog, activate the new scheme for the new target.

ActivateExtension

Step 3 - Add App Group

Click + icon in Signing & Capabilities section. Select App Group from the list of Capabilities.

AddAppgroup

New section should be added under Signing & Capabilities named App Groups. Click + icon under that.

Appgroup

Enter App group name (create unique app group name ex: group.your.domain.name)

AppgroupDetail

Step 4 - Edit Podfile

In ios folder of your flutter project and open Podfile. Paste the following code and replace the extension name you just created:

target 'Your Extension Name here' do use_modular_headers! pod 'HMSBroadcastExtensionSDK' end

Podfile

In terminal change directory to ios and run pod install command.

Step 5 - Edit SampleHandler

Expand Runner > ExtensionName and open SampleHandler file.

SampleHandler

Replace the code with the code below and pass app group name to the respected field:

import ReplayKit import HMSBroadcastExtensionSDK class SampleHandler: RPBroadcastSampleHandler { let screenRenderer = HMSScreenRenderer(appGroup: "Enter App Group Name") override func broadcastStarted(withSetupInfo setupInfo: [String : NSObject]?) { // User has requested to start the broadcast. Setup info from the UI extension can be supplied but optional. } override func broadcastPaused() { // User has requested to pause the broadcast. Samples will stop being delivered. } override func broadcastResumed() { // User has requested to resume the broadcast. Samples delivery will resume. } override func broadcastFinished() { // User has requested to finish the broadcast. screenRenderer.invalidate() } override func processSampleBuffer(_ sampleBuffer: CMSampleBuffer, with sampleBufferType: RPSampleBufferType) { switch sampleBufferType { case RPSampleBufferType.video: // Handle video sample buffer if let error = screenRenderer.process(sampleBuffer) { if error.code == .noActiveMeeting { finishBroadcastWithError(NSError(domain: "ScreenShare", code: error.code.rawValue, userInfo: [NSLocalizedFailureReasonErrorKey : "You are not in a meeting."])) } } break case RPSampleBufferType.audioApp: // Handle audio sample buffer for app audio break case RPSampleBufferType.audioMic: // Handle audio sample buffer for mic audio break @unknown default: // Handle other sample buffer types fatalError("Unknown type of sample buffer") } } }

Key Notes

To start Screenshare from iOS devices you need to pass App Group and Preferred Extension name to HMSSDK constructor as shown below.

You can find appGroup and preferredExtension name in Xcode under Signing and Capabilities section under Target > yourExtensionName.

Once you have the correct App Group & Preferred Extension values created in Xcode & linked to your Apple Developer Account, you can now use them to start Screenshare from iOS devices (iPhone / iPad).

// Pass the correct App Group & Preferred Extension parameters in HMSIOSScreenshareConfig class. HMSIOSScreenshareConfig iOSScreenshareConfig = HMSIOSScreenshareConfig( appGroup: "appGroup", // App Group value linked to your Apple Developer Account preferredExtension: "preferredExtension" // Name of the Broadcast Upload Extension Target created in Xcode ); HMSSDK hmsSDK = HMSSDK(iOSScreenshareConfig: iOSScreenshareConfig); await hmsSDK.build(); // ensure to await while invoking the `build` method

parameter

After completing the setup let's see how we can Start the Screenshare from iOS.

How to Start/Stop Screenshare from the app

To start screen share, app needs to call the startScreenshare method of HMSSDK and similar is it's counterpart stopScreenShare to stop screen share.

Following is the snippet on how to use this:

class Meeting implements HMSUpdateListener, HMSActionResultListener{ void startScreenShare() { ///[hmsActionResultListener]: an instance of a class that implements HMSActionResultListener //Here this is an instance of a class that implements HMSActionResultListener, that is, Meeting hmsSDK.startScreenShare(hmsActionResultListener: this); } void stopScreenShare() { ///[hmsActionResultListener]: an instance of a class that implements HMSActionResultListener //Here this is an instance of a class that implements HMSActionResultListener, that is, Meeting hmsSDK.stopScreenShare(hmsActionResultListener: this); } void onSuccess( {HMSActionResultListenerMethod methodType = HMSActionResultListenerMethod.unknown, Map<String, dynamic>? arguments}) { switch (methodType) { ... case HMSActionResultListenerMethod.startScreenShare: //Screen share started successfully break; case HMSActionResultListenerMethod.stopScreenShare: //Screen share stopped successfully break; } } void onException( {HMSActionResultListenerMethod methodType = HMSActionResultListenerMethod.unknown, Map<String, dynamic>? arguments, required HMSException hmsException}) { switch (methodType) { ... case HMSActionResultListenerMethod.startScreenShare: // Check the HMSException object for details about the error break; case HMSActionResultListenerMethod.stopScreenShare: // Check the HMSException object for details about the error break; } } }

How to get Screen Share Status

Application needs to call the isScreenShareActive method of HMSSDK.

This method returns a Boolean which will be true incase ScreenShare is currently active and being used, and false for inactive state.

hmsSDK.isScreenShareActive();

How to display screenshare in apps

Screen share track can be differentiated from normal video track using track's source property as track.source == "SCREEN"

A peer in Room can broadcast their Screen from any platform like Web, Android, iOS, etc. If a peer shares their Screen from Web & the viewer is on a Mobile platform, some content on the Screen can get cropped. It's necessary to configure the HMSVideoView correctly to ensure the complete Screen share content is visible without any clipping/cropping from edges.

Always create the HMSVideoView with matchParent as false and ScaleType as ScaleType.SCALE_ASPECT_FIT for correctly showing Screenshare Tracks.

Let's look at the implementation:

HMSVideoView( track: screenShareTrack, // pass the screen share track here matchParent: false, // set match parent as false to ensure Screenshare view is not cropped scaleType: ScaleType.SCALE_ASPECT_FIT, // always set to Aspect Fit for Screenshare Tracks key: Key(videoTrack.trackId), // set a unique identifier using the trackId )

To learn more about Rendering any Video, refer the guide here.

Troubleshooting Guide

For starting Screenshare from iOS devices (iPhones or iPads) following are some common setup you should already have within your Apps -

Bitcode Disabled

Bitcode has been disabled by Apple from Xcode 14 & iOS 16 and above. So 100ms packages also have Disabled Bitcode to ensure compatibility. Ensure that in your Xcode project Bitcode is Disabled for all Targets.

Disable Bitcode in Xcode

Podfile with Bitcode Disabled

You can use the following Podfile which has post_install script to Disable Bitcode for all Pods. Ensure that you modify the target for your Main App and the Broadcast Upload Extension.

In this sample Podfile, the Target names are Runner and FlutterBroadcastUploadExtension. Change these to the actual Target names defined in your Xcode project.

platform :ios, '13.0' project 'Runner', { 'Debug' => :debug, 'Profile' => :release, 'Release' => :release, } def flutter_root generated_xcode_build_settings_path = File.expand_path(File.join('..', 'Flutter', 'Generated.xcconfig'), __FILE__) unless File.exist?(generated_xcode_build_settings_path) raise "#{generated_xcode_build_settings_path} must exist. If you're running pod install manually, make sure flutter pub get is executed first" end File.foreach(generated_xcode_build_settings_path) do |line| matches = line.match(/FLUTTER_ROOT\=(.*)/) return matches[1].strip if matches end raise "FLUTTER_ROOT not found in #{generated_xcode_build_settings_path}. Try deleting Generated.xcconfig, then run flutter pub get" end require File.expand_path(File.join('packages', 'flutter_tools', 'bin', 'podhelper'), flutter_root) flutter_ios_podfile_setup # ENSURE TO SET THE CORRECT MAIN APP TARGET NAME BELOW target 'Runner' do use_modular_headers! use_frameworks! flutter_install_all_ios_pods File.dirname(File.realpath(__FILE__)) end # ENSURE TO SET THE CORRECT SCREENSHARE EXTENSION TARGET NAME BELOW target 'FlutterBroadcastUploadExtension' do use_modular_headers! use_frameworks! pod 'HMSBroadcastExtensionSDK' end # Post install script to Disable Bitcode from all Pods post_install do |installer| installer.pods_project.targets.each do |target| flutter_additional_ios_build_settings(target) target.build_configurations.each do |config| config.build_settings['ENABLE_BITCODE'] = 'NO' end end end

Background Modes Enabled

In majority use-cases, playing audio from Room would be required when app is in Background Mode. Mostly, if users are starting Screenshare from their iPhones/iPads they would want to continue listening to audio from the Room. So, ensure that you have Background Modes Enabled in your Xcode project.

Enable Background Modes

App Groups Enabled

Ensure that you have enabled App Groups for both your Main App Target & the newly created Broadcast Extension Target. If the same App Group is not enabled on both Targets then the App & the Screenshare Extension won't be able to communicate & starting Screenshare from your iOS device will fail. Also, ensure that there's no typo / spelling mis-matches between the App Group enabled on Main App Target & the Screenshare Broadcast Extension Target.

Same App Group for both Targets

Permission Denied EXC_BAD_ACCESS error

100ms Example Apps already have configurations for starting Screenshare on iOS devices. The values for App Group & Preferred Extension used in 100ms Example Apps cannot be reused by any other apps as they won't be linked to your Apple Developer Account.

If the 100ms values for App Group which is "group.flutterhms" & Preferred Extension which is "FlutterBroadcastUploadExtension" are resued by your apps then you will have an Xcode exception containing Permission Denied EXC_BAD_ACCESS error message similar to the one shown below -

CFMessagePort: bootstrap_register(): failed 1100 (0x44c) 'Permission denied', port = 0xbc03, name = 'group.flutterhms.88C5E70E-F40F-4C36-A48C-E65736E85CAC.audio.mach.port' See /usr/include/servers/bootstrap_defs.h for the error codes. * thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x8) frame #0: 0x00000001c03ddc78 CoreFoundation_CFGetNonObjCTypeID + 92 ...

To resolve this issue ensure that you create a Unique App Group which is linked to your Apple Developer Account. If you do not have an Active Apple Developer Account or do not have permissions to create a New App Group, then you won't be able to this feature.

Running on Simulator

Starting Screenshare from an iOS Simulator is not supported by Apple. You can start Screenshare only from an actual iOS device like an iPhone or iPad.

iOS Deployment Target Version

100ms Flutter Package is supported for iOS 12 and above versions. Ensure that the Minimum iOS Deployment Target Version is set to 12.0 or above in your Xcode project & the Podfile.

Flutter version 3.3.0+

100ms Flutter Package is supported for Flutter versions 3.3.0 or above. You can check your current Flutter version by running the flutter doctor command.

Flutter Doctor

Role has Screenshare Permission

Ensure that the Role used to Join the Room has Screenshare permission Enabled from the 100ms Dashboard. If the Screenshare permission is not Enabled from the Dashboard, any users joining with this Role won't be able to start Screenshare. These users would still be able to see Screenshare performed by other Peers who have Screenshare permissions.

Screen Share Permission


Have a suggestion? Recommend changes ->

Was this helpful?

1234