PIP (Picture-In-Picture) Mode

100ms React Native SDK provides support for creating Picture in Picture mode experience for video calls.

PIP Mode lets the user watch the room video in a small window pinned to a corner of the screen while navigating between apps or browsing content on the main screen. It is a great feature to provide a seamless experience to the users while they are in a video call. It is especially useful when the user wants to multitask while being in a video call.

Due to inherent platform differences, the PIP setup is different for Android & iOS platforms.

Minimum Requirements

• Minimum version required to support PiP is Android 8.0 (API level 26)
• Minimum react-native-hms SDK version required is 1.10.8
• Minimum iOS version required to support PiP is iOS 15 or above
• [iOS Only] Your app should have com.apple.developer.avfoundation.multitasking-camera-access entitlement on iOS to use PiP mode during video calls. Without this entitlement, the app will not be able to access Camera while in PiP or Background mode. When your app enters a multitasking mode, this entitlement allows your app to continue using the camera. Before submitting an app with this entitlement to the App Store, you need permission to use the entitlement. To request permission, see Multitasking Camera Access. After you receive permission, add the entitlement to your app by opening the entitlement file in Xcode.

How Picture in Picture works during a video call?

When your app goes into the background, it can no longer access the camera and publish it to other peers in the room. Also, you can't see the video of other participants in the call if your app is in the background. Both of these issues are resolved by implementing PiP mode for video calling in your app.

While enabling PIP Mode you can configure the PIP Window Aspect Ratio, Scale Type, Buttons shown to perform actions, etc using the HMSPIPConfig interface.

Actionable buttons like Mute/Unmute Audio/Video, Leave Room for local peers can be customized while enabling PIP Mode.

The HMSPIPConfig interface has the following properties:

• autoEnterPipMode?: boolean - To automatically enter PIP Mode when the app goes to background.

• aspectRatio?: [number, number] - The Aspect Ratio of the PIP Window which will be appear. Default is [16, 9] on Android & [9, 16] on iOS. Other values can be [3, 4], [1, 1], [4, 3] ratios.

• scaleType?: HMSVideoViewMode - [iOS Only] The scale type of the PIP window. Default is ASPECT_FILL. Other values can be ASPECT_FIT, ASPECT_BALANCED.

• useActiveSpeaker?: boolean - [iOS Only] To use the active speaker's video track in PIP mode. Default is true, so the currently speaking Peer's video track will be shown in PIP mode.

• endButton?: boolean - [Android Only] To enable/disable showing of a button through which user can leave the room. Default is false, so the button is not visible by default.

• videoButton?: boolean - [Android Only] To enable/disable showing of a button through which user can Mute/Unmute their Video Camera. Default is false, so the button is not visible by default.

• audioButton?: boolean - [Android Only] To enable/disable showing of a button through which user can Mute/Unmute their Video Camera. Default is false, so the button is not visible by default.

Android

How to add PiP support on Android?

  <activity
    ....
    android:supportsPictureInPicture="true"
    android:configChanges="keyboard|keyboardHidden|orientation|screenSize|smallestScreenSize|screenLayout|uiMode"
    ... />

import com.reactnativehmssdk.HMSManager;
public class MainActivity extends ReactActivity {

  ...

  @Override
  public void onPictureInPictureModeChanged(boolean isInPictureInPictureMode, Configuration newConfig) {
    super.onPictureInPictureModeChanged(isInPictureInPictureMode, newConfig);
    HMSManager.Companion.onPictureInPictureModeChanged(isInPictureInPictureMode, newConfig);
  }

  @Override
  protected void onResume() {
    super.onResume();
    HMSManager.Companion.onResume();
  }

  @Override
  protected void onUserLeaveHint() {
    super.onUserLeaveHint();
    HMSManager.Companion.onUserLeaveHint();
  }
}

// You can also check if PIP is supported on device or not.
const isPIPSupported = await hmsInstance.isPipModeSupported();

const isEnabled = await hmsInstance.enterPipMode();
if (isEnabled) {
    // App has entered into PIP Mode
    setPIPModeActive(true);
}

const isEnabled = await hmsInstance?.enterPipMode({
                          aspectRatio: [16, 9],   // for 16:9 aspect ratio
                          endButton: true,        // for showing/hiding leave room button
                          videoButton: true,      // for showing/hiding local Video button
                          audioButton: true,      // for showing/hiding local Audio button
                        });

  import { AppState, AppStateStatus } from 'react-native';

  ...

  const appStateRef = useRef(AppState.currentState);

  useEffect(() => {
    // We will register an AppState listener when App is in PIP Mode
    if (pipModeActive) {
      appStateRef.current = AppState.currentState;

      const appStateListener = (nextAppState: AppStateStatus) => {
        if (
          appStateRef.current.match(/inactive|background/) &&
          nextAppState === "active"
        ) {
          // Now, App is not in PIP mode
          setPIPModeActive(false);
        }

        appStateRef.current = nextAppState;
      };

      AppState.addEventListener('change', appStateListener);

      return () => {
        AppState.removeEventListener('change', appStateListener);
      }
    }
  }, [pipModeActive]);

  // Getting trackId to show in PIP mode
  // getPreferredTrackId function returns trackId of preferred video track
  const pipTrackId = getPreferredTrackId()

  ...

  // If PIP mode is active, showing only single peer
  if (isPipModeActive) {
    return (
      <HmsView
        ...
        trackId={pipTrackId}
        ...
      />
    )
  }

  // Showing tiles for all the peers
  return <MultiplePeers />

  ...

iOS

const pipConfig: HMSPIPConfig = {
  aspectRatio: [9, 16],
  autoEnterPipMode: true,
  useActiveSpeaker: true,
  scaleType: HMSVideoViewMode.ASPECT_FILL,
  endButton: false, // this option is not available on iOS
  audioButton: false, // this option is not available on iOS
  videoButton: false, // this option is not available on iOS
};

hmsInstance.setPipParams(pipConfig);

const pipConfig: HMSPIPConfig = {
  ...
  useActiveSpeaker: true, // iOS Only
  ...
};

hmsInstance.setActiveSpeakerInIOSPIP(true); // to use active speaker's video track in PIP mode

hmsInstance.changeIOSPIPVideoTrack(videoTrack);

PiP Helper Methods

/**
 * Attaches a listener that gets notified whenever Picture-in-Picture (PIP) mode changes.
 *
 * Call this function to subscribe to PIP mode changes. The provided callback will be
 * invoked with an object containing an `isInPictureInPictureMode` property indicating
 * the current PIP mode state.
 *
 * @example
 * onPIPModeChangedListener((data) => {
 *   if (data.isInPictureInPictureMode) {
 *     console.log("App entered PIP mode");
 *   } else {
 *     console.log("App exited PIP mode");
 *   }
 * });
 */
const onPIPModeChangedListener = async (): Promise<{ isInPictureInPictureMode: boolean }> => {
  // ... (implementation details)
};

/**
 * Checks if the current device supports Picture-in-Picture (PIP) mode.
 *
 * This function asynchronously determines whether the device running the application
 * can utilize PIP mode. It resolves to `true` if PIP is supported, `undefined` otherwise.
 *
 * @returns A Promise resolving to `true` if PIP is supported, `undefined` otherwise.
 */
async isPipModeSupported(): Promise<undefined | boolean> {
  // ... (implementation details)
};

/**
 * Attempts to manually enter Picture-in-Picture (PIP) mode on the device.
 *
 * This function triggers the application to enter PIP mode. It accepts an optional
 * `HMSPIPConfig` object for configuration (details omitted for brevity). The function
 * resolves to `true` if PIP mode is successfully entered, `undefined` otherwise.
 *
 * @param data An optional HMSPIPConfig object for PIP mode configuration.
 * @returns A Promise resolving to `true` if PIP is entered, `undefined` otherwise.
 */
async enterPipMode(data?: HMSPIPConfig): Promise<undefined | boolean> {
  // ... (implementation details)
};

/**
 * Configures the Picture-in-Picture (PIP) window.
 *
 * This function allows you to customize the PIP window's behavior. It accepts an optional
 * `HMSPIPConfig` object for configuration (details omitted for brevity). The function
 * resolves to `true` if the configuration is successful, `undefined` otherwise.
 *
 * @param data An optional HMSPIPConfig object for PIP window configuration.
 * @returns A Promise resolving to `true` if configuration is successful, `undefined` otherwise.
 */
async setPipParams(data?: HMSPIPConfig): Promise<undefined | boolean> {
  // ... (implementation details)
};

/**
 * Sets the video track for Picture-in-Picture mode (iOS only).
 *
 * This function allows you to specify the video track you want to display within the
 * PIP window. It's exclusive to iOS devices. The `track` parameter represents the
 * `HMSVideoTrack` object you wish to display.
 *
 * @param track The HMSVideoTrack object representing the video to show in PIP.
 * @throws {Error} Throws an error if the function is called on a non-iOS device.
 */
async changeIOSPIPVideoTrack(track: HMSVideoTrack) {
  // ... (implementation details)
};

/**
 * Shows or hides the active speaker's video in the PIP window (iOS only).
 *
 * This function allows you to control the visibility of the active speaker's video
 * within the PIP window. It's exclusive to iOS devices. The `enable` parameter is a
 * boolean value indicating whether to show (`true`) or hide (`false`) the video.
 *
 * @param enable A boolean value indicating whether to show (`true`) or hide (`false`) the active speaker's video.
 * @throws {Error} Throws an error if the function is called on a non-iOS device.
 */
async setActiveSpeakerInIOSPIP(enable: boolean) {
  // ... (implementation details)
};

/**
 * This listener fires when the room is left while in Picture-in-Picture mode (Android only).
 *
 * This function serves as a listener specifically for Android devices. It's triggered
 * whenever the user leaves the room while the application is in PIP mode. The provided
 * callback receives a data object containing details about the event (details omitted
 * for brevity).
 *
 * @param data An object containing details about leaving the room in PIP mode (Android only).
 */
function onPIPRoomLeaveListener(data): void {
  // ... (implementation details)
}

Checkout Video of PIP in Action on Android

👀 To see an example of Picture in Picture implementation in our example app, checkout our example project.