Synchronizing with Videos

A tutorial outlining how to achieve full Handy video synchronization.

Overview

In our fourth tutorial, we’ll attach a video element to the page, bind it to the Handy UI, and then we’ll have the Handy synchronize with the video.

Invoke the Handy UI

As usual, we’ll invoke the Handy UI on our page using the code from our previous tutorial:

Script and video files

For our purposes, we’ll use our script for testing Handy synchronization. It can be found at the following URL:

https://sweettecheu.s3.eu-central-1.amazonaws.com/testsync/sync_video_2021.csv

Additionally, we’ll use the corresponding test video as part of the synchronization. It can be found at the following URL:

https://sweettecheu.s3.eu-central-1.amazonaws.com/testsync/sync_video_2021.mp4

Timing and order of operations

We’ll build on the code from our previous tutorials to achieve video synchronization.

Adding the <video> element

The Handy UI will need to bind to a <video> element to achieve timing synchronization and to have play/pause control.

Let’s create a video element using the synchronization test footage as the source. We’ll give it an ID so we can reference it easily later.

<video id="sync-video" width="100%" controls src="https://sweettecheu.s3.eu-central-1.amazonaws.com/testsync/sync_video_2021.mp4"/>

This will render as the following:

Of course, this will have no interactivity with the Handy until we bind it to the Handy UI.

Binding the video with the Handy UI

To bind the <video> element, we call the setVideoPlayer method from the Handy SDK. The input to the method will be a reference to the video element using its ID.

HANDY.setVideoPlayer(document.getElementById('sync-video'));

Binding a video to the Handy SDK means that the video and the Handy motion will be synchronized and that the video player can be used to start, pause, and skip the script running inside the Handy.

We’ll expand our example from our previous tutorial to add video synchronization functionality.

Altogether the code will look like the following:

<script>
function setScriptAndPlay() {
  var scriptURL = "https://sweettecheu.s3.eu-central-1.amazonaws.com/testsync/sync_video_2021.csv";
  HANDY.setVideoPlayer(document.getElementById('sync-video-2'));
  document.getElementById("play-btn").innerText = "Setting script...";
  HANDY.setScript(scriptURL).then((value) => {
    HANDY.hsspPlay(0, HANDY.getEstimatedServerTime);
    document.getElementById('sync-video-2').play();
    document.getElementById("play-btn").innerText = "Playing script and video";
  });
}

  function stopPlay(){
    HANDY.hampStop();
    document.getElementById("play-btn").innerText = "Set Script and Play Video";
    document.getElementById('sync-video-2').pause();
  }
</script>

<div>
  <video id="sync-video-2" width="100%" controls src="https://sweettecheu.s3.eu-central-1.amazonaws.com/testsync/sync_video_2021.mp4"/>
</div>
<button id="play-btn" onClick="setScriptAndPlay()">Set script and play</button>
<button onClick='stopPlay()'>Pause</button>

Rendered on the page it looks like the following example:

By pressing Set script and play, the Handy SDK will send the script to the Handy and begin video playback when everything is ready.

You’ll notice that the movement of your Handy will mirror the movement of the points on the graph. Play around by playing and pausing the video, as well as skipping to different points in the video.

Concluding thoughts

This concludes our set of tutorials for getting up and running with some synchronization basics. We recommend that you explore our SDK reference to find other functionalities of the Handy for more complicated use cases.