Video Player

createVideoPlayer() decodes a video into native frames: IOSurface-backed CVPixelBuffers on Apple platforms, AHardwareBuffers on Android. The frames are sampled by WebGPU without any copy through the CPU, with importExternalTexture or importSharedTextureMemory, copied into a regular texture with copyExternalImageToTexture, and drawn by React Native Skia.

const player = createVideoPlayer(url);
player.play();

const render = () => {
  // The most recently decoded frame, or null when there is no new frame
  const frame = player.copyLatestFrame();
  if (frame) {
    const texture = device.importExternalTexture({
      source: frame,
      rotation: player.rotation,
    });
    // ...sample the texture in a render pass and submit
    texture.destroy();
    frame.release();
  }
  requestAnimationFrame(render);
};
requestAnimationFrame(render);

createVideoPlayer

createVideoPlayer(source: string, options?: CreateVideoPlayerOptions): VideoPlayer

source is a URL (http, https, file) or a file path. The player starts paused: call play() to start decoding.

VideoPlayer

Playback is controlled like an HTMLMediaElement; the difference is that frames are pulled with copyLatestFrame() instead of being displayed.

The metadata of a remote video loads asynchronously, so duration, videoWidth, videoHeight, rotation and frameRate read 0 until it has. Frames carry their own width and height, which is what a render loop should use.

NativeVideoFrame

A frame holds a native buffer and keeps it alive until release() is called (or until the object is garbage collected). A player can only keep a few frames in flight: release each frame once the GPU work that samples it has been submitted, or copyLatestFrame() stops returning new frames.

handle is the raw pointer of the buffer (an IOSurfaceRef or an AHardwareBuffer*) as a BigInt, which importSharedTextureMemory({ handle }) takes.

Copying a frame into a texture

queue.copyExternalImageToTexture({ source: frame, rotation: player.rotation }, { texture }, size) renders a frame into a regular texture, on both platforms and whatever the frame's layout, with the rotation applied. See the native frame copy.

Pixel formats

Other sources

createVideoFrameFromNativeBuffer(pointer) wraps a buffer produced by another library (a CVPixelBufferRef on Apple platforms, an AHardwareBuffer* on Android, for instance VisionCamera's frame.getNativeBuffer().pointer) into a NativeVideoFrame that keeps it alive until released.

Examples

• ImportExternalTexture: a video drawn through importExternalTexture on both platforms
• SharedTextureMemory: BGRA frames bound as a regular sampled texture
• Video in a Skia canvas: frames drawn with React Native Skia