React Native WebGPU

Native Buffers

Share pixels between WebGPU, other GPU APIs and the screen without a copy.

A NativeBuffer is an AHardwareBuffer on Android and an IOSurface on Apple platforms.

It does not belong to any GPU device: any GPU API can write it or read it, and a NativeBufferView can display it, without copies or CPU readbacks. Since each of them works asynchronously on the GPU, every handoff comes with a fence that signals when the previous user is done with the buffer.

This page shows how these pieces fit together. The Native Buffer and Native Extensions API pages document each call in detail.

The flow

A device uses a buffer in one of two ways: as a GPUTexture to read and write, or as a GPUExternalTexture to read only.

  1. NativeBuffer is the buffer itself, with its size, its pixel format and release(). A camera, a video decoder or createNativeBuffer() produces it, and it works without any device.
  2. GPUSharedTextureMemory is the buffer bound to a device: Dawn's import of it. A buffer is imported once per device, and stays imported for as long as you use it.
  3. GPUTexture is what memory.createTexture() returns: a regular WebGPU texture whose pixels are the buffer's, no copy involved. It is read and write: render into it, sample it, copy from it. It is used only inside an access window: between memory.beginAccess() and memory.endAccess(), the device owns the buffer.
  4. GPUExternalTexture comes from the WebGPU specification, where it samples a video frame. It is read only: it converts YUV to RGB while sampling, with the frame's rotation and mirroring applied, and it is the only way to read a camera or video frame in a YUV format. importExternalTexture({ source: buffer }) imports the buffer for this one frame and releases it when you destroy the external texture.

To read every frame of a camera or a decoder without importing its buffers again each time, an external texture can also be built over a texture of a shared memory; see Importing a buffer once.

Two more types sit next to the flow:

  • GPUSharedFence says when some GPU work on the buffer is done (a sync fd on Android, an MTLSharedEvent on Apple platforms). memory.endAccess() returns the fences of the access, so the next user, another API or the screen, can wait for them.
  • NativeBufferView displays a buffer as a React Native view. It has no swap chain and no device of its own.

Where buffers come from

Your own rendering. createNativeBuffer() allocates an empty color buffer that WebGPU or Skia renders into, for instance to display it in a NativeBufferView (see Rendering into a buffer). The 200 icons example draws each icon with Skia into a buffer of its own and compares it with one Skia <Canvas> per icon.

Video. React Native Skia Video decodes and encodes video with Skia, and its React Native Skia v3 version moves frames between the platform decoders and encoders and Skia through React Native WebGPU, without CPU copies.

Camera and video frames come from a pool: the producer writes the next frames into the same few buffers, and gets a buffer back once you release its frame. Release pooled frames as soon as the GPU work that reads them has been submitted; a producer that runs out of buffers stops delivering frames. A buffer you allocate yourself stays yours until you release it.

Reading a buffer on the GPU

You want toUse
Sample a frame in your shader, YUV includedimportExternalTexture({ source: buffer })
Do the same for every frame of a camera or a decoder, importing each buffer onceimportSharedTextureMemory(), then importExternalTexture({ source: texture })
Use a color buffer as a regular texture, without a copyimportSharedTextureMemory(), then memory.createTexture()
Keep a copy of the frame in a texture of your owncopyExternalImageToTexture()

The first row, for a frame from a camera or a video decoder:

const  = .({ : ,  });
(); // ...and queue.submit()
.();
.(); // the producer can reuse the buffer

The copy of the last row runs on the GPU, so the frame can be released right after; it is the path to hand a frame to a library that wants a texture it owns, such as Skia or Three.js.

Rendering into a buffer

Allocate a buffer with createNativeBuffer(), import it with importSharedTextureMemory(), and render into its texture inside an access window:

const  = (512, 512);
const  = .({ :  });
const  = .();

// false: a new buffer, nothing in it to keep
.(, false);
(); // encode passes into the texture and submit
// The fences signal once the GPU has finished writing the buffer
const  = .();

When endAccess() returns, the work is submitted but may not be done yet: whoever reads the buffer next (the screen, another device, a native library) waits for its fences first. NativeBufferView takes them as they are, with present(frame, done).

Displaying a buffer

NativeBufferView shows a buffer as a regular React Native view. It reports its size in physical pixels, and you present a buffer of that size through its ref:

export const  = () => {
  const  = <NativeBufferViewRef>(null);
  const { ,  } = ();

  (() => {
    if (! || !.) {
      return;
    }
    const  = (., .);
    const  = .({ :  });
    const  = .();
    .(, false);
    ();
    // Shown once the GPU work is done. The view keeps its own reference on
    // the buffer and on the fences.
    ..(, .());
    .();
    .();
    .();
  }, []);

  return (
    <
      ={}
      ={}
      ={{ : 96, : 96 }}
    />
  );
};

present() returns right away. The view waits for the fences off the main thread and shows the buffer on the next vsync after they signal. Until the producer catches up with a resize, the view scales the buffer it has according to its resizeMode.

NativeBufferView or Canvas?

A Canvas has a swap chain: a few buffers it cycles through to show a new frame on every vsync. A NativeBufferView holds a single buffer, the last one presented. That makes it the better fit for content that is drawn once or updated now and then, such as icons, thumbnails, a chart or a filtered photo, especially when a screen has many of them. Animated content belongs in a Canvas.

Never write a displayed buffer

The view reads the buffer whenever the system composites the screen, so a buffer must not be written again once it has been presented. Render each update into a new buffer and present that one. The view keeps its own reference on the buffer, so a buffer from createNativeBuffer() can be released right after present().

A pooled frame, from a camera or a video decoder, breaks this rule on its own: its producer writes a later frame into the buffer as soon as the frame is released, even while the view still shows it. Copy such a frame into a buffer of your own and present the copy:

const  = (., .);
const  = .({ :  });
const  = .();
.(, false);
..(
  { :  },
  {  },
  [., .],
);
const  = .();
.(); // back to the decoder or the camera
.(, );
.();
.();
.();

Presenting from a worklet

ref.current.viewId identifies the view on any runtime. On a worklet runtime, such as a frame processor or a Reanimated worklet, present with the RNWebGPU global, which the worklet captures:

RNWebGPU.presentNativeBuffer(viewId, frame, memory.endAccess(texture));

Synchronization

GPU work is asynchronous, so each handoff of a buffer comes with a fence that says when the previous user is done with it.

  • From the producer to your device. On Android, a frame carries acquireFenceFd: the decoder or the camera signals it once the buffer is written. importExternalTexture(), copyExternalImageToTexture() and present() wait for it on their own. With importSharedTextureMemory(), pass it to beginAccess() once you import it with importSharedFence({ type: "sync-fd", handle: frame.acquireFenceFd }); see Importing a buffer once. On Apple platforms, frames are complete when you receive them.
  • From your device to the next user. endAccess() returns the fences that signal once the work submitted in the access is done. Pass the result to present(), export each fence to a native consumer with fence.export(), or destroy them.
  • Inside the access window. A texture created from a buffer can only be used between beginAccess() and endAccess(), and a texture can have only one access open at a time.

beginAccess() and present() take their own reference on the fences they are given, so you can destroy a fence right after passing it.

Lifetimes

ObjectKeeps aliveRelease with
NativeBufferThe platform bufferrelease(), or garbage collection. Release pooled frames as soon as the GPU work that reads them is submitted.
GPUSharedTextureMemoryThe import (Dawn holds its own reference on the buffer)destroy(), after endAccess() and after destroying its textures
GPUTexture of a memoryThe texture over the bufferdestroy(), before the memory
GPUSharedFenceThe native fencedestroy(), once it has been handed to beginAccess() or present()
GPUExternalTexture of a frameThe import of the frame and its accessdestroy(), after the queue.submit() that sampled it
GPUExternalTexture of a memory textureNothing: it opens no access of its owndestroy() when you drop the buffer, before the texture and the memory
NativeBufferViewThe last buffer presentedReplaced by the next present(), released on unmount and on reload

Garbage collection eventually releases all of these, but it can run long after the object is unreachable. A runtime that imports a buffer per frame should release eagerly.

Devices

requestDevice() enables the Dawn features behind native buffers whenever the adapter supports them, so nothing has to be requested. A device created by another library and wrapped with importDevice() has to request them itself, as React Native Skia's device does:

PlatformFeatures
iOS, macOSshared-texture-memory-iosurface, shared-fence-mtl-shared-event, dawn-multi-planar-formats
Androidshared-texture-memory-ahardware-buffer, shared-fence-sync-fd, opaque-ycbcr-android-for-external-texture

On the rare drivers and emulators that cannot import native buffers, the import throws an error naming what is missing. Catch it to degrade.

Platform notes

  • YUV frames. On both platforms, a YUV frame (from a camera or a video decoder) is read through an external texture, over the frame or over a texture of its shared memory, or copied with copyExternalImageToTexture(). On Android, its layout is known only to the driver. Color frames, such as BGRA ones on Apple platforms, also work as regular textures through importSharedTextureMemory().
  • Android 8 and 9. NativeBufferView copies the pixels of each buffer it is given through the CPU. From Android 10 on, it shows the buffer without a copy.
  • High bit depth. A createNativeBuffer() buffer in rgba16float gives more precision, not extended range: NativeBufferView shows values in [0, 1] and clamps the rest.

Examples

  • NativeBufferView: renders into a buffer of the view's size and re-renders when the view is resized
  • 200 icons: Skia draws 200 icons into native buffers displayed by NativeBufferViews, compared with one Skia <Canvas> per icon
  • Vision Camera: WGSL effects on live camera frames