Article 5

Video & Capture

Use Processing's Video library to access a webcam, perform frame differencing for motion detection, apply chroma key, and build a live pixelation effect.

⏱ 16 min video webcam capture pixels motion-detection

Processing’s Video library unlocks live webcam input and video file playback. Working with camera pixels directly gives you a powerful foundation for motion detection, augmented reality effects, and creative filters. This tutorial walks through every step from opening a capture device to building a real-time pixelation effect.

Installing the Video Library

Processing 4 ships with a GStreamer-based Video library. If it is not already installed:

Open Sketch → Import Library → Manage Libraries
Search for Video
Click Install on the entry by The Processing Foundation

After installation, restart Processing and add the import at the top of your sketch.

Opening a Capture Device

import processing.video.*;

Capture cam;

void setup() {
  size(640, 480);

  // List all available capture devices to the console
  String[] cameras = Capture.list();
  if (cameras == null || cameras.length == 0) {
    println("No cameras found.");
    exit();
    return;
  }
  for (int i = 0; i < cameras.length; i++) {
    println(i + ": " + cameras[i]);
  }

  // Open the first device at sketch resolution
  cam = new Capture(this, width, height);
  cam.start();
}

Capture.list() returns a string array of every device descriptor your system exposes. If you have multiple cameras, pass one of those strings as the third argument instead of using the default.

Reading Frames

Frames arrive asynchronously. Check cam.available() in draw() before reading:

void draw() {
  if (cam.available()) {
    cam.read();
  }
  image(cam, 0, 0);
}

You can also use the callback method captureEvent(Capture c) which fires automatically when a new frame is ready — useful when you want to decouple capture from drawing.

void captureEvent(Capture c) {
  c.read();
}

Working with Pixels

After cam.read(), call cam.loadPixels() to make the pixel array available. Each element in cam.pixels[] is a 32-bit int encoding alpha, red, green, and blue:

cam.loadPixels();
for (int i = 0; i < cam.pixels.length; i++) {
  color c = cam.pixels[i];
  float r = red(c);
  float g = green(c);
  float b = blue(c);
  float bright = (r + g + b) / 3.0;
  cam.pixels[i] = color(bright);  // convert to greyscale
}
cam.updatePixels();
image(cam, 0, 0);

For direct integer-level access (much faster in large loops), unpack the channels manually:

int px = cam.pixels[i];
int r  = (px >> 16) & 0xFF;
int g  = (px >>  8) & 0xFF;
int b  =  px        & 0xFF;

Frame Differencing for Motion Detection

Subtract the previous frame from the current frame. Pixels that changed a lot are where motion occurred.

import processing.video.*;

Capture cam;
PImage prevFrame;
float threshold = 30;

void setup() {
  size(640, 480);
  cam = new Capture(this, width, height);
  cam.start();
  prevFrame = createImage(width, height, RGB);
}

void draw() {
  if (cam.available()) cam.read();

  cam.loadPixels();
  prevFrame.loadPixels();
  loadPixels();

  for (int i = 0; i < pixels.length; i++) {
    color curr = cam.pixels[i];
    color prev = prevFrame.pixels[i];

    float dr = abs(red(curr)   - red(prev));
    float dg = abs(green(curr) - green(prev));
    float db = abs(blue(curr)  - blue(prev));
    float diff = (dr + dg + db) / 3.0;

    pixels[i] = diff > threshold ? color(255) : color(0);
  }

  updatePixels();
  prevFrame.copy(cam, 0, 0, width, height, 0, 0, width, height);
}

Bright regions in the output indicate movement. Lowering threshold makes the detector more sensitive.

Chroma Key (Green Screen)

Replace every pixel whose green channel dominates with a background image:

PImage background;

void chromaKey(PImage src, PImage bg, float sensitivity) {
  src.loadPixels();
  bg.loadPixels();
  loadPixels();

  for (int i = 0; i < src.pixels.length; i++) {
    color c = src.pixels[i];
    float r = red(c), g = green(c), b = blue(c);

    // pixel is "green" when green channel is much larger than red and blue
    boolean isGreen = (g - r > sensitivity) && (g - b > sensitivity);
    pixels[i] = isGreen ? bg.pixels[i] : c;
  }
  updatePixels();
}

Call this in draw() after reading the camera. Adjust sensitivity (try values from 20 to 80) based on your lighting conditions.

Full Example: Live Pixelation Effect

import processing.video.*;

Capture cam;
int blockSize = 16;

void setup() {
  size(640, 480);
  String[] cams = Capture.list();
  if (cams == null || cams.length == 0) { exit(); return; }
  cam = new Capture(this, width, height);
  cam.start();
  noStroke();
}

void draw() {
  if (cam.available()) cam.read();

  // Dynamic block size driven by mouse X position
  blockSize = (int) map(mouseX, 0, width, 2, 60);

  cam.loadPixels();

  // Sample one colour per block and fill a rectangle
  for (int y = 0; y < height; y += blockSize) {
    for (int x = 0; x < width; x += blockSize) {
      // clamp the sample point inside the camera image
      int sx = constrain(x + blockSize / 2, 0, cam.width  - 1);
      int sy = constrain(y + blockSize / 2, 0, cam.height - 1);

      color c = cam.pixels[sy * cam.width + sx];
      fill(c);
      rect(x, y, blockSize, blockSize);
    }
  }

  // Optional: show block size as a HUD
  fill(255, 220);
  textSize(14);
  text("Block: " + blockSize + "px  |  Move mouse left/right", 10, height - 10);
}

void keyPressed() {
  if (key == 's') saveFrame("pixelation-####.png");
}

Move the mouse from left to right to increase the block size from fine detail to coarse mosaic. Press s to save a frame as a PNG.

Tips and Common Pitfalls

Mirror the image. Webcam feeds are often mirrored. Flip horizontally with scale(-1, 1) and translate(-width, 0) before drawing.

Performance. Pixel-level operations in pure Java are slow. For real-time effects, keep the sketch at a manageable resolution (320×240 or 640×480) or use pixelDensity(1) on Retina displays to avoid quadrupling the pixel count.

Available() check. Always wrap cam.read() inside if (cam.available()). Calling read() when no new frame has arrived returns a blank image.

Library version mismatches. On Linux, if you see GStreamer errors in the console, try different camera descriptor strings from Capture.list() — some backends expose the same device under multiple names.

Key Takeaways

Capture.list() enumerates devices; pass the desired descriptor string to the Capture constructor.
Check cam.available() before calling cam.read() each frame.
After cam.loadPixels(), iterate cam.pixels[] directly for pixel-level manipulation.
Frame differencing computes motion as the per-channel absolute difference between consecutive frames.
Chroma key tests whether the green channel exceeds red and blue by a threshold, then swaps in a background pixel.