Hardware Integration
Connect Processing to Arduino over Serial, control DMX lighting via ArtNet, and send and receive MIDI using The MidiBus library.
Processing occupies a useful position in the hardware stack: it runs on a full computer with network access and a graphics GPU, but it can also exchange data in real time with microcontrollers, lighting rigs, and audio hardware. This article covers the three most common integration patterns.
Arduino over Serial
The processing.serial library provides a bidirectional byte channel. A robust protocol frames messages with a start byte, length, payload, and checksum so partial reads are detected.
Arduino Firmware (sensor → Processing)
// Arduino sketch: sends analog pin A0 every 20ms
void setup() {
Serial.begin(115200);
}
void loop() {
int val = analogRead(A0); // 0–1023
Serial.print('<'); // start marker
Serial.print(val);
Serial.println('>'); // end marker
delay(20);
}
Processing Serial Reader
import processing.serial.*;
Serial port;
String buffer = "";
float sensorValue = 0;
void setup() {
size(800, 400);
printArray(Serial.list()); // find your port index
port = new Serial(this, Serial.list()[0], 115200);
port.bufferUntil('>'); // trigger serialEvent on '>'
}
void draw() {
background(20);
float barW = map(sensorValue, 0, 1023, 0, width);
fill(0, 180, 255);
noStroke();
rect(0, height/2 - 20, barW, 40);
fill(255);
text("Sensor: " + (int)sensorValue, 10, 30);
}
void serialEvent(Serial p) {
String raw = p.readString();
if (raw == null) return;
buffer += raw;
int start = buffer.lastIndexOf('<');
int end = buffer.lastIndexOf('>');
if (start >= 0 && end > start) {
String payload = buffer.substring(start + 1, end).trim();
try {
sensorValue = float(payload);
} catch (Exception e) {}
buffer = "";
}
}
Processing → Arduino (actuator control)
Send a single byte value (0–255) to control LED brightness:
void mouseMoved() {
int brightness = (int)map(mouseX, 0, width, 0, 255);
port.write(brightness); // Arduino reads Serial.read()
}
On the Arduino side, analogWrite(LED_PIN, Serial.read()) sets PWM duty cycle directly.
DMX Lighting via ArtNet
ArtNet is DMX over UDP. Processing sends ArtNet packets over your local network to a node that converts them to RS-485 DMX. Install the ArtNet for Processing library by Sadao Usui or use the raw UDP approach:
import java.net.*;
import java.nio.ByteBuffer;
DatagramSocket socket;
InetAddress artnetIP;
final int ARTNET_PORT = 6454;
byte[] dmx = new byte[512]; // 512 DMX channels
void setup() {
try {
socket = new DatagramSocket();
artnetIP = InetAddress.getByName("192.168.1.100"); // node IP
} catch (Exception e) { e.printStackTrace(); }
}
void sendDMX() {
// Build ArtNet OpDmx packet (18-byte header + 512 channel data)
byte[] packet = new byte[530];
// "Art-Net\0"
byte[] id = {'A','r','t','-','N','e','t',0};
System.arraycopy(id, 0, packet, 0, 8);
packet[8] = 0x00; // OpCode low byte (OpDmx = 0x5000)
packet[9] = 0x50; // OpCode high byte
packet[10] = 0; // ProtVer high
packet[11] = 14; // ProtVer low
packet[12] = 0; // Sequence
packet[13] = 0; // Physical
packet[14] = 0; // Universe low
packet[15] = 0; // Universe high
packet[16] = 2; // Length high (512 = 0x0200)
packet[17] = 0; // Length low
System.arraycopy(dmx, 0, packet, 18, 512);
try {
socket.send(new DatagramPacket(packet, packet.length, artnetIP, ARTNET_PORT));
} catch (Exception e) { e.printStackTrace(); }
}
void draw() {
background(0);
// Map mouse X to a wash fixture at DMX address 1 (RGB = ch 1,2,3)
dmx[0] = (byte)map(mouseX, 0, width, 0, 255); // Red
dmx[1] = (byte)map(mouseY, 0, height, 0, 255); // Green
dmx[2] = 128; // Blue constant
sendDMX();
}
MIDI with The MidiBus
Install The MidiBus library from the Contribution Manager. It exposes incoming MIDI events as Processing callbacks.
import themidibus.*;
MidiBus bus;
void setup() {
size(800, 600);
MidiBus.list(); // prints available MIDI devices to console
// Use device names or indices from the list:
bus = new MidiBus(this, "IAC Driver Bus 1", "IAC Driver Bus 1");
}
// --- Incoming MIDI callbacks ---
void noteOn(int channel, int pitch, int velocity) {
println("Note ON ch=" + channel + " pitch=" + pitch + " vel=" + velocity);
// trigger visual event
launchNote(pitch, velocity);
}
void noteOff(int channel, int pitch, int velocity) {
println("Note OFF ch=" + channel + " pitch=" + pitch);
}
void controllerChange(int channel, int number, int value) {
// CC messages: knobs, faders, mod wheel
println("CC ch=" + channel + " num=" + number + " val=" + value);
}
Sending MIDI CC Messages
void mouseMoved() {
int ccValue = (int)map(mouseX, 0, width, 0, 127);
bus.sendControllerChange(0, 1, ccValue); // ch=0, CC#1 (mod wheel), value
}
void keyPressed() {
if (key == 'a') bus.sendNoteOn(0, 60, 100); // middle C
if (key == 's') bus.sendNoteOff(0, 60, 0);
}
Firmata Protocol
Firmata lets Processing control Arduino GPIO without writing custom firmware. Upload StandardFirmata to your Arduino from the Arduino IDE (File > Examples > Firmata > StandardFirmata), then use the Arduino (Firmata) Processing library:
import cc.arduino.*;
import processing.serial.*;
Arduino arduino;
final int LED_PIN = 13;
final int SENSOR_PIN = 0;
void setup() {
size(600, 400);
arduino = new Arduino(this, Arduino.list()[0], 57600);
arduino.pinMode(LED_PIN, Arduino.OUTPUT);
}
void draw() {
background(20);
int sensorVal = arduino.analogRead(SENSOR_PIN);
float mapped = map(sensorVal, 0, 1023, 0, 255);
arduino.analogWrite(LED_PIN, (int)mapped); // PWM on pin 13
fill(mapped, 100, 255 - mapped);
ellipse(width/2, height/2, mapped, mapped);
text("Analog: " + sensorVal, 10, 20);
}
Firmata’s PWM resolution is 8-bit (0–255) for analogWrite. Digital pins use Arduino.HIGH / Arduino.LOW.
Full Example: MIDI Controller Sketch
Incoming MIDI notes trigger animated visual events. Mouse X/Y sends continuous CC messages that control parameters in a connected synthesiser.
import themidibus.*;
MidiBus bus;
ArrayList<NoteEvent> events = new ArrayList<NoteEvent>();
int[] noteColors = new int[128];
void setup() {
size(1280, 720, P2D);
colorMode(HSB, 360, 100, 100, 100);
background(0);
MidiBus.list();
bus = new MidiBus(this, 0, 1); // input 0, output 1
// Pre-assign hue per pitch class
for (int i = 0; i < 128; i++) {
noteColors[i] = (i % 12) * 30; // 12 pitch classes → 12 hues
}
}
void draw() {
// Slowly dim background for trail
noStroke();
fill(0, 0, 0, 12);
rect(0, 0, width, height);
// Update and draw note events
for (int i = events.size() - 1; i >= 0; i--) {
NoteEvent e = events.get(i);
e.update();
e.draw();
if (e.isDead()) events.remove(i);
}
// Send mouse position as CC on channel 1
int ccX = (int)map(mouseX, 0, width, 0, 127);
int ccY = (int)map(mouseY, 0, height, 0, 127);
bus.sendControllerChange(0, 74, ccX); // CC74 = filter cutoff
bus.sendControllerChange(0, 71, ccY); // CC71 = resonance
}
void noteOn(int channel, int pitch, int velocity) {
float x = map(pitch, 0, 127, 50, width - 50);
float y = map(velocity, 0, 127, height - 50, 50);
float sz = map(velocity, 0, 127, 20, 180);
events.add(new NoteEvent(x, y, sz, noteColors[pitch]));
}
class NoteEvent {
float x, y, size, hue, alpha;
NoteEvent(float x, float y, float size, float hue) {
this.x = x;
this.y = y;
this.size = size;
this.hue = hue;
alpha = 100;
}
void update() {
size *= 1.02;
alpha *= 0.93;
}
void draw() {
noFill();
stroke(hue, 90, 95, alpha);
strokeWeight(2);
ellipse(x, y, size, size);
}
boolean isDead() { return alpha < 1; }
}
Connect a MIDI keyboard or controller, confirm device indices via MidiBus.list(), and launch the sketch. Each note press spawns an expanding ring at a pitch-mapped horizontal position, while mouse movement continuously modulates synthesiser parameters over CC.