Interactivity: Raycaster, Controls, Events

A scene isn't useful until the user can touch it. This article is about the three tools that turn a passive render into an app: Raycaster (clicking on 3D objects), Controls (OrbitControls, TransformControls, DragControls), and pointer events (unified mouse / touch / pen handling).

In the demo, hover any planet and it lights up. Click one and the camera flies to it. Click empty space to return. Orbit with drag, zoom with scroll.

The mental model: a ray cast into the scene

The screen is a 2D projection of a 3D world. "What's under the mouse?" is answered by reversing the projection: shoot a ray from the camera, through the clicked pixel, into the scene, and see what it hits first. That's what a Raycaster is.

const raycaster = new THREE.Raycaster();
const pointer   = new THREE.Vector2();

canvas.addEventListener('pointermove', (e) => {
  const rect = canvas.getBoundingClientRect();

  // Convert pixel coords → normalized device coords (-1 to +1)
  pointer.x =  ((e.clientX - rect.left) / rect.width)  * 2 - 1;
  pointer.y = -((e.clientY - rect.top)  / rect.height) * 2 + 1;
});

// Inside the loop:
raycaster.setFromCamera(pointer, camera);
const hits = raycaster.intersectObjects(scene.children, true);
if (hits.length) hover(hits[0].object);
else             hover(null);

Three numbers to remember:

• Normalized device coordinates (NDC) — the raycaster wants (-1, -1) at bottom-left and (+1, +1) at top-right. Notice the - sign on the Y conversion — the browser's Y goes down, NDC's goes up.
• The second argument to intersectObjects — pass true for "recurse into children". Almost always you want this.
• Hits are sorted nearest-first. Use hits[0] for "what the user actually clicked on through the scene".

What each hit contains

hits[0] = {
  object:    Mesh,              // the hit mesh
  distance:  3.42,              // camera-to-hit, world units
  point:     Vector3,           // hit position in world space
  face:      { a, b, c, normal }, // the triangle that was hit
  uv:        Vector2,           // texture coordinate at hit
  faceIndex: 214,
  instanceId: 7,                // if object is an InstancedMesh
}

point is the one you'll use most — placing decals, stickers, markers, or aiming the camera. uv is gold for texture-based interactions (painting, click zones on a texture).

Click detection — the right way

"Click" sounds simple. It isn't. You have to separate intent to click from drag to orbit.

let downAt = null;
let downPos = new THREE.Vector2();

canvas.addEventListener('pointerdown', (e) => {
  downAt = performance.now();
  downPos.set(e.clientX, e.clientY);
});

canvas.addEventListener('pointerup', (e) => {
  if (!downAt) return;
  const held   = performance.now() - downAt;
  const moved  = downPos.distanceTo(new THREE.Vector2(e.clientX, e.clientY));
  downAt = null;

  // Click = short press, tiny movement. Otherwise it's an orbit drag.
  if (held < 400 && moved < 4) {
    handleClick(e);
  }
});

4 pixels and 400ms are good defaults. Touch users move more; bump to 8 and 500 for mobile-first UIs.

Hover + click highlighting

Three ways to highlight a hovered/selected object, each in the demo dropdown:

The outline-pass version

import { EffectComposer } from 'three/addons/postprocessing/EffectComposer.js';
import { RenderPass }     from 'three/addons/postprocessing/RenderPass.js';
import { OutlinePass }    from 'three/addons/postprocessing/OutlinePass.js';

const composer = new EffectComposer(renderer);
composer.addPass(new RenderPass(scene, camera));

const outline = new OutlinePass(
  new THREE.Vector2(width, height), scene, camera
);
outline.visibleEdgeColor.set('#38bdf8');
outline.edgeStrength = 5;
outline.edgeThickness = 1.5;
composer.addPass(outline);

// When you pick a new object:
outline.selectedObjects = [hoveredMesh];

// In the loop, use composer instead of renderer:
composer.render();

We'll go deeper on post-processing in Article 07. The demo uses the simpler emissive-boost approach so you don't need a composer to see it.

Controls: OrbitControls and friends

Three.js ships six control classes in addons. The two you'll use in 95% of cases:

OrbitControls — the one you'll use most

import { OrbitControls } from 'three/addons/controls/OrbitControls.js';

const controls = new OrbitControls(camera, renderer.domElement);
controls.enableDamping = true;           // smooth inertia — use this
controls.dampingFactor = 0.08;
controls.minDistance = 1;
controls.maxDistance = 50;
controls.minPolarAngle = 0.1;                // prevent flipping over the top
controls.maxPolarAngle = Math.PI / 2 - 0.05; // keep above horizon
controls.target.set(0, 1, 0);

// In the loop:
controls.update();                           // required when damping is on

The damping call is the one everyone forgets. Without controls.update() in the animation loop, inertia never settles — the camera twitches once and stops.

Flying the camera to a target (the demo trick)

When you click a planet in the demo, we tween the orbit target and camera position. It's not a built-in, it's 15 lines:

let tween = null;

function flyTo(object) {
  const target = new THREE.Vector3();
  object.getWorldPosition(target);

  // Camera sits 2.5 units "back" from the target along current camera direction.
  const offset = camera.position.clone().sub(controls.target).normalize().multiplyScalar(2.5);
  const desiredCamera = target.clone().add(offset);

  tween = {
    t: 0,
    fromTarget: controls.target.clone(),
    toTarget:   target,
    fromCam:    camera.position.clone(),
    toCam:      desiredCamera,
  };
}

// In the loop:
if (tween) {
  tween.t = Math.min(1, tween.t + 0.04);
  const e = 1 - Math.pow(1 - tween.t, 3);   // ease-out cubic
  controls.target.lerpVectors(tween.fromTarget, tween.toTarget, e);
  camera.position.lerpVectors(tween.fromCam, tween.toCam, e);
  if (tween.t === 1) tween = null;
}

lerpVectors is Three.js's most underrated method. It rewrites the target vector to a linear interpolation between two others. Combined with an easing curve, it gives you cinematic camera moves in 4 lines.

Pointer events over mouse events

Never listen to mousedown, touchstart, and PointerEvent separately. Use pointer events only:

Pointer events unify mouse, touch, and pen, give you the same clientX/clientY API for all three, and expose pressure for pens. They've been stable since 2019 in every modern browser.

Preventing browser gestures

canvas.style.touchAction = 'none';   // block default touch pan/zoom

canvas.addEventListener('pointerdown', (e) => {
  canvas.setPointerCapture(e.pointerId);  // we keep getting events even if finger leaves
});

touchAction: none on the canvas alone (not the whole page) prevents browser pinch/pan from stealing events. setPointerCapture is essential for drag operations — without it, a fast drag off the canvas drops the pointer.

Selective picking: layers, raycast override, bounding pre-pass

By default the raycaster tests every triangle in every mesh under the root. With dense scenes, that's slow. Three strategies to make picking cheap:

1. Limit to a picking array

// Only planets are pickable, not background stars:
const pickable = scene.getObjectByName('Planets').children;
const hits = raycaster.intersectObjects(pickable, false);

2. Layers — per-camera visibility, also affects raycaster

const PICK_LAYER = 1;
raycaster.layers.set(PICK_LAYER);
mesh.layers.enable(PICK_LAYER);
// Only meshes that also enabled PICK_LAYER are tested.

3. BVH for huge scenes

For scenes with tens of thousands of triangles per mesh, the community package three-mesh-bvh precomputes a bounding volume hierarchy — picking goes from O(n) to O(log n). Drop-in replacement for BufferGeometry.raycast. Google it when you need it.

DragControls — drop-in drag-and-drop

import { DragControls } from 'three/addons/controls/DragControls.js';

const drag = new DragControls([cube, sphere], camera, renderer.domElement);

drag.addEventListener('dragstart', () => orbit.enabled = false);
drag.addEventListener('dragend',   () => orbit.enabled = true);

The disable/enable toggle is critical — both controls own the canvas, and without it dragging an object also orbits the camera.

TransformControls — the gizmo

import { TransformControls } from 'three/addons/controls/TransformControls.js';

const tc = new TransformControls(camera, renderer.domElement);
tc.attach(mesh);
tc.setMode('translate');   // 'translate' | 'rotate' | 'scale'
scene.add(tc);

tc.addEventListener('dragging-changed', (e) => {
  orbit.enabled = !e.value;   // don't orbit while dragging the gizmo
});

Gives you Blender-style drag handles for translate / rotate / scale. Tab through the modes with a key listener and you've built a minimal scene editor.

Common first-time pitfalls

• Raycaster hits nothing. Y sign in NDC conversion is wrong. Or you're passing scene.children without true for recursive.
• Highlighting flickers. Running traverse-based highlight every frame without tracking the previous target. Store prevHover and only update on change.
• Click fires after a drag. You're not separating click from drag. Check the moved < 4px / held < 400ms pattern above.
• OrbitControls with damping doesn't feel smooth. You forgot controls.update() in the loop.
• Touch drag breaks on mobile. canvas.style.touchAction = 'none' is missing.
• TransformControls won't grab. You didn't scene.add(tc). The gizmo itself is a 3D object and needs to be in the scene.

Exercises

1. Add a right-click-to-delete behavior. On contextmenu event, raycast, and remove the hit mesh from the scene (don't forget dispose).
2. Build a measurement tool: click two points in space (use hit.point), draw a line between them with their distance labeled. Hint: Line + CSS2DRenderer for the label.
3. Add camera bookmarks: press 1–9 to save the current camera pose, shift+1–9 to fly back to it. Combines the lerp pattern with key events.

What's next

Article 07 — Post-Processing: EffectComposer. Bloom, SSAO, depth-of-field, the outline pass we deferred to next time, and a custom shader pass. The cosmetics article.