Season 10 · Article 09

AI-Assisted 3D Authoring

Claude/GPT generates shaders. Chat-to-scene editing. Copilot for Three.js. The workflow where AI is your pair-programmer for 3D.

1. Shader generation via Claude/GPT

Prompt: "Write a fragment shader for an iridescent soap bubble effect on a sphere.
Input: vec3 vNormal, vec3 vViewDirection. Output: gl_FragColor."

Response:
precision highp float;
varying vec3 vNormal; varying vec3 vViewDirection;
void main() {
  float fresnel = pow(1.0 - dot(normalize(vNormal), vViewDirection), 2.0);
  vec3 iridescent = 0.5 + 0.5 * cos(fresnel * 6.28 + vec3(0,2,4));
  gl_FragColor = vec4(iridescent * (0.3 + fresnel), fresnel);
}

Usable output in seconds. Iterate via chat: "make it more subtle", "add noise", "add Fresnel power slider."

2. Chat-to-scene editing

An editor where the user says "add three red cubes to the left, make the ground blue" and the AI translates to code that mutates the scene.

// System prompt
You control a Three.js scene. Available tools:
- add(shape, color, position, scale)
- remove(id)
- setMaterial(id, color, roughness, metalness)
- setLight(type, color, position, intensity)

// User
"Add three glowing red cubes floating at y=2"

// LLM response
tool_calls: [
  { name: 'add', args: { shape: 'box', color: '#ff0000', position: [-2,2,0], scale: [0.5,0.5,0.5], emissive: '#aa0000' } },
  { name: 'add', args: { shape: 'box', color: '#ff0000', position: [0,2,0], ... } },
  { name: 'add', args: { shape: 'box', color: '#ff0000', position: [2,2,0], ... } },
]

3. Vibe-coding Three.js

Claude Code / Cursor / Continue: point it at your Three.js project. Type "make the camera shake on hit". It reads files, writes the effect.

Caveats: LLMs trained on older Three.js may write r100 API. Correct them in the prompt: "Use Three.js r170+ API."

4. Prompt patterns that work

• Show code examples from recent Three.js in the prompt. LLM mimics pattern.
• Specify the surrounding context: "This is a WebGPURenderer project using TSL."
• Iteratively refine: "The shader works but is too bright. Reduce emission 50%."
• Ask for step-by-step: "explain what you're doing" → catches bugs early.

5. Where LLMs struggle

• Fresh API versions: Three.js evolves fast. Check deprecations.
• TSL (very new): model knowledge uneven. Paste docs inline.
• Perf tradeoffs: generated code is rarely optimal.
• Weird coordinate systems: glTF Y-up vs Blender Z-up confuses models.

6. Templates for your team

# .cursor/rules or CLAUDE.md in your repo
- Target Three.js r170+ only.
- Use named imports from 'three'.
- For shaders, prefer TSL over ShaderMaterial for new code.
- Always include proper disposal on scene teardown.
- Physics uses Rapier.
- Assets via GLTFLoader + DRACOLoader + KTX2Loader.
- Ship code must pass: npm run lint, npm run test, npm run build.

LLMs respect this. Quality of output 2-3× higher.

7. Current landscape (2026)

• Claude Code: terminal pair-programmer.
• Cursor: VS Code + LLM in-editor.
• Continue: open source Cursor alt.
• v0 by Vercel: UI→React code (some 3D).
• ChatGPT Code Interpreter: runs code for debugging.

8. Code review

Always review AI output before committing. It hallucinates methods, mismixes APIs, introduces performance traps.

Good use: scaffold, iterate. Bad use: merge without reading.

9. Takeaways

• LLMs generate shaders, scene edits, Three.js boilerplate.
• Prompt with recent-version context + code examples.
• Function calling → chat-to-scene editing.
• CLAUDE.md / .cursor/rules standardize output.
• Review everything. LLMs hallucinate.
• Claude Code, Cursor, Continue — pick your workflow.