3D Physics Visualization Skill

Expert guidance for building high-performance 3D physics simulations using Three.js, React Three Fiber, Drei, and react-three/rapier. This skill provides comprehensive workflows for both client-side physics and backend-driven physics visualization architectures.

When to Use This Skill

Activate this skill when the user asks about:

•Creating 3D scenes, environments, or visualizations with Three.js or React Three Fiber
•Adding physics simulations (gravity, collisions, rigid bodies, joints)
•Building backend-driven physics visualization (backend computes, frontend renders)
•Working with WebGL, 3D graphics, or rendering pipelines
•Optimizing 3D performance (frame rates, memory management, LOD)
•Implementing camera controls, lighting, shadows, or materials
•Using Drei helpers (OrbitControls, Environment, Html, etc.)
•Debugging Three.js applications or physics behavior
•Setting up real-time synchronization via WebSocket for physics state
•Creating interactive 3D objects, animations, or user interactions

Architecture Patterns

Pattern 1: Backend-Driven Physics (Recommended for Complex Simulations)

Use when:

•Physics calculations are computationally intensive
•Need server-side validation or multiplayer synchronization
•Want to reduce client-side CPU/GPU load
•Building production-grade physics simulation platforms

Architecture:

code

Backend (C#, Node.js, Python)         Frontend (React Three Fiber)
├── Physics Engine                    ├── Three.js Scene
│   ├── Rapier/PhysX/Bullet          │   ├── Camera & Lights
│   ├── Collision Detection          │   └── Visual Objects
│   └── State Updates                 ├── WebSocket Client
└── WebSocket Server ────────────────>└── State Interpolation

Key Components:

•Backend: Runs physics simulation, sends state updates via WebSocket
•Frontend: Receives state, renders visuals, interpolates between updates
•No @react-three/rapier on frontend (rendering only)

See backend-driven-workflow.md for detailed implementation.

Pattern 2: Client-Side Physics (Simple Interactions)

Use when:

•Simple physics interactions (drag, drop, bounce)
•Offline/standalone applications
•Prototyping or demos
•Limited number of physics objects (<50)

Architecture:

code

Frontend (React Three Fiber + Rapier)
├── Three.js Scene
├── Physics World (@react-three/rapier)
├── Rigid Bodies & Colliders
└── User Interactions

Key Components:

•All physics computed in browser using WebAssembly
•Direct integration with React Three Fiber

Prerequisites

Required Dependencies

json

{
  "dependencies": {
    "three": "^0.160.0",
    "@react-three/fiber": "^8.15.0",
    "@react-three/drei": "^9.92.0",
    "@react-three/rapier": "^1.2.0"
  }
}

Installation

bash

npm install three @react-three/fiber @react-three/drei @react-three/rapier

Environment Setup

Ensure Next.js or React app is configured for:

•ES modules support
•WebGL context (automatic in modern browsers)
•WebSocket support (for backend-driven approach)

Core Workflows

Workflow 1: Create Basic 3D Scene

Steps:

•

Set up Canvas (React Three Fiber root component)

tsx

import { Canvas } from '@react-three/fiber';

export function Scene() {
  return (
    <Canvas>
      {/* 3D content here */}
    </Canvas>
  );
}

•

Add Lighting

tsx

<Canvas>
  <ambientLight intensity={0.5} />
  <directionalLight position={[10, 10, 5]} intensity={1} />
</Canvas>

•

Create 3D Objects

tsx

function Box() {
  return (
    <mesh>
      <boxGeometry args={[1, 1, 1]} />
      <meshStandardMaterial color="orange" />
    </mesh>
  );
}

•

Add Camera Controls

tsx

import { OrbitControls } from '@react-three/drei';

<Canvas>
  <OrbitControls />
  {/* objects */}
</Canvas>

Workflow 2: Add Client-Side Physics

Steps:

•

Wrap scene with Physics provider

tsx

import { Physics } from '@react-three/rapier';

<Canvas>
  <Physics gravity={[0, -9.81, 0]}>
    {/* physics objects */}
  </Physics>
</Canvas>

•

Create Rigid Bodies

tsx

import { RigidBody } from '@react-three/rapier';

<RigidBody position={[0, 5, 0]} colliders="cuboid">
  <mesh>
    <boxGeometry args={[1, 1, 1]} />
    <meshStandardMaterial color="blue" />
  </mesh>
</RigidBody>

•

Add Ground Plane

tsx

<RigidBody type="fixed" position={[0, 0, 0]}>
  <mesh rotation={[-Math.PI / 2, 0, 0]}>
    <planeGeometry args={[100, 100]} />
    <meshStandardMaterial color="green" />
  </mesh>
</RigidBody>

•

Enable Debug Visualization

tsx

<Physics debug>
  {/* See collision shapes */}
</Physics>

Workflow 3: Backend-Driven Physics Visualization

Steps:

•

Create Backend Communication Service

tsx

// services/SimulationService.ts
export class SimulationService {
  private ws: WebSocket;
  
  connect() {
    this.ws = new WebSocket('ws://localhost:5195/physics');
    this.ws.onmessage = this.handleStateUpdate;
  }
  
  private handleStateUpdate = (event: MessageEvent) => {
    const state = JSON.parse(event.data);
    this.onUpdate?.(state);
  };
}

•

Create State Synchronization Hook

tsx

function usePhysicsSync(simulationId: string) {
  const [objects, setObjects] = useState([]);
  
  useEffect(() => {
    const service = new SimulationService();
    service.connect();
    service.onUpdate((state) => setObjects(state.objects));
    return () => service.disconnect();
  }, [simulationId]);
  
  return { objects };
}

•

Render Visual Objects (No Physics)

tsx

function VisualObject({ position, rotation }) {
  return (
    <mesh position={position} rotation={rotation}>
      <boxGeometry args={[1, 1, 1]} />
      <meshStandardMaterial color="red" />
    </mesh>
  );
}

•

Add Smooth Interpolation

tsx

function InterpolatedObject({ targetPosition, targetRotation }) {
  const meshRef = useRef();
  
  useFrame(() => {
    meshRef.current.position.lerp(targetPosition, 0.1);
    meshRef.current.quaternion.slerp(targetQuaternion, 0.1);
  });
  
  return <mesh ref={meshRef}>{/* geometry */}</mesh>;
}

See complete examples in templates.

Workflow 4: Optimize Performance

Steps:

•

Enable Shadows Selectively

tsx

<Canvas shadows>
  <mesh castShadow>
    {/* Only important objects */}
  </mesh>
</Canvas>

•

Use Level of Detail (LOD)

tsx

import { Detailed } from '@react-three/drei';

<Detailed distances={[0, 20, 50]}>
  <HighPolyModel />
  <MediumPolyModel />
  <LowPolyModel />
</Detailed>

•

Dispose Resources Properly

tsx

useEffect(() => {
  return () => {
    geometry.dispose();
    material.dispose();
    texture.dispose();
  };
}, []);

•
Profile Performance
- •Use React DevTools Profiler
- •Monitor FPS with <Stats /> from drei
- •Check Three.js memory usage

Common Drei Helpers

Helper	Use Case
`<OrbitControls />`	Camera rotation around target
`<Environment preset="sunset" />`	Quick HDRI lighting
`<Html />`	Embed HTML in 3D space
`<Text3D />`	3D text rendering
`<Float />`	Floating animation effect
`<ContactShadows />`	Cheap ground shadows
`<Stats />`	FPS/memory monitoring

Troubleshooting

Issue	Solution
Objects not visible	Check camera position, add lighting, verify mesh inside Canvas
Physics not working	Ensure `<Physics>` wraps objects, check RigidBody configuration
Poor performance	Reduce shadow-casting objects, implement LOD, dispose resources
WebSocket disconnects	Add reconnection logic, validate backend is running
Jerky animations	Implement interpolation with `useFrame`, check update frequency
Memory leaks	Dispose geometries/materials in cleanup, check for event listeners
Black screen	Check WebGL support, verify shaders compile, inspect console errors

Type Definitions

Essential TypeScript interfaces for physics state:

typescript

interface PhysicsObject {
  id: string;
  position: [number, number, number];
  rotation: [number, number, number];
  velocity: [number, number, number];
  type: 'dynamic' | 'fixed' | 'kinematic';
}

interface SimulationState {
  timestamp: number;
  objects: PhysicsObject[];
  isPaused: boolean;
}

References

Comprehensive Guides

•React Three Fiber Hooks Guide - Deep dive into useFrame, useThree, useLoader, and custom hooks
•Drei Components Comprehensive Guide - 30+ components with usage examples and best practices
•Rapier Physics Basics - Physics concepts, body types, colliders, joints, and client-side implementation
•Backend-Driven Workflow Guide - Complete architecture, WebSocket protocol, synchronization strategies
•Performance Optimization Guide - Geometry, materials, rendering, and memory optimization
•Three.js Common Patterns - Lighting, materials, animations, and interactions

Official Documentation

Quick Start Templates

•Basic 3D Scene - Minimal Three.js + R3F setup
•Client-Side Physics Demo - Simple physics playground
•Backend-Driven Viewer - WebSocket-based physics visualization
•Performance Optimized - LOD, shadows, memory management

Best Practices

•Always dispose Three.js resources - Prevent memory leaks
•Use useFrame for animations - Avoid state updates per frame
•Batch backend updates - Send multiple object states in one message
•Interpolate between states - Smooth 60 FPS even with 20 FPS backend updates
•Enable debug mode during development - <Physics debug> visualizes colliders
•Profile early and often - Use <Stats /> to catch performance issues
•Prefer declarative patterns - Use React Three Fiber patterns, not imperative Three.js code

Security Considerations

•Backend Validation: Validate all physics parameters on backend
•WebSocket Security: Use WSS (secure WebSocket) in production
•Rate Limiting: Prevent command spam from frontend
•State Validation: Validate all incoming state from backend before rendering

License

This skill is provided under the Apache License 2.0. See LICENSE.txt for complete terms.