artiscene/

A workflow of agents for simulation-ready articulated scenes.

artiscene composes LLM-generated articulated assets into scenes authored in USD, assigns per-link physical materials and per-joint drives, and validates the result in NVIDIA Isaac Sim.

View the code How it works
● stage: /World/Sceneusdchecker: pass
5 stages3 LLM design, 1 deterministic authoring, 1 review
6 layersper furniture asset: interface → physics → physx → materials → geometry → instances
30 scenesevaluation corpus across 6 domains
$3.22mean generation cost per scene

Generated scenes

Each clip is a fly-through of a scene generated from the prompt shown beneath it.

The five-stage pipeline

The three design stages each run a designer–critic–planner loop, in which the designer proposes, the critic scores, and the planner accepts, revises, or rolls back. The authoring stage is deterministic Python; no language model writes USD directly.

Three panels: annotated 2D shell plan, grey blockout of furniture placement, final rendered scene
An office scene at three points in the pipeline: the shell plan, furniture placement, and the composed scene.
  1. 01ShellLLM

    Plans the room box, openings and lighting, and selects wall and floor PBR textures by CLIP retrieval from the ambientCG catalog.

  2. 02FurnitureLLM

    Retrieves articulated records from the Articraft catalog, places them, adds props and on-surface items.

  3. 03PhysicsLLM

    Assigns per-slot mass, per-link friction and material, and per-joint drive parameters.

  4. 04SceneAuthordeterministic

    URDF → layered USD assets → composed scene_isaac.usda, gated by usdchecker.

  5. 05SceneReviewLLM

    Critic and planner score the authored scene from rendered previews; a scene that fails review is marked as degraded rather than discarded.

Engineering detail

The pipeline authors the attributes PhysX reads, and verifies on the Isaac host that they arrive.

Isaac Sim stage panel showing prismatic drawer joints with drive force, damping and limits Rendered scene with a drawer articulated open

Joints and drives

Each drawer and door carries a prismatic or revolute joint with limits, drive force, and damping. The screenshot shows the composed scene open in Isaac Sim, where the drawer joints resolve as PhysicsPrismaticJoint prims with a 40 N linear drive.

Visual mesh of a cabinet Single convex hull approximation CoACD convex decomposition into colored pieces

Collision that matches the shape

From left to right: the visual mesh, a single convex hull, and the CoACD decomposition. A single hull seals a cabinet shut; convex decomposition preserves the cavities, so drawers can open under simulation.

Six-layer USD assets

Each furniture piece is authored as a layered USD package, so physics tuning and materials can be replaced without touching geometry.

slots/desk_with_drawer_00/
  interface.usda    # reference target, sublayers the rest
  physics.usda      # links, joints, drives, mass
  physx.usda        # PhysX tuning overlay
  materials.usda    # per-link physics + visual materials
  geometries.usdc   # binary mesh payloads
  instances.usda    # collision/visual gprims + bindings
Dozens of articulated assets from the Articraft catalog laid out in a grey render

The furniture catalog

Furniture comes from Articraft, a catalog of URDF records with semantic parts and physical joints, retrieved by category. A retrieval miss degrades to a fallback asset and is recorded in the session trace.

Scene in Isaac Sim with all drawers and doors closed Same scene with drawers and doors driven open by their joint drives

At rest and actuated in Isaac Sim

The same composed scene at rest and with its joint drives actuated: drawers slide and doors swing under PhysX, using the drive parameters authored by the Physics stage.

Results

The benchmark comprises thirty scenes across six domains (offices, kitchens, laboratories, robotics cells, studios and workshops), carrying on average 25 articulated assets and 106 movable joints each. Every scene passes through structural, spatial and simulation validation tiers.

Scenes passing usdchecker30 / 30
Movable joints with complete drive and limit specification3,170 / 3,170
Scenes loading in Isaac Sim 4.5 without fatal error30 / 30
Isaac Asset Validator, PhysicsRules and RobotRules pass30 / 30
Mean stable fraction under a fifteen-second gravity settle0.98
Finite joints actuated to 95% of catalog range2,102 / 2,319 (90.6%)
Mean generation cost per scene$3.22

A controlled incline test confirms that PhysX consumes the authored friction materials: a block carrying friction µ slides on a 30° incline only when µ falls below the slip threshold, at every value tested.

Failure modes

Two failure modes recur across the evaluation corpus.

Four top-down views of a kitchen scene across critic-planner revision rounds
A kitchen across four critic–planner revision rounds. Each revision raises the layout score by removing objects, and the room drifts toward emptiness; the critic rewards tidiness more reliably than function.
Top-down view of a large room containing mostly floor lamps and scattered stools
A seating area populated by floor lamps and stools. When the catalog lacks the requested furniture, the designer converges on its densest categories rather than reporting the gap.

Every session records a full trace: per-call LLM logs, per-stage checkpoints, and a cost ledger. Both cases above were diagnosed from those artifacts.

Acknowledgements

This project uses Articraft (Zhou et al., 2026) as its articulated furniture catalog, and NVIDIA Isaac Sim for simulation and validation. PBR textures come from ambientCG.