Simulation scenes of battery packs built from teardown photographs instead of CAD, for synthetic-only perception and robotic disassembly
Robotic EV-battery disassembly needs a perception model per battery design, and synthetic training needs a simulation scene per design. Production CAD is generally unavailable, so those scenes are hand-built, at about 100 to 138 hours per pack.
We build the scene from photographs instead. Each rigid component is photographed on a white background during the teardown, reconstructed from a single image, scaled to metric size with one physical measurement and assembled in NVIDIA Isaac Sim with the teardown photos as the layout reference. No CAD geometry or CAD-derived dimensions are used. Wiring and connectors are not reconstructed and are outside the scope of the comparison. Trained on synthetic renders only, the resulting model segments real images of two production packs with pack- and component-dependent differences to the CAD route, and provides the perception for a physical disassembly cell.
From the physical teardown to masks the robot can use, without any CAD geometry. Shown on the Audi e-tron.
PhotoEach component is photographed from several viewpoints on a white background during the teardown. Every reconstruction uses one photo; the candidate that best matches the real part is kept. One physical measurement sets its metric scale; further measurements only check it.
Assembled sceneThe parts are placed into a scene that mirrors the pack, with the teardown photos as the layout reference. The ordered teardown states define which parts are present at each stage.
Domain randomization of camera pose, lighting, environment and materials yields images with pixel-perfect masks. RF-DETR-Seg is trained on these alone.
The model segments real Zivid images of the pack. Each mask becomes a 4-DoF grasp through a class-specific geometric policy, tested in a disassembly cell.
Hands-on time: 5 h for the e-tron and 5.5 h for the Kona, including scene assembly, against 100 to 138 h for a hand-built CAD scene.
Drag to turn a pack. Slide the pink handle between the scene's appearance and its components coloured by class.
Rendered turntables of the simulation scenes; the classes are the scenes' semantic labels.
Hand-annotated Zivid frames of the two production packs. All models were trained on synthetic renders only. Large parts and fasteners are separate models: the fastener models run tiled over the full-resolution image. Selected examples; the full test sets are scored in the table below.
Real-image instance segmentation with RF-DETR-Seg, trained on synthetic data only: 80 Audi e-tron frames and 106 Hyundai Kona frames, with SAM-initialised, manually refined annotations.
| Pack and scope | CAD | Photo | |
|---|---|---|---|
| Audi e-tron, shared classes, AP50 | 0.890 | 0.961 | |
| Audi e-tron, fasteners (tiled), AP50 | 0.844 | 0.769 | |
| Hyundai Kona, shared classes, AP50 | 0.758 | 0.717 | |
| Kona module study, AP (Pixal3D / TRELLIS.2) | 0.829 | 0.815 / 0.702 |
Means over classes present in both scenes (e-tron 4, Kona 8); fasteners scored separately. Pack rows use RF-DETR-Seg medium, the module study the nano variant. Paired frame-level bootstrap intervals are in the paper.
Estimated from timestamped project records. CAD hours are active in-CAD interaction time for modelling and assembly (edits less than 15 minutes apart) and exclude simulator integration and initial material authoring. Photo-to-sim hours include photography and scene construction in the simulator but not reconstruction compute (about 13 min e-tron, 26 min Kona of unattended GPU time). The physical teardown is excluded from both routes.
Each predicted instance mask is converted by a class-specific geometric policy into a 4-DoF top-down grasp, back-projected with registered depth and transformed into the robot base frame. Execution is operator-gated.
The clip shows the full sequence on the real Hyundai Kona pack: the four polystyrene spacers and the four battery modules are picked from one pack and placed into a second one. The same pack position is run with each perception model; in this clip the one trained on the hand-built CAD scene runs first, then the one trained on the photo-to-sim scene. The inset shows what the robot saw: the overview capture, a close-up before each suction pick, and the module overview before the gripper picks.
| Measured on the robot camera | Photo-to-sim | CAD (reference) |
|---|---|---|
| Modules found in the overview capture | 80 / 80 | 80 / 80 |
| Polystyrene spacers found in the overview capture | 79 / 80 | 80 / 80 |
| Detections beyond the parts present | 0 | 0 |
The robot evaluation covers 20 positions of one Hyundai Kona pack, each run once with each model in alternating order, with the same capture settings and grasp policy; only the segmentation weights differ. It scores detection of the four modules and four polystyrene spacers in the overview capture, not pick success: execution is operator-gated, and runs aborted for planning or mechanical reasons were repeated and do not count against either model. The one miss was a fully visible spacer at position 17 that the photo-trained model left unmasked. The paper presents this as a proof of concept.
@inproceedings{isoaho2027photographs,
title = {From Photographs to Segmentation: {CAD}-Independent {3D} Scene Construction for {EV} Battery Disassembly},
author = {Isoaho, Jesse and R{\"a}tz, Raphael and El Ghazouali, Safouane and Ochsenbein, Christian},
booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
year = {2027}
}