The researchers additionally expanded the system’s talents by including texture and coloration choices. For instance, utilizing an look immediate like “Electrical guitar in metallic purple,” LegoGPT can generate a guitar mannequin, with bricks assigned a purple coloration.
Testing with robots and people
To show their designs labored in actual life, the researchers had robots assemble the AI-created Lego fashions. They used a dual-robot arm system with pressure sensors to select up and place bricks in line with the AI-generated directions.
Human testers additionally constructed among the designs by hand, displaying that the AI creates genuinely buildable fashions. “Our experiments present that LegoGPT produces steady, numerous, and aesthetically pleasing Lego designs that align intently with the enter textual content prompts,” the staff famous in its paper.
When examined towards different AI programs for 3D creation, LegoGPT stands out by way of its deal with structural integrity. The staff examined towards a number of alternate options, together with LLaMA-Mesh and different 3D era fashions, and located its strategy produced the very best share of steady constructions.
A video of two robotic arms constructing a LegoGPT creation, supplied by the researchers.
Nonetheless, there are some limitations. The present model of LegoGPT solely works inside a 20×20×20 constructing area and makes use of a mere eight normal brick sorts. “Our technique at the moment helps a hard and fast set of generally used Lego bricks,” the staff acknowledged. “In future work, we plan to broaden the brick library to incorporate a broader vary of dimensions and brick sorts, corresponding to slopes and tiles.”
The researchers additionally hope to scale up their coaching dataset to incorporate extra objects than the 21 classes at the moment out there. In the meantime, others can actually construct on their work—the researchers launched their dataset, code, and fashions on their project website and GitHub.
