Published on November 17, 2023, 3:40 am

“Transforming Designs Into Code: Tldraw’S ‘Make It Real’ Brings Ai To Software Development”

On Wednesday, a collaborative whiteboard app maker called “tldraw” caused quite a stir in the online community with the release of their new feature, “Make it Real.” This innovative feature utilizes AI technology to bring user-drawn software designs to life. By leveraging OpenAI’s GPT-4V API, tldraw’s “Make it Real” can transform vector drawings into functional Tailwind CSS and JavaScript web code. The result is the ability to replicate user interfaces and even create simple implementations of games like Breakout.

The response to this new feature has been overwhelmingly positive. Designers such as Kevin Cannon expressed their amazement at the creation of functioning sliders that rotate objects on screen, an interface for changing object colors, and even a working game of tic-tac-toe. Other users followed suit, showcasing their own creations such as clones of Breakout, dial clocks that tick, snake games, and even visual state charts.

If you’re interested in experimenting with this groundbreaking functionality yourself, you can try out a live demo of Make It Real online. However, it’s worth noting that running the demo requires an API key from OpenAI. While this may be a small inconvenience for some users, it’s important to exercise caution as providing your API key can pose security risks. If your key were to fall into the wrong hands, unauthorized access could lead to substantial charges since OpenAI bills based on data usage.

Tldraw was developed by Steve Ruiz in London and is an open source collaborative whiteboard tool. It offers users an infinite canvas for drawing, text input, and media integration—all without requiring a login. Launched in 2021 with $2.7 million in seed funding and support from GitHub sponsors, Tldraw has quickly gained traction among creative professionals looking for an intuitive digital workspace. The integration of OpenAI’s GPT-4V API into Tldraw came about with the help of Sawyer Hood’s design prototype called “draw-a-ui.”

At the heart of this impressive AI-driven functionality is the GPT-4V, an advanced version of OpenAI’s large language model. This specially designed model can interpret visual images and use them as prompts for generating code. AI expert Simon Willison explains that “Make it Real” works by generating a base64-encoded PNG of the drawn components and passing it to GPT-4V Vision along with a system prompt. The prompt provides instructions for converting the image into functioning code using Tailwind.

The possibilities presented by GPT-4V and its integration into tools like Tldraw seem nearly endless. As more individuals experiment with this technology and combine it with other frameworks, we can expect to witness more novel applications of OpenAI’s vision-parsing capabilities in the near future. One such example is a developer who recently utilized the GPT-4V API to create real-time narration of a video feed using a simulated AI-generated David Attenborough voice—an impressive achievement in itself.

Looking ahead, it appears that this combination of AI technology and intuitive design tools could represent a significant shift in software development and interface design. With “Make it Real,” creating functional prototypes may become as simple as visual mock-ups, allowing AI models to handle the rest. As developer Michael Dubakov expressed when showcasing his own Make It Real creation, “OK, @tldraw is officially insane. It is really interesting where we end up in 5 years… I can’t keep up with innovation pace anymore.”

In conclusion, tldraw’s new feature “Make it Real” demonstrates the immense potential of generative AI technology in simplifying software development processes. With its ability to transform vector drawings into functional code, this innovative tool opens up new possibilities for designers and developers alike. As we continue to explore and harness the power of generative AI, we can anticipate even more groundbreaking applications in the near future.

Share.

Comments are closed.