Expert Picks: The Must-Reads for Leveraging LLMs in Your Design Process
Why do Large Language Models matter to designers?
For this week’s blog, I wanted to make a primer for designers who are interested in LLMs. The basic idea was to provide links to essential resources for anyone interested in knowing more but too overwhelmed to know where to start.
To do this I reached out to my network — colleagues, former colleagues, advisors, and collaborators — and asked them to share their favorite LLM articles, with a specific design focus.
From their response, I was able to bring together some compelling resources related LLMs for design — from a general overviews, practical and unexpected use cases, and business cases.
So, without further ado, here are the 10+ Must Reads on LLMs for Designers, picked by experts
#1 — Background on GPT Models in the AEC
Expert Pick — “GPT models in construction industry”
As a structural engineer, I wanted to share this article because it shows how the construction industry hasn’t really caught up with using AI tools like GPT models — technologies, which have been making waves in other areas such as business and education. It gives a lot of real examples of how AI models could be game-changers at various stages of construction projects, from planning to demolition. The findings demonstrate the urgent need for our industry to start paying attention and thinking about how to bring AI into their workflows.
#2 — How to Choose the best LLM model?
Expert Pick — “You don’t need hosted LLMs, do you?”
This is the best article I have found that explains differences, advantages and drawbacks of commercial (OpenAI, Google, Anthropic, etc) and open source (LLAMA, Mistral, etc) LLMs. It compares the most essential parameters such as cost, speed of development, privacy and others between these two distinctive LLM paradigms in plain English. This is a must-read for anyone who plans to start exploring the world of LLM and could be a valuable guide in the question of which model type to choose. — Aleksei
#3 — How to use LLMs for Architectural Design?
Expert Pick — Architext: Language-Driven Architectural Design
I chose this article because I think everyone should know about Theodoros Galanos, a pioneer of LLMs in the architecture space, even from the pre-ChatGPT era. His early work on LLMs was incredibly inspiring to me because it showcased the potential of LLMs to transform the way architects design more innovative, efficient, and sustainable buildings! — Rutvik
#4 — How Much Money can LLMs save?
Article: The economic potential of generative AI
If your company is thinking about investing in generative AI tools, McKinsey has produced an excellent report that outlines some practical uses of LLMs and the value they will generate for the economy. At least the start of your journey, this report can be a great guide for exploring potential investment in this rapidly evolving area because it offers high-level market metrics and quantifies the value and impact they can bring
#5 — How to use LLMs for Structural Design?
Expert Pick — Enhancing structural form-finding through a text-based AI engine coupled with computational graphic statics
As a structural engineer, I love this example because it shows that you can combine LLMs with graphic static s— providing a new way to freely explore structural designs through text queries. In my opinion, this methodology, can be extended to other use cases where designers are looking to innovate or personalize solutions — Alex
#6- How to use LLMs for human-centric design?
Expert Pick — Reading Users’ Minds from What They Say: An Investigation into LLM-based Empathic Mental Inference
Did you know LLMs have design super powers? Here, researchers explore using LLMs to figure out what users really need in order to make it easier to design products that resonate. Since understanding a small group of users doesn’t always give us the full picture for everyone else, the study uses a special method to see if LLMs can understand user needs as well as human designers do. Turns out, they can! This opens up a big opportunity to use LLMs to make designs that are more about the user
#7 — LLMs will be ‘multi-modal’ and capable of working with information beyond text
Expert Pick — Apple researchers achieve breakthroughs in multimodal AI as company ramps up investments
Looking ahead, Large language models (LLMs) will be extended to be trained on both text and images. By using different types of data, and developing new types of AI models, this article explains how researchers are leveraging LLMs to develop AI systems which can work with types of information beyond text.
#8 — Using LLMs to simplify working with technical documents.
Expert Pick — Interrogate your technical documentation using free and paid LLMs
Everyone interested in LLMs needs to also know about Retrieval Augmented Generation (RAG). This article shows how RAG is used in conjunction with LLMs to smoothly navigation within structural engineering code in Europe. This approach can also be applied in any industry where there is a vast amount of internal and external documentation and need to quickly navigate it! — Aleksei
#9 -Using LLMs to Spot Risks in Construction Contracts
Expert Pick — Construction contract risk identification based on knowledge-augmented language models
This is a great example of how LLMs can help save money by spotting risks in construction contracts — tasks that are currently performed manually and can lead to mistakes and overlooked problems. Given that disputes in construction can last years and cost millions, finding a smart way to manage these risks is crucial. Technology-wise, it also provides an interesting explanation on how we can make an existing LLM smarter by feeding it extra information without having to build one from scratch — Mussie
#10 — Using LLMs and Synthetic Data to Create Parametric 3D Models
Article: Can Large Language Models Generate 3D Shapes with Sharp Features and Parametric Control?
I am impressed by this study, which utilizes Large Language Models (LLMs) and synthetic data to generate parametric 3D models. For me, the use of synthetic data in this example, shows the potential of adapting and applying our existing code base to build exciting new AI products for our enterprise clients. — Camiel
11 — LLMs are limited in a lot of ways.
Yann LeCun on a vision to make AI systems learn and reason like animals and humans
With all the hype surrounding LLMs, we often give LLMs more credit that they are due; especially with respect to planning, reasoning, and executing actions. To do these things, AI will need to be able to perceive and collect meaningful information from an environment, and be able to recommend the best action for a given situation. Today, even the most powerful of LLMs accessible today lack these fundamental components. Industry decision-makers need to be aware of these limitations when considering investment in LLMs. — Houssame
The Road Ahead
I hope these expert-recommended readings offer a solid foundation on the world of LLMs, especially if you’re a designer, planner, or architect. We aim to inspire you with ideas on how these emerging tools can bring new opportunities to your practice, and at the very least, help you stay updated on some of the recent AI developments in our field.
Thank you to our contributors!
- Rutvik Deshpande; MIT Urban Risk Lab
- Aleksei Kondratenko DBF
- Mussie Birhane, ++39 SRL
- Camiel Weijenberg, DBF
- Houssame E Hsain, DBF
- Prof. Jianxi Luo, Data-Driven Innovation Lab
- Prof. Mark Tam, University of Hong Kong
Join the Conversation
What are your thoughts on the intersection of AI and design? Have you found any key resources on LLMs that have profoundly influenced your understanding or approach to design?
You are welcome to join the conversation, by sharing your experiences and insights in the comments below or reaching out to me directly sayjel@digitalbluefoam.com.
About DBF
DBF is a Singapore-based technology company and creator of a AI-powered platform for city and building design and development. www.digitalbluefoam.com
