Generating conceptual landscape design via text-to-image generative AI model

Abstract

This study explores the integration of text-to-image generative AI, particularly Stable Diffusion, in conjunction with ControlNet and LoRA models in conceptual landscape design. Traditional methods in landscape design are often time-consuming and limited by the designer’s individual creativity, also often lacking efficiency in the exploration of diverse design solutions. By leveraging AI tools, we demonstrate a workflow that efficiently generates detailed and visually coherent landscape designs, including natural parks, city plazas, and courtyard gardens. Through both qualitative and quantitative evaluations, our results indicate that fine-tuned models produce superior designs compared to non-fine-tuned models, maintaining spatial consistency, control over scale, and relevant landscape elements. This research advances the efficiency of conceptual design processes and underscores the potential of AI in enhancing creativity and innovation in landscape architecture.

Keywords

Text-to-image generative AI conceptual design generative design stable diffusion landscape design

Get full access to this article

View all access options for this article.

References

Andreasen

Hansen

Cash

(2015) Conceptual Design: Interpretations, Mindset and Models. Berlin: Springer International Publishing. DOI: 10.1007/978-3-319-19839-2.

Aničin

Stojmenović

(2023) Bias analysis in stable diffusion and MidJourney models. In: Nandan Mohanty

Garcia Diaz

Satish Kumar

GAE

(eds) Intelligent Systems and Machine Learning. Nature Switzerland: Springer, 378–388. DOI: 10.1007/978-3-031-35081-8_32.

Bell

(2019) Elements of Visual Design in the Landscape. London: Routledge.

Bendel

(2023) Image synthesis from an ethical perspective. AI & Society. DOI: 10.1007/s00146-023-01780-4.

Booth

(2012) Residential Landscape Architecture. Hoboken, NJ: Prentice Hall. https://thuvienso.dau.edu.vn:88/handle/123456789/9001.

Boudier

Beltrán

Michelangelo

(2024) How to generate new versions of an original character? an application of LoRA and DreamBooth fine-tuning of diffusion models. https://hdl.handle.net/10230/68765.

Buxton

(2010) Sketching User Experiences: Getting the Design Right and the Right Design. Burlington, MA: Morgan Kaufmann.

Cao

Liu

, et al. (2023) A comprehensive survey of AI-generated content (AIGC): a History of generative AI from GAN to ChatGPT (arXiv:2303.04226). DOI: 10.48550/arXiv.2303.04226.

Chen

Zhao

Yao

, et al. (2023) Generative design of outdoor green spaces based on generative adversarial networks. Buildings 13(4): 1083. DOI: 10.3390/buildings13041083.

10.

Creswell

White

Dumoulin

, et al. (2018) Generative adversarial networks: an overview. IEEE Signal Processing Magazine 35(1): 53–65. DOI: 10.1109/MSP.2017.2765202.

11.

Danhaive

Mueller

(2021) Design subspace learning: structural design space exploration using performance-conditioned generative modeling. Automation in Construction 127: 103664. DOI: 10.1016/j.autcon.2021.103664.

12.

Dee

(2001) Form and Fabric in Landscape Architecture: A Visual Introduction. London: Taylor & Francis.

13.

Derevyanko

Zalevska

(2023) Comparative analysis of neural networks Midjourney, stable diffusion, and DALL-E and ways of their implementation in the educational process of students of design specialities. Scientific Bulletin of Mukachevo State University. Series “Pedagogy and Psychology 9: 36. DOI: 10.52534/msu-pp3.2023.36.

14.

Niyato

, et al. (2023, March 23). Diffusion-based reinforcement learning for edge-enabled AI-generated content services. https://arxiv.org/abs/2303.13052v3.

15.

Fernberg

Chamberlain

(2023) Artificial intelligence in landscape architecture: a literature review. Landscape Journal 42(1): 13–35. DOI: 10.3368/lj.42.1.13.

16.

Fui-Hoon Nah

Zheng

Cai

, et al. (2023) Generative AI and ChatGPT: applications, challenges, and AI-human collaboration. Journal of Information Technology Case and Application Research 25(3): 277–304. DOI: 10.1080/15228053.2023.2233814.

17.

Gonzalez

(2018) Deep convolutional neural networks [lecture notes]. IEEE Signal Processing Magazine 35(6): 79–87. DOI: 10.1109/MSP.2018.2842646.

18.

Goodfellow

Pouget-Abadie

Mirza

, et al. (2014) Generative adversarial nets. Advances in Neural Information Processing Systems 27. Available at: https://proceedings.neurips.cc/paper_files/paper/2014/hash/5ca3e9b122f61f8f06494c97b1afccf3-Abstract.html.

19.

Hakimshafaei

(2023) Survey of Generative AI in Architecture and Design. MS. Santa Cruz: University of California. https://www.proquest.com/docview/2812330639/abstract/4B3973AF789249C9PQ/1.

20.

Han

Chen

(2024) Automatic generation of standard nursing unit floor plan in general hospital based on stable diffusion. Buildings 14(9): 2601. DOI: 10.3390/buildings14092601.

21.

Hanafy

(2023) Artificial intelligence’s effects on design process creativity: “A study on used A.I. text-to-image in architecture”. Journal of Building Engineering 80: 107999. DOI: 10.1016/j.jobe.2023.107999.

22.

Holden

(1984) Basic elements of landscape architectural design. Landscape and Planning 11(3): 260–261.

23.

Hoşer

Köymen

(2023) Analysis of text-to-image artificial intelligence Systems in terms of contribution to interior coloring. Bilişim Teknolojileri Dergisi 16(4): 275–283. DOI: 10.17671/gazibtd.1252993.

24.

Huang

Zhang

Yao

, et al. (2022) Accelerated environmental performance-driven urban design with generative adversarial network. Building and Environment 224: 109575. DOI: 10.1016/j.buildenv.2022.109575.

25.

Huynh-Thu

Ghanbari

(2008) Scope of validity of PSNR in image/video quality assessment. Electronics Letters 44(13): 800–801.

26.

Jiang

Webster

, et al. (2024) Automated site planning using CAIN-GAN model. Automation in Construction 159: 105286. DOI: 10.1016/j.autcon.2024.105286.

27.

Kim

Yang

Min

(2024) DALS: diffusion-based artistic landscape sketch. Mathematics 12(2): 238. DOI: 10.3390/math12020238.

28.

Kingma

Welling

(2022) Auto-encoding variational Bayes (arXiv:1312.6114). DOI: 10.48550/arXiv.1312.6114.

29.

Kotturi

Anderson

Ford

, et al. (2024) Deconstructing the veneer of simplicity: Co-designing introductory generative AI workshops with local entrepreneurs. In Proceedings of the CHI conference on human factors in computing systems, Honolulu, HI, May 11–16, 2024: 1–16. DOI: 10.1145/3613904.3642191.

30.

Kulkarni

Shivananda

Kulkarni

, et al. (2023) Diffusion model and generative AI for images. In: Kulkarni

Shivananda

Kulkarni

Gudivada

(eds) Applied Generative AI for Beginners: Practical Knowledge on Diffusion Models, ChatGPT, and Other LLMs. New York, NY: Apress, 155–177. DOI: 10.1007/978-1-4842-9994-4_8.

31.

Kwon

, et al. (2023, October 2) DataInf: efficiently estimating data influence in LoRA-tuned LLMs and diffusion models. https://arxiv.org/abs/2310.00902v1.

32.

Lamac

(2023) Text-to-image AI as a tool for the designer’s ideation process. https://aaltodoc.aalto.fi/handle/123456789/125920.

33.

LeCun

Bengio

Hinton

(2015) Deep learning. Nature 521(7553): 436–444. DOI: 10.1038/nature14539.

34.

Lee

J-K

Yoo

Cha

(2024) Generative early architectural visualizations: incorporating architect’s style-trained models. Journal of Computational Design and Engineering 11: 40–59. DOI: 10.1093/jcde/qwae065.

35.

(2024) Generating daylight-driven architectural design via diffusion models (arXiv:2404.13353). DOI: 10.48550/arXiv.2404.13353.

36.

(2024) Sketch-to-Architecture: generative AI-aided architectural design (arXiv:2403.20186). https://arxiv.org/abs/2403.20186.

37.

Mandapuram

Gutlapalli

Bodepudi

, et al. (2018) Investigating the prospects of generative artificial intelligence. Asian Journal of Humanity, Art and Literature 5(2): 2. DOI: 10.18034/ajhal.v5i2.659.

38.

Norton

Brouwer

(2009) Chapter 4—the planning, design and construction process. In: Hessler

Lehner

NDM

(eds) Planning and Designing Research Animal Facilities. Cambridge, MA: Academic Press: 17–44. DOI: 10.1016/B978-0-12-369517-8.00004-9.

39.

Oppenlaender

(2022) The creativity of text-to-image generation. In Proceedings of the 25th International Academic Mindtrek Conference: 192–202. DOI: 10.1145/3569219.3569352.

40.

Paananen

Oppenlaender

Visuri

. (2023). Using text-to-image generation for architectural design ideation (arXiv:2304.10182). arXiv. DOI: 10.48550/arXiv.2304.10182.

41.

Phillips

Jiao

Clubb

(2024) Testing the capability of AI art tools for urban design. IEEE Computer Graphics and Applications 44(2): 37–45. DOI: 10.1109/MCG.2024.3356169.

42.

Pressman

(2012) Designing Architecture: The Elements of Process. London: Routledge.

43.

Rane

(2023) ChatGPT and similar generative artificial intelligence (AI) for smart industry: role, challenges and opportunities for industry 4.0, industry 5.0 and society 5.0. (SSRN Scholarly Paper 4603234) DOI: 10.2139/ssrn.4603234.

44.

Reid

(2007) From Concept to Form in Landscape Design. Hoboken, NJ: John Wiley & Sons.

45.

Rombach

Blattmann

Lorenz

, et al. (2022) High-Resolution Image Synthesis with Latent Diffusion Models: 10684–10695. https://openaccess.thecvf.com/content/CVPR2022/html/Rombach_High-Resolution_Image_Synthesis_With_Latent_Diffusion_Models_CVPR_2022_paper.html.

46.

Salter

Gann

(2003) Sources of ideas for innovation in engineering design. Research Policy 32(8): 1309–1324. DOI: 10.1016/S0048-7333(02)00119-1.

47.

Shaw

Goodchild

, et al. (2024) Human dynamics research in GIScience: challenges and opportunities. Computational Urban Science 4(1): 31.

48.

Smith

Hsu

Y-C

Zhang

, et al. (2023, April 12). Continual diffusion: continual customization of text-to-image diffusion with C-LoRA. https://arxiv.org/abs/2304.06027v1.

49.

Souter-Brown

(2014) Landscape and Urban Design for Health and Well-Being: Using Healing, Sensory and Therapeutic Gardens. London: Routledge.

50.

Stiles

(1994) Landscape theory: a missing link between landscape planning and landscape design? Landscape and Urban Planning 30(3): 139–149. DOI: 10.1016/0169-2046(94)90053-1.

51.

Stupariu

M-S

Cushman

Pleşoianu

A-I

, et al. (2022) Machine learning in landscape ecological analysis: a review of recent approaches. Landscape Ecology 37(5): 1227–1250. DOI: 10.1007/s10980-021-01366-9.

52.

Sun

Qiu

, et al. (2019) How to fine-tune BERT for text classification? In: Sun

Huang

, et al. (eds) Chinese Computational Linguistics. Berlin: Springer International Publishing: 194–206. DOI: 10.1007/978-3-030-32381-3_16.

53.

Suthersan

Horst

Schnobrich

, et al. (2016) Remediation Engineering: Design Concepts. 2nd edition. Boca Raton, FL: CRC Press.

54.

Thompson

Sorvig

(2000) Sustainable landscape construction: a guide to green building outdoors. https://trid.trb.org/view/680366.

55.

Wan

Shi

(2021) Research on urban renewal public space design based on convolutional neural network model. Security and Communication Networks 2021: 1–9. DOI: 10.1155/2021/9504188.

56.

Wang

Bovik

Sheikh

, et al. (2004) Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing 13(4): 600–612. DOI: 10.1109/TIP.2003.819861.

57.

Gan

Chen

, et al. (2023) AI-generated content (AIGC): a survey (arXiv:2304.06632). DOI: 10.48550/arXiv.2304.06632.

58.

Xiangli

Rao

, et al. (2021) BlockPlanner: city block generation with vectorized graph representation. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV): 5057–5066. DOI: 10.1109/ICCV48922.2021.00503.

59.

Yang

Zhang

Song

, et al. (2023) Diffusion models: a comprehensive survey of methods and applications. ACM Computing Surveys 56(4): 105:1–105:39. DOI: 10.1145/3626235.

60.

Duan

Peng

(2021) Spatiotemporal prediction of theft risk with deep inception-residual networks. Smart Cities 4(1): 204–216.

61.

(2022) MasterplanGAN: facilitating the smart rendering of urban master plans via generative adversarial networks. Environment and Planning B: Urban Analytics and City Science 49(3): 794–814.

62.

Jamonnak

Van Zandt

, et al. (2024) Developing campus digital twin using interactive visual analytics approach. Frontiers of Urban and Rural Planning 2(1): 9.

63.

Yee

(2012) Architectural Drawing: A Visual Compendium of Types and Methods. Hoboken, NJ: John Wiley & Sons.

64.

Yelavich

Adams

(2014) Design as Future-Making. London: Bloomsbury Publishing.

65.

Zhang

Agrawala

(2023). Adding conditional control to text-to-image diffusion models (arXiv:2302.05543). DOI: 10.48550/arXiv.2302.05543.

66.

Zhang

Isola

Efros

, et al. (2018) The unreasonable effectiveness of deep features as a perceptual metric. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition: 586–595. DOI: 10.1109/CVPR.2018.00068.

67.

Zhang

, et al. (2023) Text-to-image diffusion models in generative AI: a survey (arXiv:2303.07909). DOI: 10.48550/arXiv.2303.07909.

68.

Zhang

Tian

Zhang

, et al. (2024a) Aided design of bridge aesthetics based on stable diffusion fine-tuning. DOI: 10.48550/ARXIV.2409.15812.

69.

Zhang

Gambardella

, et al. (2024b) Taming stable diffusion for text to 360° panorama image generation. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR): 6347–6357. DOI: 10.1109/CVPR52733.2024.00607.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

1.55 MB