Deep learning in design workflows: The elusive design pixel

Abstract

The recent wave of developments and research in the field of deep learning and artificial intelligence is causing the border between the intuitive and deterministic domains to be redrawn, especially in computer vision and natural language processing. As designers frequently invoke vision and language in the context of design, this article takes a step back to ask if deep learning’s capabilities might be applied to design workflows, especially in architecture. In addition to addressing this general question, the article discusses one of several prototypes, BIMToVec, developed to examine the use of deep learning in design. It employs techniques like those used in natural language processing to interpret building information models. The article also proposes a homogeneous data format, provisionally called a design pixel, which can store design information as spatial-semantic maps. This would make designers’ intuitive thoughts more accessible to deep learning algorithms while also allowing designers to communicate abstractly with design software.

Keywords

Associative logic creative processes deep learning embedding vectors BIMToVec homogeneous design data format design pixel idea persistence

Get full access to this article

View all access options for this article.

References

Kalay

YE.

Architecture’s new media: principles, theories, and methods of computer-aided design. Cambridge, MA: The MIT Press, 2004.

Coyne

Tools for exploring associative reasoning in design. In: McCullough

Mitchell

Purcell

(eds) The electronic design studio: architectural knowledge and media in the computer era. Cambridge, MA: The MIT Press, 1990, pp. 91–106.

Van der Maaten

Hinton

. Visualizing data using t-SNE. J Mach Learn Res 2008; 9: 2579–2605.

Metz

Visualizing with t-SNE, https://indico.io/blog/visualizing-with-t-sne/ (accessed 28 April 2018).

LeCun

Cortes

The MNIST database of handwritten digits, http://yann.lecun.com/exdb/mnist/ (1998, accessed 1 December 2017).

Goodfellow

Pouget-Abadie

Mirza

et al . Generative adversarial networks. arXiv:14062661, http://arxiv.org/abs/1406.2661 (2014, accessed 31 January 2017).

Gero

Maher

ML.

Mutation and analogy to support creativity in computer aided design. In: Schmitt

(ed.) CAAD Futures. Zurich: ETH, 1991, pp. 261–270.

Simon

. The sciences of the artificial. 3rd ed. Cambridge, MA: The MIT press, 1996.

Wolfram Alpha: Total words in English language, http://www.wolframalpha.com/input/?i=total+words+in+english (accessed 28 January 2018).

10.

Wolfram Alpha: Average word length in English, http://www.wolframalpha.com/input/?i=average+english+word+length (accessed 28 January 2018).

11.

Bengio

Ducharme

Vincent

et al . A neural probabilistic language model. J Mach Learn Res 2003; 3: 1137–1155.

12.

DURAARK. DURAARK Datasets, http://duraark.eu/data-repository/ (accessed 14 July 2018).

13.

Open IFC Model Repository, http://openifcmodel.cs.auckland.ac.nz/ (accessed 14 July 2018).

14.

OpenBIM, http://www.openbim.org/ (accessed 14 July 2018).

15.

Schnabel

Labutov

Mimno

et al . Evaluation methods for unsupervised word embeddings. In: Proceedings of the 2015 conference on empirical methods in natural language processing, Lisbon, 17–21 September 2015, pp. 298–307. Stroudsburg, PA: The association for computational linguistics.

16.

Russakovsky

Deng

et al . ImageNet large scale visual recognition challenge. arXiv:14090575, http://arxiv.org/abs/1409.0575 (2014, accessed 22 July 2018).

17.

Mahankali

Assessing the potential applications of deep learning in design. Ann Arbor: University of Washington, 2018.

18.

Statt

Google’s Al doodle bot will transform your crude drawings into glorious clip art, https://www.theverge.com/2017/4/11/15263434/google-ai-autodraw-doodle-bot-drawing-image-recognition (2017, accessed 29 January 2018).

19.

Kiros

Zhu

Salakhutdinov

et al . Skip-thought vectors. arXiv: 1506.06726, http://arxiv.org/abs/1506.06726 (2015, accessed 1 March 2018).

20.

Mikolov

Sutskever

Chen

et al . Distributed representations of words and phrases and their compositionality. arXiv:13104546, http://arxiv.org/abs/1310.4546 (2013, accessed 30 May 2015).

21.

Lipton

Berkowitz

Elkan

CA.

A critical review of recurrent neural networks for sequence learning. arXiv:150600019, http://arxiv.org/abs/1506.00019 (2015, accessed 30 April 2018).

22.

Bahdanau

Cho

Bengio

Neural machine translation by jointly learning to align and translate. arXiv:1409.0473, http://arxiv.org/abs/1409.0473 (2014, accessed 28 April 2018).