In recent years, deep learning and computer vision have been applied to solve complex problems across many domains. In urban studies, these technologies have been instrumental in the development of smart cities and autonomous vehicles. However, a knowledge gap is present when it comes to informal urban regions in less developed countries. How can deep learning and artificial intelligence untangle the complexities of informality to advance urban modelling? In this paper, we introduce a framework for multipurpose realistic-dynamic urban modelling using deep convolutional neural networks. The purpose of the framework is twofold: (1) to sense and detect informality and slums in urban scenes from aerial and street-level images and (2) to detect pedestrian and transport modes. The model has been trained on images of urban scenes in cities across the globe. The framework shows strong validation performance in the identification of planned and unplanned regions, despite broad variations in the classified images. The algorithms of the URBAN-i model are coded in Python and the trained models can be applied to images of any urban setting, including informal settlements and slum regions.
Badrinarayanan V, Kendall A and Cipolla R (2017) SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 39(12): 2481–2495.
2.
BattyM (1984) Pseudo-dynamic Urban Models.
Cardiff:
University of Wales Institute of Science and Technology. Available at: http://www.casa.ucl.ac.uk/PhD.pdf.
3.
BattyM (1997)
Cellular automata and urban form: A primer.Journal of the American Planning Association63(2): 266–274.
4.
BattyM (2005)
Agents, cells, and cities: New representational models for simulating multiscale urban dynamics.Environment and Planning A37(8): 1373–1394.
5.
BattyM (2008)
The size, scale, and shape of cities.Science319(5864): 769–771.
6.
BattyMCouclelisHEichenM (1997) Urban Systems as Cellular Automata.
London:
SAGE Publications Sage UK.
7.
BattyMLongleyP (1994) Fractal Cities: A Geometry of Form and Function.
New York:
Academic Press.
8.
BattyMTorrensPM (2001) Modelling complexity: The limits to prediction. Cybergeo. Epub ahead of print 17 April 2017. DOI: 10.4000/cybergeo.1035.
9.
BattyMXieY (1996)
Preliminary evidence for a theory of the fractal city.Environment and Planning A28(10): 1745–1762.
10.
BettencourtL (2013)
The origins of scaling in cities.Science340(6139): 1438–1441.
11.
BettencourtLWestG (2010)
A unified theory of urban living.Nature467(7318): 912–913.
12.
Chen L-C, Papandreou G, Kokkinos I, et al. (2016) Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs. arXiv:1412.7062 [cs]. Available at: http://arxiv.org/abs/1412.7062 (accessed 23 April 2019).
13.
ChenL-CPapandreouGSchroffF; et al. (2017) Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv:1706.05587.
14.
CrandallDJBackstromLHuttenlocherD; et al. (2009) Mapping the world’s photos. In: Proceedings of the 18th international conference on World Wide Web, 2009, pp.761–770. New York: ACM.
15.
DahlGESainathTNHintonGE (2013) Improving deep neural networks for LVCSR using rectified linear units and dropout. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.8609–8613. New York: IEEE.
16.
de AlmeidaCMBattyMMonteiroAMV, et al. (2003)
Stochastic cellular automata modeling of urban land use dynamics: Empirical development and estimation.Computers, Environment and Urban Systems27(5): 481–509.
17.
deGroot M and Brown E (2017) A PyTorch Implementation of Single Shot MultiBox Detector. Available at: https://github.com/amdegroot/ssd.pytorch (accessed 22 February 2019).
18.
Demir I, Koperski K, Lindenbaum D, et al. (2018) DeepGlobe 2018: A Challenge to Parse the Earth through Satellite Images. arXiv:1805.06561 [cs]. Available at: http://arxiv.org/abs/1805.06561 (accessed 23 April 2019).
19.
DubeyANaikNParikhD, et al. (2016) Deep learning the city: Quantifying urban perception at a global scale. In: European conference on computer vision, pp.196–212. Berlin: Springer.
20.
EveringhamMEslamiSMAVan GoolL, et al. (2015)
The Pascal visual object classes challenge: A retrospective.International Journal of Computer Vision111(1): 98–136.
21.
Frankhauser P (1998) The fractal approach. A new tool for the spatial analysis of urban agglomerations. Population. New methodological approaches in the social sciences 10(1): 205–240.
22.
FriesenJTaubenböckHWurmM, et al. (2018)
The similar size of slums.Habitat International73: 79–88.
23.
GallagherAJoshiDYuJ, et al. (2009) Geo-location inference from image content and user tags. In: IEEE computer society conference on computer vision and pattern recognition workshops, CVPR workshops 2009, pp.55–62. New York: IEEE.
24.
GarcíaCMeana-LloriánDG-BusteloBC, et al. (2017)
Midgar: Detection of people through computer vision in the internet of things scenarios to improve the security in smart cities, smart towns, and smart homes.Future Generation Computer Systems76: 301–313.
25.
GirshickRDonahueJDarrellT; et al. (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: 2014. arXiv:1311.2524 [cs.CV]. Available at: https://arxiv.org/pdf/1311.2524.pdf.
26.
Glorot X, Bordes A and Bengio Y (2011) Deep sparse rectifier neural networks. In: Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, Fort Lauderdale, FL, USA., 2011, pp. 315–323. JMLR: W&CP 15.
27.
GuoYLiuYOerlemansA, et al. (2016)
Deep learning for visual understanding: A review.Neurocomputing187: 27–48.
28.
HaysJEfrosAA (2015) Large-scale image geolocalization. In: ChoiJFriedlandG (eds) Multimodal Location Estimation of Videos and Images.
Cham:
Springer International Publishing, pp.41–62.
29.
HeKZhangXRenS, et al. (2015) Deep residual learning for image recognition. arXiv:1512.03385v1. Available at: https://arxiv.org/pdf/1512.03385.pdf.
30.
HeppenstallAJCrooksATSeeLM, et al. (eds) (2012) Agent-Based Models of Geographical Systems.
Dordrecht:
Springer Netherlands.
31.
HintonGEOsinderoSTehY-W (2006)
A fast learning algorithm for deep belief nets.Neural Computation18(7): 1527–1554.
32.
Huang G, Liu Z, Maaten L van der, et al. (2017) Densely Connected Convolutional Networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, July 2017, pp. 2261–2269. IEEE.
33.
IsalgueACochHSerraR (2007)
Scaling laws and the modern city.Physica A: Statistical Mechanics and its Applications382(2): 643–649.
34.
Kingma DP and Ba J (2014) Adam: A Method for Stochastic Optimization. arXiv:1412.6980 [cs]. Available at: http://arxiv.org/abs/1412.6980 (accessed 23 April 2019).
KufferMPfefferKSliuzasR (2016)
Slums from space – 15 years of slum mapping using remote sensing.Remote Sensing8(6): 455.
37.
KufferMWangJNagenborgM, et al. (2018)
The scope of earth-observation to improve the consistency of the SDG slum indicator.ISPRS International Journal of Geo-Information7(11): 428.
38.
LawSPaigeBRussellC (2018) Take a look around: Using street view and satellite images to estimate house prices. arXiv:1807.07155 [cs, econ]. Available at: http://arxiv.org/abs/1807.07155 (accessed 17 October 2018).
39.
LeCunYBengioYHintonG (2015)
Deep learning.Nature521(7553): 436–444.
40.
LiXWangZJWYangCLJ; et al. (2017) FoveaNet: Perspective-aware urban scene parsing. arXiv preprint arXiv:1708.02421.
41.
LinGMilanAShenC, et al. (2017) RefineNet: Multi-path refinement networks for high-resolution semantic segmentation. In: IEEE conference on computer vision and pattern recognition (CVPR), 2017.
42.
LinT-YMaireMBelongieS; et al. (2014) Microsoft COCO: Common objects in context. arXiv:1405.0312 [cs]. Available at: http://arxiv.org/abs/1405.0312 (accessed 30 May 2018).
43.
LiuWAnguelovDErhanD, et al. (2016) SSD: Single shot multibox detector. In: European conference on computer vision, pp.21–37. Berlin: Springer.
44.
Long J, Shelhamer E and Darrell T (2015) Fully convolutional networks for semantic segmentation. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, June 2015, pp. 3431–3440. IEEE.
45.
LynchK (1960) The Image of the City.
Cambridge, MA:
The Technology Press and Harvard University Press.
46.
MahabirRCroitoruACrooksA, et al. (2018)
A critical review of high and very high-resolution remote sensing approaches for detecting and mapping slums: Trends, challenges and emerging opportunities.Urban Science2(1): 8.
47.
MbogaNPerselloCBergadoJ; et al. (2017) Detection of informal settlements from VHR satellite images using convolutional neural networks. In: 2017 IEEE international geoscience and remote sensing symposium (IGARSS). Available at: https://ieeexplore.ieee.org/document/8128166.
48.
MurcioRMasucciAPArcauteE, et al. (2015)
Multifractal to monofractal evolution of the London street network. Physical Review E92(6): 062130.
49.
NaikNKominersSDRaskarR, et al. (2017)
Computer vision uncovers predictors of physical urban change.Proceedings of the National Academy of Sciences114(29): 7571–7576.
50.
NaikNRaskarRHidalgoCA (2016)
Cities are physical too: Using computer vision to measure the quality and impact of urban appearance.American Economic Review106(5): 128–132.
51.
OlivaATorralbaA (2006) Building the gist of a scene: The role of global image features in recognition. In: S. Martinez-Conde, S.L. Macknik, L.M. Martinez, J.-M. Alonso, P.U. Tse (eds) Progress in Brain Research. Amsterdam: Elsevier, pp.23–36.
52.
PengCZhangXYuG; et al. (2017) Large kernel matters–improve semantic segmentation by global convolutional network. arXiv preprint arXiv:1703.02719.
53.
PerselloCSteinA (2017)
Deep fully convolutional networks for the detection of informal settlements in VHR images.IEEE Geoscience and Remote Sensing Letters14(12): 2325–2329.
54.
QuackTLeibeBVan GoolL (2008) World-scale mining of objects and events from community photo collections. In: Proceedings of the 2008 international conference on content-based image and video retrieval, pp.47–56. New York: ACM.
55.
RenSHeKGirshickR, et al. (2016) Faster R-CNN: Towards real-time object detection with region proposal networks. In: 2016. arXiv:1506.01497v3.
56.
RussakovskyODengJSuH, et al. (2015)
ImageNet large scale visual recognition challenge.International Journal of Computer Vision115(3): 211–252.
57.
SalessesPSchechtnerKHidalgoCA (2013)
The collaborative image of the city: Mapping the inequality of urban perception.PLoS ONE8(7): e68400.
SchererDMüllerABehnkeS (2010) Evaluation of pooling operations in convolutional architectures for object recognition. In: International conference on artificial neural networks, 2010, pp.92–101. Berlin: Springer.
60.
SimonyanKZissermanA (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.
61.
SrivastavaNHintonGKrizhevskyA, et al. (2014)
Dropout: A simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research15(1): 1929–1958.
62.
StasollaMGambaP (2007)
Exploiting spatial patterns for informal settlement detection in arid environments using optical spaceborne data. International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences36(3): W49A.
Zhao H, Shi J, Qi X, et al. (2017) Pyramid Scene Parsing Network. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, July 2017, pp. 6230–6239. IEEE. DOI: 10.1109/CVPR.2017.660.
69.
ZhouBLiuLOlivaA, et al. (2014) Recognizing city identity via attribute analysis of geo-tagged images. In: European conference on computer vision, pp.519–534. Berlin: Springer.
