Those who work in the design, development, and management of cities are often limited by the scarcity of data. Particularly in the Global South, urban databases may be insufficient, out of date, or simply not available. However, digital technology is making it possible to fill gaps and build substantial datasets using “urban clues,” or attributes, gathered in high-resolution imagery by sky- and street-based cameras. Aided by machine learning, it is possible to detect specific building characteristics (purpose, condition, size, material, and construction)—yielding an array of geolocated details about the built environment. The resulting composite view can be made available, as we have done, in an open-source portal for use in urban management. The insights gained in this way may help address common urban management challenges, such as locating homes vulnerable to hazards such as flooding or earthquakes, identifying urban sprawl and informal housing, prioritizing infrastructure investments, and guiding public program support. This approach has been applied in Colombia, Guatemala, Indonesia, Mexico, Paraguay, Peru, St Lucia, and St Maarten.
AjamiAKufferMPerselloC, et al. (2019) Identifying a slums’ degree of deprivation from VHR images using convolutional neural networks. Remote Sensing11(11): 1282. DOI: 10.3390/rs11111282.
2.
AlbertAKaurJGonzalezMC (2017) Using convolutional networks and satellite imagery to identify patterns in urban environments at a large scale. Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Halifax NS Canada: Association for Computing Machinery, 1357–1366. DOI: 10.1145/3097983.3098070.
3.
BennettRMUngerE-MLemmenC, et al. (2021) Land administration maintenance: a review of the persistent problem and emerging fit-for-purpose solutions. Land10(5): 509. DOI: 10.3390/land10050509.
4.
BiljeckiFItoK (2021) Street view imagery in urban analytics and GIS: a review. Landscape and Urban Planning215: 104217. DOI: 10.1016/j.landurbplan.2021.104217.
5.
CaoRZhuJTuW, et al. (2018) Integrating aerial and street view images for urban land use classification. Remote Sensing10(10): 1553. DOI: 10.3390/rs10101553.
6.
ChenLYaoXLiuY, et al. (2020) Measuring impacts of urban environmental elements on housing prices based on multisource data—a case study of Shanghai, China. ISPRS International Journal of Geo-Information9(2): 106. DOI: 10.3390/ijgi9020106.
7.
CinnamonJJahiuL (2021) Panoramic street-level imagery in data-driven urban research: a comprehensive global review of applications, techniques, and practical considerations. ISPRS International Journal of Geo-Information10(7): 471. DOI: 10.3390/ijgi10070471.
8.
ClayG (1973) Close-up: How to Read the American City. Chicago: The University of Chicago Press.
9.
DakinKXieWParkinsonS, et al. (2020) Built environment attributes and crime: an automated machine learning approach. Crime Science9(1): 12. DOI: 10.1186/s40163-020-00122-9.
10.
DengCMaJ (2015) Viewing urban decay from the sky: a multi-scale analysis of residential vacancy in a shrinking U.S. city. Landscape and Urban Planning141: 88–99. DOI: 10.1016/j.landurbplan.2015.05.002.
11.
DingJZhangJZhanZ, et al. (2022) A precision efficient method for collapsed building detection in post-earthquake UAV images based on the improved NMS algorithm and Faster R-CNN. Remote Sensing14: 663, MDPI AG. DOI: 10.3390/rs14030663.
12.
ElaminAEl-RabbanyA (2022) UAV-based multi-sensor data fusion for urban land cover mapping using a Deep Convolutional Neural Network. Remote Sensing14: 4298, MDPI AG. DOI: 10.3390/rs14174298.
13.
EstevaAKuprelBNovoaRA, et al. (2017) Dermatologist-level classification of skin cancer with deep neural networks. Nature542(7639): 115–118. DOI: 10.1038/nature21056.
14.
FuXJiaTZhangX, et al. (2019) Do street-level scene perceptions affect housing prices in Chinese megacities? An analysis using open access datasets and deep learning. PLoS One14(5): e0217505. DOI: 10.1371/journal.pone.0217505.
15.
GlaeserELKominersSDLucaM, et al. (2018) Big data and big cities: the promises and limitations of improved measures of urban life. Economic Inquiry56(1): 114–137. DOI: 10.1111/ecin.12364.
16.
GonzalezDRueda-PlataDAcevedoAB, et al. (2020) Automatic detection of building typology using deep learning methods on street level images. Building and Environment177: 106805. DOI: 10.1016/j.buildenv.2020.106805.
17.
GrippaTGeorganosSZarouguiS, et al. (2018) Mapping urban land use at street block level using OpenStreetMap, remote sensing data, and spatial metrics. ISPRS International Journal of Geo-Information7(7): 246. DOI: 10.3390/ijgi7070246.
18.
HeKGkioxariGDollárP, et al. (2017) Mask R-CNN. 2017 IEEE International Conference on Computer Vision (ICCV)2980–2988. DOI: 10.1109/ICCV.2017.322.
19.
HermosillaTPalomar-VázquezJBalaguer-BeserÁ, et al. (2014) Using street based metrics to characterize urban typologies. Computers, Environment and Urban Systems44: 68–79. DOI: 10.1016/j.compenvurbsys.2013.12.002.
20.
HoffmannEJWangYWernerM, et al. (2019) Model fusion for building type classification from aerial and street view images. Remote Sensing11(11): 1259. DOI: 10.3390/rs11111259.
21.
IbrahimMRHaworthJChengT (2021) From urban scenes to mapping slums, transport modes, and pedestrians in cities using deep learning and computer vision. Environment and Planning B: Urban Analytics and City Science48(1): 76–93. DOI: 10.1177/2399808319846517.
22.
JacobsJ (1961) The Death and Life of Great American Cities. New York: Random House.
23.
JacobsAB (1985) Looking at Cities. Cambridge: Harvard University Press.
24.
KangJKörnerMWangY, et al. (2018) Building instance classification using street view images. ISPRS Journal of Photogrammetry and Remote Sensing145: 44–59. DOI: 10.1016/j.isprsjprs.2018.02.006.
25.
KochDDespotovicMSakeenaM, et al. (2018) Visual estimation of building condition with Patch-level ConvNets. In: Proceedings of the 2018 ACM Workshop on Multimedia for Real Estate Tech. New York: Association for Computing Machinery, 12–17. DOI: 10.1145/3210499.3210526.
26.
KoevaMGasukuOLengoiboniM, et al. (2021) Remote sensing for property valuation: a data source comparison in support of fair land taxation in Rwanda. Remote Sensing13(18): 3563. DOI: 10.3390/rs13183563.
27.
KufferMWangJThomsonDR, et al. (2021) Spatial information gaps on deprived urban areas (slums) in low-and-middle-income countries: a user-centered approach. Urban Science5: 72. DOI: 10.3390/urbansci5040072.
28.
LawSPaigeBRussellC (2019) Take a look around: using street view and satellite images to estimate house prices. ACM Transactions on Intelligent Systems and Technology10(5): 1–19. DOI: 10.1145/3342240.
29.
LeeSMaisonneuveNCrandallD, et al. (2015) Linking past to present: discovering style in two centuries of architecture. In: 2015 IEEE International Conference on Computational Photography (ICCP). Piscataway: IEEE, 1–10. DOI: 10.1109/ICCPHOT.2015.7168368.
30.
LindenthalTJohnsonEB (2021) Machine learning, architectural styles and property values. The Journal of Real Estate Finance and Economics. DOI: 10.1007/s11146-021-09845-1.
31.
LynchK (1960) The Image of the City. Cambridge: The MIT Press.
32.
MaskeliūnasRKatkevičiusAPlonisD, et al. (2022) Building façade style classification from UAV imagery using a pareto-optimized deep learning network. Electronics11: 3450, MDPI AG. DOI: 10.3390/electronics11213450.
33.
Movshovitz-AttiasYYuQStumpeMC, et al. (2015) Ontological supervision for fine grained classification of Street View storefronts. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 1693–1702. DOI: 10.1109/CVPR.2015.7298778.
34.
NingHYeXChenZ, et al. (2022) Sidewalk extraction using aerial and street view images. Environment and Planning B: Urban Analytics and City Science49(1): 7–22. DOI: 10.1177/2399808321995817.
35.
OwusuMKufferMBelgiuM, et al. (2021) Towards user-driven earth observation-based slum mapping. Computers, Environment and Urban Systems89: 101681. DOI: 10.1016/j.compenvurbsys.2021.101681.
36.
PratomoJKufferMMartinezJ, et al. (2017) Coupling uncertainties with accuracy assessment in object-based slum detections, case study: Jakarta, Indonesia. Remote Sensing9(11): 1164. DOI: 10.3390/rs9111164.
37.
RonnebergerOFischerPBroxT (2015) U-Net: convolutional networks for biomedical image segmentation. In: NavabNHorneggerJWellsW, et al. (eds) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015 Lecture Notes in Computer Science. Cham: Springer, Vol. 9351. DOI: 10.1007/978-3-319-24574-4_28.
38.
SariturkBKumbasarDSekerDZ (2023) Comparative analysis of different CNN models for building segmentation from satellite and UAV images. American Society for Photogrammetry and Remote Sensing89(2): 97–105. DOI: 10.14358/PERS.22-00084R2.
39.
SimonyanKZissermanA (2015) Very deep convolutional networks for large-scale image recognition. In: 3rd International Conference on Learning Representations (ICLR 2015). San Diego: Conference Track Proceedings, 1–14.
40.
StokesECSetoKC (2019) Characterizing and measuring urban landscapes for sustainability. Environmental Research Letters14(4): 045002. DOI: 10.1088/1748-9326/aafab8.
41.
SunMZhangFDuarteF (2021) Automatic building age prediction from street view images. In: 2021 7th IEEE International Conference on Network Intelligence and Digital Content (IC-NIDC). Piscataway: IEEE, 102–106. DOI: 10.1109/IC-NIDC54101.2021.9660554.
42.
UNDESA (United Nations Department of Economic and Social Affairs) (2019) World Urbanization Prospects: The 2018 Revision. New York: UNDESA.
43.
VanderhaegenSCantersF (2017) Mapping urban form and function at city block level using spatial metrics. Landscape and Urban Planning167: 399–409. DOI: 10.1016/j.landurbplan.2017.05.023.
44.
WangCAntosSETrivenoLM (2021) Automatic detection of unreinforced masonry buildings from street view images using deep learning-based image segmentation. Automation in Construction132: 103968. DOI: 10.1016/j.autcon.2021.103968.
45.
WangCAntosSEGosling-GoldsmithJ, et al. (2024) Assessing climate disaster vulnerability in Peru and Colombia using street view imagery: a pilot study. Buildings14(1): 14. https://www.mdpi.com/2075-5309/14/1/14.
46.
WangWShiYZhangJ, et al. (2023) Traditional village building extraction based on improved Mask R-CNN: a case study of Beijing, China. Remote Sensing15(10): 2616. DOI: 10.3390/rs15102616.
47.
WangYLiSTengF, et al. (2022) Improved Mask R-CNN for rural building roof type recognition from UAV high-resolution images: a case study in Hunan Province, China. Remote Sensing14: 265, MDPI AG. DOI: 10.3390/rs14020265.d.
48.
WegnerJDBransonSHallD, et al. (2016) Cataloging public objects using aerial and street-level images — urban trees. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 6014–6023. DOI: 10.1109/CVPR.2016.647.
49.
WilliamsSH (1954) Urban aesthetics: an approach to the study of the aesthetic characteristics of cities. Town Planning Review25(2): 95–113.
50.
WuS-SQiuXUseryEL, et al. (2009) Using geometrical, textural, and contextual information of land parcels for classification of detailed urban land use. Annals of the Association of American Geographers99(1): 76–98. DOI: 10.1080/00045600802459028.
51.
YuQWangCMcKennaF, et al. (2020) Rapid visual screening of soft-story buildings from street view images using deep learning classification. Earthquake Engineering and Engineering Vibration19(4): 827–838. DOI: 10.1007/s11803-020-0598-2.
52.
ZeppelzauerMDespotovicMSakeenaM, et al. (2018) Automatic prediction of building age from photographs. In: Proceedings of the 2018 ACM on International Conference on Multimedia Retrieval. New York: Association for Computing Machinery, 126–134. DOI: 10.1145/3206025.3206060.
53.
ZhangWLiWZhangC, et al. (2017) Parcel-based urban land use classification in megacity using airborne LiDAR, high resolution orthoimagery, and Google Street View. Computers, Environment and Urban Systems64: 215–228. DOI: 10.1016/j.compenvurbsys.2017.03.001.
54.
ZouSWangL (2021) Detecting individual abandoned houses from Google Street View: a hierarchical deep learning approach. ISPRS Journal of Photogrammetry and Remote Sensing175: 298–310. DOI: 10.1016/j.isprsjprs.2021.03.020.
