Urban climate zone classification using convolutional neural network and ground-level images

Abstract

Urban climate risks have a wide range of impacts on the health of more than 50% of the world’s population, which is a critical issue relating to climate change. To support urban climate study and categorise different urban environments and their atmospheric impacts in a consistent way, the Local Climate Zone (LCZ) classification scheme has been developed. The World Urban Database and Access Portal Tools project aims to map the LCZ of cities across the globe. However, previous classification approaches based on satellite images have limitations regarding the characterisation of three-dimensional features such as building heights. This study aims to apply convolutional neural networks to classify LCZ types based on ground-level images, which can provide more detail of the urban environments. Validation results have shown an overall accuracy of 69.6%. The new method outperformed previous satellite-based studies for classifying the LCZ types Compact Mid-rise, Sparsely Built, Heavy Industry, and Bare Rock or Paved.

Keywords

Urban climate Local Climate Zone convolutional neural network transfer learning Google Street View

Get full access to this article

View all access options for this article.

References

Alexander

Mills

(2014) Local climate classification and Dublin’s urban heat island. Atmosphere 5(4): 755–774.

Andrew

Wulder

Nelson

(2014) Potential contributions of remote sensing to ecosystem service assessments. Progress in Physical Geography 38(3): 328–353.

Arnds

Böhner

Bechtel

(2017) Spatio-temporal variance and meteorological drivers of the urban heat island in a European city. Theoretical and Applied Climatology 128(1–2): 43–61.

Bechtel

(2011) Multitemporal Landsat data for urban heat island assessment and classification of local climate zones. In: Urban Remote Sensing Event (JURSE), Munich, Germany, 11–13 April 2011, pp. 129–132. IEEE.

Bechtel

Daneke

(2012) Classification of local climate zones based on multiple earth observation data. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 5(4): 1191–1202.

Bechtel

Alexander

Beck

. (2019) Generating WUDAPT level 0 data–Current status of production and evaluation. Urban Climate 27: 24–45. Available at: https://www.sciencedirect.com/science/article/pii/S221209551830302X

Bechtel

Alexander

Böhner

. (2015) Mapping local climate zones for a worldwide database of the form and function of cities. ISPRS International Journal of Geo-Information 4(1): 199–219.

Bechtel

Demuzere

Sismanidis

. (2017) Quality of crowdsourced data on urban morphology—The Human Influence Experiment (HUMINEX). Urban Science 1(2): 15.

Bengio

(2012) Deep learning of representations for unsupervised and transfer learning. Unsupervised and Transfer Learning Challenges in Machine Learning 27: 17–36.

10.

Bengio

Bastien

Bergeron

. (2011) Deep learners benefit more from out-of-distribution examples. In: Geoffrey

Gordon

David

Dunson

Miroslav

Dudík

(eds) International conference on artificial intelligence and statistics. Fort Lauderdale, FL, USA, 11–13 April 2011, pp. 164–172.

11.

Caruana

(1995) Learning many related tasks at the same time with backpropagation. Advances in Neural Information Processing Systems 657–664. Available at: http://papers.nips.cc/paper/959-learning-many-related-tasks-at-the-same-time-with-backpropagation.pdf

12.

Chapman

(2008) An introduction to ‘upside-down’ remote sensing. Progress in Physical Geography: Earth and Environment 32(5): 529–542.

13.

Ching

Mills

Bechtel

. (2018) World urban database and access portal tools (WUDAPT), an urban weather, climate and environmental modeling infrastructure for the Anthropocene. Bulletin of the American Meteorological Society 99(9): 1907–1924.

14.

Ching

See

Mills

. (2014) WUDAPT: Facilitating advanced urban canopy modeling for weather, climate and air quality applications. In: 94th American Meterological Society Annual Meeting Georgia, USA, 2–6 February 2014.

15.

Coutts

Tapper

Beringer

. (2013) Watering our cities: The capacity for Water Sensitive Urban Design to support urban cooling and improve human thermal comfort in the Australian context. Progress in Physical Geography 37(1): 2–28.

16.

Crossman

Bryan

King

(2011) Contribution of site assessment toward prioritising investment in natural capital. Environmental Modelling & Software 26(1): 30–37.

17.

Danylo

See

Bechtel

. (2016) Contributing to WUDAPT: A local climate zone classification of two cities in Ukraine. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 9(5): 1841–1853.

18.

Donahue

Jia

Vinyals

. (2013) DeCAF: A deep convolutional activation feature for generic visual recognition. arXiv:1310.1531 32(1): 647–655.

19.

Dubey

Naik

Parikh

. (2016) Deep learning the city: Quantifying urban perception at a global scale. In: Leibe

Bastian

Matas

Jiri

Sebe

Nicu

Welling

Max

(eds) European conference on computer vision Amsterdam, The Netherlands, 11–14 October 2016, pp. 196–212. Springer.

20.

Field

Barros

Mach

. (2014) Climate change 2014: Impacts, adaptation, and vulnerability. Contribution of working group II to the fifth assessment report of the intergovernmental panel on climate change. IPCC Working Group II.

21.

Garcia

Delakis

(2002) A neural architecture for fast and robust face detection. In: Proceedings 16th international conference on pattern recognition, vol. 2, Quebec City, Canada, 11–15 August 2002, 44–47. IEEE.

22.

Geletič

Lehnert

(2016) GIS-based delineation of local climate zones: The case of medium-sized Central European cities. Moravian Geographical Reports 24(3): 2–12.

23.

Gerard

Petit

Smith

. (2010) Land cover change in Europe between 1950 and 2000 determined employing aerial photography. Progress in Physical Geography 34(2): 183–205.

24.

Giri

Pengra

Zhu

. (2007) Monitoring mangrove forest dynamics of the Sundarbans in Bangladesh and India using multi-temporal satellite data from 1973 to 2000. Estuarine, Coastal and Shelf Science 73(1–2): 91–100.

25.

Glaeser

Kominers

Luca

. (2018) Big data and big cities: The promises and limitations of improved measures of urban life. Economic Inquiry 56(1): 114–137.

26.

Gong

F-Y

Zeng

Z-C

Zhang

. (2018) Mapping sky, tree, and building view factors of street canyons in a high-density urban environment. Building and Environment 134: 155–167. DOI: 10.1016/j.buildenv.2018.02.042

27.

GRSS (2017) 2017 IEEE GRSS Data Fusion Contest results. Available at: https://www.grss-ieee.org/community/technical-committees/data-fusion/2017-ieee-grss-data-fusion-contest-results/. (accessed 23 November 2017).

28.

Heilig

(2012) World urbanization prospects: The 2011 revision. New York: United Nations, Department of Economic and Social Affairs (DESA), Population Division, Population Estimates and Projections Section.

29.

Heumann

(2011) Satellite remote sensing of mangrove forests: Recent advances and future opportunities. Progress in Physical Geography 35(1): 87–108.

30.

Jaroszweski

Hooper

Chapman

(2014) The impact of climate change on urban transport resilience in a changing world. Progress in Physical Geography 38(4): 448–463.

31.

Jiang

Deal

Pan

. (2017) Remotely-sensed imagery vs. eye-level photography: Evaluating associations among measurements of tree cover density. Landscape and Urban Planning 157: 270–281. DOI: 10.1016/j.landurbplan.2016.07.010

32.

Kaloustian

Bechtel

(2016) Local climatic zoning and urban heat island in Beirut. Procedia Engineering 169: 216–223. DOI: 10.1016/j.proeng.2016.10.026

33.

Karayev

Trentacoste

Han

. (2013) Recognizing image style. Available at: https://arxiv.org/abs/1311.3715

34.

LeCun

Bottou

Bengio

. (1998) Gradient-based learning applied to document recognition. Proceedings of the IEEE 86(11): 2278–2324.

35.

Ratti

(2018) Mapping the spatial distribution of shade provision of street trees in Boston using Google Street View panoramas. Urban Forestry & Urban Greening 31: 109–119. DOI: 10.1016/j.ufug.2018.02.013

36.

Zhang

. (2015) Assessing street-level urban greenery using Google Street View and a modified green view index. Urban Forestry & Urban Greening 14(3): 675–685.

37.

Long

Shelhamer

Darrell

(2015) Fully convolutional networks for semantic segmentation. CVPR (to appear). Available at: https://www.cv-foundation.org/openaccess/content_cvpr_2015/html/Long_Fully_Convolutional_Networks_2015_CVPR_paper.html

38.

Mills

(2009) Micro-and mesoclimatology. Progress in Physical Geography 33(5): 711–717.

39.

Mitraka

Del Frate

Chrysoulakis

. (2015) Exploiting earth observation data products for mapping local climate zones. In: Urban Remote Sensing Event (JURSE), Lausanne, Switzerland, 30 March-1 April 2015, pp. 1–4. IEEE.

40.

Oke

(1982) The energetic basis of the urban heat island. Quarterly Journal of the Royal Meteorological Society 108(455): 1–24.

41.

Osadchy

Cun

Miller

(2007) Synergistic face detection and pose estimation with energy-based models. The Journal of Machine Learning Research 8: 1197–1215.

42.

Pan

Yang

(2010) A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering 22(10): 1345–1359.

43.

Perera

Emmanuel

(2016) A Local Climate Zone based approach to urban planning in Colombo, Sri Lanka. Urban Climate. DOI: 10.1016/j.uclim.2016.11.006

44.

Ren

Wang

Cai

. (2016) The accuracy of LCZ maps generated by the world urban database and access portal tools (WUDAPT) method: A case study of Hong Kong. In: 4th international conference on countermeasure urban heat islands, Singapore, 30 May–1 June 2016, pp. 1–11.

45.

Richards

Edwards

(2017) Quantifying street tree regulating ecosystem services using Google street view. Ecological Indicators 77: 31–40. DOI: 10.1016/j.ecolind.2017.01.028

46.

Russakovsky

Deng

. (2015) ImageNet large scale visual recognition challenge. International Journal of Computer Vision 115(3): 211–252.

47.

See

Perger

Duerauer

. (2015) Developing a community-based worldwide urban morphology and materials database (WUDAPT) using remote sensing and crowdsourcing for improved urban climate modelling. In: Urban Remote Sensing Event (JURSE), 2015 Joint. IEEE.

48.

Sermanet

Eigen

Zhang

. (2013) Overfeat: Integrated recognition, localization and detection using convolutional networks. Available at: https://arxiv.org/abs/1312.6229

49.

Skarbit

Stewart

Unger

. (2017) Employing an urban meteorological network to monitor air temperature conditions in the ‘local climate zones’ of Szeged, Hungary. International Journal of Climatology. 37(1): 582–596.

50.

Souch

Grimmond

(2006) Applied climatology: urban climate. Progress in Physical Geography 30(2): 270–279.

51.

Stewart

Oke

(2012) Local climate zones for urban temperature studies. Bulletin of the American Meteorological Society 93(12): 1879–1900.

52.

Stewart

Oke

Krayenhoff

(2014) Evaluation of the ‘local climate zone’ scheme using temperature observations and model simulations. International Journal of Climatology 34(4): 1062–1080.

53.

Strigl

Kofler

Podlipnig

(2010) Performance and scalability of GPU-based convolutional neural networks. In: 18th Euromicro international conference on parallel, distributed and network-based processing (PDP), Pisa, 17–19 February 2010, pp. 317–324.

54.

Szegedy

Vanhoucke

Ioffe

. (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition.

55.

Tuia

Moser

Le Saux

. (2017) 2017 IEEE GRSS Data Fusion Contest: Open data for global multimodal land use classification (technical committees). IEEE Geoscience and Remote Sensing Magazine 5(1): 70–73.

56.

Verdonck

M-L

Okujeni

van der Linden

. (2017) Influence of neighbourhood information on ‘Local Climate Zone’ mapping in heterogeneous cities. International Journal of Applied Earth Observation and Geoinformation 62: 102–113. DOI: 10.1016/j.jag.2017.05.017.

57.

Wegner

Branson

Hall

. (2016) Cataloging public objects using aerial and street-level images-urban trees. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Las Vegas, NV, USA, 27–30 June 2016, pp. 6014–6023.

58.

Zhu

. (2017a) Automatic land cover classification of geo-tagged field photos by deep learning. Environmental Modelling & Software 91: 127–134. DOI: 10.1016/j.envsoft.2017.02.004

59.

Ren

Cai

. (2017b) Classification of local climate zones using ASTER and Landsat data for high-density cities. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. DOI: 10.1109/JSTARS.2017.2683484.

60.

Yin

Cheng

Wang

. (2015) ‘Big data’ for pedestrian volume: Exploring the use of Google Street View images for pedestrian counts. Applied Geography 63: 337–345. DOI: 10.1016/j.apgeog.2015.07.010

61.

Zhang

. (2017) Parcel-based urban land use classification in megacity using airborne LiDAR, high resolution orthoimagery, and Google Street View. Computers, Environment and Urban Systems 64: 215–228. DOI: 10.1016/j.compenvurbsys.2017.03.001

62.

Zhongli

Hanqiu

(2016) A study of urban heat island intensity based on local climate zones: A case study in Fuzhou, China. In: 4th international workshop on earth observation and remote sensing applications (EORSA), Guangzhou, China, 4–6 July 2016, pp. 250–254. IEEE.