Xiaoxiang Zhu - Selected Publications#
1. Zhu et al. (2022). The Urban Morphology on Our Planet - Global perspectives from Space. Remote Sensing of Environment, 269, pp. 112794.
Urbanization is the second-largest megatrend, right after climate change. Accurate measurements of urban morphological and demographic figures are at the core of many international endeavours to address issues of urbanization, such as the United Nations’ call for “Sustainable Cities and Communities”. In many countries – particularly developing countries –, however, this database does not yet exist. Here, we demonstrate a novel deep learning and big data analytics approach to fuse freely available global radar and multi-spectral satellite data, acquired by the Sentinel-1 and Sentinel-2 satellites. Via this approach, Zhu et al. created the first-ever global and quality controlled urban local climate zones classification covering all cities across the globe with a population greater than 300,000 and made it available to the community. Statistical analysis of the data quantifies a global inequality problem: approximately 40% of the area defined as compact or light/large low-rise accommodates about 60% of the total population, whereas approximately 30% of the area defined as sparsely built accommodates only about 10% of the total population. Beyond, patterns of urban morphology were discovered from the global classification map, confirming a morphologic relationship to the geographical region and related cultural heritage. The open access of the resulting LCZ maps encourages research on the global change process of urbanization, as a multidisciplinary crowd of researchers will use this baseline for spatial perspective in their work. In addition, it can serve as a unique dataset for stakeholders such as the United Nations to improve their spatial assessments of urbanization.
2. Zhu et al. (2021), "Deep Learning Meets SAR: Concepts, Models, Pitfalls, and Perspectives," in IEEE Geoscience and Remote Sensing Magazine, doi: 10.1109/MGRS.2020.3046356.
Deep learning in remote sensing has become an international hype, but it is mostly limited to the evaluation of optical data. Although deep learning has been introduced in Synthetic Aperture Radar (SAR) data processing, despite successful first attempts, its huge potential remains locked. In this paper, Zhu et al. provided an introduction on the most relevant deep learning models and concepts, point out possible pitfalls by analyzing special characteristics of SAR data, review the state-of-the-art of deep learning applied to SAR in depth, summarize available benchmarks, and recommend some important future research directions. With this effort, Zhu et al. hope to stimulate more research in this interesting yet under-exploited research field and to pave the way for use of deep learning in big SAR data processing workflows.
3. Gawlikowski, J., Tassi, C. R. N., Ali, M., Lee, J., Humt, M., Feng, J., Kruspe, A., Triebel, R., Jung, P., Roscher, R., Shahzad, M., Yang, W., Bamler, R., Zhu, X. (2023). A survey of uncertainty in deep neural networks, Artificial Intelligence Review, https://doi.org/10.1007/s10462-023-10562-9.
This work gives a comprehensive overview of uncertainty estimation in neural networks, reviews recent advances in the field, highlights current challenges, and identifies potential research opportunities. It is intended to give anyone interested in uncertainty estimation in neural networks a broad overview and introduction, without presupposing prior knowledge in this field. Although only published in 2023, it has been cited for 485 times.
4. Zhu et al. (2020). So2Sat LCZ42: A Benchmark Data Set for the Classification of Global Local Climate Zones [Software and Data Sets]. IEEE Geoscience and Remote Sensing Magazine, 8(3), pp. 76–89.
In an unprecedented effort Zhu et al. generated a scientific dataset of two unique selling points:
(i) It is the most rigorously and carefully labelled reference dataset in EO. Over one month 15 domain experts carefully designed the labelling workflow, the error mitigation strategy, the validation methods, and conducted the data labelling. It consists of manually assigned Local Climate Zone (LCZ) labels of about half a million Sentinel-1 and Sentinel-2 image patches globally distributed in 42 urban agglomerations covering all the inhabited continents and 10 cultural zones.
(ii) It is the first EO dataset that provides a quantitative measure of the label uncertainty. This was achieved by letting a group of domain experts cast 10 independent votes on 19 cities in the dataset. I have not seen this on any other reference dataset in remote sensing.
The dataset is considered as an idea benchmark for cutting-edge AI4EO research on quantification of uncertainty, data fusion, automated machine learning and computing in AI. It has already drawn great international attention from different communities. This dataset was included in TensorFlow which is the most widely used deep learning framework developed by Google.
5. Camps-Valls G., Tuia D., Zhu X., Reichstein M. (2020): “Deep learning for the Earth Sciences: A comprehensive approach to remote sensing, climate science and geosciences”, UK: Wiley & Sons, Sep 2020.
Big Earth observation data offers new perspectives in modeling the Earth system. To unlock its potential, machine learning plays a vital role in extracting geoinformation from the big EO data and integrating them with physical models. Among others deep learning is a fundamental technique in modern artificial intelligence and is being applied to disciplines across the scientific spectrum; the earth sciences are no exception. This book provides a pioneering, unifying perspective to the application of deep learning methods because it’s the first to bring together insights from the world’s leading experts on this issue. The book is intended for informed readers and will enable PhD students and researchers to quickly become familiar with modern approaches. Zhu is one of the four editors of the book, along with Gustau Camps-Valls from Universitat de València, Spain, Devis Tuia, from Swiss Federal Institute of Technology Lausanne, Switzerland and Markus Reichstein from the Max Planck Institute, Germany.
6. Shi, Li, Zhu (2020). Building Segmentation Through a Gated Graph Convolutional Neural Network with Deep Structured Feature Embedding. ISPRS Journal of Photogrammetry and Remote Sensing, 159, pp. 184–197.
The new dimension of ongoing global migration into the cities poses fundamental challenges to our societies across the globe. Despite of increasing efforts, global urban mapping still drags behind the geometric, thematic and temporal resolutions of geo-information needed to address these challenges. For example, it is estimated to be more than 2 billion buildings in the world. By June 2021, only 457 buildings have building footprints in OpenStreetMap – the biggest open urban data base. To close this gap, Zhu and her team develop machine learning methods and big data analytics solutions to create the first time the global building footprints from Planet scope satellite data. This paper describes the core machine learning algorithm of this endeavor.
Outlook: To date, the global building footprint, named Global OpenBuilding Map is already produced and the corresponding scientific paper is under review.
7. Mou, Bruzzone, Zhu (2019), Learning Spectral-Spatial-Temporal Features via a Recurrent Convolutional Neural Network for Change Detection in Multispectral Imagery, IEEE TGRS 57(2), pp. 924-935.
Change detection is a central problem in EO, in particular considering the free and open multisensory satellite data provided on a weekly basis through ESA’s Copernicus Program. In this paper a novel and generic recurrent convolutional neural network architecture is proposed, which is trained to learn a joint spectral-spatial-temporal feature representation in a unified framework for this task. Recently published, it has already been cited 422 times. Mou is Zhu’s PhD student.
8. Zhu et al. (2017), Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources, IEEE Geoscience and Remote Sensing Magazine 5(4), pp. 8-36.
Central to the paradigm shift toward data-intensive science, deep learning (DL) has set new standards in Earth Observation (EO) data analysis. But the field of research has also triggered a hype – a hunt for "low hanging fruits". In this article, Zhu highlights what makes DL special in remote sensing, reviews recent advances, emphasizes that the high quality requirements for geoinformation and the diversity of EO data require innovative and EO-specific DL approaches and provides resources to make DL in remote sensing readily applicable. More importantly, she encourages young scientists to use it to tackle important challenges, such as monitoring of global change and urbanization.
Her therein visionary thoughts and critical view are widely acknowledged by the international peers. In four years it has been cited 2446 times. Dr. Zhu received many invitations to keynote speeches. Today she is one of the world's best known, influential and successful scientists in this field, which now operates under the acronym "AI4EO" introduced by the ESA, where she is an AI visiting professor.
9. Zhu et al. (2016): Geodetic SAR Tomography, IEEE TGRS 54(1), pp. 18-35.
The paper introduces geodetic SAR tomography by combining SAR tomography and SAR imaging geodesy leading to extremely high 3D/4D point densities and absolute positioning accuracies (cm – dm). The paper was submitted, reviewed and published within less than a month which underlines its novelty and impact. It was the most popular article of the journal from Nov. 2015 - Mar. 2016, and has received the DLR Science Award 2016 and the IEEE TGRS Best Paper Award 2017.
10. Zhu and Bamler (2010): Tomographic SAR Inversion by L1 Norm Regularization – The Compressive Sensing Approach, IEEE TGRS, pp. 3839-3846.
In this fundamental work compressive sensing and sparse priors are exploited for the first time to improve resolution of SAR tomography; a superresolution factor of 15 is achieved. With this work it became possible to reconstruct dynamic (3D + temporal) city models with an unprecedented point density of 1 million/km2 and with accuracy in the order of mm/year, which opened the door for large scale high density urban infrastructure deformation monitoring. Because of this work, Zhu has won the Technology Review Germany “Innovators under 35” and several research awards. The paper has been cited 550 times.
Zhu extend this work considerably: (i) incorporating nonlinear motion components (Zhu X., Bamler R. (2011), IEEE GRSL; 115 citations), (ii) deriving super-resolution factors and localization accuracies (Zhu X., Bamler R. (2012), IEEE TGRS; 343 citations) and (iii) exploiting stacks of extremely small numbers of 3-5 bistatic TanDEM-X acquisitions (Shi et al. (2020), IEEE TGRS). These are currently the only SAR tomographic data of global coverage. Hence, Zhu’s work makes global tomographic building reconstruction possible.
