CLIM-SEG: A Generizable Segmentation Model for Heat and Flood Risk
2021 - 2023
SIERA
UMERA
Project Leads
PI: John E. Fernandez
Lead Researcher: Norhan Bayomi
Project Team
Anushka Ray,
MIT MEng, 2022 – Course 6
Katherine Y Xu,
MIT MEng, 2022 – Course 6
Supported By
Bank of Bangkok, Thailand
A novel framework for high-resolution urban heat and flood risk assessment that integrates semantic segmentation of aerial imagery with environmental, socioeconomic, and building data to produce comprehensive climate risk evaluations at the census tract level.
CLIM-SEG is a framework developed by the Urban Metabolism Group for assessing urban heat and flood risk at the census tract level, addressing critical gaps in current climate risk modeling. The framework combines semantic segmentation of aerial imagery with a weighted sum approach that synthesizes land cover data with hazard and vulnerability factors, producing risk scores ranging from 0 to 1. This low-cost and efficient approach enables urban planners to prioritize resources for flood mitigation and heat adaptation using publicly available data sources.
At the core of CLIM-SEG is a fine-tuned Segmenter model, a transformer-based encoder-decoder architecture, that performs semantic segmentation on aerial imagery to classify pixels into six land cover categories: building, pavement, tree, grass, earth, and water. The model was trained on a custom-curated dataset of 545 aerial images, including 145 manually annotated segmentation maps from Apple Maps and Nearmap, achieving a pixel accuracy of 97.85% and an Intersection over Union (IoU) of 0.9578 for key urban features. The fine-tuned Segmenter significantly outperformed baseline CNN (PSPNet) and vision transformer (KNet) architectures in identifying built and natural environment features from aerial imagery.
The extracted land cover features are integrated with environmental datasets (elevation, slope, flow accumulation from GMTED2010 and HydroSHEDS), land surface temperature data (from the Yale Center for Earth Observation’s SUHI dataset), socioeconomic vulnerability indicators (from 2020 Census data covering eight vulnerable population groups), and building characteristics (age, height, type, and air conditioning presence from the Boston Buildings Dataset). Heat and flood risk models aggregate these factors through a weighted sum of hazard and vulnerability scores, each normalized and combined with equal weights of 0.5.
Boston serves as the primary case study, selected because the city experiences severe urban heat islands with temperatures rising up to 10°F in summer and faces projected increases in coastal and riverine flooding, with over 11,000 structures expected to be affected by 2070. Results indicate that census tracts in the South End have the highest flood risk (weighted score of 0.825), while census tracts in the Fenway–Kenmore neighborhood show the highest heat risk (score of 0.991). Both findings were validated against the Boston Climate Ready Map for flood risk and the Risk Factor platform for heat risk, demonstrating strong alignment with established assessments.
The framework’s generalizability was tested by applying the Boston-trained segmentation model to aerial images of New York City, including Manhattan, Queens, and Brooklyn. Despite significant differences in urban morphology, particularly Manhattan’s denser, taller building stock and more complex rooftop geometries, the model produced qualitatively strong segmentation results, correctly handling heavily shadowed areas and varied land cover patterns. This cross-city evaluation demonstrates the potential for CLIM-SEG to be scaled to other urban areas confronting climate change impacts.
By leveraging publicly available datasets and frequently updated aerial imagery, CLIM-SEG provides a cost-effective, data-driven tool for climate risk assessment that can be readily adapted to new urban contexts. The framework contributes to more targeted and equitable adaptation strategies by identifying census tracts where vulnerable populations intersect with high hazard exposure, supporting urban planners and policymakers in prioritizing interventions such as green infrastructure expansion, improved drainage systems, and cooling center placement.
