Download PDF Version

About this keynote: This post summarizes methods and open challenges for segmenting ultra-high resolution (UHR) images—from patch-based inference pipelines to scalable transformer architectures and remote sensing benchmarks.

TL;DR

  • TODO: 3–5 bullets summarizing the post.

Motivation

Ultra-high resolution images—remote sensing tiles, whole-slide pathology, aerial orthophotos, and gigapixel panoramas—carry fine-grained structure that standard segmentation pipelines cannot process in a single forward pass.

Problem Formulation

Given an image (\mathbf{I} \in \mathbb{R}^{H \times W \times C}) with (H \times W) far beyond GPU memory limits, the goal is to predict a dense label map (\mathbf{Y} \in \mathbb{L}^{H \times W}).

Key challenges:

  1. Memory: A full-image forward pass is infeasible at native resolution.
  2. Context vs. detail: Local patches preserve detail but may miss global semantics; downsampled views capture context but blur boundaries.
  3. Annotation cost: Pixel labels at full resolution are expensive to collect and review.
  4. Boundary consistency: Overlapping patch predictions must be fused without seam artifacts.

Patch-Based Methods

Sliding-window and tile-based inference remain the most widely deployed baseline for UHR segmentation.

  • TODO: U-Net [Ronneberger et al., 2015] and fully convolutional designs.
  • TODO: Overlap-tile strategy, blending weights, and boundary handling.
  • TODO: Representative remote sensing / pathology pipelines.

Multi-Scale Architectures

Methods that explicitly combine local and global context:

  • TODO: Feature pyramid and encoder–decoder variants for large images.
  • TODO: Hierarchical or coarse-to-fine refinement.
  • TODO: Cross-scale attention and memory-efficient long-range modeling.

Foundation Models

Recent segment-anything style models and vision backbones pretrained at scale:

  • TODO: SAM-style promptable segmentation [Kirillov et al., 2023].
  • TODO: Adapting foundation models to UHR via tiling, memory banks, or sparse attention.
  • TODO: Open-vocabulary and weakly supervised extensions.

Remote Sensing

Remote sensing is a primary application domain for UHR segmentation due to sensor resolution and scene extent.

  • TODO: Land-cover / land-use mapping at city or regional scale.
  • TODO: Building footprint extraction and road parsing.
  • TODO: Dataset and benchmark notes (e.g., LoveDA, OpenEarthMap, etc.).

Evaluation

  • TODO: Metrics (mIoU, F1, boundary-aware scores).
  • TODO: Train/validation tiling vs. full-scene test evaluation.
  • TODO: Efficiency metrics: latency, GPU memory, and throughput on gigapixel inputs.

Takeaways

  1. TODO: When patch-based inference is enough—and when it fails.
  2. TODO: Design axes: context aggregation, fusion strategy, and annotation efficiency.
  3. TODO: Practical checklist for deploying UHR segmentation systems.

References

Ronneberger, O., Fischer, P., & Brox, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. In Medical Image Computing and Computer-Assisted Intervention (MICCAI), pp. 234–241.
Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W.-Y., Dollár, P., & Girshick, R. (2023). Segment Anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 4015–4026.

Contact: bili_sakura@zju.edu.cn