WRR Lab
Image-to-Image Translation · Remote Sensing

PID-LDM: Physics-Informed Latent Diffusion for RGB-to-Thermal Image Translation in Precision Agriculture

A physics-informed latent diffusion model that synthesizes canopy thermal imagery directly from low-cost RGB UAV photographs, then turns the predicted temperature into a Crop Water Stress Index (CWSI) map, bringing thermal-camera-grade water-stress monitoring within reach of RGB-only missions.

Boaz B. Tulu1, Yiannis Ampatzidis2, Changying Li3, Willingthon Pavan3, Golmar Golmohammadi4, Haimanote K. Bayabil1
1Agricultural & Biological Engineering, Tropical REC, UF/IFAS, Homestead, FL  ·  2Agricultural & Biological Engineering, Southwest Florida REC, UF/IFAS, Immokalee, FL  ·  3Agricultural & Biological Engineering, UF, Gainesville, FL  ·  4Soil, Water, and Ecosystem Sciences, UF, Gainesville, FL

Try RGB→Thermal

Pick one of the bundled RGB UAV plot images below to see the model's predicted thermal map.

Pix2Pix · MAE 2.31 °C · thermal 0 to 55 °C

Loading sample gallery…

The live demo serves the locally-hosted Pix2Pix baseline (test MAE 2.31 °C). The paper's headline model, PID-LDM, reaches 0.90 °C MAE. Custom image upload is planned but not enabled yet.

Overview

Cost-effective remote sensing of canopy temperature is critical for precision irrigation, yet thermal cameras remain three to ten times more expensive than visible-band sensors and require specialized payloads. Recent work shows thermal imagery can be synthesized from RGB, but adversarial generators treat the task as pure pattern matching and lightweight regression networks collapse to a single deterministic mapping, neither of which captures the distribution of physically plausible thermal images consistent with a given RGB observation.

PID-LDM is a physics-informed latent diffusion model proposed to address this gap. It operates in the latent space of a thermal variational autoencoder, applies a denoising U-Net conditioned on a learned RGB embedding and on a frozen physics-aware feature map from a temperature-emissivity-vector network (TeVNet), and is trained with a composite objective that combines latent-space noise prediction with a physics-reconstruction loss. Inference uses deterministic DDIM sampling, which preserves the radiative-transfer constraint while producing a coherent thermal image. Predicted thermal is calibrated to a fixed 0 to 55 °C scale.

Method

Three-season UAV dataset

Trained and evaluated on 2,710 paired RGB and thermal UAV images of sweet corn and green beans grown under four irrigation treatments (25, 50, 75, 100% of full irrigation) at the UF/IFAS Tropical REC.

Physics-informed diffusion

A latent diffusion U-Net is conditioned on an RGB embedding and a frozen TeVNet physics prior, trained with noise prediction plus a physics-reconstruction loss on low-noise timesteps.

Deterministic inference

DDIM sampling yields stable, single-pass prediction at roughly 250 ms per 256x256 patch on a laptop GPU, then canopy temperature is converted to a CWSI map.

Results

Three generative paradigms were compared on a 271-image held-out test set: paired Pix2Pix (cGAN), unpaired CycleGAN, and the physics-informed PID-LDM.

ModelTest MAE (°C) ↓CWSI error ↓Type
CycleGAN3.910.215Unpaired GAN
Pix2Pix2.310.095Paired GAN (served in demo)
PID-LDM0.900.011Physics-informed diffusion

PID-LDM attained a mean absolute temperature error of 0.90 °C (RMSE 1.18 °C, SSIM 0.87, PSNR 34.2 dB), a 61.1% reduction over Pix2Pix and 77.0% over CycleGAN, and a lower error than each GAN baseline on at least 97% of test images. Its CWSI predictions agreed with ground truth to within 0.011 on average, an order of magnitude better than either GAN.

From RGB to crop water stress

The end-to-end pipeline this work delivers.

1 · RGB capture

Low-cost RGB UAV imagery of the field is acquired, with no thermal sensor required.

2 · Thermal synthesis

PID-LDM predicts a calibrated canopy-temperature map (°C) from each RGB image.

3 · CWSI mapping

Predicted canopy temperature is converted to the Crop Water Stress Index, highlighting water-stressed zones for irrigation scheduling.

The Crop Water Stress Index normalizes canopy temperature between a well-watered (wet) and a non-transpiring (dry) reference, so a value near 0 indicates a well-watered canopy and a value near 1 indicates severe water stress. Because the thermal layer is generated from RGB, this index can be produced from inexpensive imagery and applied retrospectively to RGB-only archives.

Code & data

The trained models, inference code, and dataset are available upon request.

Code, on request Dataset, on request Request access