Try RGB→Thermal
Pick one of the bundled RGB UAV plot images below to see the model's predicted thermal map.
Loading sample gallery…
The live demo serves the locally-hosted Pix2Pix baseline (test MAE 2.31 °C). The paper's headline model, PID-LDM, reaches 0.90 °C MAE. Custom image upload is planned but not enabled yet.
Overview
Cost-effective remote sensing of canopy temperature is critical for precision irrigation, yet thermal cameras remain three to ten times more expensive than visible-band sensors and require specialized payloads. Recent work shows thermal imagery can be synthesized from RGB, but adversarial generators treat the task as pure pattern matching and lightweight regression networks collapse to a single deterministic mapping, neither of which captures the distribution of physically plausible thermal images consistent with a given RGB observation.
PID-LDM is a physics-informed latent diffusion model proposed to address this gap. It operates in the latent space of a thermal variational autoencoder, applies a denoising U-Net conditioned on a learned RGB embedding and on a frozen physics-aware feature map from a temperature-emissivity-vector network (TeVNet), and is trained with a composite objective that combines latent-space noise prediction with a physics-reconstruction loss. Inference uses deterministic DDIM sampling, which preserves the radiative-transfer constraint while producing a coherent thermal image. Predicted thermal is calibrated to a fixed 0 to 55 °C scale.
Method
Three-season UAV dataset
Trained and evaluated on 2,710 paired RGB and thermal UAV images of sweet corn and green beans grown under four irrigation treatments (25, 50, 75, 100% of full irrigation) at the UF/IFAS Tropical REC.
Physics-informed diffusion
A latent diffusion U-Net is conditioned on an RGB embedding and a frozen TeVNet physics prior, trained with noise prediction plus a physics-reconstruction loss on low-noise timesteps.
Deterministic inference
DDIM sampling yields stable, single-pass prediction at roughly 250 ms per 256x256 patch on a laptop GPU, then canopy temperature is converted to a CWSI map.
Results
Three generative paradigms were compared on a 271-image held-out test set: paired Pix2Pix (cGAN), unpaired CycleGAN, and the physics-informed PID-LDM.
| Model | Test MAE (°C) ↓ | CWSI error ↓ | Type |
|---|---|---|---|
| CycleGAN | 3.91 | 0.215 | Unpaired GAN |
| Pix2Pix | 2.31 | 0.095 | Paired GAN (served in demo) |
| PID-LDM | 0.90 | 0.011 | Physics-informed diffusion |
PID-LDM attained a mean absolute temperature error of 0.90 °C (RMSE 1.18 °C, SSIM 0.87, PSNR 34.2 dB), a 61.1% reduction over Pix2Pix and 77.0% over CycleGAN, and a lower error than each GAN baseline on at least 97% of test images. Its CWSI predictions agreed with ground truth to within 0.011 on average, an order of magnitude better than either GAN.
From RGB to crop water stress
The end-to-end pipeline this work delivers.
1 · RGB capture
Low-cost RGB UAV imagery of the field is acquired, with no thermal sensor required.
2 · Thermal synthesis
PID-LDM predicts a calibrated canopy-temperature map (°C) from each RGB image.
3 · CWSI mapping
Predicted canopy temperature is converted to the Crop Water Stress Index, highlighting water-stressed zones for irrigation scheduling.
The Crop Water Stress Index normalizes canopy temperature between a well-watered (wet) and a non-transpiring (dry) reference, so a value near 0 indicates a well-watered canopy and a value near 1 indicates severe water stress. Because the thermal layer is generated from RGB, this index can be produced from inexpensive imagery and applied retrospectively to RGB-only archives.
Code & data
The trained models, inference code, and dataset are available upon request.