Filters for a compact crop–weed sensor, searched by NSGA-II with and without LLM agents.
From even spacing to the searched design
Hyperspectral cameras separate crops and weeds well but are too expensive for sprayers, so a compact sensor measures a few bands through optical filters whose centres and widths must be chosen. NSGA-II searched them on public greenhouse images of three crops and four weeds. Told a budget of only 200 evaluations, and not how to spend it, the agents left the evolutionary search for local refinement around the selected design and gained 1.4–1.7 points of balanced accuracy, though not significantly after correction.
WeedCube: 160 images, 224 bands, seven species
Signal-to-noise ratio (log scale)
Images were recorded with a Specim FX10 push-broom camera on a laboratory scanner under halogen light. Bands below 430 nm were left out of the design space because their median signal-to-noise ratio on vegetation pixels was 6.7, against 277 between 750 and 900 nm.
15 grouped cross-validation partitions
Stratified grouped five-fold cross-validation repeated three times; repeated scans of one scene (joined squares) were merged into 156 scene groups and never fall on both sides of a split. Each species was imaged in its own sessions, so absolute accuracies are upper bounds, while comparisons between methods remain informative.
Drag the filters; the classifier retrains in your browser
Filter outputs of each species' mean spectrum
Test pixels as the classifier sees them
Each filter averages the reflectance under its normalised Gaussian response (centres 430 to 1000 nm, FWHM 5.5 to 50 nm). An LDA classifier with Ledoit–Wolf shrinkage is trained on the outputs; the nominal objective A is its balanced accuracy (BA), and the deployment objective R scores the same classifier under 16 draws of illumination gain (0.9 to 1.1), spectral tilt (up to 5% per 300 nm), calibration error (up to 2% per channel) and shot noise (SNR 50 for a 10 nm filter).
Here the LDA is trained on 100 pixels per species from the 128 training images of partition 0 and scored on 60 per species from its 32 test images; the study used 300 pixels per training image and three inner folds.
Confusion matrix on the test pixels, nominal (percent of each species' pixels)
NSGA-II search, partition 0, ten filters
Hypervolume
Filters rule S would select now
Each generation of 100 designs costs 100 evaluations, and the budget of 4100 for ten filters was fixed by a pilot before any test image was scored (0.10–0.21 s per evaluation). Rule S takes, among the designs that no other design beats on both objectives, the one with the highest R whose A lies within one point of the best A (shaded strip). Differential evolution maximised A only and found designs about as accurate but far less robust.
Selected designs of the 15 partitions
One real run with a budget of 200, partition 0
Solid: requests. Dashed: checked replies.
Budget of 200 evaluations
Evaluated designs (inner folds)
The agents were built on the raia platform (Raia LLC) on one language model: Spectral Intelligence (regions and seed designs), Validation (assessments), Filter Optimisation (the next action) and Single Design (all three roles). They could run the NSGA-II engine, refine a design, write designs, ask for diagnostics or stop, but never change the objectives, constraints, budget or the final rule S. Diagnostics are evaluations that showed the agents how a design reacts to each deployment condition; they were charged to the budget.
How the evaluations were spent
Actions on partition 0, in order
Shares are means over 15 partitions with ten filters. With the full budget the agents left most evaluations to NSGA-II; told 200, all 30 runs refined locally, and 65 of 71 refinements were centred on the design rule S would have selected. Their instructions did not say which action suits a small budget.
Accuracy against the number of evaluations
Test accuracy of the designs
Accuracy and cost after 200 evaluations
Agent as governor of three optimisers, 1000 evaluations
Replies valid against the schema
Pre-specified comparisons
Authors. Boaz Tulu and Haimanote Bayabil, Department of Agricultural and Biological Engineering, Tropical Research and Education Center, University of Florida, Homestead, FL, USA. Contact: btulu@ufl.edu.
Data. WeedCube by Ram, Mettler, Howatt, Ostlie and Sun, Data in Brief 56, 110837 (2024), doi:10.1016/j.dib.2024.110837; dataset doi:10.15482/USDA.ADC/25306255.v1.
Computing. HiPerGator, University of Florida; agents on the raia platform (Raia LLC). Code will be available upon request.
This page. Runs offline and sends no data anywhere. Numbers in "Try a design" are computed in your browser on a small sample; all others come from the study.
For the sensor, ten filters searched for nominal and deployment accuracy kept 9.5 points more accuracy under simulated deployment conditions than designs searched for accuracy alone, and lost only 0.15 points when built from catalogue filters. All images come from greenhouses under one lamp with one camera, each species in its own sessions, so accuracies are upper bounds and transfer to field light is untested.