Do Satellite Crop Maps See Double?
No, and that's a problem.
If you're eating lettuce in North America this month, it is highly likely that it was grown in Yuma County, Arizona. As winter turns to spring and summer, lettuce production moves north and west.
Curious whether Arizona lettuce production had changed over time, I went to the satellite data app on my website to look at crop maps. This app shows the Cropland Data Layer (CDL), which estimates the crop planted on each 30mx30m pixel in the country.
I fell into a rabbit hole.
Comparing 2012 and 2022, I saw two things that surprised me. First, there appeared to be a lot less lettuce in 2012 than 2022. Second, there was a lot of durum wheat in 2012, but apparently not much in 2022. Durum is the variety of wheat used in pasta. More than 90% of US durum wheat is grown in Montana and North Dakota, but most of the remaining 10% is grown in Arizona or in California's Imperial Valley. Durum grown in the southwest is often referred to as "Desert Durum".
Aside. It really is fun to look at satellite crop maps. Try it for yourself by clicking here. All you need is a web browser. One drawback of our app, which uses Google Earth Engine, is that it labels only a subset of crops. To identify a color that is not shown on the app, you can go to USDA's web app, which is slower but allows you to query any pixel.
So, what has happened to Arizona lettuce and durum wheat production in recent years?
USDA does not publish county-level survey data on lettuce acreage and production. It does publish those data for durum wheat, although not every year. Comparing the number of wheat acres in the survey data to the satellite data, we see close matches in some years (e.g., 2015, 2016, 2020). In other years, the satellite estimates are much lower than the survey (e.g., 2010, 2011, 2013). In 2022, the survey estimates 36,000 acres of durum wheat, compared to 21,000 estimated by the satellite. (I computed acreage in the CDL by summing pixels. I did no data cleaning.)
After growing lettuce in the winter, many Yuma County farmers plant a summer crop, such as durum wheat or cotton. This practice is known as double cropping --- two crops per year.
The satellite image below from April 2022 shows a lot of green fields in the main lettuce-growing area in Yuma County (black circle), implying that a summer crop had been planted. By July, many of those fields were brown, suggesting an early summer harvest. Desert durum is typically harvested in early summer, in contrast to other summer crops such as cotton, which are harvested later.
The CDL satellite reports a category for pixels that are double cropped with lettuce and durum wheat. Perhaps it sometimes classifies durum wheat in the "Durum Wheat" category and other times in the "Double Crop Lettuce /Durum Wheat" category. Combining these two categories explains some of the discrepancy between the data sources for 2010-14, but it still seems the satellite is missing a lot of desert durum.
Looking across all crops, the CDL satellite estimates about 175,000 acres of crops each year in Yuma County. However, the large variation from year to year suggests significant misclassification, especially surrounding double cropping. Other apparent CDL errors include the absence of citrus in 2008 and 2010 (those trees did not disappear and reappear) and the phantom appearance of winter wheat in 2011.
Estimated lettuce acres fluctuate wildly, most likely because lettuce fields are sometimes categorized as lettuce and other times as double cropped. Since 2002, the five-yearly agricultural census reports about 70,000 acres of lettuce per year. This suggests that most of the fields classified as lettuce, durum wheat, or doubled-cropped lettuce actually grew lettuce in the winter. Conversely, it suggests that most of the durum wheat produced in Yuma County is double cropped with lettuce.
The USDA reports error rates for the CDL that it estimates using ground truth data. The error rates are published annually by state in the CDL metadata. They include "omission errors", which are the percent of fields that were planted to a crop but which the CDL misses and "commission errors", which are the percent of fields classified as a certain crop but which were planted to something else.
Plotting the reported error rates alongside the estimated acreage, we see some years in which high error rates coincide with suspicious acreage estimates. For example, the reported error rates are relatively high for durum wheat in 2010-2014, lettuce in every year but especially 2011, and citrus in 2008 and 2010. In other years, reported error rates fluctuate without corresponding large changes in total acreage
Error rates for double cropped lettuce and durum wheat are often not reported because no acres are given this classification. Moreover, it appears that the reported error rates are incorrect in many cases. In particular, this analysis suggests that the true omission errors for double cropped lettuce and durum wheat are very high because most such fields are being classified as something else.
The CDL satellite data are not seeing double cropping when they should be. This is a potential pitfall for researchers and something an enterprising data scientist could fix.
I made the figures in this article using this R code.