19 July 2002
Validation of Satellite Measurements Using Simple, Inexpensive Handheld Instruments
By Forrest M. Mims III
Geronimo Creek Observatory
Seguin, Texas 78155 USA
forrest.mims@ieee.org
This paper was presented at the first annual Citizen Science Conference, Philadelphia, June 28-29, 2002
Abstract
Various satellites measure the atmosphere's aerosol optical depth (AOD). Because of instrumental degradation and calibration drift, it is important to regularly validate data provided by satellite remote sensing instruments. Serious students can use GLOBE Sun photometers to play an important role in such comparisons by making observations during satellite overpasses when the solar disk is unobscured by clouds. Here I report that on 5 clear days a carefully calibrated GLOBE Sun photometer has precisely validated AOD measurements by the MODIS instrument aboard NASA's Terra satellite. The correlations of the green AOT measured by GLOBE and MODIS within 35 and 10 minutes of the MODIS overpass are, respectively, good (r2 = 0.89) and remarkably high (r2 = 0.99). Similarly, the correlations of the red AOD measured within 35 and 10 minutes of the MODIS overpass are, respectively, good (r2 = 0.80) and remarkably high (r2 = 0.997). These findings compare well with the results of a comparison of MODIS and some 30 automated Cimel Sun photometers in the AERONET network (Chu et al., 2000). Very similar biases in the MODIS AOD were identified by the NASA study and the work reported here.
Introduction
The harsh environment of space can degrade even the most carefully designed instruments that monitor the Earth from orbiting satellites. Therefore it is essential to periodically compare measurements from space with observations made at or near the surface. This process is known as ground truth verification or validation. When there is good agreement in measurements of the same parameter made at the surface and from an overhead satellite, confidence is ascribed to satellite measurements made over similar sites elsewhere. Amateur scientists and students can make a significant contribution to validating various satellites.
Background
Every day a dozen or more research satellites launched by various countries fly over every square meter of Earth. Since 1990 I have checked the accuracy of measurements made by several of these satellites during overpasses of Texas, Hawaii, Brazil, Alaska, Switzerland, Japan and various other sites.
Instruments in these satellites take pictures and make many kinds of measurements. My interest is in satellites that measure the ozone, water vapor and haze than blanket our planet. Ozone plays vital roles in regulating the temperature of the atmosphere and the ultraviolet radiation from the Sun. Water vapor condenses into clouds and rain. It also provides the greenhouse effect that prevents the Earth from freezing. Haze is more complicated than ozone and water vapor. It can be formed naturally or by people. It can consist of dust, smoke and many kinds of chemicals. Water vapor often increases haze.
In 1990 I began using a homemade ozone instrument, TOPS (Total Ozone Portable Spectrometer) to track the Total Ozone Mapping Spectrometer (TOMS) aboard NASA's Nimbus-7. TOPS was originally developed for "The Amateur Scientist" column in Scientific American. It was eventually described elsewhere (Mims, 1992a). In 1991, the TOMS data began to depart from the measurements made here. Since the TOPS ozone algorithm was derived from an empirical comparison with TOMS ozone measurements, there was good reason to consider the drift to be real.
TOPS has identified various other anomalies in ozone satellites, including periodic look-angle errors that characterized one of the TOMS series satellites. MICROTOPS II, an advanced microprocessor-controlled version of the original TOPS, has tracked the stability of Earth Probe TOMS since 1996, and MICROTOPS II ozone amounts are typically within 1% of those measured by this satellite instrument. There is also very close agreement in solar ultraviolet inferred by satellite and MICROTOPS II ozone measurements.
Initially NASA was reluctant to accept the finding of an error or drift in the TOMS data. Eventually, however, the difference between TOPS and the satellite was traced to a calibration drift in the satellite's ozone instrument. Because two teams of scientists in Antarctica had noticed differences between their ozone measurements and those by the satellite, a short paper about this finding was submitted to Nature (Mims, 1993). The drift in TOMS ozone measurements went undetected because a drift in the satellite's orbit precluded periodic sunlight calibration tests. In later years my instruments found a periodic error in a second ozone satellite. NASA also found this error.
In 1990 I began measuring the amount of haze in a vertical column through the atmosphere using a very simple homemade instrument that, like TOPS, was originally developed during my brief tenure writing "The Amateur Scientist" in Scientific American. Shawn Carlson kindly described a simple version of the instrument during his tenure at "The Amateur Scientist" (Carlson, 1997).
In 1992 the scientist responsible for a satellite instrument that measures haze (NOAA's AVHRR or Advanced High Resolution Radiometer) reported that the atmosphere had returned to normal after the 1991 eruption of Mount Pinatubo. His haze findings were very different from mine, for there was still considerable Pinatubo haze over South Texas and Hawaii. It could even be seen during every sunrise and sunset. After the two of us exchanged data, it was obvious that the AVHRR was significantly underestimating haze. The error was subsequently corrected.
Since last August I have been tracking several measurements made by Terra, NASA's flagship environmental satellite, in my role as associate principle investigator for the GLOBE Program's haze project. Ground measurements of haze made here and from space by Terra show strikingly close agreement (r2 > 0.99) when observations are made within 10 minutes of the satellite overpass. The findings here agree closely with a recent NASA study that is described below. The satellite slightly, but consistently, underestimates the amount of haze, and this is easily corrected.
While checking haze measurements made by Terra, I also checked its measurements of water vapor using an instrument built for around $20 in 1990. Calibrations made at Hawaii's Mauna Loa Observatory show that this instrument is very accurate. A paper describing 12 years of water vapor measurements made here with this instrument will soon be published (Mims, 2002). There is good agreement with the satellite on most days. But on warm summer days when the total column water vapor exceeds 40 mm, Terra's water vapor measurements are much too high. Upper air soundings at Del Rio, Texas, and Washington, D.C. and CIMEL Sun photometer measurements at an AERONET site confirm this problem, and the science team has been informed.
A Role for Amateur Scientists and Students
Based on the success of satellite validation here at Geronimo Creek Observatory (GCO) since 1990, it is likely that amateur scientists and serious students can also perform satellite validation. They require only a calibrated Sun photometer and the time to make regular observations. Because current networks of automated instruments that perform satellite validation are so limited, a volunteer network of amateur scientists and students could play a major role in validating various satellite instruments.
The GLOBE program (www.globe.gov) is a major international network of schools that measure various environmental parameters. Satellite validation is among the applications proposed for GLOBE Sun photometers that measure the aerosol optical depth (AOD)of a column through the atmosphere (Brooks & Mims, 2001). AOD can be considered a haze index that specifies the transparency of the atmosphere. The GLOBE Sun photometers are based on a simple, inexpensive design that provides stable, drift-free measurements of AOD when carefully operated (Mims, 1992b).
Case Study: Validating Terra's MODIS Instrument
Amateur scientists can select from a variety of satellites to validate. Among the most relevant is Terra, the first in a series of sophisticated remote sensing satellites developed by NASA and its contractors as part of NASA's Earth Observing System (EOS). One of Terra's main instruments is MODIS (Moderate Resolution Imaging Radiometer), a spectroradiometer that scans the Earth below the satellite at 36 wavelengths ranging from the blue to the middle infrared. MODIS is the first instrument specifically designed to measure AOT from space. It does so at specific wavelengths using algorithms that account for the reflectance of the Earth and the passage of sunlight down through the atmosphere and back up toward space. Near-infrared channels are used to calculate the total column water vapor.
Terra was launched on December 18, 1999, into a Sun synchronous, polar orbit that places it overhead before noon at least once each day. On May 4, 2002, a second MODIS was successfully launched aboard Aqua, the second EOS satellite. Aqua's orbit places it overhead after local noon. Thus, MODIS measurements from Aqua will permit the monitoring of changes in AOD and water vapor that may have occurred after the pre-noon Terra overpass.
A Successful Protocol for Validating MODIS
The success of prior validation studies using handheld Sun photometers suggested that validation of MODIS AOD observations would be straightforward. However, a comparison of MODIS AOD measurements with GLOBE AOD measurements at local solar noon, the time GLOBE observations are made at GCO, showed rather poor agreement.(The protocols for some other GLOBE atmospheric studies specify that measurements should be made near local solar noon. However, the GLOBE aerosol protocol does not impose this restriction.) The problem was traced to differences in the timing of the GLOBE measurements and the Terra overpasses, which typically occur from 30 to 60 minutes before solar noon.
Ozone measurements from space are easily validated from the ground because the abundance of ozone in a column through the atmosphere changes rather slowly over a period of minutes or even hours. However, AOD can change significantly in a fraction of a second when cloud fragments move over and obscure part or all of the solar disk. Clouds of dust and smoke can cause significant fluctuations in AOD over periods of seconds to minutes. Thus, AOD validation requires conditions in which aerosols in the sky are reasonably stable and there is little chance of cloud interference. It is also very important that ground measurements be made at or within a reasonable time of the MODIS overpass. With these and other considerations in mind, the following protocol was established for a pilot MODIS validation program:
1. Determine the MODIS overpass time using NASA's Spacecraft Overpass Predictor (http://eospso.gsfc.nasa.gov/eos_homepage/sattrack.html). This web-accessible program is simple, and middle school and older students should be able to use it after some simple direction from a teacher or peer. It requires only that the ground site's geographic coordinates be entered. Various options are also available. The program responds with details about the time on the requested day that the satellite comes into view over the horizon, reaches its maximum angle over the site, and then descends behind the horizon. The program also provides details about the azimuthal position of the satellite during the overpass window.
2. Use a GLOBE Sun photometer calibrated by the Langley method at Mauna Loa Observatory to perform ground observations during or as close as possible to the MODIS overpass time.
3. Store the GLOBE instrument in a closed container and out of direct sunlight until it is used. This significantly reduces heating that can cause temporary calibration drift.
4. Use care when pointing the Sun photometer at the Sun. Because measurements were made with various instruments, there was not time to follow the established GLOBE protocol of three separate measurements. Instead, the peak measurement over a period of about 5 seconds was recorded for each channel. This abbreviated measurement procedure reduced the instrument's exposure to ambient temperature.
5. Record the time of the measurement using a watch that has been recently calibrated against a GPS receiver or a radio or Internet time signal (e.g., www.time.gov).
6. For this work the ozone column was measured with a well-calibrated MICROTOPS II. The ozone amount measured by a satellite such as Earth Probe TOMS may also be used. See http://toms.gsfc.nasa.gov/index.html. The climatological mean for ozone over a specific location may also be used. Failing to subtract the optical depth caused by ozone will cause important errors for both the green and red channels of the GLOBE Sun photometers.
7. Record the temperature, barometric pressure, dew point and relative humidity.
8.Note the sky conditions, especially the presence of clouds, cloud haze, smoke and dust. Crepuscular rays caused by cloud shadows should be noted, for they provide an important indicator of haze.
9. Photograph the sky using a 186-degree full-sky (fisheye) lens. Also, photograph the solar aureole (Mims).
10. Enter into a spreadsheet program that calculates AOT the date, time, barometric pressure and the raw data for each GLOBE Sun photometer measurement. Subtract from the AOT the optical depth of the ozone layer to arrive at the final AOT.
11. Retrieve MODIS data from the web when they become available. Plot the mean AOT for the 550 nm and 660 nm MODIS channels against the AOT for the 520 nm and 620 nm GLOBE channels. Perform linear regressions of the paired data sets for various selections of days to determine the level of agreement. Assess the data and consider deleting outliers only on a case-by-case basis. Observed cloud haze or variable smoke or dust during a MODIS overpass is good cause for eliminating a data point.
This protocol works reasonably well when both MODIS and the ground Sun photometer are making observations during the overpass window. This is not always the case. For various reasons, there are no MODIS data for some days when sky conditions were pristine. On some days when ground measurements were not made during the overpass window due to clouds in the area, MODIS measurements were made anyway. The ground observer does not know in advance if MODIS data for a particular overpass session will be available. During this study, there were periods of months when data were either not available or before data was published on the MODIS web site.
|
Figure 1. Aerosol optical depth (AOD) measured at Geronimo Creek Observatory (GCO) in South-Central Texas (29.6 N, 97.9 W) by the green channels of a GLOBE Sun photometer (520 nm) and from space by MODIS (550 nm) within (a) 60 minutes,(b) 35 minutes and (c) 10 minutes of the center of the overpass window. Click image to enlarge |
Results and Discussion
Validation measurements were begun on 8 August 2001 and continue to the present time. Until February 2002, the GLOBE measurements were conducted only at or near solar noon. Thereafter, on clear days the regular solar noon schedule was supplemented with measurements during the MODIS overpass window. days. This paper reports results of 26 comparisons of the GLOBE and MODIS green and red channels made within 60 minutes of the center of the MODIS overpass window from 24 September 2001 to 2 March 2002. Eleven of the comparisons occurred within 35 minutes and 5 occurred within 10 minutes of the overpass. The NASA study required ground observations within 30 minutes of MODIS overpass. Because a cluster of the GLOBE observations occurred between 30 and 35 minutes, they were included in a 35 minute category for comparison with the NASA analysis.
Figure 1 is a set of 3 scatter graphs that compare the green AOD measured by GLOBE at 520 nm and by MODIS at 550 nm within 60 minutes (a), 35 minutes (b), and 10 minutes (c) of the peak overpass time. The correlation of the observations within 60 minutes of the overpass is poor (r2 = 0.63). The correlation for observations within 35 minutes of the overpass is good (r2 = 0.89). The correlation for observations within 10 minutes of overpass is remarkably high (r2 = 0.99).
Very similar results were obtained for the GLOBE and MODIS red channels. Figure 2 is a set of 3 scatter graphs that compare the red AOD measured by GLOBE at 620 nm and by MODIS at 660 nm within 60 minutes (a), 35 minutes (b), and 10 minutes (c). The correlation of the observations within 35 minutes of the overpass is good (r2 = 0.80). The correlation for observations within 10 minutes of overpass is as remarkably high (r2 = 0.997)as for the green channels.
|
Figure 2. AOD measured at GCO by the red channels of a GLOBE Sun photometer (620 nm) and from space by MODIS (660 nm) within (a) 60 minutes,(b) 35 minutes and (c) 10 minutes of the center of the overpass window. Click image to enlarge. |
These findings compare well with the results of a comparison of MODIS and some 30 automated CIMEL Sun photometers in the AERONET network (Chu et al., 2000). That study validated blue and red MODIS channels. The correlation for the AERONET and MODIS red channels for the Central United States (Fig. 3(a)) within 30 minutes of overpass was surprisingly close to the comparison of the GLOBE and MODIS red channels within 35 minutes of GCO, r2 = 0.72 and 0.80, respectively.
Figures 1 and 2 show an offset in the data that suggests a bias in the AOT measurements by either GLOBE or MODIS. The slope of the regression line in Fig. 3 (a) of the NASA study is very similar to that in Figs. 1 and 2 herein, which suggests that the AOT measured by MODIS is biased slightly low. The results of the regressions are:
1. AERONET-MODIS Comparison (Central U.S. within 30 minutes of overpass):
MODIS AOT = _0.01 + 0.89 GLOBE AOT (within 30)
2. GLOBE-MODIS Comparison (South-Central Texas within 35 minutes of overpass):
MODIS AOT = 0.002 + 0.95 GLOBE AOT
3. GLOBE-MODIS Comparison (South-Central Texas within 10 minutes of overpass):
MODIS AOT = 0.027 + 0.79 GLOBE AOT
The mean of the regressions for GLOBE and MODIS within 35 and 10 minutes of peak overpass is MODIS AOT = 0.001 + 0.87 GLOBE AOT, which almost matches the NASA regression for CIMEL observations within 30 minutes of overpass.
Figure 3 shows solar aureole and full-sky photographs made during the MODIS overpass on each of the 5 days. The significance of these images is that the pair for 4 November 2001 (a.2 and b.2) shows cumulus clouds and cloud haze very close to the Sun. The pair for 1 April 2002 (a.5 and b.5) show a line of cloud haze east of the Sun. In spite of the cloud haze, the comparison with MODIS was excellent. This suggests that the cloud haze was uniformly distributed over the MODIS field of view on both days. The sky is much clearer in the remaining 3 pairs of images.
|
Figure 3. Digital imagery of the solar aureole (a.1-a.5) and the full-sky (b.1-b.5) during MODIS overpasses within 10 minutes of GLOBE Sun photometer observations at GCO. The position of MODIS at the center of its overpass is denoted by + in the full sky images. Click image to enlarge. |
Only 2 days of GLOBE-MODIS observations within 60 minutes of overpass were deleted, 14 February and 2 March 2002. Sky photographs for both days reveal considerable cloud cover on 2 March and a cloud deck to the east and west on 14 February. Thus, while the Sun was open during the GLOBE observations, the MODIS data were probably affected by the cloud coverage.
While these results are very promising, they require further analysis and more comparisons. For example, the slight difference in wavelengths between the GLOBE instrument and MODIS are not considered. These differences will cause slight changes in the results given here.
Ongoing measurements with the GLOBE instrument have yet to be compared with MODIS because of delays in the posting of MODIS data on the web. Measurements within 10 minutes of the MODIS overpass will continue indefinitely when conditions permit.
Finally, AOT measurements during MODIS overpasses are also made with various filter and LED microprocessor-controlled MICROTOPS Sun photometers. An AEROCET laser particle counter is used to measure the particles per liter of ambient air during the overpass. Finally, measurements of the ratio of diffuse sky to total sky irradiance at various wavelengths are made at solar noon. These measurements are made with LEDs installed in small housings with Teflon diffusers. (Teflon is a Trademark of the DuPont Company.) These supplemental data have yet to be fully analyzed. Soon these measurements will also be performed by an automated, temperature-regulated Yankee Multi-Filter Rotating Shadowband Radiometer in which all 7 detectors are LEDs (including 2 each of the GLOBE green and red LEDs).
Conclusions
Amateur scientists and serious students can perform a high quality validation of a sophisticated remote sensing satellite. In the case study described here, a carefully calibrated GLOBE Sun photometer has precisely validated a sequence of MODIS measurements of green and red AOT on 5 days having a green AOT of from 0.02 to 0.19 and a red AOT of from 0.01 to 0.11. The correlations for the comparisons are r2 = 0.989 (green) and 0.997 (red). The GLOBE measurements were made within 10 minutes of the center of the MODIS overpass window.
A NASA study (Chu et al., 2001) that compared AERONET CIMEL Sun photometer measurements within 30 minutes of the MODIS overpass yielded results surprisingly close to a comparison of 11 days of GLOBE and MODIS red measurements within 35 minutes of GCO, r2 = 0.72 and 0.80, respectively. The NASA paper and this present study both identified a slight, very similar bias in the MODIS AOT results for the Central U.S. The linear regression for AERONET and MODIS in the Central U.S. in the NASA study is MODIS AOT = -0.01 + 0.89 GLOBE AOT (within 30 minutes of overpass). The linear regression for GLOBE and MODIS at GCO is MODIS AOT = 0.002 + 0.95 GLOBE AOT (within 35 minutes of overpass).
These findings have important implications for satellite AOT validation, especially in light of possible calibration drift of the 660 nm channel of some AERONET instruments discussed in the NASA report (Chu et al., 2001):
1. A well calibrated, carefully operated LED Sun photometer assembled from parts costing under $70 US can validate AOT measurements made by MODIS.
2. MODIS validation is a feasible project for serious students who can find the time to make careful GLOBE Sun photometer observations within 10 minutes of the MODIS overpass.
3. Under the assumption that the bias of MODIS data over a specific area is known, GLOBE and other scientists can use MODIS observations to validate student measurements of AOT.
Ongoing work is aimed at comparing MODIS and GLOBE measurements elsewhere in the world. David Brooks of the GLOBE Program has developed important contacts in several countries and working hard to establish haze measurement programs. Here at Geronimo Creek Observatory, special emphasis is being given observations of high AOT in South Texas resulting from major episodic smoke events that occur in spring and summer and Sahara dust events that occur in summer and fall. Annual calibrations of GLOBE Sun photometers at Mauna Loa Observatory will continue to assess their long term stability. Hopefully there will be opportunities for more field trips to measure the atmospheric and biological effects of severe smoke pollution in Brazil and at major forest fires in the U.S. Meanwhile, it is hoped that this paper will motivate amateur scientists and serious students to begin satellite validation programs of their own.
Acknowledgments
I am deeply grateful to Shawn Carlson for the inspiration and motivation provided by his consistent support, encouragement and advice. I also greatly appreciate the ongoing encouragement and support of David R. Brooks, principal investigator of the GLOBE Program's haze monitoring project. Dr. Brooks developed the Excel program used for the GLOBE AOT measurements described herein. He also constructed the instrument used in this study. I am grateful to Brent Holben, Tom Eck, Yoram Kaufman, and Lorraine Remer of the Goddard Space Flight Center (GSFC) for helpful discussions. I especially appreciate Charles Ichoku of GFSC for his rapid responses to data inquiries and other questions. And I am grateful to John Barnes of Mauna Loa Observatory for permission to conduct annual calibrations of GLOBE Sun photometers. This work was sponsored primarily by the GLOBE program. The University of the Nations provided travel expenses to Hawaii and other assistance. A Rolex Award funded development of the original microprocessor-controlled versions of TOPS.
References
Brooks, D.R. and F. M. Mims III, "Development of an inexpensive handheld LED-based Sun photometer for the GLOBE program", J. Geophysical Research 106, 4733-4740, 2001.
Carlson, Shawn, "When hazy skies are rising", Scientific American 276, 106-107, 1997.
Chu, D. A., Y. J. Kaufman, C. Ichoku, L. A. Remer, D. Tame, B. N. Holben, "Validation of MODIS aerosol optical depth retrieval over land", http://aeronet.gsfc.nasa.gov:8080/PDF/Validation_MODIS_aer_over_land.pdf
Mims III, F.M., "How to Measure the Ozone Layer", Science Probe 2, 45-51, 1992b.
Mims III, F.M., "Sun Photometer with Light_Emitting Diodes as Spectrally Selective Detectors", Applied Optics 31, 33, 6965-6967, 1992a.
Mims III, F.M., "Satellite Monitoring Error", Nature 361, 505, 1993.
