NOAA's satellite archive amasses environmental data
Connecting state and local government leaders
A National Oceanic and Atmospheric Administration archive of environmental satellite data uses different types of storage for different data availability needs.
A National Oceanic and Atmospheric Administration archive of environmental satellite data uses different types of storage for different data availability needs.The Satellite Active Archive (SAA) has three levels of storage: local cache for the most recent week's worth, network-attached storage devices for data products and partial data sets, and near-line robotic tape storage for everything in the archive.SAA is the online repository for NOAA and Defense Department Polar-Orbiting Environmental Satellite (POES) data and derived data products, said Alexander W. Kidd, SAA's program manager.Kidd is the sole federal employee supervising a dozen contractors from Computer Sciences Corp. at NOAA offices in Suitland and Camp Springs, Md.SAA went online as a prototype in late 1994, when the Web was just starting to take off. The online archive became fully operational in 1995.It distributes original data sets that came from polar orbiting satellites as far back as 1978, as well as products derived from them. Visitors to the SAA Web site, at , can register for free and search on such criteria as data source, date, time and geographic region.The data sets and products such as images derived from them are free of charge. Users can select and download the spreadsheets or GIF files to desktop computers.The site includes links to extensive documentation and help files.About 11,000 online users around the world request an average of 200,000 data sets per month, totaling about 1.5T to 2T, Kidd said. SAA also sends data periodically to about 400 subscribers, about a fourth of whom work for either NOAA or NASA.At the moment, SAA takes in data from three NOAA satellites and two DOD satellites, although one of the NOAA satellites is about to be decommissioned, Kidd said.Each orbiter circles the Earth about 14 times daily. A full data set is one orbit's worth of data, dubbed a Level 1b set. It can vary from 2M to 70M, depending on the type of data.Technically, Level 1b data doesn't come straight from a satellite. Mainframes process it from the raw Level 1a data at SAA's parent organization, the National Environmental Satellite, Data and Information Service, or at the Navy's Fleet Numerical Meteorology and Oceanography Center in Monterey, Calif.A newly upgraded Informix Dynamic Server 7.3 database system from Informix Corp. of Menlo Park, Calif., holds the metadata, the indexing information and the records about registered customers.The archive uses two NetApp filer storage appliances, one a midrange model F760 and one a large-scale F840, from Network Appliance Inc. of Sunnyvale, Calif. The F760 and F840 have maximum capacities of 3T and 6T, respectively. Kidd bought only enough hard drive trays in the beginning to provide 1T and 2.5T of storage space.He recently ordered more trays to bring the F760 up to 3T, and he plans to increase the F840's capacity to 4T. Each tray can hold seven 72G drives.The F760 storage appliance has been online for the past six months without problems. 'We don't have to worry about administration,' Kidd said.The near-line storage is an IBM 3494 robotic tape subsystem with IBM 3590 format drives. Workers have dubbed the 8-year-old system Big Bird because of its large robotic arm. Kidd next year wants to replace the tape system, which now holds 40T, with a more modern tape library.The most recent seven days' worth of data remains in local cache, made up of an IBM Serial Storage Architecture disk farm attached to an IBM SP server node, Kidd said. Up to 60 percent of all requests are for data that arrived within the last week.To speed up access, NOAA officials are considering using Network Appliance devices for local cache, too, Kidd said.For regular data backup, SAA has an offline digital linear tape system from Advanced Digital Information Corp. of Redmond, Wash.A few years from now, SAA's online storage needs will grow dramatically as two new satellites start beaming information back to Earth. One polar orbiter, a joint project of NOAA, NASA and DOD, will generate 1T per day after its scheduled 2005 launch, Kidd said.
www.saa.noaa.gov
Images are free, downloadable
Open sesame
An image of the Earth's sea surface temperatures comes from a data set beamed from NOAA's environmental satellites.
www.saa.noaa.gov
Images are free, downloadable
Open sesame
NEXT STORY: @INFO.POLICY