Exascale: No. 1 priority for DOE Science Office

This week, an Energy Department official called exascale a priority. Next week, researchers meet to talk about making it a reality.

At a Jan. 30 hearing on the Department of Energy's modernization, Rep. Suzanne Bonamici (D-Ore.) asked officials how they planned to prioritize exascale computing. Research into next-generation supercomputing has pumped at least $1.8 million in DOE exascale funding into Bonamici’s home state, according to the University of Oregon.

Paul Dabbar, the under secretary for science at the DOE, reiterated what he said in a Jan. 9 subcommittee hearing when he called exascale research a priority. This time, though, he said it was the top priority for the DOE Office of Science.

Of all the priority projects under his purview at the DOE Office of Science, Dabbar said in the House Science, Space and Technology Committee hearing, “Exascale … is No. 1.”

Today's supercomputers perform operations at petascale, or a quadrillion calculations per second. But exascale will ratchet that up to one quintillion -- or 1,000,000,000,000,000,000 -- operations per second.

The DOE expects its first exascale computer, Aurora, to go online in 2021 at Argonne National Laboratory. Two other exascale machines, Frontier at Oak Ridge National Laboratory and El Capitan at Lawrence Livermore National Laboratory, are expected to be up and running between 2021 and 2022, according to a timeline presented at a recent Advanced Scientific Computing Advisory Committee meeting.

The U.S. isn't the only country chasing exascale computing. China is expected to achieve exascale levels by 2020 with the European Union and Japan expected to follow closely behind.

DOE’s Exascale Computing Project, a partnership between six national labs, university researchers and hardware vendors, has been working to ensure there are software and applications ready for exascale machines when they come online. It is focused on three areas: application development, software technology and hardware integration.

Douglas Kothe, the director of ECP and deputy associate lab director of ORNL’s Computing and Computational Sciences Directorate, said there are 25 application teams and 60 different software projects within ECP.

“The expectation is when the exascale systems roll out in the U.S. … all of these applications are going to be ready to roll on the system and begin to compute and to tackle their particular problem of interest,” Kothe said in an interview.

With exascale's computing power, researchers can run more complicated simulations faster, increasing their ability to make predictions and reduce models’ reliance on assumptions, Kothe said. Exascale computers are expected to be able to design chemical catalysts with little to no experimentation and run core simulators that speed the development of new nuclear reactors. They could also help increase the productivity of wind farms.

“Right now,” Kothe said, “wind farms are thought to [be unable to capture] 20 to 30 percent of the potential wind energy hitting the farm because of the turbine-to-turbine interactions. So being able to simulate 40 to 50 wind turbines” and combine that with other data like the effects of local weather and geography “could potentially allow wind farms to be more efficient.”

Exascale machines could also run national security applications, including modeling nuclear weapon performance based on stockpile stewardship practices, according to ECP Deputy Director Stephen Lee.

Besides the applications, ECP is preparing the software, or the “things required to execute and get useful information out of these simulations,” Lee said. This includes math libraries, solver libraries, programing models, visualization libraries and workflow tools.

But how does one create software and applications for a machine that doesn’t exist yet? With partnerships and modeling fueled by older high-performance computers, Lee explained. With these tools ECP will give birth to a new, even more powerful breed of machine.

The labs are making progress through a process called codesign, in which software and application teams work alongside hardware vendors at every step “to create new algorithms and new programing models tuned for these different architectures,” Lee said.

Researchers run simulations on different kinds of architectures and tools to predict various core algorithms and test pieces of software they know are important. Findings from simulations are shared with vendors who likewise share their insights.

It also helps to have access to the most powerful computers in the country. “We’re making heavy use of the largest systems in the DOE complex right now,” Kothe said.

Vendors are a key part of this process.  DOE wants to involve different suppliers to ensure applications can stay up and running even if one vendor falls out of the market, Kothe said.

“These applications are computer programs that are usually millions of lines of code with hundreds of man-years of investment,” he said. “So if we develop an application that only runs on one architecture -- and that architecture disappears in a few years or that vender disappears --  then we have a boutique application without a path forward.”

More vendors also means a more competitive landscape and potentially more innovation, Lee said.

Next week, ECP researchers will meet alongside vendors at the organization's second annual meeting in Knoxville, Tenn. The meeting, which runs Feb. 5-9, has been organized to address risks and challenges teams said they've encountered, Kothe said.

The agenda includes sessions like “An Introduction to Scalable Deep Learning with CANDLE,” “Algorithmic, Software, and Integration Challenges for Exascale Energy Applications,” and “Addressing Complex Memory for Exascale Systems and Applications.”

The Knoxville meeting will allow these researchers, who are working toward similar goals in disparate locations, a chance to meet and share best practices. So, for example, if one team figured out the best way to use a particular programming model, they can then share it with other teams.

“There is a cross-fertilization going on,” Kothe said.

When researchers cross the exascale finish line in a few years, however, it won’t be the end of the race. It will be just the latest step in a long line of advancements in computational power. The applications and software ECP is working on will live on to the next, post-exascale generation of supercomputers.

“Exascale is the next evolution in [a] computing power platform, in other words, a plateau,” Lee said. New goals will appear, new technology will continue to be developed. “Computing itself is transitory, new computers come and go," he said. "But software and applications are the things that are enduring.”