Super storage for scientific computing
Connecting state and local government leaders
Los Alamos National Laboratory developed systems software elements that use computational storage to achieve orders-of-magnitude performance improvement for scientific data analysis.
With a mission of solving national security challenges with science, it’s no surprise Los Alamos National Laboratory (LANL) wants to optimize computational storage by bringing processing power closer to storage devices or even into the storage system itself.
Computational Storage
Los Alamos National Laboratory
Click here for all the 2020 Public Sector Innovation winners
Complex scientific storage software stacks can’t take advantage of emerging high-speed storage devices because of insufficient compute resources on storage servers. Today, common storage system operations that require several passes over memory – such as data compression and indexing – just can’t keep up with fast storage devices and networks.
To address this, LANL developed systems software elements that use computational storage to achieve orders-of-magnitude performance improvement for scientific data analysis.
Two of the major efforts include extending the ZFS file system to use specialized devices that support the NVMe computational storage standard proposed by the Storage and Networking Industry Association. LANL announced last November a collaboration with Eideticom, a computational storage solutions provider, to create a high-performance Lustre/ZFS-based parallel file system.
LANL has made this branch publicly available and used it to deploy the world’s first storage system using compression enabled by computational storage. LANL has also developed a computational storage processing pipeline, a technique called embedded in-situ indexing, that enables scientists to attach high-value data indexes to scientific data, dramatically increasing the rate of scientific discovery.
“Memory bottlenecks make it impossible to perform critical storage system tasks, ” said Brad Settlemyer, senior scientist at LANL. With computational storage devices, the lab can “offload storage functions onto accelerators, enabling line-rate compression, improving CPU utilization and reducing memory bandwidth pressure.”
Simply adding more resources to build bigger computers does not always hasten the rate of scientific discovery, however. because complex science code can’t scale well. To support sustainable supercomputing, LANL must reimagine the design of high-performance computing platforms.
As the senior lab in the Energy Department’s system, LANL designs supercomputers such as Crossroads, which the lab announced Sept. 30, to replace Trinity, a supercomputer that increases the computational capability of the National Nuclear Security Administration’s Nuclear Security Enterprise.
LANL launched the Efficient Mission Centric Computing Consortium last year, and more than a dozen companies, universities and federal agencies joined it to study ways, including computational storage, to make supercomputers more efficient.
NEXT STORY: 21 Public Sector Innovation award winners