IT Case Study: Lawrence Livermore National Laboratory
Kim Minuzzo, LLNL Facility Project Leader
Lawrence Livermore National Laboratory Considers Improved Scalability of InfiniBand™ Architecture for World's Largest Computer
Even the biggest computer in the world has budget restrictions. Lawrence Livermore National Laboratory is looking for ways to increase performance while reducing the overhead of proprietary technologies in its data center.
Founded in 1952, Lawrence Livermore National Laboratory (LLNL) is a national nuclear security laboratory, with a mission to ensure national security and apply science and technology to the important issues of our time. LLNL manages one of the world's largest computers, a data center with more than 500 clustered server nodes, designed to give physicists and researchers the computing power to ensure national defense and solve environmental and medical challenges.
LLNL's data center has 150 terabytes of disk storage on a global parallel file system of 512 clustered nodes. With 100 to 200 users accessing the network per day, and around 5,000 per year, data mining is a non-trivial task. Although processing speeds of each node exceed 1.7 ghz, the infrastructure between each node isn't capable of maintaining the processor's rate. To get beyond current interconnect bandwidth limitations, Kim Minuzzo, LLNL Facility Project Leader, can use proprietary technologies, but the cost greatly exceeds that of a standards-based server I/O technology.
Enter InfiniBand™ Architecture
Hungry for greater performance using a standards-based interconnect, Minuzzo is intrigued by InfiniBand Architecture.
"InfiniBand makes sense as we increase our link speeds to 10 Gigabits per second," Minuzzo said. "Currently we use a mix of proprietary interconnects, which is difficult to maintain and often expensive to implement. We'd much rather use something that works heterogeneously, like an industry standard such as InfiniBand."
InfiniBand Architecture provides aggregate link bandwidth speeds of 2.5 Gbits/second to 30 Gbits/second, while scaling to thousands of nodes. Rather than sharing a common bus, InfiniBand Architecture provides a fabric, giving all connected devices a unique link to the server processors.
Minuzzo also cited improved data center manageability and node scalability as possible benefits of InfiniBand Architecture.
"The ability to add thousands of nodes with no depreciation in performance is particularly appealing," he said. "We add dozens of nodes a month to the data center, so scalability is a major concern. In addition, reducing the management overhead from many interconnect technologies to a single subnet is appealing in terms of decreasing the operating costs."
Given LLNL's need for performance, scalability and standards-based technologies, Minuzzo joined the InfiniBand Trade Association IT Program, designed to engage leading IT customers, demonstrate the new technology and solicit feedback.
"We're excited to become engaged with the InfiniBand Trade Association and learn more," he said. "Given the size of our data center, the potential benefit of InfiniBand Architecture to Lawrence Livermore is significant."