I have been itching to write on this subject ever since I first met w/ Doug Lora and Frank Chism. High-Performance computing – Wow! The first time someone explained the concept to me, I couldn’t help but visualize a scene from the movie “Blade Runner” and the futuristic feel of how Supercomputers actually work. My interest took me deeper into the heart of HPC to try to get my head around what HPC really is all about. High-performance computing systems, also referred to sometimes as “Supercomputers” are more prevalent today across prominent verticals such as Oil and Gas, Bioinformatics, Finance and Entertainment than ever before. Wikipedia described HPC as “ Supercomputers and Computer Clusters i.e. computing systems comprised of multiple (usually mass-produced) processors linked together in a single system with commercially available interconnects. Usually, computer systems in or above the teraflop-region are counted as HPC-computers” . An HPC Cluster is usually implemented to provide increased performance by splitting a computational task across many different nodes in the cluster.
Applications that run on HPC systems are prevalent in heavy-duty research and experimentation to engineering scenarios including transactional processing, data warehousing, computational fluid dynamics, virtual prototype testing etc. The evolution behind clustering technology has a lot to do with the growing adoption of this technology as well. The additional element that has made this technology very attractive is price. An HPC cluster can be implemented at a fraction of the cost today as compared to 10-15 years ago. Take a Cray Y-MP c916 supercomputer that cost close to $40 million 15 years ago. Today, you can get computing power very close to that for almost $4,000. The proof of this adoption is in the fact that every industry vertical is deploying HPC. From a “mainframe” approach that existed decades ago, the implementation trend of this technology is gravitating towards decentralized grids and clusters.
So why do I say that this is the way all computing will look - HPC is already a $9 billion growing market (source HPCwire). Evolution in this sphere is occurring at a blazing speed and the demand for HPC systems across various verticals is expected to multiply. Bottom-line – HPC will play a very key role in how computing power is used, stacked and scaled. Not only that, the development of HPC propagated file-systems will be an area that we all should watch very closely over the next few years. Let’s also fully realize the “impact” of this technology in the business space. If done right, HPC clusters hold the key to superior systems performance, while maintaining reasonable economies of scale. Delving into the benefits of these clusters, which until recently was a domain of the scientific community, is literally like lighting a fuse to an explosive. I say this with a strong ethos because we have yet to recognize all possible uses of HPC clusters with their underlying potential. According to some researchers at INIST-CNRS, “Analytic methods, statistical modeling, and pattern searching algorithms that are common in scientific computing can now be applied to the vast amounts of operational and historical data generated by business transactions to extract knowledge that can be used for competitive advantage”.
A nagging question still remained in my head as to the WHY behind the importance of HPC? I kept looking for the single reason behind the heavy investment in this area and why it’s such a critical component of highly-complex computational analysis being done. The biggest advantage or theme that emerged from wherever I looked was that HPC is one of the few tangible technologies out there, whose sheer computing power helps solve highly complex computational workloads and problems. This is not to mention the solitary advantage of using this technology – time. Time that it takes to resolve highly complex workloads is greatly reduced with a faster outcome. And anyone reading this blog knows the value of time and how it is THE most valuable element of all. And everyone knows, no matter how evolved and fast hardware can get, there will always be bleeding-edge problems that will demand processing power beyond what the best clusters can provide.
My endeavor here at OSSL, is to understand this topic from the ground up, have an open discussion on the subject matter as well as educate the audience along the way. And how do I plan to do that – well, we have started venturing into doing more with HPC and understanding the various HPC platforms and technologies out there. Over the course of the next few months I’ll be sharing more on this subject with all of you including market trends, evolution of HPC, Grid Computing Scenarios, “chip” supercomputing etc.
-Kishi