The storage industry has come a long way, but… (Part 2)
As I thought more about the topic of my last post – how far the storage industry has come since its inception – another point occurred to me. While disk drive capacity and cost achievements have been incredible, orders of magnitude improvement, disk drive performance gains are unremarkable – especially when you compare them to the significant advances in CPU and network performance.
Now, I don’t want to receive an inbox full of angry emails (angry comments are welcomed!) about this, so let me make it clear that I truly appreciate the technological challenges and the progress that has been made towards reducing disk latency and positioning times. But, at the end of the day performance improvement is less than 40x in nearly 50 years! This compares to multiple orders of magnitude improvement in CPU performance during the same period.
Amdahl’s (other) Law requires that I/O performance improve at the same rate as CPU performance to maintain balanced system performance. However, with the lag in disk drive performance I/O over the years, what we have now is a growing gap that system designers have had to cope with in their attempt to balance system performance. The result: the birth of new industries so that the system designers can add additional hardware – such as cache and RAID together with short-stroking and over-provisioning the disk drives – in an attempt to overcome the performance and reliability shortfalls due to the mechanical nature of HDD’s.
While these approaches do improve performance to some degree, they also carry a significant cost to customers. This is due not only to the cost of the additional hardware and software but increased system complexity, increased power consumption, reduced reliability, increased floor space, increased maintenance expense, and on and on. What is the true cost of HDD performance….it is anybody’s guess, but I’d argue that it is far greater than what is generally believed!
I believe that this is the most important data storage issue that needs to be addressed. In particular, how can the industry solve the I/O performance problem without even more patches (e.g., more cache) and ever increasing over provisioning?
I have some thoughts that I’ll share next time. In the meantime, I’d love to hear from you.
Amyl Ahola
