The seminars have been held in cities across North America (Toronto, Montreal, Boston, Bethesda, and Minneapolis), and the final one is scheduled for this Thursday, Dec. 3, in Milpitas, CA. Four SSD suppliers, including Pliant, will be presenting on the multiple benefits of adding SSDs to the storage infrastructure.

Not surprisingly, there’s quite a bit of expectation and discussion at the seminars that SSDs can deliver significantly higher I/O performance than hard drives. However, what’s surprising to me, having done five of the six seminars, is that people are already talking about “performance droop in SSD over time.” One of the vendors even stated, “fresh-out-of-box performance is different than steady-state performance.”

Now, I will spare you the discussion on how garbage collection affects SSD performance over time, as there are already plenty of articles on this subject. What is most troublesome is that performance droop should not occur at all. Solid state drives are supposed to alleviate performance bottlenecks, not introduce new ones. As such, a properly designed SSD controller should and must have sufficient horsepower so that a critical function like garbage collection will not impact I/O performance.

While we’re on the subject of performance, let’s talk about scalable performance.

Advanced interfaces, such as Fibre Channel (FC) and Serial Attached SCSI (SAS), provide access to two ports. For years, the secondary port has been relegated to sitting idle, used only when the primary port failed.

That’s a shame…and a waste, in my opinion.

Both the FC and SAS interfaces allow performance to scale when both ports are actively used. Given the fact that solid state drives are not physically constrained by a single read/write head, one should expect to scale performance by reading and writing to both ports at the same time!

The below charts illustrate the point:

You paid for both ports already. Why not actually use both?

If you would like to learn more about “scalable performance” and how you can best implement enterprise SSD storage in your IT infrastructure, I invite you to join me and the Bell Micro team at the seminar this Thursday. It will be held at the Crowne Plaza Hotel in Milpitas, CA.

You can find more detailed info here:  http://www.bellmicro.com/ssd/seminar.asp.

I look forward to seeing you there!

C.T. Chu

Nothing could be further from the truth.

I believe that this stems from the fact that the industry is stuck on using the HDD metric of $/GB and single drive cost as the primary measures of the cost. As I wrote in a previous post, “Storage managers getting wise to prevailing SSD limitations”, looking at historical or single drive cost metrics doesn’t accurately measure solution-level costs. So let’s try this again.

Yes, individual enterprise-class solid state drives (Enterprise Flash Drives) cost more than individual enterprise hard drives. So having stated this fact, let’s also be sure to state the fact that EFDs offer tremendous performance boosts (>100X), and can replace many 15K RPM HDDs. Budget constraints require that enterprises and data centers focus on maximizing both performance and efficiency, so transaction cost ($/IOPS) is also a key metric.

The goal is to provide a storage solution that optimizes for both $/GB and $/IOPS.

Let’s look at a typical data warehousing application from the TPC-C benchmarks (http://www.tpc.org/tpcc/results/tpcc_perf_results.asp). The storage solution must provide 640,000 transactions/minute (320,000 IOPS) for 18 TB of data. With a typical all-HDD solution, this requires:

• 1000 15K 2.5-inch HDDs (short stroked to 18GB)
• 40 rack mounted shelves
• 8000 watts to operate and (an additional) 8000 watts to cool
• Price tag = $ 450,000

Now, let’s look at how a ‘hybrid’ approach combining EFDs and existing HDDs can not only provide a lower transaction cost, but also a lower cost/GB and a lower total cost. This hybrid solution would be configured as outlined below:

Not only does the hybrid approach offer a much lower $/GB and $/IOP (and requires 34 fewer shelves), but the total cost is one-half that of the HDD-only configuration.

Did you catch that?  One-half the total cost.

At the end of the day, the numbers don’t lie. The value proposition of EFDs is simple, it provides ‘more for less’ – more performance for less cost, less power and floor space, and more reliability. And, EFDs can be managed with existing software.

What will IT managers do with all the savings?

Amyl Ahola

This ‘revelation’ appeared in a SearchStorage.com article by Beth Pariseau, “Storage admins mull SSDs at SNW.”  The article quotes multiple storage administrators who all basically believe in the benefits of SSD, but stop short of saying that the technology is ready for prime time.

Here are their top concerns: predictable performance, data integrity, the lack of consistent, industry-accepted SSD benchmarks, and cost.

Let’s quickly look at each of these:

1. Predictable performance – I covered this recently in my “’Predictable performance’ for changing business dynamics” post. This area has traditionally been a challenge for SSDs in enterprise applications because workloads are random and indeterminate. Predictability requires consistent performance, independent of whether reading or writing data, because enterprise applications typically vary the read-to-write ratio between 60/40 and 90/10. Enterprise SSDs should be able to maintain performance across this range.
2. Data integrity – I couldn’t agree more that data integrity features are critical if flash technology is to perform at enterprise levels, and the Data Integrity Field (DIF) standard is an important step in this direction. Yet, today so few storage devices support the DIF standard. Pliant began mapping toward the DIF standard early on, recognizing how important it was for enterprise-class storage systems.
3. Standardized benchmarks – In my post, “SSD jargon and the need for standards,” I listed a number of pivotal questions that must be addressed if the industry is ever to develop more accurate, relevant – and yes, consistent – SSD benchmarks. These include making sure that real performance is measured and that product lifecycle benchmarks are based on true, 100% duty cycle operation. If product life metrics are contingent on usage limitations – e.g., based on a maximum number of writes or writes per day due to limited error management capability – then the benchmarks are virtually useless.
4. Cost – Transaction cost (IOPS per $) is the key SSD metric to consider, not the old HDD industry metric of $/GB. This metric is an irrelevant measure of SSD value as a performance solution, and we expect EFDs (Enterprise Flash Drives) to complement high capacity HDDs to optimize for both $/IOP and $/GB.

With most existing vendors either falling short on a number of these points, or masking the limitations of their devices behind carefully crafted marketing spin, it’s no wonder why some storage admins are still skeptical.

This is why I continue to extol the values of EFDs, a new class of solid state storage devices designed with key enterprise considerations in mind. By definition, EFDs are designed to address all of the above issues.

And, as we prepare to announce availability of our first products shortly, my hope is that our approach will help turn the heads and change the minds of the remaining nay-sayers in the industry.

Amyl Ahola

This poses a huge problem for enterprise IT managers. Why? 

Quite simply, enterprise information demands continue to increase with no end in sight.  And, data center managers will have to do anything and everything in their power — without making significant new IT capital investments —to keep up with the increasing IT system performance demands. 

Failure to do so will be unacceptable, so what are the options?

Two things come to mind:  1) optimizing existing IT systems for increased performance; and 2) significantly reducing the energy consumption of power-hungry high RPM hard disk racks.  Is this difficult? 

It may be easier than one thinks and requires no change to the existing infrastructure, management software or systems.  By adding Enterprise Flash Drives (EFDs) to handle the performance workload of many spinning hard drives, both goals can be achieved.  The high performance of the EFD enables more I/Operformance and flexibility to meet peak periods and growing demands.  By combining EFDs with high capacity HDDs, today’s storage racks can be reduced to storage shelves saving power (up to 80%), space and money. 

I predict that beginning in 2009, EFDs will be a key tool for enterprise IT managers to survive the economic turmoil while optimizing their existing storage systems.

And, let’s face it, it’s time for a change to the traditional approach to high-performance storage solutions. 

Interested to hear your feedback, so please feel free to comment.

Amyl Ahola

So, I think a description and definition is in order. 

In the world of disk drives, enterprise-class products are distinguished from desktop and laptop products by their ability to provide superior performance and reliability.  This means that they are expected to perform flawlessly in mission critical environments.  This same requirement also holds true for enterprise SSD devices.  However, just like lower-end disk drives, SSDs designed for laptops and desktops simply can’t pass muster when expected to provide the performance and reliability required in a mission-critical enterprise environment.  There are a number of existing SSD products marketed for the enterprise, many of which are nothing more than re-packaged consumer grade (laptop) SSD technology.  In fact, many of the so-called “enterprise SSD” drives actually underperform HDDs in laptop applications…hardly what I would call enterprise class. 

Therefore, a true EFD must provide high levels of performance and reliability for flawless operation in mission critical, I/O-intensive environments.  Given the growing power and space concerns of today’s large enterprise environments, reduced energy consumption is becoming an equally important criterion for any new class of primary storage devices.  An EFD’s superior performance, energy efficiency and improved reliability allow data centers to substantially grow capacity and performance in existing installations while reducing energy needs and TCO.

Given these requirements, an Enterprise Flash Drive should, at a minimum, provide the following:

1. Superior I/O Performance – Adequate I/O performance levels to prevent bottlenecks, even during peak activity periods (generally 3-5 times greater than typical activity periods), without requiring extra hardware (i.e., cache)  while providing ample scalability for growth.  At a minimum, an EFD should deliver at least 100,000 random IOPS or more and be able to sustain this rate for typical block sizes (4K bytes or more). 
2. Exceptional Reliability – EFDs need to deliver significantly lower failure rates than disk drives, given the inherent benefit of solid state technology (no moving parts).  Performance and reliability must be predictable and sustainable at 100 percent duty cycles (24/7/365) without cycle-stealing maintenance or “housekeeping” actions.  Lifetime should exceed five years without performance or capacity degradation.  Robust reliability monitoring and reporting capabilities are essential.
3. Energy Efficiency – EFDs should meet new standards for green data center excellence of greater than 20,000 IOPS per Watt, with activity-based power management to limit energy consumption when the device is less than 100 percent utilized.
4. Cost Efficiency – Transaction costs ($/IOPS) must be substantially reduced from that of an HDD (<10%).  And, it goes without saying that an EFD must be form factor and interface compatible with HDDs (while providing similar storage capacities).

While these requirements are very demanding, I believe they only begin to define the needs and ability of solid state technology to transform future system and storage architectures.  In my opinion, the vast majority of today’s SSD products are already falling short of the true needs. 

Interested to hear what you think…

Amyl Ahola

Enterprise storage applications are demanding, and it is essential that reliability specifications are met at a 100-percent duty cycle operation on a 24/7/365 basis.  Those in the industry know that true enterprise-class disk drives are required for this environment, and that disk drives designed for low cost and low duty cycle laptop/desktop applications literally fall apart when employed in an enterprise application.  Likewise, SSDs designed for laptop/desktop applications also do not even come close to meeting the need.  So, for Enterprise Flash Drives to be accepted in the enterprise they must meet or exceed enterprise class HDD reliability. This is not a trivial task. 

The primary enterprise reliability specifications take the form of MTBF (or more meaningfully: annualized failure rate) and non-recoverable error rates (lost data).  Flash technology has three primary failure phenomenon that have a significant impact on reliability:

• Write Endurance – the limit on how many times a cell can be written/erased before it becomes damaged
• Write/Program Disturb –  writing to a given page in a Flash chip can alter bit(s) in a page that is not being written (does not damage the cell); this is sometimes referred to as “bit flip”
• Read Disturb – similar to Write Disturb, reading a page in a Flash chip can alter bit(s) in a page not being read (does not damage the cell)

A further complication is that these failure modes are not independent.  For example, the read disturb error rate is related to the number of writes or erases so that write endurance and read disturbs (and write disturbs) must be holistically considered.  It is obvious that they all contribute to non-recoverable errors, but perhaps not as obvious that they contribute to MTBF as well.  MTBF is a measurement of performance to specification, not just to some catastrophic event, as is typical with a disk drive.  This includes meeting performance and capacity specifications.

A common approach used in typical SSDs to deal with write endurance is to incorporate a wear-leveling algorithm to distribute writes across blocks within the chip(s), together with error correction (ECC), so that any damaged cells can be corrected when read.  This same ECC can then be applied for all reads to detect and correct altered bits (‘bit flips’) independently of how they became defective, i.e., write endurance, read disturb, or write disturb.  If the number of defective bits exceeds the ECC threshold, the sector(s) being read would then have to be marked as defective (non-recoverable error) and made unavailable to the system.  Depending on the amount of spare Flash capacity, at some point the resulting system capacity may well drop below the specification.

As an example, a well-known supplier of SSDs advertises an ECC that corrects up to 8 bytes in 1024 bytes, while another supplier advertises 6 bytes in 528 bytes.  At the same time, both talk about program erase/write cycles well in excess of 1 million.  However, tests show that both ECC levels would frequently result in non-recoverable errors after as few as 200,000 write/erase cycles.  These error rates result in SSD reliability falling far short of disk drive reliability in terms of non-recoverable error rate.  At the same time, overall capacity begins to erode and eventually falls below the device specification, resulting in an MTBF failure. 

And, that’s not all.  There is also a significant performance impact resulting from the management of these high error rates (It drops dramatically!).

The primary point is that enterprise-level reliability, whether it’s MTBF or non-recoverable error rate, can not be addressed with just traditional ECC.  Other techniques must be employed in addition to ECC to manage errors.  In addition, these additional techniques cannot be allowed to significantly impact performance (IOPs or bandwidth).

Sounds like a daunting task…or is it??  Stay tuned.

 Amyl Ahola

However, one topic that I’m afraid may not receive its fair share of MSM attention is the rapidly growing problem of data center power consumption. 

Here’s the issue.  According to a recent report (http://www.energystar.gov/ia/partners/prod_development/downloads/EPA_Datacenter_Report_Congress_Final1.pdf), servers and data centers account for about 1.5 percent of all U.S. energy consumption, or 61 billion kilowatt-hours (kWh).  This is more than the electricity consumed by the nation’s color televisions in a year, and about as much energy used to power 5.8 million average U.S. households.  And, at the rate our digital information requirements are growing, server/data center energy consumption will nearly double to 100 kWh by 2011, which represents about $7.4 billion in annual electricity costs. 

So what can we do to move rapidly to greener data centers?  The worst offending part of the system, the misuse of HDDs, should be among the first to be dealt with.  The fact is that many IT managers are using 3 to 4 times more HDDs than they need from a capacity perspective just to meet growing I/O performance requirements.  This “over provisioning” does not only fail to meet I/O performance needs as I noted in earlier posts, but it’s probably one of the most inefficient uses of IT technology I’ve ever seen.  Talk about a waste of space and power (not to mention money)!

With data centers under constant pressure to operate more efficiently and reduce costs, this type of waste is ridiculous, especially when there are other viable alternatives available.  One technology that deserves serious attention is the Enterprise Flash Drive, which is based on solid state technology to offer extremely high data I/O performance. 

Here’s an example of the benefits of deploying EFDs in the enterprise, without breaking the bank.  A hybrid solution combining existing hard drives (preserving some of the initial investment) with selectively deployed EFDs can greatly enhance I/O performance while eliminating the need for HDD over-provisioning.  Best of all, this type of approach can slash data center energy consumption – up to 80 percent in some cases.

Let’s face it, enterprise data centers will continue to push the envelope in terms of performance and capacity requirements.  The trick is finding ways to meet these demands in the most efficient and cost-effective way possible, and EFDs can be a great option for many organizations.

Amyl Ahola 

What I’d like to discuss is the price performance aspect of this announcement.  The systems tested were fully mirrored, making comparisons never quite “apples to apples.”  However, one needs to keep in mind that the MTBF of the drives employed require mirroring to reach any reasonable reliability level.  While I could not find any real price or performance data on the company’s web site, the reference to their SPC benchmarks provided considerable data.
 
From a pricing standpoint, the 1.03TB configuration sells for more than $36 per gigabyte (after a 40% discount from $60/GB)…and, flash-based SSD at $30/GB is considered expensive?
 
This benchmark is also said to be record-breaking with the lowest cost per SPC-1 IOPs.  I’m not suggesting that $36/GB is unreasonable, only that it illustrates the true cost of hard drives in high-performance environments.  A closer look at the benchmark is even more telling.  This “record-breaking” performance correlates to a response time of nearly 30 milliseconds.  In fact, response time increases dramatically starting at about 50% of the max IOPs, which is certainly troublesome for high transaction-rate systems.

This project was started a few years ago, apparently to address the growing price, performance and reliability gap in enterprise applications, as we have been talking about, and to hold off the encroachment of solid state storage devices.  However, with today’s technology, well designed Enterprise Flash Drives will not only be lower in cost per GB, less than 1/4th the cost per IOP, and more reliable.  And, did I mention power:  EFD’s will be well less than 1/100th the watts per IOPs.  I cannot help but be reminded of the Anderson Cooper segment on CNN:  “What were they thinking!”

Amyl Ahola

Some of the results are:

• Exponential increases in the size of operating systems and applications
• Mass deployment of low-end and midrange servers with multiple copies of data (and applications)
• Over-provisioning of storage to satisfy future needs projections (which also likely adopt the concept of free storage)
• Adoption of power-hungry DRAM cache appliances to mask HDD performance shortfalls
• Over-provisioning of HDDs to mask HDD performance shortfalls

These all result in inefficient use of storage that has many costs, not the least of which is the increasing cost of energy consumption.  Some of the energy data becoming available paints a sobering picture:

• Data centers account for 1.5% of ALL U.S. electrical consumption, and this is expected to double in a few years
• Power consumption per $1,000 of server spending has increased by a factor of 4 since 2000
• Power failure and availability is expected to halt data center operations at more than 90% of all companies over the next few years
• Fifty percent of current data centers will have insufficient power and cooling capacity this year

HDDs are clearly not the only contributor to the rapid acceleration of data center power consumption, but their inefficient use is likely one of the largest contributors.  Data that suggests more than one third of data center power consumption is storage related.

Trends and techniques such as consolidation, virtualization and thin provisioning should all contribute to improved efficiencies.  But while doing so, these approaches will put increased performance demands on the HDDs.  The result:  an increased need for higher performance (i.e., higher RPM……read that as ‘power consuming’) drives and even further over-provisioning for performance – and therefore once again increased energy consumption.

It’s time for new metrics to be considered in the data centers, which take into account energy usage to aid the system designers as they optimize their systems.  Several metrics are identified at the www.greendatastorage.com website; examples cited include activity per watt, such as transactions/Watt, IOPs/Watt, and bandwidth/Watt.

I believe that Enterprise Flash Drives (EFDs) will play a major role in reversing these trends. EFDs can provide over 1000x improvement in IOPs/Watt, and an order of magnitude or more improvement in bandwidth/Watt over the highest performing HDD’s.

Amyl Ahola

The question I’m exploring is what can be done about the growing gap between disk drive and enterprise network performance, as well as the escalating inefficiencies?   One only has to look at the root cause:  the mechanical nature of disk drives.  The solution is obvious; eliminate the mechanics. 

Easier said than done! 

The Holy Grail for primary storage has always been directly addressable low latency, non-volatile random access memory.  This remains a long way off, but it is time to begin the next evolutionary step.  Solid state technology (particularly Flash) cost and performance continues to improve geometrically, and new and even more competitive semiconductor storage technologies are around the corner.  Meanwhile, disk drive performance (seek, latency) is stagnating, with only limited foreseeable improvements and with cost per I/O leveling off or even beginning to increase with time. 

Last year Greg Schulz of the StorageIO Group predicted the increasing use of solid state technology in enterprise storage applications, saying (paraphrased) that 2008 will be the year of awareness and early adoption by vendors and early deployment by customers, while 2009 will be the broader adoption phase.   Supporting that projection, EMC has recently announced their commitment to Flash and is the first major enterprise storage company to do so (http://www.emc.com/about/news/press/us/2008/011408-1.htm).  Although it was a limited announcement with what I consider an ‘entry level’ SSD technology, it is the first step towards validation of Flash technology as an enterprise primary storage device.

Solid state storage has the potential to be transformational, relegating disk drives to applications that better match their strengths, low cost per GB and large block sequential applications (for the old timers amongst us, it should be noted this is similar to the role disk played years ago with respect to magnetic tape drives).

But Flash comes with its own set of problems…(Stay tuned)

Amyl Ahola