While I am excited that Pillar has topped the Storage Performance Council’s performance charts for $/IOP for business-class storage, I am sure it is temporary. All of the systems benchmarked use “currently available” drive technology, but both drive technology and controller technology are constantly evolving. The point of the benchmark in my opinion is that a vendor is competitive enough to be in the game. Clearly we are in the game, and are, in fact, leading the business-class pack for now.

This brings me to another point – what about EMC? Well, the truth is, EMC doesn’t play this game. Chuck Hollis pointed out in his blog that EMC finds the benchmarks to be too far off the real-world workload to consider them useful, and hence refuses to play. I think Chuck has a point, and I have blogged about it before. The problem with this is, you can’t have it both ways; you can’t just play when it suits you, and pass when it doesn’t. You either play or don’t.  EMC decided to set up their own utilization efficiency benchmark, and then roasted NetApps over the coals, along with HP and IBM, but they won’t play in other benchmarks like SPC-1.

Well if they don’t play, how did their results get posted? It’s old news, but NetApps bought an EMC Clariion, ran the benchmarks, and posted it for them! Damn that’s funny. What balls do you have to have to do that? Anyway, I suspect that’s why the Clariion showed so poorly in the configuration that we beat them in by over 2.5X…not because their product is really that bad. So please don’t use these Benchmark results and draw the conclusion that our product is 250% better than EMC Clariion; it isn’t.  It is our belief that our product is typically 20-40% better than a mid-range Clariion under real workloads and a reasonable price assumption like 37% off list price as HP used in their posting.

Wait a minute – can you really get results this inferior in a benchmark based on setup assumptions, tweaking and “configuration parameters” that figure into these things?  Really?

YES!!

Truth be told, people work hard to optimize their configs and setups to get competitive results; they don’t just happen.  So “setting one up for EMC” is a scam – nobody has the knowledge and motivation to set up a competitor’s system to perform its best. Doing so is almost as dishonest as comparing your RAID DP (the NTAP marketing term for RAID 6) system against another’s RAID 10 configuration for storage efficiency. Yikes, who would do that?

While we are excited about our first results, I also have to say that the Axiom does far better in real workloads than it does on SPC-1 benchmark workloads (We mentioned this in our press release on the subject). I believe that the Clariion will do better too, but not as good as the Axiom; perhaps the Clariion won’t even beat the NetApps machine. It turns out that if you were to construct a workload for NetApps WAFL, you couldn’t construct too much better a workload than the SPC-1 benchmark….which might give us all a tad of insight as to why EMC refuses to participate, really.

There is another point worth making here, for those who like to see BIG numbers (bragging rights).  3PAR lost to Pillar in this posting, but only for $/IOP.  3PAR actually posted the biggest IOP count of all the players. Well, remember when I said in a previous blog that these benchmarks were all fundamentally limited by spindle count? 3PAR posted a result that took over 1200 spindles and cost over 2M smackers (that is US smackers, not Euro-smackers). If we chose to build a $2M configuration of Axiom, we would post a larger number than 3PAR. So we say. Truth is, I am not sure you are ever going to see us do this, as I think it is kinda silly. It would temporarily get us bragging rights for sure, (we could post about 260,000 SPC-1 IOPS for this price) but who really gives a damn?