Some thoughts about the SAP 8-socket benchmark at HP and Sun

Quicksearch

Disclaimer

The individual owning this blog works for Oracle in Germany. The opinions expressed here are his own, are not necessarily reviewed in advance by anyone but the individual author, and neither Oracle nor any other party necessarily agrees with them.

Navigation

< Case design | PSARC 2009/177 - Solaris support for Rainbow Falls platforms >

Some thoughts about the SAP 8-socket benchmark at HP and Sun

Friday, December 11. 2009

I thought a while just write another comment about the HP785G6 benchmark in the comment section of the "Case Design" or the Do more with less" article. At the end i thought, this will be a fine article.

At first: Yes, the benchmark of Sun used 256 GByte, whereas as the HP 128 GByte. At first this sounds like benchmark not comparable to each other. One of the basics in SoHo benchmarks is to compare benchmarks with the same memory size. The problem: You can't use this knowledge here.

But: The amount of memory just matters in SAP-SD when you don't have enough. When you don't believe me, ask your favorite vendor of IT equipment and ask for SAP-SD numbers with different memory sizes. The answer will be similar. To be more exact, the speed of the memory has a bigger impact than the size of the memory, given that you have enough memory.

Another reader summarized it in a good manner: If HP could get a better result just by adding memory, they would have done it. It's difficult to increase compute power when you are already using the fastest CPU. It's difficult to increase the core count when you are already using the the CPU with the highest number of cores. But just plugging more memory into the system is easy. I haven't heard any news about HP having a shortage of memory modules.

But it think there is a different reason, why HP just used 128 GB. When you look at page 15 of the quick specs pdf of the HP DL785G6 speed you will find the following note:

When only PC2-6400 DIMMs modules are installed with a processor then memory bus speeds for 4 or fewer, 6 or 8 DIMMs per processor will operate at PC2-6400, PC2-5300 and PC2-4200 respectively. All other processor and memory configurations will operate at PC2-5300 with 4 or fewer DIMMs and PC2-4200 with

The largest memory flavor with PC2-6400 is "8 GB REG PC2-6400 2 x 4 GB". The DL785 has 64 DIMM slots. To keep the memory bus at PC6400, you can just populate 32 of them. 32x4=128GB. They've ran the system at the top configuration that allowed them to use faster DIMMs. But you just to that, when you have enough memory in the smaller configuration, because the impact of having not enough memory is much larger than the impact of 133 MHz more memory speed.

From this point it get's even more strangely: The X4640 didn's just had slower a slower CPU (200 MHz), it has also a tad slower memory (a X4640 use PC2-5300 for all memory configurations). This makes the difference between the systems even larger as suggested by the pure SAPS numbers.

For the database something similar is valid. Roundabout 3-4% percent of an SAP-SD benchmark are database-bound and basic math suggests, that even a database 25% faster has a rather small impact on the database.

Of course it would be nicer to have an exact proof by having benchmark results just differing by a single point but we have to live with it, that such results aren't available.

But Sun is in a nice situation here, when you look at the situation with some logic. The vast difference between the HP DL785 and the X4640 can be explained by three major points:

The X4640 is vastly better than the DL785.
Solaris is vastly better than Windows/Linux.
The benchmarking team at HP is just outright incompetent.

I think we can say that the third point can be erased from the list. I know HP as a competent company. As usual the an organisation of such a size is a victim of the bell curve, thus it has it share of dumb asses and outright geniuses. So your personal experience may differ, but let's just assume that HP is competent enough to tune a SAP.

So we are at:

The X4640 is vastly better than the DL785.
Solaris is vastly better than Windows/Linux.
~~The benchmarking team at HP is just outright incompetent.~~

Leaves us with "better OS" or "better HW". Either way it's nice for Sun and that's exactly the reason why we won't see an Solaris/DL785/Oracle result from HP. Such a benchmark could have two outcomes: The DL785 yields the same result than the X4640 or it delivers worse result. The first would show that Solaris is a better operating system than Linux, the second outcome would show that the X4640 is a better system. Both is positive for Sun, both outcomes are negative for HP.

When the system is 25% slower, this would really kill the DL785. If Solaris is 25% better, then this would kill all the arguments that Linux is as scaleable as an enterprise Unix. If it's a combination of both, well ... then this just proofs that Sun has a more competitive package. I'm fine with all three outcomes.

Of course you could ask "Why does Sun don't deliver an Windows/Linux result?". Well ... honestly: Would you believe us? We have our own operating system and i'm sure the prefered excuse when the benchmarks with Linux or Windows are made by Sun would be "Sun didn't tune Linux correctly, they just want to make Solaris looks good". Of course Sun won't make up the results, as this is too easy to detect and to confirm and would habe an horrible impact PR-wise. And at the end it's HPs task to show that their system is comparable, not ours.

Posted by Joerg Moellenkamp in English, Solaris, Sun/Oracle, The IT Business at 09:12 | Comments (20) | Trackbacks (0)

View as PDF: This entry | This month | Full blog

Trackbacks

Trackback specific URI for this entry

No Trackbacks

Comments

Display comments as (Linear | Threaded)

First of all, lesson number 1 of professional journalism: you don't use your publication to insult individuals who had a different opinion than your own, especially considering your affiliation with Sun. You have a history of doing this. You seem to have a grudge with me, SteveA, and even commenters on Storagemojos who said F5100 sucked.

Having said that, it is late at night, I would take a brief few seconds answer your accusations that "I will make a fool of myself"

1. OS comparison
I have found that Solaris is faster than Linux at 8 socket or above. The windows benchmark if I read correctly, used Windows Server 2008. R2 had a major vertical scalability improvement, going from 64cores to 256 cores. I don't have concrete data comparing Solaris 10 update 8 vs Windows 2008 R2, but I do have concrete data comparing Solaris 10 update 8 vs CentOS 5.4. So I can give it to you right now, that I agree with you Solaris is vastly better than Linux at 8 Sockets.

2. Ram comparison
According to http://download.sap.com/download.epd?context=B1FEF26EB0CC34664FC7E80B933FCCAC80DD88CBFAF48C8D126FB65D80D09E988311DE75E0922A14
The X4640 config mentions nothing about memory configuration, ie, memory size, memory speed, number of DIMMs. So I am suspecting that Sun used 32x8GB DIMMs vs HP's 32x4GB DIMMs. 8GB DIMMs DDR2-6400 can be found. It is called MetaRam. They are bankrupt, but Hynix uses their chips to build 8GB and 16GB modules. Rare but possible. Even if it is slower, you have shown no data showing that the SAP benchmark's working size is less than 128GB. That also ignores file system caching, which we know ZFS is a hog. Until you take have the ram out, you don't have data to support your thesis that 256GB ram didn't help the benchmark.

3. CPU comparison.
You keep on mentioning that the HP system had a 200MHz CPU advantage. You are right. However, you forgot to mention the asynchorous northbridge clock that the Opterons have(so do Nehalems).
www.sgi.com/company_info/acceleratingresults/six_core_compare.pdf
All of the non-HE Istanbul Opterons are capped at 2.2Ghz northbridge clock. That's fundamentally the bottleneck. 200Mhz CPU advantage means crap when your CPU waits on the IMC2.2Ghz.

4. X4640 is better than DL785G6
True only if you are retard. X4640 only has 4 SAS drive bays, so you have to either use Fiber channel SAN with it, or SAS Adapters. DL785G6 has 16 built in bays. If the X4640 were to use a 2U 24 bay array like the Witte or Dell MD1120, it would have been a 6U solution. Just 1U shy of the DL785.

Thanks for wasting another 15 minutes of my time. Next time tell Ginesh the benchmarketer to take half the ram out, bench it with 128GB with Solaris, and then again with Linux to show how much 128GB of addictional ram contributed to the score, and how much Solaris contributed to the score. I stand with my original assessment that X4640 and DL785 are interchangable, and both obsolete-ware in 3 months.

#1 TS on 2009-12-11 11:37 (Reply)

You still don't get it? I'm not a professional journalist. I don't want to be one. I'm an admin and a presales engineer. This blog is just a pet project. But you are correct ... i have deleted the "fool comment". But don't consider this as apology. I just thought other readers will recognize it on their own. Personally i'm still considering you as a major annoyment, but in a strange kind it's really amusing. Without this amusement i would have put you on my mental ignore list long ago. And on the other side i have to set your stuff in a context for other readers

But i will give you a hint: You have a problem, even when you have a point, you totally destroy your credibility singlehandedly in the same mail. For example this statement about "DL785 is better because of the internal disk". Just in case you don't know it ... you don't use such systems with local storage. You have a SAN or a NAS for your VMs, for your database data pools. Have you thought how to an cluster failover (aside of a shared-nothing clusteR)without shared disks? How to do a live migration without shared medium?

That you don't think in enterprise terms is just obvious in many in your comments. The F5100 has a shortfall indeed, but it isn't the one you think (which is none anyways). For someone fluent in enterprise storage architectures this one is pretty obvious.

They may be use cases for this vast amount of disks (i assume HPC tasks where you could use this as local scratch space) but for the usual usecase in IT those room is just wasted. Many enterprise customers would kill us, if we had integrated a whole load of disk slots they aren't using.

When something doesn't fit in your perspective of the world you just turn it around. Cooling down a full rack of X4640 isn't a challenge for quite a time since the industry learned how to cool a rack full with HP blades, IBM Blade Center or Sun B6000. But you want to think of a challenge, as otherwise you would have to admit, that the 4HE are quite an USP for other customers.

The same is valid for many other things you've said in all the comments: The superduper Intel SSD card with the performance numbers, that turned out to be the number of a whole load of them. Your idea to place more SSD in a storage array, but forgetting that you can't get the IOPS out of the system. Or complaining about hard disk prices at Sun, while the harddisks at Dell and HP have almost the same price

And told this, you had the great reply "I second-source the hard-disks anyway at new-egg". Obviously you never tried to open a call at any vendor with such a disk in it

And your "finger in the wound" are more like the finger in the air because of "Please explain this again, i didn't understand this".

Another example is your statement about memory: I know there are large and fast 8 GB dimms, but when you look at the support matrix of HP there aren't there. But you can't use them in in both systems. Perhaps they would even work. But they aren't supported. There isn't an order number at the HP page for such memory in this machine. And before you say this: No, you don't use second-sourced memory in mission-critical server. And it should be pretty obvious that Tier-1 vendors like HP and Sun are somewhat cautious about using a technology just available from one vendor with when they have to fulfill long term maintenance and spare part agreements. But that is the same reason why you can't jump from one CPU to the other just because one is faster or cheaper. Companies have no understanding when one system with the same name may contain an AMD, the next system a Nehalem ... and later perhaps a AMD again and they have to have spare parts for 3 different systems. Such a move has to be well decided and well planed. It's a little bit different thought model than just second-sourcing hard disks at New Egg ...

You don't plug second-sourced memory in such a system. For Sun this isn't a problem. You you would had used the time to look in the specs of the X4640 you would have seen that the memory clock is 667 MHz at 4, 6 and 8 DIMMS per CPU. So there is an advantage for HP at 4 DIMMS, a tie at 6 DIMMS and an advantage for Sun at 8 DIMMS. However using 8 GB DIMMS would limit the HP system at 667 for 4 and 6 DIMMS and 533 for 8 DIMS.

So when you consider commenting here as a waste of time, why are you still commenting here;) One of the other miracles surrounding your person

Of course i like to see any number of customers that confirms my opinion that Solaris is vastly better than Linux. Please share them with us together with an exact list of your configuration, the description of the benchmark and the rationale why this benchmark describes your workload.

At the end: I would like to know what's your profession, your job ... of course besides spreading your infinite wisdom to the world and hyping Intel products

Or to be exact: I know what you are doing ... i'm capable to use Google ... so i should specify the question more exactly: What qualifies you you to assume that you know what enterprise customers want. I would really like to know, if i discuss with with someone who had never worked in or for a corporate datacenter. How many large customers did you consulted? How many SAP systems were implemented under you direct control? How many corporate datacenters in central London, California, Tokyo, New York did you saw from the inside while solving their problems? I'm just asking, because when i'm looking at your CV your undeniable strengths seems to be a different area.

So ... my train is almost at home. Will just close now.

#1.1 Joerg M. (Homepage) on 2009-12-11 21:16 (Reply)

Presales. That explains everything.

# The X4640 is vastly better than the DL785.
# Solaris is vastly better than Windows/Linux.

BTW, only one of those statements can be true. Obviously, a salesman wouldn't understand the logic behind that.

#1.1.1 TS on 2009-12-12 14:57 (Reply)

Sorry ... but perhaps you don't know what a presales systems engineer does. Those people are the poor people, who prevent the sales reps from making promises a system can't fulfill. And honestly, when there is one job, i want to be even less than a professional journalist then it's the job of a salesman. But it knew it right from the first second, when i wrote this line, that you won't understand this.

To your comment: It looks like you can't long texts because it looks like you are reading just parts: Because if you had hesitated to read the further text, then you would have read the passage "Leaves us with "better OS" or "better HW". Either way it's nice for Sun and that's exactly the reason why we won't see an Solaris/DL785/Oracle result from HP." OR ... not AND. Even a salesman could see that ... as at least in Germany such reading skills are part of the basic education and i'm sure it's not differently in the US.

So ... nice try ... but try it again. It get's really amusing: BTW .. please share you numbers. I'm really looking forward to see them.

#1.1.1.1 Joerg M. (Homepage) on 2009-12-12 15:32 (Reply)

So which one is it? "better OS" or "better hardware"? There better Solaris is compared to Linux, the worse X4640 is relative to DL785.

What exactly do you expect me to say? I already said the most vendor neutral thing you can imagine: "interchangeable" meaning you can go either way.

#1.1.1.1.1 TS on 2009-12-12 16:05 (Reply)

Sorry ... i'm a die hard Solaris Fan, but even i'm not assuming that Solaris is that far ahead of Linux that it's able to outweight 200 MHz less CPU frequency and 133 MHz lower RAM clock and still being 25% better than Linux. I assume that the truth lies somewhere in between: I don't think that the Sun Hardware is 25% better just as i think that Solaris isn't 25% better than Linux. Thus the servers are not interchangeable.

#1.1.1.1.1.1 Joerg Moellenkamp on 2009-12-12 16:27 (Reply)

We will just let the rest of the world judge the "interchangeability". In fact, the rest of the world had already decided.

People like me are eying the Dell R910 and HP DL780G6 or Supermicro equivalents. If Sun has gears like this, it is better to let people know about it.

Think about depreciation. You buy something for $30K+ knowing that in 3 months, the Dell R910 can be had for $10-15K and beat the performance of your $30K gear, it is just not good news. You can come back and say there is always new hardware in the horizon, so waiting is pointless. You are right. So we each make our own choices then.

#1.1.1.1.1.1.1 TS on 2009-12-12 17:00 (Reply)

Lastly, if you really looked up who I am from outdated google caches, then you do know what I do. I mean it is not hard considering my full name is the email I used.(Redirects to GMail. If I really wanted to remain anonymous to you I would have used a Mailinator account.)

One comment I would leave here is this: Joerg: talk about the products, argue about the products with numbers that are not fudged. When other people say Sun hardware sucks, don't alienate and attack the person(I am sure you have attacked many in your past). Don't try to attack the person's reputation. Try to argue with logic and numbers.

I am here on this blog because opensolaris/zfs is a great piece of software.

#1.1.2 TS on 2009-12-12 15:31 (Reply)

Oh ... my list of people i've "attacked" over the blog is short. Steve A. for making up an discussion, a german reader that got "trollish" and you ... for your arrogant attitude of telling everybody what is best. I'm thinking that isn't much for 5 years of blogging, albeit i'm a fan of the old saying that you can't carry the torch of an opinion through a crowd, without burning some beards.

The original saying was "truth" instead of "opinion" but i don't want to use this word, because i think nobody is the possessor of the "truth", only of his personal view of the world his opinion. And there were are at the basic core why i don't like your comments. It's not the stuff you are saying, it's the way think about your opinion, your way seeing the world as the singular truth. We have have an saying in Germany ( i don't know if there's a similar wording in the US) . It goes like "As you shout in the forest, it get's back to you". And sorry, this blog is my forest

I see the world differently. I express my opinion. I asking experts. (That's the nice thing about working at Sun. You have many experts in you background, in some cases people who practically invented the whole business). And 8 years sun (as a Project Engineer, a Systems Engineer, as a Principal Field Technologist), 3 years being a data center planer, 3 years of implementing a whole ISP and 2 years network administration you can be sure that i was able to collect quite an amount of experience about the real world of IT. However i wouldn't tell anybody, that he or she has to tell someone else to do something. Just because i've learned that differences are vast in IT. Because i've learned that putting a product into the field in a worldwide organisation is vastly more than just throwing datasheets to sales people.

PS: Normally i'm a really calm person ...

There is a simple reason for this: If i would opt to attack everybody who isn't my opinion, this would led to less readers and the person could be a customer of Sun.

#1.1.2.1 Joerg Moellenkamp on 2009-12-12 16:20 (Reply)

http://www.c0t0d0s0.org/archives/6114-Im-starting-to-get-really-angry-.....html

Using your own words,
"Of course i didn't and Giri already answered with the links to screenshots showing the documents. The second screenshot of the benchmark whitepaper clearly shows that the widthdawn/removed benchmark of the HP rx6600 used 8 streams."

The operative word is "withdrawn" and "removed". I don't know why in the world you expect someone outside of Sun to have knowledge of a withdrawn/removed benchmark.
That's one defense I have for SteveA.

You are not a bad person. I am not exactly a schill like you had imagined. Both you and I love ZFS, and honestly, ZFS is the last shining beacon for the Sun camp for a reason. Let's leave it at that.

Please, for your own good, and for the sanity of your readers, don't insult individuals even if it's your "forest". I noticed that you removed "making a fool of yourself" comment minutes after my response. That's a right step. Just keep it that way.

#1.1.2.1.1 TS on 2009-12-12 16:37 (Reply)

#1.1.2.1.2 TS on 2009-12-12 16:37 (Reply)

Also manchmal nervt diese penetrante Rechthaberei schon. Ein wenig nerven die Benches schon wenn ich mir dann ansehe wie die Systeme zwar schnell sind, aber ebenso schnell auch kaputt sind und man wieder wochenlang auf Patches oder Firmwares wartet.
Euer Support ist mittlerweile kaum noch als solcher zu bezeichnen. Permanent überlastet und oft unbrauchbar weil überlastet. Wo ist der Sun Service hin ? Ach ja, da gibt es ja keinen Benchmark zum vorzeigen.

#2 AN on 2009-12-11 11:52 (Reply)

Um Dinge gut vergleichen zu können, sollte man nur ein Teil des Stacks verändern!

Hardware + OS + SAP + DB =Stack

Vielleicht sollte einfach mal jemand so ein Vergleich machen.....

#3 :) on 2009-12-11 12:13 (Reply)

"4. X4640 is better than DL785G6
True only if you are retard. X4640 only has 4 SAS drive bays, so you have to either use Fiber channel SAN with it, or SAS Adapters. DL785G6 has 16 built in bays. If the X4640 were to use a 2U 24 bay array like the Witte or Dell MD1120, it would have been a 6U solution. Just 1U shy of the DL785. "

So the 785 is better because it has more disks ?

Nice.

Alex

#4 Alex on 2009-12-11 17:12 (Reply)

Strange reasoning. Then HP should build a large chassi that can hold 1000 discs, and it would be the best computer in the world, according to him. Strang indeed.

#4.1 Kebabbert on 2009-12-13 17:01 (Reply)

If you use remote storage, then X4640 has a 3U space savings advantage over DL785, but if you use direct attach storage, then X4640 only has 1U space advantage. In either situation, it really doesn't matter, since datacenters are power density restrained.

So you can buy the X4640 because it is a cable-less design, or the chassis looks nice inside, or saves 3U of space(to save 1/14 of a $1000 42U rack), or it saves you 120Watts or so compared to last generation DL785G5. That's a great choice for you, when you find out that in 3 months, your 30K investment just turned into $10K value.

If you don't have a brain, I can't really help you.

#4.1.1 TS on 2009-12-13 19:12 (Reply)

1. I've corrected power consumption article ... the number was for the G6 not for the G5. I assume the party was too long on friday.

2. Let's assume that Nehalem EX will deliver on promise and those machines will have that price. Even 20.000 Euro are next to nothing when you have to do right now, and you are loosing business by not buying the adequate hardware. Given the usual prices of commercial software per seat even 30.000 Euro are negligible. Most corporations aren't really worried about such sums, as their business demand drives IT, not the price of the components. Try to explain your users that you wait for cheaper gear in 3 or 6 months, and they have to work on the slower system until that time. Of course it's a nice opportunity to wait for EX gear. But until those equipment goes through the process of corporate procurement and vendor/equipment lists ... well wait even longer.

3. Systems of this class don't use DAS in almost all cases, out of obvious reasons and i would bet that you won't see many filled DL785 drive slots.

4. The problem of power density has been solved. Cooling doors are available from many vendors, Hot Aisle containment in conjunction with in-row-cooling is starting to get mainstream. Sun have such solutions (http://www.sun.com/servers/cooling/5200/) , other vendors have similar systems .... It's not a big problem to cool down 15 kW, even when you have standard racks. This may look different at cheap rack space providers. At the moment the challenges start at 35-40 kW per Rack.

#4.1.1.1 Joerg Moellenkamp (Homepage) on 2009-12-13 23:57 (Reply)

You have to realize that I am not exactly pumping Intel only. Both Intel and AMD are "shrinking sockets" in Q1 2010, meaning they will give you a 4S system that will have more or equal horsepower than 8 socket systems. Both Nehalem-EX and Opteron 6100 4 Socket systems will be targeting 10K-15K price target. Not only that, since you shrink 8 sockets worth of CPU cores to 4 sockets, you also shrink the per socket software licensing cost and power consumption. It is a win win to wait until mid Feb 2010(I heard).

Having said that, if Sun really wanted to impress me, gut the X4640 server, and do a custom 8 Socket Opteron 6100 server. That would require custom chipset and custom 16way HT topologies, and results in 96 cores and will compete(barely) against the HP DL780G6.

There are no technical issues cooling even 16-20KW. The problem is, it is cheaper to cool 2 racks of 8kW each than it is to cool 1 rack at 16kw. So that's the inherent density issue. To each his own.

#4.1.1.1.1 TS on 2009-12-14 02:24 (Reply)

"a german reader that got "trollish""

Du denkst also immer noch an mich, schnuckelchen?

#5 Anon Ymous on 2009-12-13 03:34 (Reply)

Den ersten Troll im Blog vergisst man doch nie

#5.1 Joerg Moellenkamp on 2009-12-13 07:25 (Reply)

Add Comment

Name
Email
Homepage
In reply to
Comment	E-Mail addresses will not be displayed and will only be used for E-Mail notifications. To prevent automated Bots from commentspamming, please enter the string you see in the image below in the appropriate input box. Your comment will only be submitted if the strings match. Please ensure that your browser supports and accepts cookies, or your comment cannot be verified correctly. Enter the string from the spam-prevention image above: Standard emoticons like :-) and ;-) are converted to images. Enclosing asterisks marks text as bold (word), underscore are made via _word_.
	Remember Information? Subscribe to this entry