Re: SATA drives performance

This isn't true. IBMs IPS series controllers can the checked and configured via the ipssend utility that works very well in 2.6.x LINUX.

"Scott Marlowe" <scott(dot)marlowe(at)gmail(dot)com> wrote:

>On Thu, Dec 24, 2009 at 11:09 AM, Richard Neill <rn214(at)cam(dot)ac(dot)uk> wrote:
>>
>>
>> Jeremy Harris wrote:
>>>
>>> On 12/24/2009 05:12 PM, Richard Neill wrote:
>>>>
>>>> Of course, with a server machine, it's nearly impossible to use mdadm
>>>> raid: you are usually compelled to use a hardware raid card.
>>>
>>> Could you expand on that?
>>
>> Both of the last machines I bought (an IBM X3550 and an HP DL380) come with
>> hardware raid solutions. These are an utter nuisance because:
>>
>>  - they can only be configured from the BIOS (or with a
>>    bootable utility CD). Linux has very basic monitoring tools,
>>    but no way to reconfigure the array, or add disks to empty
>>    hot-swap slots while the system is running.
>>
>>  - If there is a Linux raid config program, it's not part of the
>>    main packaged distro, but usually a pre-built binary, available
>>    for only one release/kernel of the wrong distro.
>>
>>  - the IBM one had dodgy firmware, which, until updated, caused the
>>    disk to totally fail after a few days.
>>
>>  - you pay a lot of money for something effectively pointless, and
>>    have less control and less flexibility.
>>
>> After my experience with the X3550, I hunted for any server that would ship
>> without hardware raid, i.e. connect the 8 SATA hotswap slots direct to the
>> motherboard, or where the hardware raid could be de-activated completely,
>> and put into pass-through mode. Neither HP nor IBM make such a thing.
>
>Yep. And that's why I never order servers from them. There are
>dozens of reputable white box builders (I use Aberdeen who give me a 5
>year all parts warranty and incredible customer service, but there are
>plenty to choose from) and they build the machine I ask them to build.
> For hardware RAID I use Areca 1680 series, and they also provide me
>with machines with software RAID for lighter loads (slave dbs,
>reporting dbs, and stats dbs)
>
>--
>Sent via pgsql-performance mailing list (pgsql-performance(at)postgresql(dot)org)
>To make changes to your subscription:
>http://www.postgresql.org/mailpref/pgsql-performance

--
Message composed using K-9 mail on Android.
Apologies for improper reply quoting (not supported) by client.

Adam Tauno Williams wrote:
> This isn't true. IBMs IPS series controllers can the checked and configured via the ipssend utility that works very well in 2.6.x LINUX.
>

Unfortunately, what we got (in the IBM) was the garbage ServeRaid 8kl
card. This one is atrocious - it shipped with a hideous firmware bug.
And there is no way to bypass it.

The HP have the P400 cards, which are decent in themselves, just not as
good as software raid.

Richard

> "Scott Marlowe" <scott(dot)marlowe(at)gmail(dot)com> wrote:
>
>> On Thu, Dec 24, 2009 at 11:09 AM, Richard Neill <rn214(at)cam(dot)ac(dot)uk> wrote:
>>>
>>> Jeremy Harris wrote:
>>>> On 12/24/2009 05:12 PM, Richard Neill wrote:
>>>>> Of course, with a server machine, it's nearly impossible to use mdadm
>>>>> raid: you are usually compelled to use a hardware raid card.
>>>> Could you expand on that?
>>> Both of the last machines I bought (an IBM X3550 and an HP DL380) come with
>>> hardware raid solutions. These are an utter nuisance because:
>>>
>>> - they can only be configured from the BIOS (or with a
>>> bootable utility CD). Linux has very basic monitoring tools,
>>> but no way to reconfigure the array, or add disks to empty
>>> hot-swap slots while the system is running.
>>>
>>> - If there is a Linux raid config program, it's not part of the
>>> main packaged distro, but usually a pre-built binary, available
>>> for only one release/kernel of the wrong distro.
>>>
>>> - the IBM one had dodgy firmware, which, until updated, caused the
>>> disk to totally fail after a few days.
>>>
>>> - you pay a lot of money for something effectively pointless, and
>>> have less control and less flexibility.
>>>
>>> After my experience with the X3550, I hunted for any server that would ship
>>> without hardware raid, i.e. connect the 8 SATA hotswap slots direct to the
>>> motherboard, or where the hardware raid could be de-activated completely,
>>> and put into pass-through mode. Neither HP nor IBM make such a thing.
>> Yep. And that's why I never order servers from them. There are
>> dozens of reputable white box builders (I use Aberdeen who give me a 5
>> year all parts warranty and incredible customer service, but there are
>> plenty to choose from) and they build the machine I ask them to build.
>> For hardware RAID I use Areca 1680 series, and they also provide me
>> with machines with software RAID for lighter loads (slave dbs,
>> reporting dbs, and stats dbs)
>>
>> --
>> Sent via pgsql-performance mailing list (pgsql-performance(at)postgresql(dot)org)
>> To make changes to your subscription:
>> http://www.postgresql.org/mailpref/pgsql-performance
>
> --
> Message composed using K-9 mail on Android.
> Apologies for improper reply quoting (not supported) by client.

On Thu, Dec 24, 2009 at 3:51 PM, Richard Neill <rn214(at)cam(dot)ac(dot)uk> wrote:
>
>
> Adam Tauno Williams wrote:
>>
>> This isn't true.  IBMs IPS series controllers can the checked and
>> configured via the ipssend utility that works very well in 2.6.x LINUX.
>>
>
> Unfortunately, what we got (in the IBM) was the garbage ServeRaid 8kl card.
> This one is atrocious - it shipped with a hideous firmware bug. And there is
> no way to bypass it.
>
> The HP have the P400 cards, which are decent in themselves, just not as good
> as software raid.

Yeah, the HP400 gets pretty meh reviews here on the lists. The P600
is adequate and the P800 seems to be a good performer.

Can you replace the IBM RAID controller with some other controller?
Even just a simple 4 or 8 port SATA card with no RAID capability would
be better than something that locks up.

Personally I'd call my rep and ask him to come pick up his crap server
and give me a check to replace it if it was that bad.

Scott Marlowe wrote:
> On Thu, Dec 24, 2009 at 3:51 PM, Richard Neill <rn214(at)cam(dot)ac(dot)uk> wrote:
>>
>> Adam Tauno Williams wrote:
>>> This isn't true. IBMs IPS series controllers can the checked and
>>> configured via the ipssend utility that works very well in 2.6.x LINUX.
>>>
>> Unfortunately, what we got (in the IBM) was the garbage ServeRaid 8kl card.
>> This one is atrocious - it shipped with a hideous firmware bug. And there is
>> no way to bypass it.
>>
> Can you replace the IBM RAID controller with some other controller?
> Even just a simple 4 or 8 port SATA card with no RAID capability would
> be better than something that locks up.

A replacement would have been nice, however the 8kl is very tightly
integrated with the motherboard and the backplane. We'd have had to buy
a PCI-X card, and then get out the soldering iron to fix the cables.

To be fair, the 8kl is now working OK; also there was a note in the box
mentioning that firmware updates should be applied if available. What I
found unbelievable was that IBM shipped the server to me in a state with
known crashing firmware (a sufficiently bad bug imho to merit a product
recall), and hadn't bothered to flash it themselves in the factory.
Usually BIOS updates are only applied by the end user if there is a
specific issue to fix, and if the product line has been out for years,
but that particular server was only assembled 3 weeks ago, why would one
expect a company of IBM's standing to ship it in that state.

Richard

On Thu, Dec 24, 2009 at 5:15 PM, Richard Neill <rn214(at)cam(dot)ac(dot)uk> wrote:
>
>
> Scott Marlowe wrote:
>>
>> On Thu, Dec 24, 2009 at 3:51 PM, Richard Neill <rn214(at)cam(dot)ac(dot)uk> wrote:
>>>
>>> Adam Tauno Williams wrote:
>>>>
>>>> This isn't true.  IBMs IPS series controllers can the checked and
>>>> configured via the ipssend utility that works very well in 2.6.x LINUX.
>>>>
>>> Unfortunately, what we got (in the IBM) was the garbage ServeRaid 8kl
>>> card.
>>> This one is atrocious - it shipped with a hideous firmware bug. And there
>>> is
>>> no way to bypass it.
>>>
>> Can you replace the IBM RAID controller with some other controller?
>> Even just a simple 4 or 8 port SATA card with no RAID capability would
>> be better than something that locks up.
>
> A replacement would have been nice, however the 8kl is very tightly
> integrated with the motherboard and the backplane. We'd have had to buy a
> PCI-X card, and then get out the soldering iron to fix the cables.
>
> To be fair, the 8kl is now working OK; also there was a note in the box
> mentioning that firmware updates should be applied if available. What I
> found unbelievable was that IBM shipped the server to me in a state with
> known crashing firmware (a sufficiently bad bug imho to merit a product
> recall), and hadn't bothered to flash it themselves in the factory. Usually
> BIOS updates are only applied by the end user if there is a specific issue
> to fix, and if the product line has been out for years, but that particular
> server was only assembled 3 weeks ago, why would one expect a company of
> IBM's standing to ship it in that state.

It does kind of knock the stuffing out of the argument that buying
from the big vendors ensures good hardware experiences. I've had
similar problems from all the big vendors in the past. I can't
imagine getting treated that way by my current supplied. It's one
thing for some obscure bug in a particular ubuntu kernel to interact
poorly with a piece of equipment, but when a hardware RAID controller
arrives in a basically broken state, that's inexcusable. It's really
not too much to expect working hardware on arrival.

--- On Fri, 25/12/09, Scott Marlowe <scott(dot)marlowe(at)gmail(dot)com> wrote:

> It does kind of knock the stuffing out of the argument that
> buying
> from the big vendors ensures good hardware
> experiences.  I've had
> similar problems from all the big vendors in the
> past.  I can't
> imagine getting treated that way by my current
> supplied.  It's one
> thing for some obscure bug in a particular ubuntu kernel to
> interact
> poorly with a piece of equipment, but when a hardware RAID
> controller
> arrives in a basically broken state, that's
> inexcusable.  It's really
> not too much to expect working hardware on arrival.

Last month I found myself taking a powerdrill to our new dell boxes in order to route cables to replacement raid cards. Having to do that made me feel really unprofessional and a total cowboy, but it was either that or shitty performance.

Glyn Astill wrote:
> Last month I found myself taking a powerdrill to our new dell
> boxes in order to route cables to replacement raid cards. Having
> to do that made me feel really unprofessional and a total cowboy,
> but it was either that or shitty performance.

Can you be more specific? Which Dell server, which RAID card, what were the performance problems, and what did you buy to fix them?

We're thinking of expanding our servers, and so far have had no complaints about our Dell servers. My colleagues think we should buy more of the same, but if there's some new problem, I'd sure like to know about it.

Thanks!
Craig