Re: Why we are going to have to go DirectIO

On 12/03/2013 08:23 PM, Josh Berkus wrote:
> On 12/03/2013 10:59 AM, Joshua D. Drake wrote:
>> This seems rather half cocked. I read the article. They found a problem,
>> that really will only affect a reasonably small percentage of users,
>> created a test case, reported it, and a patch was produced.
>
> "Users with at least one file bigger than 50% of RAM" is unlikely to be
> a small percentage.
>
>>
>> Kind of like how we do it.
>
> I like to think we'd have at least researched the existing literature on
> 2Q algorithms (which is extensive) before implementing our own. Oh,
> wait, we *did*. Nor is this the first ill-considered performance hack
> pushed into mainline kernels without any real testing. It's not even
> the first *that year*.
>
> While I am angry over this -- no matter what Kernel.org fixes now, we're
> going to have to live with their mistake for 3 years -- the DirectIO
> thing isn't just me; when I've gone to Linux Kernel events to talk about
> IO, that's the response I've gotten from most Linux hackers: "you
> shouldn't be using the filesystem, use DirectIO and implement your own
> storage."
>
> That's why they don't feel that it's a problem to break the IO stack;
> they really don't believe that anyone who cares about performance should
> be using it.

reading that article I think this is an overreaction, it is not
kernel.orgs fault that distributions exist and bugs and regression
happen in all pieces of software.

We are in no way different and I would like to note that we do not have
any form of sensible performance related regression testing either.
I would even argue that there is ton more regression testing (be it
performance or otherwise) going into the linux kernel (even on a
relative scale) than we do and pointing the finger at something they are
dealing with once noticed.
If we care about our performance on various operating systems it is
_OUR_ responsibility to track that closely and automated and report back
and only if that feedback loop fails to work we are actually in a real
position to consider something as drastical as considering a platform
"undependable" or looking into alternatives (like directIO).

Stefan