Re: Machine available for community use

> ------- Original Message -------
> From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
> To: Greg Smith <gsmith(at)gregsmith(dot)com>
> Sent: 25/07/07, 18:54:50
> Subject: Re: [HACKERS] Machine available for community use
>
> Another fairly big issue is that we need to know whether measurements we
> take in August are comparable to measurements we take in October, so a
> fairly stable platform is important. As you say, a fast-changing kernel
> would make it difficult to have any confidence about comparability over
> time. That would tend to make me vote for RHEL/Centos, where long-term
> stability is an explicit development goal. Debian stable might do too,
> though I'm not as clear about their update criteria as I am about Red Hat's.

Perhaps RH could donate us a RHEL/RHN licence for this?

/D

One thing to take into account is I dont have physical access to the
box (It is in TX, I am in PA). All installs but Gentoo will be
performed by a well trained NOC monkey. *cough*

On 7/25/07, Dave Page <dpage(at)postgresql(dot)org> wrote:
>
>
> > ------- Original Message -------
> > From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
> > To: Greg Smith <gsmith(at)gregsmith(dot)com>
> > Sent: 25/07/07, 18:54:50
> > Subject: Re: [HACKERS] Machine available for community use
> >
> > Another fairly big issue is that we need to know whether measurements we
> > take in August are comparable to measurements we take in October, so a
> > fairly stable platform is important. As you say, a fast-changing kernel
> > would make it difficult to have any confidence about comparability over
> > time. That would tend to make me vote for RHEL/Centos, where long-term
> > stability is an explicit development goal. Debian stable might do too,
> > though I'm not as clear about their update criteria as I am about Red Hat's.
>
> Perhaps RH could donate us a RHEL/RHN licence for this?
>
> /D
>
> ---------------------------(end of broadcast)---------------------------
> TIP 7: You can help support the PostgreSQL project by donating at
>
> http://www.postgresql.org/about/donate
>

"Dave Page" <dpage(at)postgresql(dot)org> writes:
> Perhaps RH could donate us a RHEL/RHN licence for this?

I could ask, if there's consensus we want it. It sounded like more
people like Debian, though.

regards, tom lane

Gavin M. Roy wrote:
> One thing to take into account is I dont have physical access to the
> box (It is in TX, I am in PA). All installs but Gentoo will be
> performed by a well trained NOC monkey. *cough*

iLO ?

Stefan

Tom Lane wrote:
> "Dave Page" <dpage(at)postgresql(dot)org> writes:
>> Perhaps RH could donate us a RHEL/RHN licence for this?
>
> I could ask, if there's consensus we want it. It sounded like more
> people like Debian, though.

well a RHEL/RHN licence would not be a bad thing either (and I guess
it's also a fairly common database-on-linux platform).

Stefan

On Wed, 2007-07-25 at 14:32 -0400, Tom Lane wrote:
> "Dave Page" <dpage(at)postgresql(dot)org> writes:
> > Perhaps RH could donate us a RHEL/RHN licence for this?
>
> I could ask, if there's consensus we want it.

Please.

> It sounded like more
> people like Debian, though.

Well, if you don't we probably will go Debian.

--
Simon Riggs
EnterpriseDB http://www.enterprisedb.com

If RH can sponsor a license of RHEL I'm inclined to go there. Not
that it was offered, but I think Dave's suggestion was Tom could field
that for the box if inclined. If I'm wrong, let me know. If that
can't happen, would people prefer CentOS or Ubuntu Server? The people
I'm most concerned with are the people who will actually use it. If
you consider yourself one of those people, pipe in now, I will tally
votes and go from there. From a Gentoo side, I would have kept things
pretty stable, but I'd rather developers be comfortable with the
environment which will encourage you to use it. I'm not interested in
running Debian, which I'm happy to talk about off topic, in private,
if anyone cares enough to want to discuss it.

What I'm most interested in to touch on Simon's request is SMP
scaling. From another Hackers thread this month, which I can dig up,
I've walked away with the impression that after 4 cores, we don't see
the same level of per-processor performance improvement that we see <=
4 cores. What you actually do is up to you, we want to provide this
to the hacker community to use as they see fit to continue to improve
PostgreSQL which is integral to our operation. Any performance,
scalability or even advocacy efforts (read benchmarking) will benefit
myYearbook.

Gavin

On 7/25/07, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
> On Wed, 2007-07-25 at 14:32 -0400, Tom Lane wrote:
> > "Dave Page" <dpage(at)postgresql(dot)org> writes:
> > > Perhaps RH could donate us a RHEL/RHN licence for this?
> >
> > I could ask, if there's consensus we want it.
>
> Please.
>
> > It sounded like more
> > people like Debian, though.
>
> Well, if you don't we probably will go Debian.
>
> --
> Simon Riggs
> EnterpriseDB http://www.enterprisedb.com
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 6: explain analyze is your friend
>

"Gavin M. Roy" <gmr(at)myyearbook(dot)com> writes:
> If RH can sponsor a license of RHEL I'm inclined to go there.

I'm checking into this, but it may take a few days to get an answer
(particularly since I'm planning to take Friday through Monday off).

regards, tom lane

Tom Lane wrote:
> "Gavin M. Roy" <gmr(at)myyearbook(dot)com> writes:
>> If RH can sponsor a license of RHEL I'm inclined to go there.
>
> I'm checking into this, but it may take a few days to get an answer
> (particularly since I'm planning to take Friday through Monday off).
>

Well if we go RHEL why not CentOS5 and just call it good?

Joshua D. Drake

> regards, tom lane
>
> ---------------------------(end of broadcast)---------------------------
> TIP 5: don't forget to increase your free space map settings
>

--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/

Where in Texas? I might be able to assist on-site if needed (though I
don't know much about linuxes).

On Jul 25, 2007, at 11:31 AM, Gavin M. Roy wrote:

> One thing to take into account is I dont have physical access to the
> box (It is in TX, I am in PA). All installs but Gentoo will be
> performed by a well trained NOC monkey. *cough*
>
> On 7/25/07, Dave Page <dpage(at)postgresql(dot)org> wrote:
>>
>>
>> > ------- Original Message -------
>> > From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
>> > To: Greg Smith <gsmith(at)gregsmith(dot)com>
>> > Sent: 25/07/07, 18:54:50
>> > Subject: Re: [HACKERS] Machine available for community use
>> >
>> > Another fairly big issue is that we need to know whether
>> measurements we
>> > take in August are comparable to measurements we take in
>> October, so a
>> > fairly stable platform is important. As you say, a fast-
>> changing kernel
>> > would make it difficult to have any confidence about
>> comparability over
>> > time. That would tend to make me vote for RHEL/Centos, where
>> long-term
>> > stability is an explicit development goal. Debian stable might
>> do too,
>> > though I'm not as clear about their update criteria as I am
>> about Red Hat's.
>>
>> Perhaps RH could donate us a RHEL/RHN licence for this?
>>
>> /D
>>
>> ---------------------------(end of
>> broadcast)---------------------------
>> TIP 7: You can help support the PostgreSQL project by donating at
>>
>> http://www.postgresql.org/about/donate
>>
>
> ---------------------------(end of
> broadcast)---------------------------
> TIP 6: explain analyze is your friend
>

--
Jim Nasby jim(at)nasby(dot)net
EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)

Hi,

On Wed, 2007-07-25 at 20:22 -0700, Joshua D. Drake wrote:
> > I'm checking into this, but it may take a few days to get an answer
> > (particularly since I'm planning to take Friday through Monday off).
>
> Well if we go RHEL why not CentOS5 and just call it good?

...because RHEL and CentOS are not really that identical. They are just
binary-compilant.

RHEL has better performance than CentOS -- I guess it is the compiler
options that Red Hat is using while compiling their RPMs.

I have performed a test using OSDL test suite a few months ago on a
system that has:

* 8 x86_64 CPUs @ 3200.263
* 16 Gigabytes of RAM
* PostgreSQL 8.1.5 (PGDG packages)

and RHEL performed much better than CentOS.

Regards,
--
Devrim GÜNDÜZ
PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Managed Services, Shared and Dedicated Hosting
Co-Authors: plPHP, ODBCng - http://www.commandprompt.com/

Devrim GÜNDÜZ wrote:
> Hi,

> RHEL has better performance than CentOS -- I guess it is the compiler
> options that Red Hat is using while compiling their RPMs.
>
> I have performed a test using OSDL test suite a few months ago on a
> system that has:
>
> * 8 x86_64 CPUs @ 3200.263
> * 16 Gigabytes of RAM
> * PostgreSQL 8.1.5 (PGDG packages)
>
> and RHEL performed much better than CentOS.

Not to be unkind, but I doubt that on an identical configuration.

Joshua D. Drake

>
> Regards,

--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/

Hi,

On Mon, 2007-07-30 at 19:14 -0700, Joshua D. Drake wrote:
> > and RHEL performed much better than CentOS.
>
> Not to be unkind, but I doubt that on an identical configuration.

Since I don't have the permission to distribute the benchmark results, I
will be happy to spend time for re-running these tests if someone
provides me an identical machine.

Each test took 1-2 days -- I will insist that CentOS performs poorer
than RHEL.

BTW, I will ask for permission to distribute the graphs that I produced
using gnuplot -- Maybe those graphs will give us some light.

Regards,
--
Devrim GÜNDÜZ
PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Managed Services, Shared and Dedicated Hosting
Co-Authors: plPHP, ODBCng - http://www.commandprompt.com/

Devrim =?ISO-8859-1?Q?G=DCND=DCZ?= <devrim(at)CommandPrompt(dot)com> writes:
> On Mon, 2007-07-30 at 19:14 -0700, Joshua D. Drake wrote:
>>> and RHEL performed much better than CentOS.

>> Not to be unkind, but I doubt that on an identical configuration.

> Each test took 1-2 days -- I will insist that CentOS performs poorer
> than RHEL.

I'm finding that hard to believe too. There isn't any "secret sauce"
in the RHEL build process --- the CentOS guys should have been able to
duplicate the RHEL RPMs exactly. Now it's possible that CentOS had
lagged in updating some performance-relevant package; did you compare
package versions across both OSes?

regards, tom lane

Hi,

On Mon, 2007-07-30 at 23:36 -0400, Tom Lane wrote:
> > Each test took 1-2 days -- I will insist that CentOS performs poorer
> > than RHEL.
>
> I'm finding that hard to believe too.

I have felt the same, that's why I repeated the test twice.

> There isn't any "secret sauce" in the RHEL build process

Really? Are the compiler options, etc, public?

> --- the CentOS guys should have been able to duplicate the RHEL RPMs
> exactly. Now it's possible that CentOS had lagged in updating some
> performance-relevant package; did you compare package versions across
> both OSes?

Actually I did not compare -- But both of them were 4.3 (RHEL 4.3 and
CentOS 4.3). I'm assuming that they have the same package versions,
right?

BTW, they were stock 4.3 -- no updates, etc.

I hope I will be able to publish only the graphs, so that community will
take a look what is going on.

Regards,

--
Devrim GÜNDÜZ
PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Managed Services, Shared and Dedicated Hosting
Co-Authors: plPHP, ODBCng - http://www.commandprompt.com/

Devrim =?ISO-8859-1?Q?G=DCND=DCZ?= <devrim(at)CommandPrompt(dot)com> writes:
> On Mon, 2007-07-30 at 23:36 -0400, Tom Lane wrote:
>> There isn't any "secret sauce" in the RHEL build process

> Really? Are the compiler options, etc, public?

Certainly. If you doubt it, try comparing pg_config output for the RHEL
and CentOS packages. (And if the CFLAGS entries are different, you
should be mentioning it to the CentOS package maintainer, not me.)

> Actually I did not compare -- But both of them were 4.3 (RHEL 4.3 and
> CentOS 4.3). I'm assuming that they have the same package versions,
> right?

> BTW, they were stock 4.3 -- no updates, etc.

RHEL 4.3 was obsoleted more than a year ago, so I'd like to think that
nobody finds "no update" comparisons to be very relevant today ...

regards, tom lane

Hi,

On Tue, 2007-07-31 at 01:54 -0400, Tom Lane wrote:
> > Really? Are the compiler options, etc, public?
>
> Certainly. If you doubt it, try comparing pg_config output for the
> RHEL and CentOS packages.

As I wrote before, I used PGDG packages for both -- What I'm suspecting
is the other packages like kernel, etc.

> > BTW, they were stock 4.3 -- no updates, etc.
>
> RHEL 4.3 was obsoleted more than a year ago, so I'd like to think that
> nobody finds "no update" comparisons to be very relevant today ...

I was referring to 4.3 isos of both distros, with no updates by that
time.

Regards,
--
Devrim GÜNDÜZ
PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Managed Services, Shared and Dedicated Hosting
Co-Authors: plPHP, ODBCng - http://www.commandprompt.com/

On 7/31/07, Devrim GÜNDÜZ <devrim(at)commandprompt(dot)com> wrote:
> Hi,
>
> On Mon, 2007-07-30 at 19:14 -0700, Joshua D. Drake wrote:
> > > and RHEL performed much better than CentOS.
> >
> > Not to be unkind, but I doubt that on an identical configuration.
>
> Since I don't have the permission to distribute the benchmark results, I
> will be happy to spend time for re-running these tests if someone
> provides me an identical machine.
>
> Each test took 1-2 days -- I will insist that CentOS performs poorer
> than RHEL.

Would it be possibe to include Unbreakable Linux in such test?
Out of curiosity of course. :-)

Regards,
Dawid

On Mon, 30 Jul 2007, Devrim GNDZ wrote:

> I have performed a test using OSDL test suite a few months ago on a
> system that has:
> * 8 x86_64 CPUs @ 3200.263...
> and RHEL [4.3] performed much better than CentOS [4.3]

RHEL 4 update 3 included some reworking of the x86_64 kernel, like adding
the kernel-largesmp for many CPU systems. I would not be surprised to
find that the first CentOS release based on that may not have achieved a
perfect rebuild because of all that, and since you didn't do any updates
from the initial ISO images you were basically running the CentOS beta for
that feature set.

I think it's accurate to say "sometimes CentOS releases have bugs that
make them perform worse than the RHEL they're derived from", and would not
dispute your results accordingly. I've seen fuzzy periods where CentOS
had a release out to match a new RHEL version, but it wasn't quite right
until after CentOS released an update or two. There can be some lag
there, particularly in the period after a new major release. Right now,
for example, I still don't completely trust the CentOS build based on the
recent RHEL 5, and have been following the developer mailing lists to get
a feel for when things have settled down. It is one of the risks that
goes along with using CentOS, and removing it by using a genuine RHEL
certainly has value.

At the same time, I've done a fair amount of benchmarking work on machines
that switched from RHEL<->CentOS where performance was completely
identical. I'd need to see a lot more than one test result suggesting
otherwise before I'd believe that CentOS is slower in general than the
RHEL it's derived from.

--
* Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD

Folks,

Hey, this is looking like a serious case of "Bike Shedding". That is, a dozen
people are arguing about what color to paint the bike shed instead of getting
it built.[1]

Given that there are much more substantial issues: what performance software
to install and how to install it, how to set up authentication and
time-sharing for running tests, whether we can set up automated perf testing,
getting money so some of our unfunded performance developers can work on it,
etc., is the "which Linux distro" question worth spending our time on?

[1] http://www.bikeshed.com/

--
Josh Berkus
PostgreSQL @ Sun
San Francisco

Josh Berkus <josh(at)agliodbs(dot)com> writes:
> Hey, this is looking like a serious case of "Bike Shedding". That is, a dozen
> people are arguing about what color to paint the bike shed instead of getting
> it built.[1]

FWIW, it's looking like Red Hat will donate a RHEL/RHN subscription if
we want one, though I don't have final approval quite yet.

regards, tom lane

On Tue, 31 Jul 2007, Josh Berkus wrote:

> That is, a dozen people are arguing about what color to paint the bike
> shed instead of getting it built.

Until there's an OS installed on it and it's on a network, the machine
essentially doesn't exist--so there was no way to work on the
building--and there was a clearly a gap between what Gavin was planning to
do and what the aggregate hacker community wanted. If there's a
bike-shedding analogy here, the argument has been about what type of
foundation to build the shed on. The design of the shed itself may be
much more complicated than that part, but if you put it someplace that's
not level you may not ever get what you wanted no matter how much work you
put into it later. That's why I thought it was important to at least talk
through the Linux distribution topic, so everyone was aware of the
trade-offs involved.

--
* Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD

Tom,

> FWIW, it's looking like Red Hat will donate a RHEL/RHN subscription if
> we want one, though I don't have final approval quite yet.

Great. Any chance of a machine? Can RH exert some leverage with Dell?

We could use up to 8 servers for performance testing, so I'm asking
everyone.

--
--Josh

Josh Berkus
PostgreSQL @ Sun
San Francisco

Tom Lane wrote:
>
> FWIW, it's looking like Red Hat will donate a RHEL/RHN subscription if
> we want one, though I don't have final approval quite yet.
>
>
One possible point favoring the use of Centos over RHEL - its a little
easier for community members to reproduce or test any findings... i.e.
you don't have to get a RHEL sub!

Cheers

Mark

Let us know when/if and we'll pay command prompt to install the base OS on
the system. All that we're waiting on at this point is the final on the OS.

Gavin

On 7/31/07, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>
> Josh Berkus <josh(at)agliodbs(dot)com> writes:
> > Hey, this is looking like a serious case of "Bike Shedding". That is, a
> dozen
> > people are arguing about what color to paint the bike shed instead of
> getting
> > it built.[1]
>
> FWIW, it's looking like Red Hat will donate a RHEL/RHN subscription if
> we want one, though I don't have final approval quite yet.
>
> regards, tom lane
>
> ---------------------------(end of broadcast)---------------------------
> TIP 7: You can help support the PostgreSQL project by donating at
>
> http://www.postgresql.org/about/donate
>