Re: sh -> pl

Folks,

I've noticed that a big hunk of our build system has gratuitous
dependencies on some variety of shell and on tools like sed, none of
which makes Windows developers feel welcome. I know people are
working toward a cmake or other more cross-platform toolchain.

My proposal is a lot more modest, and doesn't conflict with the larger
one. I'd like to move the above stuff to self-contained perl would
help to make things more cross-platform and clean up, no offense to
the fine authors, some pretty crufty code in there.

Comments? Objections?

Cheers,
David.
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter
Skype: davidfetter XMPP: david(dot)fetter(at)gmail(dot)com

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

David Fetter wrote:
> Folks,
>
> I've noticed that a big hunk of our build system has gratuitous
> dependencies on some variety of shell and on tools like sed, none of
> which makes Windows developers feel welcome. I know people are
> working toward a cmake or other more cross-platform toolchain.
>
> My proposal is a lot more modest, and doesn't conflict with the larger
> one. I'd like to move the above stuff to self-contained perl would
> help to make things more cross-platform and clean up, no offense to
> the fine authors, some pretty crufty code in there.
>
>
>

Give us some examples. (If you think the sed scripts are crufty, check
out some of the awk we use.)

In the case of sed, there is a standard perl replacement called psed
which can be used in at least simple cases.

But nothing we do to the make-based build system will make Windows
developers wanting to use MSVC feel any better, so I'm not quite sure
exactly what you would achieve.

Is perl currently required to build from tarball? If not, you would be
placing an additional build requirement and there may still be a few odd
build environments that don't sport perl by default.

cheers

andrew

David Fetter <david(at)fetter(dot)org> writes:
> My proposal is a lot more modest, and doesn't conflict with the larger
> one. I'd like to move the above stuff to self-contained perl would
> help to make things more cross-platform and clean up, no offense to
> the fine authors, some pretty crufty code in there.

This seems quite useless. Make scripts always rely on a shell
environment; you won't be buying any portability at all.

We might or might not want to switch to cmake, but I don't see any
value in half-measures.

regards, tom lane

On Tue, Jun 17, 2008 at 10:19:59AM -0400, Andrew Dunstan wrote:
> David Fetter wrote:
>> Folks,
>>
>> I've noticed that a big hunk of our build system has gratuitous
>> dependencies on some variety of shell and on tools like sed, none
>> of which makes Windows developers feel welcome. I know people are
>> working toward a cmake or other more cross-platform toolchain. My
>> proposal is a lot more modest, and doesn't conflict with the larger
>> one. I'd like to move the above stuff to self-contained perl would
>> help to make things more cross-platform and clean up, no offense to
>> the fine authors, some pretty crufty code in there.
>
> Give us some examples.

That new version stamper calls out to sed, when perl is perfectly
capable of doing the same work itself and not spawning 30 shells in
the process.

> (If you think the sed scripts are crufty, check out some of the awk
> we use.)

Another cleanup opportunity :)

> In the case of sed, there is a standard perl replacement called psed which
> can be used in at least simple cases.
>
> But nothing we do to the make-based build system will make Windows
> developers wanting to use MSVC feel any better, so I'm not quite sure
> exactly what you would achieve.

Well, it'll wind up with a build system that's documented a lot better
than it is :)

> Is perl currently required to build from tarball? If not, you would
> be placing an additional build requirement and there may still be a
> few odd build environments that don't sport perl by default.

This is 2008, and it's silly to pretend we need to support this
"requirement" on systems where people are building Postgres.

Cheers,
David.
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter
Skype: davidfetter XMPP: david(dot)fetter(at)gmail(dot)com

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

David Fetter wrote:
> On Tue, Jun 17, 2008 at 10:19:59AM -0400, Andrew Dunstan wrote:
>
>> David Fetter wrote:
>>
>>> Folks,
>>>
>>> I've noticed that a big hunk of our build system has gratuitous
>>> dependencies on some variety of shell and on tools like sed, none
>>> of which makes Windows developers feel welcome. I know people are
>>> working toward a cmake or other more cross-platform toolchain. My
>>> proposal is a lot more modest, and doesn't conflict with the larger
>>> one. I'd like to move the above stuff to self-contained perl would
>>> help to make things more cross-platform and clean up, no offense to
>>> the fine authors, some pretty crufty code in there.
>>>
>> Give us some examples.
>>
>
> That new version stamper calls out to sed, when perl is perfectly
> capable of doing the same work itself and not spawning 30 shells in
> the process.
>

I thought you might be thinking of that. In fact, you are completely
mischaracterising this script, which is not any part of the build
system. It is a maintenance tool, probably of practical interest to
about three people, and there is no requirement for it to be cross-platform.

>> Is perl currently required to build from tarball? If not, you would
>> be placing an additional build requirement and there may still be a
>> few odd build environments that don't sport perl by default.
>>
>
> This is 2008, and it's silly to pretend we need to support this
> "requirement" on systems where people are building Postgres.
>
>
>

Weren't you the person who just wanted not to be painted into a corner?
In general, I am in favor of having as few build dependencies as
possible. So should you be.

cheers

andrew

David Fetter wrote:

> That new version stamper calls out to sed, when perl is perfectly
> capable of doing the same work itself and not spawning 30 shells in
> the process.

That's great. Please send a patch to improve the stamper. (Are you
really worried about its performance, given that it runs about once a
month on average?)

> > (If you think the sed scripts are crufty, check out some of the awk
> > we use.)
>
> Another cleanup opportunity :)

I'm not sure it's the same case here, because the stamper is supposed to
run on very few machines (mostly just Marc's) whereas the rest of the
stuff is supposed to run on many others.

> > Is perl currently required to build from tarball? If not, you would
> > be placing an additional build requirement and there may still be a
> > few odd build environments that don't sport perl by default.
>
> This is 2008, and it's silly to pretend we need to support this
> "requirement" on systems where people are building Postgres.

Maybe, or maybe not. Do these platforms all have Perl?

gypsy_moth Solaris 8 SUN Studio 8 sparc
warthog UnixWare 7.1.4 cc 4.2 isa
canary NetBSD 1.6 gcc 2.95.3 x86
kudu Solaris 9 Sun WorkShop 6 update 2 C 5.3 x86
spoonbill OpenBSD OpenBSD 4.2 gcc gcc 3.3.5 Sparc64
grebe AIX 5.3 GCC 4.0.1 PPC
osprey NetBSD 2.0 gcc 3.3.3 m68k

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

David Fetter wrote:
> On Tue, Jun 17, 2008 at 10:19:59AM -0400, Andrew Dunstan wrote:
>> David Fetter wrote:
>>> Folks,

> Well, it'll wind up with a build system that's documented a lot better
> than it is :)
>
>> Is perl currently required to build from tarball? If not, you would
>> be placing an additional build requirement and there may still be a
>> few odd build environments that don't sport perl by default.
>
> This is 2008, and it's silly to pretend we need to support this
> "requirement" on systems where people are building Postgres.

I am curious what your overall proposal includes? Would I do:

perl Makefile.PL; make?

Or would things like autoconf and bison still be required?

Or are you just presenting to remove all the underlying nits that are
tied to all the different unix derived utilities?

Sincerely,

Joshua D. Drake

Alvaro Herrera <alvherre(at)commandprompt(dot)com> writes:
> David Fetter wrote:
>> This is 2008, and it's silly to pretend we need to support this
>> "requirement" on systems where people are building Postgres.

> Maybe, or maybe not. Do these platforms all have Perl?

In this connection it might be worth pointing to the Red Hat/Fedora
definition of the standard minimum build environment:
https://fedoraproject.org/wiki/Packaging/Guidelines#Exceptions
which I can assure you is no spur-of-the-moment list but was
very carefully chosen.

It has awk, and sed ... but not perl.

regards, tom lane

On Tue, 17 Jun 2008, Alvaro Herrera wrote:

> Maybe, or maybe not. Do these platforms all have Perl?
>

Of course. They're all buildfarm clients and the buildfarm script is
perl.

Kris Jurka

Alvaro Herrera wrote:
> Maybe, or maybe not. Do these platforms all have Perl?
> gypsy_moth Solaris 8 SUN Studio 8 spar
If the moths don't have perl, we'll add it, no problem - don't let that
stop anything.

(On a separate note, we have had some problems internally with DNS, so
some reporting has failed from some of out moths, it is a Sun internal
infrastructure problem which we are looking into.)

-Jørgen
--
Sun Database Group

http://blogs.sun.com/austvik

Jorgen Austvik wrote:
> Alvaro Herrera wrote:
>> Maybe, or maybe not. Do these platforms all have Perl?
>> gypsy_moth Solaris 8 SUN Studio 8 spar
> If the moths don't have perl, we'll add it, no problem - don't let
> that stop anything.
>
>

Of course they have perl - the buildfarm script is perl.

cheers

andrew

David Fetter wrote:
> On Tue, Jun 17, 2008 at 10:19:59AM -0400, Andrew Dunstan wrote:
> > David Fetter wrote:
> >> Folks,
> >>
> >> I've noticed that a big hunk of our build system has gratuitous
> >> dependencies on some variety of shell and on tools like sed, none
> >> of which makes Windows developers feel welcome. I know people are
> >> working toward a cmake or other more cross-platform toolchain. My
> >> proposal is a lot more modest, and doesn't conflict with the larger
> >> one. I'd like to move the above stuff to self-contained perl would
> >> help to make things more cross-platform and clean up, no offense to
> >> the fine authors, some pretty crufty code in there.
> >
> > Give us some examples.
>
> That new version stamper calls out to sed, when perl is perfectly
> capable of doing the same work itself and not spawning 30 shells in
> the process.

This does remind me of the "Useless Use of Cat Award" from
comp.unix.shell:

http://partmaps.org/era/unix/award.html

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Bruce Momjian wrote:
> David Fetter wrote:
>
>> On Tue, Jun 17, 2008 at 10:19:59AM -0400, Andrew Dunstan wrote:
>>
>>> David Fetter wrote:
>>>
>>>> Folks,
>>>>
>>>> I've noticed that a big hunk of our build system has gratuitous
>>>> dependencies on some variety of shell and on tools like sed, none
>>>> of which makes Windows developers feel welcome. I know people are
>>>> working toward a cmake or other more cross-platform toolchain. My
>>>> proposal is a lot more modest, and doesn't conflict with the larger
>>>> one. I'd like to move the above stuff to self-contained perl would
>>>> help to make things more cross-platform and clean up, no offense to
>>>> the fine authors, some pretty crufty code in there.
>>>>
>>> Give us some examples.
>>>
>> That new version stamper calls out to sed, when perl is perfectly
>> capable of doing the same work itself and not spawning 30 shells in
>> the process.
>>
>
> This does remind me of the "Useless Use of Cat Award" from
> comp.unix.shell:
>
> http://partmaps.org/era/unix/award.html
>

Well, yes. I was confused before. I thought David was referring to the
new add_cvs_markers script (which BTW does not spawn any useless shells
at all).

This script (version_stamp.pl) is already perl, and so David's criticism
of it is slightly justified, although it spawns sh and sed 5 times, not
30. It is at best a very minor inefficiency - something that mildly
annoys those of us well at home in perl and is blithely and not
unreasonably ignored by everyone else.

Note also that this script is not part of the build process either, so
the Windows developer argument hardly holds here.

cheers

andrew