Re: Function to do runtime relative directory mapping

Peter Eisentraut wrote:
> Claudio Natoli wrote:
> > I'm yet to see a convincing argument for why we can't adopt the
> > "binary-location/../share" approach as submitted late March. AFAICS,
> > it was rejected on the basis that it was not platform independent (no
> > arguments there) and that we could not rely on the ".." approach.
>
> The only objection was that it hardcodes the layout already in the
> source, which gives us no flexibility at all to try out different
> installation layouts. If you want to compute the relative paths from
> bindir to libdir etc. at build time based on actual configure
> options, then I see no problem with that.

But we want to resolve the locations at run-time, not build or configure
time. For win32, I'm yet to see why this approach is egregious.

Do you have an alternative solution to propose?

Cheers,
Claudio

---
Certain disclaimers and policies apply to all email sent from Memetrics.
For the full text of these disclaimers and policies see
<a
href="http://www.memetrics.com/emailpolicy.html">http://www.memetrics.com/em
ailpolicy.html</a>

Claudio Natoli wrote:
> Peter Eisentraut wrote:
> > Claudio Natoli wrote:
> > > I'm yet to see a convincing argument for why we can't adopt the
> > > "binary-location/../share" approach as submitted late March.
> > > AFAICS, it was rejected on the basis that it was not platform
> > > independent (no arguments there) and that we could not rely on
> > > the ".." approach.
> >
> > The only objection was that it hardcodes the layout already in the
> > source, which gives us no flexibility at all to try out different
> > installation layouts. If you want to compute the relative paths
> > from bindir to libdir etc. at build time based on actual configure
> > options, then I see no problem with that.
>
> But we want to resolve the locations at run-time, not build or
> configure time.

If that is your intention then your original proposal was wrong to begin
with, because it resolves the locations even before build time.

Claudio Natoli said:
>
>
> Peter Eisentraut wrote:
>> Claudio Natoli wrote:
>> > I'm yet to see a convincing argument for why we can't adopt the
>> > "binary-location/../share" approach as submitted late March. AFAICS,
>> > it was rejected on the basis that it was not platform independent
>> > (no arguments there) and that we could not rely on the ".."
>> > approach.
>>
>> The only objection was that it hardcodes the layout already in the
>> source, which gives us no flexibility at all to try out different
>> installation layouts. If you want to compute the relative paths from
>> bindir to libdir etc. at build time based on actual configure
>> options, then I see no problem with that.
>
> But we want to resolve the locations at run-time, not build or
> configure time. For win32, I'm yet to see why this approach is
> egregious.
>
> Do you have an alternative solution to propose?
>

I hope we are at cross purposes here, or else Peter's suggestion won't
fly - we need to be able to decouple some of these things from
configure/build time and defer them to installation/runtime. Any other
result will have us attracting curses from on high from the whole Windows
community, and other binary packagers won't get what I understand some
want.

How about if we have a configuration flag --enable-relocation which would
require a fixed layout based on an indeterminate root. This would have the
following effects:

. if prefix did not contain 'postgres' or 'pgsql' then 'postgresql' would
be appended.
. all *dir configure options would be forbidden - they would be based on
the prefix as now, and since it would contain 'postgres' the simple layout
we want would be used.
. binaries would have a DEFINE which would mean they would know they
should look for other binaries and shared files in locations which are
fixed relative to their own location rather than in the hardcoded
locations.

None of this should need a single #ifdef WIN32 :-)

(Would we need to turn off rpath for Unix in such a case? I suspect we
would.)

cheers

andrew

Andrew Dunstan wrote:
> Claudio Natoli said:
> >
> >
> > Peter Eisentraut wrote:
> >> Claudio Natoli wrote:
> >> > I'm yet to see a convincing argument for why we can't adopt the
> >> > "binary-location/../share" approach as submitted late March. AFAICS,
> >> > it was rejected on the basis that it was not platform independent
> >> > (no arguments there) and that we could not rely on the ".."
> >> > approach.
> >>
> >> The only objection was that it hardcodes the layout already in the
> >> source, which gives us no flexibility at all to try out different
> >> installation layouts. If you want to compute the relative paths from
> >> bindir to libdir etc. at build time based on actual configure
> >> options, then I see no problem with that.
> >
> > But we want to resolve the locations at run-time, not build or
> > configure time. For win32, I'm yet to see why this approach is
> > egregious.
> >
> > Do you have an alternative solution to propose?
> >
>
> I hope we are at cross purposes here, or else Peter's suggestion won't
> fly - we need to be able to decouple some of these things from
> configure/build time and defer them to installation/runtime. Any other
> result will have us attracting curses from on high from the whole Windows
> community, and other binary packagers won't get what I understand some
> want.
>
> How about if we have a configuration flag --enable-relocation which would
> require a fixed layout based on an indeterminate root. This would have the
> following effects:
>
> . if prefix did not contain 'postgres' or 'pgsql' then 'postgresql' would
> be appended.
> . all *dir configure options would be forbidden - they would be based on
> the prefix as now, and since it would contain 'postgres' the simple layout
> we want would be used.
> . binaries would have a DEFINE which would mean they would know they
> should look for other binaries and shared files in locations which are
> fixed relative to their own location rather than in the hardcoded
> locations.

Rather than turn off configure options, we can easily make this work for
arbitary configure flags.

Everything on Win32 drives off of the directory of the binary, which we
can probe for at runtime.

If they use the configure defaults, /lib is ../lib relative to /bin. If
they specify:

bin /usr/local/pgsql/bin
lib /usr/local/lib/pgsql

then lib is ../../lib/pgsql relative to bin.

It is easy to write a C function that will give us the relative path
from bin to any of the compiled in locations like /lib or /share, and we
can use that instead of an abolute path.

Not sure how we can do this for Unix except perhaps probe PATH.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> Not sure how we can do this for Unix except perhaps probe PATH.

Huh? We have always determined the full path of the executable ---
see FindExec().

I guess what you are saying is we should have a configure-time option to
address configured directories via relative paths from the executable's
directory, rather than absolute paths? Seems reasonable ...

regards, tom lane

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > Not sure how we can do this for Unix except perhaps probe PATH.
>
> Huh? We have always determined the full path of the executable ---
> see FindExec().

Oh, OK. I forgot that.

> I guess what you are saying is we should have a configure-time option to
> address configured directories via relative paths from the executable's
> directory, rather than absolute paths? Seems reasonable ...

Yep. In fact, why would we not use that by default?

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> Tom Lane wrote:
>> I guess what you are saying is we should have a configure-time option to
>> address configured directories via relative paths from the executable's
>> directory, rather than absolute paths? Seems reasonable ...

> Yep. In fact, why would we not use that by default?

Because it'll be slower. Instead of
/usr/local/pgsql/lib
we'd be using something like
/usr/local/pgsql/bin/../lib
which is not too bad here but would get worse if the directories are not
so close.

But perhaps we can arrange for the path to be simplified down to an
absolute form when it's constructed at backend startup? You'd need a
routine anyway to combine the bindir path (determined by FindExec) with
the relative path provided by configure, so maybe this routine could be
smart about leading ../ in the configure path.

We'd also need to give some thought to pg_config output. I think I
would like to be able to see the relative path computed by configure
as well as the effective actual path ... maybe a switch to specify which
way to print it?

regards, tom lane

Tom Lane wrote:
> We'd also need to give some thought to pg_config output. I think I
> would like to be able to see the relative path computed by configure
> as well as the effective actual path ... maybe a switch to specify
> which way to print it?

What use could printing the relative path possibly have? It would only
be relative to the location of pg_config, which is of no interest to
the user of the printed information. Let's not make this more
complicated than it needs to be.

Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> Tom Lane wrote:
>> We'd also need to give some thought to pg_config output. I think I
>> would like to be able to see the relative path computed by configure
>> as well as the effective actual path ... maybe a switch to specify
>> which way to print it?

> What use could printing the relative path possibly have?

Debugging. If I can't see it, I *know* I'm going to wish I could.

regards, tom lane

Tom Lane wrote:
> > What use could printing the relative path possibly have?
>
> Debugging. If I can't see it, I *know* I'm going to wish I could.

Well, you can just look inside, it's not that big. Supporting extra
options might make the script twice as big and thus make it harder to
just look at the whole thing.

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > Tom Lane wrote:
> >> I guess what you are saying is we should have a configure-time option to
> >> address configured directories via relative paths from the executable's
> >> directory, rather than absolute paths? Seems reasonable ...
>
> > Yep. In fact, why would we not use that by default?
>
> Because it'll be slower. Instead of
> /usr/local/pgsql/lib
> we'd be using something like
> /usr/local/pgsql/bin/../lib
> which is not too bad here but would get worse if the directories are not
> so close.
>
> But perhaps we can arrange for the path to be simplified down to an
> absolute form when it's constructed at backend startup? You'd need a
> routine anyway to combine the bindir path (determined by FindExec) with
> the relative path provided by configure, so maybe this routine could be
> smart about leading ../ in the configure path.

I wrote relative_path() which does the mapping from compiled src/dst to
a new path. For example:

$ tst1 /usr/local/pgsql/bin /usr/local/pgsql/lib /users/fred/pgsql/bin
/users/fred/pgsql/lib

$ tst1 /a/b/c /a/d /f/g/h
/f/d

This can be used in the backend to map from your bindir to a relative
libdir at runtime, and share too. It returns [mp]alloc'ed values, or
NULL if it can't do the tranformation.

Patch attached.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian wrote:

>+ if (toupper(*src) != toupper(*src))
>
>

Shouldn't this be

if (toupper(*src) != toupper(*dst))

?

For completeness, you should probably also check for network drive paths ("\\machine\sharename\foo").

I also think we should just canonicalise everything early, and then never have to worry about / vs \ again.

I must confess I think this scheme is overkill - I can't think of a use case where one would want a relocatable installation which would any pattern other than the one we are thinking of for the windows binary installer. Are we taking flexibility too far?

cheers

andrew

Andrew Dunstan wrote:
> Bruce Momjian wrote:
>
> >+ if (toupper(*src) != toupper(*src))
> >
> >
>
> Shouldn't this be
>
> if (toupper(*src) != toupper(*dst))
>
> ?
>

Yep, fixed.

> For completeness, you should probably also check for network
> drive paths ("\\machine\sharename\foo").

Oh, OK.

>
> I also think we should just canonicalise everything early, and
> then never have to worry about / vs \ again.
>
> I must confess I think this scheme is overkill - I can't think
> of a use case where one would want a relocatable installation
> which would any pattern other than the one we are thinking of
> for the windows binary installer. Are we taking flexibility too
> far?

This code might be used on Unix too.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian wrote:

>Andrew Dunstan wrote:
>
>
>>I must confess I think this scheme is overkill - I can't think
>>of a use case where one would want a relocatable installation
>>which would any pattern other than the one we are thinking of
>>for the windows binary installer. Are we taking flexibility too
>>far?
>>
>>
>
>This code might be used on Unix too.
>
>
>

I know. I can't think of a Unix situation where you would want anything
else either.

If you want it relocatable, I would think that you would want

movable-root/
/bin
/lib
/share
/whatever

and then you could just pick up that root and put it somewhere else and
it would still work, Windows or Unix, it wouldn't matter, and you
wouldn't break anything else. I guess using some other scheme you could
pick up the various bits from under, say, /usr and put them in
/usr/local, but it strikes me as being very messy and likely to be error
prone.

Maybe it's just a failure of my imagination.

cheers

andrew

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> Andrew Dunstan wrote:
>> Shouldn't this be
>> if (toupper(*src) != toupper(*dst))

> Yep, fixed.

Also, all of the ctype.h calls need to be like

toupper((unsigned char) *src)

to avoid problems with high-bit-set characters.

regards, tom lane

pgman(at)candle(dot)pha(dot)pa(dot)us (Bruce Momjian) wrote in message > extern void canonicalize_path(char *path);
> + #ifndef WIN32
> + #define ISSEP(c) ((c) == '/')
> + #else
> + #define ISSEP(c) ((c) == '/' || (c) == '\\')
> + #endif

I've seen references to this "canonicalization" of the Windows \
character for the win32 port. I don't claim to be a programming guru,
but I recently read Jeff Cogswell's C++ for Dummies and he says (p.
546 "Separating a path name") that you can use Unix-style / to
separate directories in your programming and Windows will know what to
do. In other words, you don't need \\ to separate directory names. /
should work just fine.

Perhaps MinGW and the gcc compiler on Windows behaves differently(?)

Then again, perhaps there are other factors that I'm not considering.

-Tony

Tony Reina said:
> pgman(at)candle(dot)pha(dot)pa(dot)us (Bruce Momjian) wrote in message > extern void
> canonicalize_path(char *path);
>> + #ifndef WIN32
>> + #define ISSEP(c) ((c) == '/')
>> + #else
>> + #define ISSEP(c) ((c) == '/' || (c) == '\\')
>> + #endif
>
>
> I've seen references to this "canonicalization" of the Windows \
> character for the win32 port. I don't claim to be a programming guru,
> but I recently read Jeff Cogswell's C++ for Dummies and he says (p. 546
> "Separating a path name") that you can use Unix-style / to
> separate directories in your programming and Windows will know what to
> do. In other words, you don't need \\ to separate directory names. /
> should work just fine.
>
> Perhaps MinGW and the gcc compiler on Windows behaves differently(?)
>
> Then again, perhaps there are other factors that I'm not considering.
>

It has nothing to do with the compiler - it's a function of the underlying
MSVCRT library. And you are correct. See initdb.c for an example - after
canonicalising it can and does happily use / as the separator. The only
trouble you have is in dealing with Windows command interpreters, when you
have to make sure that you quote the path (you have to do that anyway, as
the path might also include spaces).

cheers

andrew

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > Tom Lane wrote:
> >> I guess what you are saying is we should have a configure-time option to
> >> address configured directories via relative paths from the executable's
> >> directory, rather than absolute paths? Seems reasonable ...
>
> > Yep. In fact, why would we not use that by default?
>
> Because it'll be slower. Instead of
> /usr/local/pgsql/lib
> we'd be using something like
> /usr/local/pgsql/bin/../lib
> which is not too bad here but would get worse if the directories are not
> so close.
>
> But perhaps we can arrange for the path to be simplified down to an
> absolute form when it's constructed at backend startup? You'd need a
> routine anyway to combine the bindir path (determined by FindExec) with
> the relative path provided by configure, so maybe this routine could be
> smart about leading ../ in the configure path.
>
> We'd also need to give some thought to pg_config output. I think I
> would like to be able to see the relative path computed by configure
> as well as the effective actual path ... maybe a switch to specify which
> way to print it?

Does this pg_config addition need to be done?

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Peter Eisentraut wrote:
> Tom Lane wrote:
> > > What use could printing the relative path possibly have?
> >
> > Debugging. If I can't see it, I *know* I'm going to wish I could.
>
> Well, you can just look inside, it's not that big. Supporting extra
> options might make the script twice as big and thus make it harder to
> just look at the whole thing.

OK, this is a followup from Peter on the pg_config issue. We only do a
relative path if they used the defaults and put it all under a single
directory so there is probably little need for a pg_config addition.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073