Re: Fixed directory locations in installs

I have been looking at our use of fixed directory specifications in
binaries.

Right now we have libdir (dynamic_library_path) predefined in the
compile, with a GUC to override it. initdb also needs to be able to
find its input files, and that can be overridden by an initdb flag.

Adding the timezone database, backends now need to know where /share is,
not just initdb. Seems we need a new GUC variable for that too.

Also, Win32 installs are going to want to be more directory independent
than Unix.

Right now if we move /lib or (or in the future /share) we need to edit
postgresql.conf. Is there an easier way to do this? Should initdb be
setting libdir and sharedir in GUC, rather than leave these as
compiled-in defaults?

Because Win32 can probe for the location of the binary, it seems it
should check to see if it can find libdir and sharedir own its own and
set those GUC values accordingly as part of initdb.

Basically I am sugesting that initdb set these defaults as GUC
variables, rather than having the backend use compiled-in defaults, and
Win32 can also use the binary location to find them and set the GUC
values.

In other words:

#dynamic_library_path = '$libdir'

could maybe become:

#dynamic_library_path = '/usr/local/pgsql/lib'

and if it doesn't match the default, the comment is removed:

dynamic_library_path = '/usr/local/pgsql/lib'

initdb would make these adjustments. Same for sharedir.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian wrote:
> Also, Win32 installs are going to want to be more directory
> independent than Unix.

Why?

> Because Win32 can probe for the location of the binary, it seems it
> should check to see if it can find libdir and sharedir own its own
> and set those GUC values accordingly as part of initdb.

This is just going to open up the possibility of silently finding the
wrong files.

Peter Eisentraut wrote:
> Bruce Momjian wrote:
> > Also, Win32 installs are going to want to be more directory
> > independent than Unix.
>
> Why?

Because when I install Win32 stuff on my machine via an installer, it
says "Where do you want the files" and puts it in C:\ or C:\Program
Files or whatever I supply. We need to have that functionality.

> > Because Win32 can probe for the location of the binary, it seems it
> > should check to see if it can find libdir and sharedir own its own
> > and set those GUC values accordingly as part of initdb.
>
> This is just going to open up the possibility of silently finding the
> wrong files.

Yes, it does. We need to check if the directory actually exists. If
the files we need don't exist in there, we will throw an error anyway,
and they will have to use a flag to specify the location. Win32 is
going to install everything under one directory anyway, so it should
work fine in most cases.

The only problem case would be where you have an initdb binary that sits
in a directory tree where the ../share and ../lib have files that initdb
needs, but they don't match the initdb version or something, but that
seems like a wacked-out configuration by any measure.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Peter Eisentraut wrote:

>Bruce Momjian wrote:
>
>
>>Also, Win32 installs are going to want to be more directory
>>independent than Unix.
>>
>>
>
>Why?
>

Common practice, for one thing. Windows programs are typically
relocatable, and Windows admins regard programs that rely on hardcoded
paths very poorly indeed.

The usual location used by an installer is something like
%ProgramFiles%/progname or %ProgramFiles%/progname/version - the
definition of %ProgramFiles% is determined by the machine it is being
installed on, not by the installer, and certainly can't be set at
compile time. The machine might not even have a C: drive, for instance.

But this is not only a Windows issue, as Tom reminded us recently. If I
understood him correctly, there have been calls for relocatable
installations from other binary packagers.

>
>
>
>>Because Win32 can probe for the location of the binary, it seems it
>>should check to see if it can find libdir and sharedir own its own
>>and set those GUC values accordingly as part of initdb.
>>
>>
>
>This is just going to open up the possibility of silently finding the
>wrong files.
>
>

Maybe it could be improved by using more version markers?

cheers

andrew

Bruce Momjian wrote:

>
>In other words:
>
> #dynamic_library_path = '$libdir'
>
>could maybe become:
>
> #dynamic_library_path = '/usr/local/pgsql/lib'
>

Not sure I follow - dynamic_library_path is a colon-separated set of
paths. How will somone using $libdir have that resolved? ISTM we need to
have another GUC var which sets it, rather than have it compiled in and
not able to be overridden. It's not really an initdb-time thing either -
it's an install-time thing - so the installer could add the right path
to postgresql.conf.sample. Of course, if people move directories around
after installation they are asking to shoot themselves in the foot, but
that's true now anyway.

cheers

andrew

Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> Peter Eisentraut wrote:
>> This is just going to open up the possibility of silently finding the
>> wrong files.

> Maybe it could be improved by using more version markers?

AFAICS the sharedir will already be sufficiently checked by means of
initdb's check on the postgres.bki version marker. In some sense, the
sharedir used by initdb is the *right* one for an installation by
definition --- I'm not even convinced that we should allow people to
fool with this after the fact. (However, it's probably not worth the
trouble to develop a non-GUC mechanism to transmit the setting from
initdb to backend.)

We could add a version-marker file to libdir, but it'd not really be a
sufficient check since people might copy their own shlibs in there from
a prior installation without recompiling; and as soon as someone adds
more directories to dynamic_library_path, all bets are off anyway.
We've seen the "wrong version of plpgsql.so" symptom often enough that
I've thought seriously of insisting on a backend-version marker embedded
right into shlibs loaded by the backend. This'd be easy enough if we
were willing to demand a source code addition in loadable modules, say
a macro
BACKEND_VERSION_MARKER
which'd compile to some sort of preinitialized global variable or constant
function returning a version string. I haven't been able to think of a
way to insert such a marker without source-level cooperation though.

regards, tom lane

I wrote:
> AFAICS the sharedir will already be sufficiently checked by means of
> initdb's check on the postgres.bki version marker. In some sense, the
> sharedir used by initdb is the *right* one for an installation by
> definition --- I'm not even convinced that we should allow people to
> fool with this after the fact.

Actually, looking at the present contents of the sharedir, I'm not even
sure that the backend needs to access it at all. Most of the files in
there are used only by initdb. The only thing that seems needed after
initdb is unknown.pltcl, and I'd not have a problem with moving that to
libdir.

regards, tom lane

Tom Lane wrote:
> I wrote:
> > AFAICS the sharedir will already be sufficiently checked by means of
> > initdb's check on the postgres.bki version marker. In some sense, the
> > sharedir used by initdb is the *right* one for an installation by
> > definition --- I'm not even convinced that we should allow people to
> > fool with this after the fact.
>
> Actually, looking at the present contents of the sharedir, I'm not even
> sure that the backend needs to access it at all. Most of the files in
> there are used only by initdb. The only thing that seems needed after
> initdb is unknown.pltcl, and I'd not have a problem with moving that to
> libdir.

Are you aware timezone is now in /share, at least in Win32, and maybe on
Unix some day?

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian wrote:
> Tom Lane wrote:
> > I wrote:
> > > AFAICS the sharedir will already be sufficiently checked by means of
> > > initdb's check on the postgres.bki version marker. In some sense, the
> > > sharedir used by initdb is the *right* one for an installation by
> > > definition --- I'm not even convinced that we should allow people to
> > > fool with this after the fact.
> >
> > Actually, looking at the present contents of the sharedir, I'm not even
> > sure that the backend needs to access it at all. Most of the files in
> > there are used only by initdb. The only thing that seems needed after
> > initdb is unknown.pltcl, and I'd not have a problem with moving that to
> > libdir.
>
> Are you aware timezone is now in /share, at least in Win32, and maybe on
> Unix some day?

We could move the timezone under /lib if that would make things easier.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Andrew Dunstan wrote:
> Common practice, for one thing. Windows programs are typically
> relocatable, and Windows admins regard programs that rely on
> hardcoded paths very poorly indeed.

OK, but how can that work in general? How do other programs handle
this? I don't think we should design a solution that goes like "ok, if
we move that file from share to lib then we could get away with it for
now". That will only postpone the potential problems. There needs to
be a definite and fixed place where programs can go looking to find the
files they need. Maybe it should be stored in that registry thing?

Peter Eisentraut wrote:

>Andrew Dunstan wrote:
>
>
>>Common practice, for one thing. Windows programs are typically
>>relocatable, and Windows admins regard programs that rely on
>>hardcoded paths very poorly indeed.
>>
>>
>
>OK, but how can that work in general? How do other programs handle
>this? I don't think we should design a solution that goes like "ok, if
>we move that file from share to lib then we could get away with it for
>now". That will only postpone the potential problems. There needs to
>be a definite and fixed place where programs can go looking to find the
>files they need. Maybe it should be stored in that registry thing?
>
>
>
>

Not portable. Also, storing things in the registry is pretty much the
Windows equivalent of storing things in /etc, an idea which has lately
been treated with less than universal approval, and suffers from almost
exactly the same problems. In particular, we need to ensure that a) you
don't need to be root/Administrator to install, and b) you can reliably
have multiple installations coexisting.

cheers

andrew