Re: Relocatable locale

Magnus found out that LOCALEDIR wasn't being handled in a relocatable
manner.

This patch fixes that. It also adjusts the get_*_path functions to
limit values to MAXPGPATH.

I have two questions. First, setlocale() seemed to be inconsistently
set inside and outside of ENABLE_NLS. I assume the proper location is
inside. Second, libpq has a locale setting for error messages, but a
libpq program could be in any directory, so I see no way to make that
relocatable. Instead, I just use the hardcoded path. I could make it
relocatable, but that seems to error-prone, plus I would have to look up
the exec path and stuff, and it seemed too complicated.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> I have two questions. First, setlocale() seemed to be inconsistently
> set inside and outside of ENABLE_NLS. I assume the proper location is
> inside.

Please do *not* go adding setlocale calls that were not there before.
You *will* break things.

> ! setlocale(LC_ALL, "");
^^^^^^^^^^^^^^^^^^^^^

Putting this call in the backend is a very serious mistake. It might be
okay in clients, but not in the backend.

regards, tom lane

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > I have two questions. First, setlocale() seemed to be inconsistently
> > set inside and outside of ENABLE_NLS. I assume the proper location is
> > inside.
>
> Please do *not* go adding setlocale calls that were not there before.
> You *will* break things.
>
> > ! setlocale(LC_ALL, "");
> ^^^^^^^^^^^^^^^^^^^^^
>
> Putting this call in the backend is a very serious mistake. It might be
> okay in clients, but not in the backend.

OK, patch applied. Turns out it was only added for the backend
("postgres").

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> OK, patch applied. Turns out it was only added for the backend
> ("postgres").

Hm? Isn't that code going to be executed in postmaster, bootstrap,
checkpoint processes, etc etc?

I don't really believe that path.c has any business doing this at
all, in any program.

regards, tom lane

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > OK, patch applied. Turns out it was only added for the backend
> > ("postgres").
>
> Hm? Isn't that code going to be executed in postmaster, bootstrap,
> checkpoint processes, etc etc?
>
> I don't really believe that path.c has any business doing this at
> all, in any program.

Well, all our client apps used to do it in their own code. Now they
call set_pglocale to do it centrally and relocabably.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> Tom Lane wrote:
>> I don't really believe that path.c has any business doing this at
>> all, in any program.

> Well, all our client apps used to do it in their own code. Now they
> call set_pglocale to do it centrally and relocabably.

I don't care if the clients do it. I don't want main.c doing it.

regards, tom lane

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > Tom Lane wrote:
> >> I don't really believe that path.c has any business doing this at
> >> all, in any program.
>
> > Well, all our client apps used to do it in their own code. Now they
> > call set_pglocale to do it centrally and relocabably.
>
> I don't care if the clients do it. I don't want main.c doing it.

Well, as coded now, it will behave the same.

The old code has in main.c:

- bindtextdomain("postgres", LOCALEDIR);
- textdomain("postgres");

and now it has a call to set_pgsetlocale which does:

if (find_my_exec(argv0, my_exec_path) < 0)
return;

get_locale_path(argv0, path);
bindtextdomain(app, path);
textdomain(app);

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> Well, as coded now, it will behave the same.

No it won't: as you have it, the postmaster and everything else that
goes through main.c will execute a setlocale call, which was not there
before for very good reasons.

regards, tom lane

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > Well, as coded now, it will behave the same.
>
> No it won't: as you have it, the postmaster and everything else that
> goes through main.c will execute a setlocale call, which was not there
> before for very good reasons.

I don't understand. I moved it up little in the file, but those calls
were happening in that file before, just a little lower.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> I don't understand. I moved it up little in the file, but those calls
> were happening in that file before, just a little lower.

No, *that* call wasn't happening at all. The calls that were there
were setting certain limited, safe LC categories. You added a call
that sets *all* LC categories, including ones we do not want changed.
See the discussion in src/backend/utils/adt/pg_locale.c.

regards, tom lane

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > I don't understand. I moved it up little in the file, but those calls
> > were happening in that file before, just a little lower.
>
> No, *that* call wasn't happening at all. The calls that were there
> were setting certain limited, safe LC categories. You added a call
> that sets *all* LC categories, including ones we do not want changed.
> See the discussion in src/backend/utils/adt/pg_locale.c.

But I added code to path.c to skip that if the app is "postgres". Why
doesn't that work?

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> But I added code to path.c to skip that if the app is "postgres". Why
> doesn't that work?

That will work fine ... for a standalone backend. Not so fine for
postmaster or bootstrap or other cases that go through main.c.

regards, tom lane

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > But I added code to path.c to skip that if the app is "postgres". Why
> > doesn't that work?
>
> That will work fine ... for a standalone backend. Not so fine for
> postmaster or bootstrap or other cases that go through main.c.

But they all use the app name of "postgres". They always did.

That is not 'progname' but a hardcoded appname that is used in the
main.c call.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> But they all use the app name of "postgres". They always did.

Oh! Sorry, I'm barking up the wrong tree then. I was thinking you were
looking at the exec_path.

regards, tom lane

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > But they all use the app name of "postgres". They always did.
>
> Oh! Sorry, I'm barking up the wrong tree then. I was thinking you were
> looking at the exec_path.

Nope, hard-coded, as they all are. I could pull from argv[0], but that
doesn't work in too many cases. The main.c call is:

set_pglocale(argv[0], "postgres");

and is the same no matter what is coming through main.c.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian wrote:
> I have two questions. First, setlocale() seemed to be inconsistently
> set inside and outside of ENABLE_NLS. I assume the proper location
> is inside.

No, in case of doubt it's outside.

> Second, libpq has a locale setting for error messages,
> but a libpq program could be in any directory, so I see no way to
> make that relocatable. Instead, I just use the hardcoded path. I
> could make it relocatable, but that seems to error-prone, plus I
> would have to look up the exec path and stuff, and it seemed too
> complicated.

Hmm, so this says that a relocatable install isn't in fact possible?

Peter Eisentraut wrote:
> Bruce Momjian wrote:
> > I have two questions. First, setlocale() seemed to be inconsistently
> > set inside and outside of ENABLE_NLS. I assume the proper location
> > is inside.
>
> No, in case of doubt it's outside.

OK, moved. New code is:

void
set_pglocale(const char *argv0, const char *app)
{
#ifdef ENABLE_NLS
char path[MAXPGPATH];
char my_exec_path[MAXPGPATH];
#endif

/* don't set LC_ALL in the backend */
if (strcmp(app, "postgres") != 0)
setlocale(LC_ALL, "");

#ifdef ENABLE_NLS
if (find_my_exec(argv0, my_exec_path) < 0)
return;

get_locale_path(my_exec_path, path);
bindtextdomain(app, path);
textdomain(app);
#endif
}

>
> > Second, libpq has a locale setting for error messages,
> > but a libpq program could be in any directory, so I see no way to
> > make that relocatable. Instead, I just use the hardcoded path. I
> > could make it relocatable, but that seems to error-prone, plus I
> > would have to look up the exec path and stuff, and it seemed too
> > complicated.
>
> Hmm, so this says that a relocatable install isn't in fact possible?

Yep. I don't see how you can have a library know where to look for its
localized messages. The only thing I can think of is an environment
variable to override the hard-coded default. How is that?

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian wrote:

> Yep. I don't see how you can have a library know where to look for its
>
>localized messages. The only thing I can think of is an environment
>variable to override the hard-coded default. How is that?
>
>
>

I'm confused. Can you explain the problem more clearly, please? Do we
need to distinguish known apps with a known (relative) message location,
from unknown libpq clients?

cheers

andrew

Andrew Dunstan wrote:
> I'm confused. Can you explain the problem more clearly, please? Do we
> need to distinguish known apps with a known (relative) message
> location, from unknown libpq clients?

The problem boils down to the fact that libpq needs to find its data
files (in this case locale data, but it could be anything), but it
doesn't know its own location, so it can't use a relative path
reference as has been proposed.

Peter Eisentraut wrote:
> Andrew Dunstan wrote:
> > I'm confused. Can you explain the problem more clearly, please? Do we
> > need to distinguish known apps with a known (relative) message
> > location, from unknown libpq clients?
>
> The problem boils down to the fact that libpq needs to find its data
> files (in this case locale data, but it could be anything), but it
> doesn't know its own location, so it can't use a relative path
> reference as has been proposed.

I am wondering if we should use environment variables, and have our apps
use putenv() to set it to the proper relative path.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian wrote:

>Peter Eisentraut wrote:
>
>
>>Andrew Dunstan wrote:
>>
>>
>>>I'm confused. Can you explain the problem more clearly, please? Do we
>>>need to distinguish known apps with a known (relative) message
>>>location, from unknown libpq clients?
>>>
>>>
>>The problem boils down to the fact that libpq needs to find its data
>>files (in this case locale data, but it could be anything), but it
>>doesn't know its own location, so it can't use a relative path
>>reference as has been proposed.
>>
>>
>
>I am wondering if we should use environment variables, and have our apps
>use putenv() to set it to the proper relative path.
>
>

It seems a bit roundabout, but might make sense. Maybe they shouldn't
override any value already present in the environment, just set it if
there is nothing there? Then the library could use the environment if
set, or fall back on a hardcoded path.

cheers

andrew

Andrew Dunstan wrote:
>
>
> Bruce Momjian wrote:
>
> >Peter Eisentraut wrote:
> >
> >
> >>Andrew Dunstan wrote:
> >>
> >>
> >>>I'm confused. Can you explain the problem more clearly, please? Do we
> >>>need to distinguish known apps with a known (relative) message
> >>>location, from unknown libpq clients?
> >>>
> >>>
> >>The problem boils down to the fact that libpq needs to find its data
> >>files (in this case locale data, but it could be anything), but it
> >>doesn't know its own location, so it can't use a relative path
> >>reference as has been proposed.
> >>
> >>
> >
> >I am wondering if we should use environment variables, and have our apps
> >use putenv() to set it to the proper relative path.
> >
> >
>
> It seems a bit roundabout, but might make sense. Maybe they shouldn't
> override any value already present in the environment, just set it if
> there is nothing there? Then the library could use the environment if
> set, or fall back on a hardcoded path.

Yes, that is what I was thinking. Call it PGLOCALEDIR.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> Peter Eisentraut wrote:
>> The problem boils down to the fact that libpq needs to find its data
>> files (in this case locale data, but it could be anything), but it
>> doesn't know its own location, so it can't use a relative path
>> reference as has been proposed.

> I am wondering if we should use environment variables, and have our apps
> use putenv() to set it to the proper relative path.

*Our* apps are not the problem. The problem comes when libpq is
embedded in somebody else's app --- which is not going to be that
cooperative, and would have no good way to know what path to supply
if it did want to be cooperative.

I can't say I care for the environment variable idea either; though
there may not be any better alternative.

regards, tom lane

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > Peter Eisentraut wrote:
> >> The problem boils down to the fact that libpq needs to find its data
> >> files (in this case locale data, but it could be anything), but it
> >> doesn't know its own location, so it can't use a relative path
> >> reference as has been proposed.
>
> > I am wondering if we should use environment variables, and have our apps
> > use putenv() to set it to the proper relative path.
>
> *Our* apps are not the problem. The problem comes when libpq is

Well, right now our apps don't work, so it is a problem. Of course the
bigger problem is other apps. How do libs find related files? I can't
think if a common API for that.

> I can't say I care for the environment variable idea either; though
> there may not be any better alternative.

I think the best we can do is putenv() for our apps, and document the
variable for other apps.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> I think the best we can do is putenv() for our apps, and document the
> variable for other apps.

Document what? How will another app find libpq's locale file?
AFAICS *it can't*. Documenting a useless variable is not a solution.

regards, tom lane

Tom Lane said:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
>> I think the best we can do is putenv() for our apps, and document the
>> variable for other apps.
>
> Document what? How will another app find libpq's locale file?
> AFAICS *it can't*. Documenting a useless variable is not a solution.
>

You wouldn't expect the environment var to be set by an app in those
cases - it would be set by a sysadmin or an installer on a system-wide
basis when pg is installed in other than the hardcoded location. At least,
that's the way I understood Bruce's suggestion.

cheers

andrew

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > I think the best we can do is putenv() for our apps, and document the
> > variable for other apps.
>
> Document what? How will another app find libpq's locale file?
> AFAICS *it can't*. Documenting a useless variable is not a solution.

I figured our apps could use putenv to set the locale directory, and
users of other apps could set the environment variable before running
the app.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Andrew Dunstan wrote:
> Tom Lane said:
> > Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> >> I think the best we can do is putenv() for our apps, and document the
> >> variable for other apps.
> >
> > Document what? How will another app find libpq's locale file?
> > AFAICS *it can't*. Documenting a useless variable is not a solution.
> >
>
> You wouldn't expect the environment var to be set by an app in those
> cases - it would be set by a sysadmin or an installer on a system-wide
> basis when pg is installed in other than the hardcoded location. At least,
> that's the way I understood Bruce's suggestion.

Right.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian wrote:
> I am wondering if we should use environment variables, and have our
> apps use putenv() to set it to the proper relative path.

The problem isn't our applications, it's someone else's applications.
Are you saying that from now on, setting certain environment variables
is part of the required startup for libpq programs? Remember that on
Windows in generall all installations are relocated away from the
default location.

Andrew Dunstan wrote:
> You wouldn't expect the environment var to be set by an app in those
> cases - it would be set by a sysadmin or an installer on a
> system-wide basis when pg is installed in other than the hardcoded
> location. At least, that's the way I understood Bruce's suggestion.

Strange. I don't suppose that PostgreSQL is the first package using
gettext on Windows. How does everyone else handle this?

Peter Eisentraut wrote:
> Bruce Momjian wrote:
> > I am wondering if we should use environment variables, and have our
> > apps use putenv() to set it to the proper relative path.
>
> The problem isn't our applications, it's someone else's applications.
> Are you saying that from now on, setting certain environment variables
> is part of the required startup for libpq programs? Remember that on
> Windows in generall all installations are relocated away from the
> default location.

Well, if they want locale-specific error messages from libpq in their
applications, the have to supply those locale files somehow. They could
supply them in the same directory as their binaries and use putenv()
and find_my_exec().

Sure, it is a hack, but what other option do we have? Do we somehow
move the locale files into the library?

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Am Donnerstag, 27. Mai 2004 07:31 schrieb Bruce Momjian:
> Sure, it is a hack, but what other option do we have? Do we somehow
> move the locale files into the library?

On Windows we use the feature that DLLs can locate themselves (described by
Magnus Hagander), and on Unix we forget about pretending to offer relocatable
installations.

With no one coming up with a better idea, this applied patch adds PGETC
(for pg_service.conf) and PGLOCALE (for locale dir) environment variable
processing to libpq.

The patch also adds code to our client apps so we set the environment
variable directly based on our binary location, unless it is already
set. This will allow our applications to emit proper locale messages
that are generated in libpq.

---------------------------------------------------------------------------

Bruce Momjian wrote:
> Andrew Dunstan wrote:
> >
> >
> > Bruce Momjian wrote:
> >
> > >Peter Eisentraut wrote:
> > >
> > >
> > >>Andrew Dunstan wrote:
> > >>
> > >>
> > >>>I'm confused. Can you explain the problem more clearly, please? Do we
> > >>>need to distinguish known apps with a known (relative) message
> > >>>location, from unknown libpq clients?
> > >>>
> > >>>
> > >>The problem boils down to the fact that libpq needs to find its data
> > >>files (in this case locale data, but it could be anything), but it
> > >>doesn't know its own location, so it can't use a relative path
> > >>reference as has been proposed.
> > >>
> > >>
> > >
> > >I am wondering if we should use environment variables, and have our apps
> > >use putenv() to set it to the proper relative path.
> > >
> > >
> >
> > It seems a bit roundabout, but might make sense. Maybe they shouldn't
> > override any value already present in the environment, just set it if
> > there is nothing there? Then the library could use the environment if
> > set, or fall back on a hardcoded path.
>
> Yes, that is what I was thinking. Call it PGLOCALEDIR.
>
> --
> Bruce Momjian | http://candle.pha.pa.us
> pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
> + If your life is a hard drive, | 13 Roberts Road
> + Christ can be your backup. | Newtown Square, Pennsylvania 19073
>
> ---------------------------(end of broadcast)---------------------------
> TIP 8: explain analyze is your friend
>

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian wrote:
> With no one coming up with a better idea, this applied patch adds
> PGETC (for pg_service.conf) and PGLOCALE (for locale dir) environment
> variable processing to libpq.

Considering that in other places these locations are referred to as
"sysconfdir" and "localdir", it would make sense to name the variables
"PGSYSCONFDIR" and "PGLOCALEDIR", so we stick to one set of terms.

Peter Eisentraut wrote:
> Bruce Momjian wrote:
> > With no one coming up with a better idea, this applied patch adds
> > PGETC (for pg_service.conf) and PGLOCALE (for locale dir) environment
> > variable processing to libpq.
>
> Considering that in other places these locations are referred to as
> "sysconfdir" and "localdir", it would make sense to name the variables
> "PGSYSCONFDIR" and "PGLOCALEDIR", so we stick to one set of terms.

OK, change made.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073