Re: initdb failure with Postgres 8.4.4

Lists: pgsql-hackers
From: "BRUSSER Michael" <Michael(dot)BRUSSER(at)3ds(dot)com>
To: <pgsql-hackers(at)postgresql(dot)org>
Subject: initdb failure with Postgres 8.4.4
Date: 2010-12-09 17:06:34
Message-ID: 29EA9BFEF7E7FC4F988818CA0C5C78C2059327@AG-DCC-MBX03.dsone.3ds.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Initdb fails for me when host machine has no access to the Postgres build location.

LOG: could not open directory .../install/share/timezone": No such file or directory
LOG: could not open directory .../install/share/timezone": No such file or directory
WARNING: could not open directory .../install/share/timezonesets": No such file or directory
HINT: This may indicate an incomplete PostgreSQL installation,
or that the file .../bin.Linux/postgres" has been moved away from its proper location.
FATAL: invalid value for parameter "timezone_abbreviations": "Default"
DEBUG: shmem_exit(1): 0 callbacks to make
DEBUG: proc_exit(1): 0 callbacks to make
DEBUG: exit(1)

The postgres executable is in the proper place.
The truncated paths .../install/share/timezone, etc point to the original build location.

The timezone and other files are available in the local directory:
share % ls -1
conversion_create.sql
information_schema.sql
pg_hba.conf.sample
pg_ident.conf.sample
pg_service.conf.sample
postgres.bki
postgres.description
postgresql.conf.sample
postgres.shdescription
psqlrc.sample
recovery.conf.sample
snowball_create.sql
sql_features.txt
system_views.sql
timezone
timezonesets
tsearch_data

I call initdb with the -L option pointing to that directory, from initdb log I can see that it is indeed being used for certain things -

POSTGRES_BKI=.../share/postgres.bki
POSTGRES_DESCR=.../share/postgres.description
POSTGRES_SHDESCR=.../share/postgres.shdescription
POSTGRESQL_CONF_SAMPLE=.../share/postgresql.conf.sample
PG_HBA_SAMPLE=.../share/pg_hba.conf.sample
PG_IDENT_SAMPLE=.../share/pg_ident.conf.sample

I am wondering why the timezone files cannot be found there as well?
This is quite urgent and any help will be greatly appreciated.
Michael.
.

This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged. If you are not one of the named recipients or have received this email in error, (i) you should not read, disclose, or copy it, (ii) please notify sender of your receipt by reply email and delete this email and all attachments, (iii) Dassault Systemes does not accept or assume any liability or responsibility for any use of or reliance on this email.For other languages, go to http://www.3ds.com/terms/email-disclaimer.


From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: "BRUSSER Michael" <Michael(dot)BRUSSER(at)3ds(dot)com>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: initdb failure with Postgres 8.4.4
Date: 2010-12-09 19:56:15
Message-ID: 11046.1291924575@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

"BRUSSER Michael" <Michael(dot)BRUSSER(at)3ds(dot)com> writes:
> Initdb fails for me when host machine has no access to the Postgres build location.

> LOG: could not open directory .../install/share/timezone": No such file or directory

Moving the install tree works for me. Did you do something odd with the
--with-system-tzdata configuration option?

regards, tom lane


From: "BRUSSER Michael" <Michael(dot)BRUSSER(at)3ds(dot)com>
To: "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: initdb failure with Postgres 8.4.4
Date: 2010-12-09 20:36:51
Message-ID: 29EA9BFEF7E7FC4F988818CA0C5C78C20594F8@AG-DCC-MBX03.dsone.3ds.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

No, we do not use --with-system-tzdata option.
I looked at the makefile and at the output of pg_config. We may need to do some cleanup there, but I did not pick any clues.
The problem occurs on all our UNIX platforms.
Is there anything I could do to shed more light on it?
I can post the output of pg_config if it helps, or try to run initdb with truss or strace.

Thanks,
Michael.

-----Original Message-----
From: Tom Lane [mailto:tgl(at)sss(dot)pgh(dot)pa(dot)us]
Sent: Thursday, December 09, 2010 2:56 PM
To: BRUSSER Michael
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: [HACKERS] initdb failure with Postgres 8.4.4

"BRUSSER Michael" <Michael(dot)BRUSSER(at)3ds(dot)com> writes:
> Initdb fails for me when host machine has no access to the Postgres build location.

> LOG: could not open directory .../install/share/timezone": No such file or directory

Moving the install tree works for me. Did you do something odd with the
--with-system-tzdata configuration option?

regards, tom lane

This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged. If you are not one of the named recipients or have received this email in error, (i) you should not read, disclose, or copy it, (ii) please notify sender of your receipt by reply email and delete this email and all attachments, (iii) Dassault Systemes does not accept or assume any liability or responsibility for any use of or reliance on this email.For other languages, go to http://www.3ds.com/terms/email-disclaimer.


From: Andrew Dunstan <andrew(at)dunslane(dot)net>
To: BRUSSER Michael <Michael(dot)BRUSSER(at)3ds(dot)com>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: initdb failure with Postgres 8.4.4
Date: 2010-12-09 21:14:48
Message-ID: 4D0146C8.9090408@dunslane.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 12/09/2010 03:36 PM, BRUSSER Michael wrote:
> No, we do not use --with-system-tzdata option.
> I looked at the makefile and at the output of pg_config. We may need to do some cleanup there, but I did not pick any clues.
> The problem occurs on all our UNIX platforms.
> Is there anything I could do to shed more light on it?
> I can post the output of pg_config if it helps, or try to run initdb with truss or strace.
>

Yes, please show us the pg_config (and please don't top-post).

cheers

andrew


From: "BRUSSER Michael" <Michael(dot)BRUSSER(at)3ds(dot)com>
To: "Andrew Dunstan" <andrew(at)dunslane(dot)net>
Cc: "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us>, <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: initdb failure with Postgres 8.4.4
Date: 2010-12-10 09:08:26
Message-ID: 29EA9BFEF7E7FC4F988818CA0C5C78C20597A2@AG-DCC-MBX03.dsone.3ds.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 12/09/2010 03:36 PM, BRUSSER Michael wrote:
> No, we do not use --with-system-tzdata option.
> I looked at the makefile and at the output of pg_config. We may need to do some cleanup there, but I did not pick any clues.
> The problem occurs on all our UNIX platforms.
> Is there anything I could do to shed more light on it?
> I can post the output of pg_config if it helps, or try to run initdb with truss or strace.
>

Yes, please show us the pg_config (and please don't top-post).
cheers
andrew

---------------------------------------------------------------------------------------------------

I tried to build Postgres with minimal options outside of our normal build framework to isolate possible interference, but got the same result.

configure \
--prefix=$prefix \
--without-readline \
--without-zlib \
--without-tcl \
--without-perl \
--disable-rpath \
2>&1 | tee _configure.log

- gmake, gmake check, gmake install -- all clean

- initdb:
initdb -D /home/<local_path>/pgdata -L /home/<local_path>/share -E UTF8 --locale=C \
-d 2> log2 > log1

log1 ends with line
creating template1 database in /home/<local_path>/pgdata/base/1 ...

In log2 I got the same errors as I mentioned earlier:
LOG: could not open directory "/home/<build-path>/.../share/timezone": No such file or directory
... ...

Directories timezone and timezonesets exist in /home/<local_path>/share, pointed to by the -L option
The local share dir has the same contents as the build share dir.

% pg_config
BINDIR = /home/<build-path>/build/Linux/qe_x86_64/bin
DOCDIR = /home/<build-path>/build/Linux/qe_x86_64/share/doc
HTMLDIR = /home/<build-path>/build/Linux/qe_x86_64/share/doc
INCLUDEDIR = /home/<build-path>/build/Linux/qe_x86_64/include
PKGINCLUDEDIR = /home/<build-path>/build/Linux/qe_x86_64/include
INCLUDEDIR-SERVER = /home/<build-path>/build/Linux/qe_x86_64/include/server
LIBDIR = /home/<build-path>/build/Linux/qe_x86_64/lib
PKGLIBDIR = /home/<build-path>/build/Linux/qe_x86_64/lib
LOCALEDIR = /home/<build-path>/build/Linux/qe_x86_64/share/locale
MANDIR = /home/<build-path>/build/Linux/qe_x86_64/share/man
SHAREDIR = /home/<build-path>/build/Linux/qe_x86_64/share
SYSCONFDIR = /home/<build-path>/build/Linux/qe_x86_64/etc
PGXS = /home/<build-path>/build/Linux/qe_x86_64/lib/pgxs/src/makefiles/pgxs.mk
CONFIGURE = '--prefix=/home/<build-path>/build/Linux/qe_x86_64' '--without-readline' '--without-zlib'
CC = gcc
CPPFLAGS = -D_GNU_SOURCE
CFLAGS = -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -fno-strict-aliasing -fwrapv
CFLAGS_SL = -fpic
LDFLAGS = -Wl,--as-needed -Wl,-rpath,'/home/<build-path>/build/Linux/qe_x86_64/lib'
LDFLAGS_SL =
LIBS = -lpgport -lcrypt -ldl -lm
VERSION = PostgreSQL 8.4.4

This is probably not helpful, but just in case
% strings initdb | grep <build-path>
/home/<build-path>/build/Linux/qe_x86_64/bin
/home/<build-path>/build/Linux/qe_x86_64/share/man
/home/<build-path>/build/Linux/qe_x86_64/share/doc/
/home/<build-path>/build/Linux/qe_x86_64/share/locale
/home/<build-path>/build/Linux/qe_x86_64/lib
/home/<build-path>/build/Linux/qe_x86_64/include/server
/home/<build-path>/build/Linux/qe_x86_64/include
/home/<build-path>/build/Linux/qe_x86_64/etc
/home/<build-path>/build/Linux/qe_x86_64/share

Thank you,
Michael.

This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged. If you are not one of the named recipients or have received this email in error, (i) you should not read, disclose, or copy it, (ii) please notify sender of your receipt by reply email and delete this email and all attachments, (iii) Dassault Systemes does not accept or assume any liability or responsibility for any use of or reliance on this email.For other languages, go to http://www.3ds.com/terms/email-disclaimer.


From: Robert Haas <robertmhaas(at)gmail(dot)com>
To: BRUSSER Michael <Michael(dot)BRUSSER(at)3ds(dot)com>
Cc: Andrew Dunstan <andrew(at)dunslane(dot)net>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: initdb failure with Postgres 8.4.4
Date: 2010-12-10 12:32:42
Message-ID: AANLkTinrVoYDEGs0vbhq_hU_i_+44cGq96973GqD6dJM@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Fri, Dec 10, 2010 at 4:08 AM, BRUSSER Michael
<Michael(dot)BRUSSER(at)3ds(dot)com> wrote:
> - initdb:
> initdb  -D  /home/<local_path>/pgdata   -L   /home/<local_path>/share -E UTF8 --locale=C  \
>  -d  2> log2  > log1
>
> log1 ends with line
>  creating template1 database in /home/<local_path>/pgdata/base/1 ...
>
> In log2 I got the same errors as I mentioned earlier:
>  LOG:  could not open directory "/home/<build-path>/.../share/timezone": No such file or directory
> ... ...

Does it really have a pathname with three dots here?

I'd have more confidence we knew what we were looking at if you sent
the actual log, rather than an edited version of the lines you think
are most important.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


From: Andrew Dunstan <andrew(at)dunslane(dot)net>
To: Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: BRUSSER Michael <Michael(dot)BRUSSER(at)3ds(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: initdb failure with Postgres 8.4.4
Date: 2010-12-10 12:50:13
Message-ID: 4D022205.8070305@dunslane.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 12/10/2010 07:32 AM, Robert Haas wrote:
> On Fri, Dec 10, 2010 at 4:08 AM, BRUSSER Michael
> <Michael(dot)BRUSSER(at)3ds(dot)com> wrote:
>> - initdb:
>> initdb -D /home/<local_path>/pgdata -L /home/<local_path>/share -E UTF8 --locale=C \
>> -d 2> log2> log1
>>
>> log1 ends with line
>> creating template1 database in /home/<local_path>/pgdata/base/1 ...
>>
>> In log2 I got the same errors as I mentioned earlier:
>> LOG: could not open directory "/home/<build-path>/.../share/timezone": No such file or directory
>> ... ...
> Does it really have a pathname with three dots here?
>
> I'd have more confidence we knew what we were looking at if you sent
> the actual log, rather than an edited version of the lines you think
> are most important.
>

And why the heck use initdb's -L? initdb knows quite well where to get
its files from, if you haven't mucked up the installation. I don't think
I've ever seen a case in production where it's been necessary.

cheers

andrew


From: "BRUSSER Michael" <Michael(dot)BRUSSER(at)3ds(dot)com>
To: "Andrew Dunstan" <andrew(at)dunslane(dot)net>, "Robert Haas" <robertmhaas(at)gmail(dot)com>
Cc: "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us>, <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: initdb failure with Postgres 8.4.4
Date: 2010-12-10 14:00:42
Message-ID: 29EA9BFEF7E7FC4F988818CA0C5C78C205988D@AG-DCC-MBX03.dsone.3ds.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

>>
>> In log2 I got the same errors as I mentioned earlier:
>> LOG: could not open directory "/home/<build-path>/.../share/timezone": No such file or directory
>> ... ...
> Does it really have a pathname with three dots here?
>
> I'd have more confidence we knew what we were looking at if you sent
> the actual log, rather than an edited version of the lines you think
> are most important.
>

And why the heck use initdb's -L? initdb knows quite well where to get
its files from, if you haven't mucked up the installation. I don't think
I've ever seen a case in production where it's been necessary.

-------------------------------------
1) The pathnames referenced in the log do not have any dots, I just truncated them -
they are absolute pathnames to where the timezone files are locates in the source or build tree
I'll post the entire log if it is still needed after this ...

2) We have been using 'initdb -L' since Postgres 7.2
The way it works here we build Postgres and it gets bundled as a part of a larger application, which can be installed anywhere.
Perhaps this makes it difficult for initdb to "know quite well where to get its files from".

If I remove the -L switch initdb breaks very early before it gets to the timezones: share_path points back to the build location,
which is not available, so it cannot locate postgres.bki and other input files.

The timezones piece seems to be the only problem - if I compile with --with-system-tzdata=/usr/share/zoneinfo
and then use initdb -L installation succeeds when the source/build tree is not accessible.

Not claiming any knowledge in this area - would it be reasonable to expect that if -L option works for other input files it should
also work for timezones?

Thanks for all your help

This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged. If you are not one of the named recipients or have received this email in error, (i) you should not read, disclose, or copy it, (ii) please notify sender of your receipt by reply email and delete this email and all attachments, (iii) Dassault Systemes does not accept or assume any liability or responsibility for any use of or reliance on this email.For other languages, go to http://www.3ds.com/terms/email-disclaimer.


From: Robert Haas <robertmhaas(at)gmail(dot)com>
To: BRUSSER Michael <Michael(dot)BRUSSER(at)3ds(dot)com>
Cc: Andrew Dunstan <andrew(at)dunslane(dot)net>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: initdb failure with Postgres 8.4.4
Date: 2010-12-10 14:55:17
Message-ID: AANLkTikUM4Off0mCNnU+AiidJ8-uJuFNiioziCrtYWeh@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Fri, Dec 10, 2010 at 9:00 AM, BRUSSER Michael
<Michael(dot)BRUSSER(at)3ds(dot)com> wrote:
> 1) The pathnames referenced in the log do not have any dots, I just truncated them -
>     they are absolute pathnames to where the timezone files are locates in the source or build tree
>     I'll post the entire log if it is still needed after this ...

Well, why are you configuring the prefix to be inside your build tree?
This is an odd thing to do. The prefix should be set to where you
expect to install the files, not where you're building them.

CONFIGURE = '--prefix=/home/<build-path>/build/Linux/qe_x86_64'
'--without-readline' '--without-zlib'

Still:

>     Not claiming any knowledge in this area - would it be reasonable to expect that if -L option works for other input files it should
>     also work for timezones?

...this seems reasonable.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


From: Andrew Dunstan <andrew(at)dunslane(dot)net>
To: Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: BRUSSER Michael <Michael(dot)BRUSSER(at)3ds(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: initdb failure with Postgres 8.4.4
Date: 2010-12-10 15:25:33
Message-ID: 4D02466D.7040703@dunslane.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 12/10/2010 09:55 AM, Robert Haas wrote:
> On Fri, Dec 10, 2010 at 9:00 AM, BRUSSER Michael
> <Michael(dot)BRUSSER(at)3ds(dot)com> wrote:
>> 1) The pathnames referenced in the log do not have any dots, I just truncated them -
>> they are absolute pathnames to where the timezone files are locates in the source or build tree
>> I'll post the entire log if it is still needed after this ...
> Well, why are you configuring the prefix to be inside your build tree?
> This is an odd thing to do. The prefix should be set to where you
> expect to install the files, not where you're building them.
>
> CONFIGURE = '--prefix=/home/<build-path>/build/Linux/qe_x86_64'
> '--without-readline' '--without-zlib'
>
> Still:
>
>> Not claiming any knowledge in this area - would it be reasonable to expect that if -L option works for other input files it should
>> also work for timezones?
> ...this seems reasonable.

OK, this has nothing at all to do with the absence of the build path. It
has to do with using a non-standard sharedir.I have reproduced it thus:

[andrew(at)aurelia blurflinst]$ mv share/ foo
[andrew(at)aurelia blurflinst]$ bin/initdb -L `pwd`/foo/postgresql data3
The files belonging to this database system will be owned by user
"andrew".
This user must also own the server process.

The database cluster will be initialized with locale en_US.utf8.
The default database encoding has accordingly been set to UTF8.
The default text search configuration will be set to "english".

creating directory data3 ... ok
creating subdirectories ... ok
selecting default max_connections ... 10
selecting default shared_buffers ... 400kB
creating configuration files ... ok
creating template1 database in data3/base/1 ... WARNING: could not
open directory
"/home/andrew/pgl/blurflinst/share/postgresql/timezonesets": No such
file or directory
HINT: This may indicate an incomplete PostgreSQL installation, or
that the file "/home/andrew/pgl/blurflinst/bin/postgres" has been
moved away from its proper location.
FATAL: invalid value for parameter "timezone_abbreviations": "Default"
child process exited with exit code 1
initdb: removing data directory "data3"
[andrew(at)aurelia blurflinst]$

I will dig a bit further.

cheers

andrew


From: "BRUSSER Michael" <Michael(dot)BRUSSER(at)3ds(dot)com>
To: "Robert Haas" <robertmhaas(at)gmail(dot)com>
Cc: "Andrew Dunstan" <andrew(at)dunslane(dot)net>, "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us>, <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: initdb failure with Postgres 8.4.4
Date: 2010-12-10 15:26:07
Message-ID: 29EA9BFEF7E7FC4F988818CA0C5C78C2059970@AG-DCC-MBX03.dsone.3ds.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Well, why are you configuring the prefix to be inside your build tree?
This is an odd thing to do. The prefix should be set to where you
expect to install the files, not where you're building them.

CONFIGURE = '--prefix=/home/<build-path>/build/Linux/qe_x86_64'
'--without-readline' '--without-zlib'

Still:

> Not claiming any knowledge in this area - would it be reasonable to expect that if -L option works for other input files it should
> also work for timezones?

...this seems reasonable.

---------------------------------------------------------

It is odd, but this only reflects some quick hack I did to have a test case outside of our standard build framework.
I wanted to make sure the odd problem with timezones is not coming from our overloaded build environment.
Production makefile does it much better.

If I don't find a better option - do you see any disadvantage in using --with-system-tzdata and OS timezone info on Solaris and Linux?

This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged. If you are not one of the named recipients or have received this email in error, (i) you should not read, disclose, or copy it, (ii) please notify sender of your receipt by reply email and delete this email and all attachments, (iii) Dassault Systemes does not accept or assume any liability or responsibility for any use of or reliance on this email.For other languages, go to http://www.3ds.com/terms/email-disclaimer.


From: Andrew Dunstan <andrew(at)dunslane(dot)net>
To: Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: BRUSSER Michael <Michael(dot)BRUSSER(at)3ds(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: initdb failure with Postgres 8.4.4
Date: 2010-12-10 15:54:39
Message-ID: 4D024D3F.7030909@dunslane.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 12/10/2010 10:25 AM, Andrew Dunstan wrote:
>
>
>>
>>> Not claiming any knowledge in this area - would it be
>>> reasonable to expect that if -L option works for other input files
>>> it should
>>> also work for timezones?
>> ...this seems reasonable.
>
>
> OK, this has nothing at all to do with the absence of the build path.
> It has to do with using a non-standard sharedir.I have reproduced it
> thus:
>
>

[snip]

> I will dig a bit further.
>
>

Here's my understanding.

It's not initdb that's really complaining. The timezone files are not
inputs to initdb. It's the postgres that initdb invokes that's complaining.

Postges will look for the share file in two places: the configured
install directory or a share directory whose path is calculated relative
to its own location. initdb's -L flag doesn't override that, it only
overrides where initdb itself looks for files (such as the BKI file).
The bottom line I think is that if you intend to use a non-standard
layout you need to specify the paths for everything and then not move
them after installation. If you want the installation to be movable,
just specify --prefix, but then if you move it you have to move the
whole thing together. You can't just relocate one directory and expect
it to work. It won't.

cheers

andrew


From: Robert Haas <robertmhaas(at)gmail(dot)com>
To: Andrew Dunstan <andrew(at)dunslane(dot)net>
Cc: BRUSSER Michael <Michael(dot)BRUSSER(at)3ds(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: initdb failure with Postgres 8.4.4
Date: 2010-12-10 16:07:46
Message-ID: AANLkTimk_od6YSrdDoUsQfB-7UTjitH325YfSPk52=1x@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Fri, Dec 10, 2010 at 10:54 AM, Andrew Dunstan <andrew(at)dunslane(dot)net> wrote:
> Here's my understanding.
>
> It's not initdb that's really complaining. The timezone files are not inputs
> to initdb. It's the postgres that initdb invokes that's complaining.

That was my impression, too, from the log that was sent.

> Postges will look for the share file in two places: the configured install
> directory or a share directory whose path is calculated relative to its own
> location. initdb's -L flag doesn't override that, it only overrides where
> initdb itself looks for files (such as the BKI file). The bottom line I
> think is that if you intend to use a non-standard layout you need to specify
> the paths for everything and then not move them after installation. If you
> want the installation to be movable, just specify --prefix, but then if you
> move it you have to move the whole thing together. You can't just relocate
> one directory and expect it to work. It won't.

So in theory we could have a GUC under "file locations" to override
this, similarly to data_directory or hba_file or ident_file. But
since it's been like this for a really long time (I think), I wouldn't
be inclined to go monkeying with it unless more than one person
complains.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: Andrew Dunstan <andrew(at)dunslane(dot)net>, BRUSSER Michael <Michael(dot)BRUSSER(at)3ds(dot)com>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: initdb failure with Postgres 8.4.4
Date: 2010-12-10 16:19:32
Message-ID: 24138.1291997972@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Robert Haas <robertmhaas(at)gmail(dot)com> writes:
> So in theory we could have a GUC under "file locations" to override
> this, similarly to data_directory or hba_file or ident_file. But
> since it's been like this for a really long time (I think), I wouldn't
> be inclined to go monkeying with it unless more than one person
> complains.

Even if we did see more than one request, I'd be inclined not to change
it. The installation layout stuff is too complicated and fragile
already; and AFAICS it does cover every *reasonable* use case.

regards, tom lane


From: Andrew Dunstan <andrew(at)dunslane(dot)net>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, BRUSSER Michael <Michael(dot)BRUSSER(at)3ds(dot)com>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: initdb failure with Postgres 8.4.4
Date: 2010-12-10 18:03:21
Message-ID: 4D026B69.7050901@dunslane.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 12/10/2010 11:19 AM, Tom Lane wrote:
> Robert Haas<robertmhaas(at)gmail(dot)com> writes:
>> So in theory we could have a GUC under "file locations" to override
>> this, similarly to data_directory or hba_file or ident_file. But
>> since it's been like this for a really long time (I think), I wouldn't
>> be inclined to go monkeying with it unless more than one person
>> complains.
> Even if we did see more than one request, I'd be inclined not to change
> it. The installation layout stuff is too complicated and fragile
> already; and AFAICS it does cover every *reasonable* use case.
>
>

I agree, although I think Robert's "really long time" is in fact "since
8.0, before which we didnt provide for relocation at all."

cheers

andrew


From: "BRUSSER Michael" <Michael(dot)BRUSSER(at)3ds(dot)com>
To: "Andrew Dunstan" <andrew(at)dunslane(dot)net>, "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: "Robert Haas" <robertmhaas(at)gmail(dot)com>, <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: initdb failure with Postgres 8.4.4
Date: 2010-12-10 19:29:20
Message-ID: 29EA9BFEF7E7FC4F988818CA0C5C78C2059C2B@AG-DCC-MBX03.dsone.3ds.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 12/10/2010 11:19 AM, Tom Lane wrote:
> Robert Haas<robertmhaas(at)gmail(dot)com> writes:
>> So in theory we could have a GUC under "file locations" to override
>> this, similarly to data_directory or hba_file or ident_file. But
>> since it's been like this for a really long time (I think), I wouldn't
>> be inclined to go monkeying with it unless more than one person
>> complains.
> Even if we did see more than one request, I'd be inclined not to change
> it. The installation layout stuff is too complicated and fragile
> already; and AFAICS it does cover every *reasonable* use case.
>
>

I agree, although I think Robert's "really long time" is in fact "since
8.0, before which we didnt provide for relocation at all."

cheers
andrew
--------------------------------------

Thanks, everyone!!!
I'm still not sure I completely understand how to package all components to allow independent installation,
but I guess this is what this weekend is good for :(

Michael

This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged. If you are not one of the named recipients or have received this email in error, (i) you should not read, disclose, or copy it, (ii) please notify sender of your receipt by reply email and delete this email and all attachments, (iii) Dassault Systemes does not accept or assume any liability or responsibility for any use of or reliance on this email.For other languages, go to http://www.3ds.com/terms/email-disclaimer.


From: "BRUSSER Michael" <Michael(dot)BRUSSER(at)3ds(dot)com>
To: "Robert Haas" <robertmhaas(at)gmail(dot)com>, "Andrew Dunstan" <andrew(at)dunslane(dot)net>
Cc: "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us>, <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: initdb failure with Postgres 8.4.4
Date: 2010-12-13 15:34:49
Message-ID: 29EA9BFEF7E7FC4F988818CA0C5C78C205A92F@AG-DCC-MBX03.dsone.3ds.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Fri, Dec 10, 2010 at 10:54 AM, Andrew Dunstan <andrew(at)dunslane(dot)net> wrote:
> Here's my understanding.
>
> It's not initdb that's really complaining. The timezone files are not inputs
> to initdb. It's the postgres that initdb invokes that's complaining.

>> Postges will look for the share file in two places: the configured install
>> directory or a share directory whose path is calculated relative to its own location

>.initdb's -L flag doesn't override that, it only overrides where
> initdb itself looks for files (such as the BKI file). The bottom line I
> think is that if you intend to use a non-standard layout you need to specify
> the paths for everything and then not move them after installation. If you
> want the installation to be movable, just specify --prefix, but then if you
> move it you have to move the whole thing together. You can't just relocate
> one directory and expect it to work. It won't.

I'm not sure if our current approach would work with v8.4. This is what we do in the nutshell:
- build Postgres
- do not run install
- collect all generated libraries, executables and input files and pack them along with other app
- distribute the tar-ball to the customer
- untar and install the app
the installation script at some point calls initdb, create database, createlang, create user,
it creates config files... done.

I guess this eliminates option 1 - using "the configured install directory"

I tried to figure out the second option "share directory whose path is calculated relative to its own location"
and ran initdb with "strace". Chances are I overlooked something, but I only saw Postgres trying to open the
timezone files in the original build path.

Do you see some way to support remote deployment with the approach I described?
Thanks,
Michael.

This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged. If you are not one of the named recipients or have received this email in error, (i) you should not read, disclose, or copy it, (ii) please notify sender of your receipt by reply email and delete this email and all attachments, (iii) Dassault Systemes does not accept or assume any liability or responsibility for any use of or reliance on this email.For other languages, go to http://www.3ds.com/terms/email-disclaimer.


From: Alvaro Herrera <alvherre(at)commandprompt(dot)com>
To: BRUSSER Michael <michael(dot)brusser(at)3ds(dot)com>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, Andrew Dunstan <andrew(at)dunslane(dot)net>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: initdb failure with Postgres 8.4.4
Date: 2010-12-13 15:53:54
Message-ID: 1292255503-sup-8485@alvh.no-ip.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Excerpts from BRUSSER Michael's message of lun dic 13 12:34:49 -0300 2010:

> I'm not sure if our current approach would work with v8.4. This is what we do in the nutshell:
> - build Postgres
> - do not run install
> - collect all generated libraries, executables and input files and pack them along with other app
> - distribute the tar-ball to the customer
> - untar and install the app
> the installation script at some point calls initdb, create database, createlang, create user,
> it creates config files... done.

I think this procedure needs to be fixed. It should be:
- build Postgres
- install
- collect the files, pack
- distribute
- unpack to the destination directory

Since our packaging is (supposed to be) relocatable, you can install in
the customer's machine at a different destination directory, and it
should work. If it doesn't, report it as a bug.

--
Álvaro Herrera <alvherre(at)commandprompt(dot)com>
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support


From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: "BRUSSER Michael" <Michael(dot)BRUSSER(at)3ds(dot)com>
Cc: "Robert Haas" <robertmhaas(at)gmail(dot)com>, "Andrew Dunstan" <andrew(at)dunslane(dot)net>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: initdb failure with Postgres 8.4.4
Date: 2010-12-13 15:58:52
Message-ID: 8338.1292255932@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

"BRUSSER Michael" <Michael(dot)BRUSSER(at)3ds(dot)com> writes:
> I'm not sure if our current approach would work with v8.4. This is what we do in the nutshell:
> - build Postgres
> - do not run install
> - collect all generated libraries, executables and input files and pack them along with other app
> - distribute the tar-ball to the customer
> - untar and install the app
> the installation script at some point calls initdb, create database, createlang, create user,
> it creates config files... done.

If that ever worked, with any version of Postgres, I'd be astonished.

Here is what you should do:

1. Build Postgres with --prefix set to some empty directory.
2. Run install.
3. Bundle up the resulting install tree as part of your tarball. Do not
editorialize upon the relative locations of its contents.
4. Drop the install tree wherever you want on the target machine.

regards, tom lane


From: "BRUSSER Michael" <Michael(dot)BRUSSER(at)3ds(dot)com>
To: "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: "Robert Haas" <robertmhaas(at)gmail(dot)com>, "Andrew Dunstan" <andrew(at)dunslane(dot)net>, <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: initdb failure with Postgres 8.4.4
Date: 2010-12-13 17:16:37
Message-ID: 29EA9BFEF7E7FC4F988818CA0C5C78C205AAEE@AG-DCC-MBX03.dsone.3ds.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Here is what you should do:

1. Build Postgres with --prefix set to some empty directory.
2. Run install.
3. Bundle up the resulting install tree as part of your tarball. Do not
editorialize upon the relative locations of its contents.
4. Drop the install tree wherever you want on the target machine.

-------------------------------------------------

Would it be a completely crazy idea if I try to modify Postgres to look at some env. var
(similar to PGDATA) and if exists and path is valid look there for the timezone files?

Michael

This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged. If you are not one of the named recipients or have received this email in error, (i) you should not read, disclose, or copy it, (ii) please notify sender of your receipt by reply email and delete this email and all attachments, (iii) Dassault Systemes does not accept or assume any liability or responsibility for any use of or reliance on this email.For other languages, go to http://www.3ds.com/terms/email-disclaimer.


From: Andrew Dunstan <andrew(at)dunslane(dot)net>
To: BRUSSER Michael <Michael(dot)BRUSSER(at)3ds(dot)com>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Robert Haas <robertmhaas(at)gmail(dot)com>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: initdb failure with Postgres 8.4.4
Date: 2010-12-13 17:26:38
Message-ID: 4D06574E.2060907@dunslane.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 12/13/2010 12:16 PM, BRUSSER Michael wrote:
> Here is what you should do:
>
> 1. Build Postgres with --prefix set to some empty directory.
> 2. Run install.
> 3. Bundle up the resulting install tree as part of your tarball. Do not
> editorialize upon the relative locations of its contents.
> 4. Drop the install tree wherever you want on the target machine.
>
> -------------------------------------------------
>
> Would it be a completely crazy idea if I try to modify Postgres to look at some env. var
> (similar to PGDATA) and if exists and path is valid look there for the timezone files?
>

Yes, that's only the first problem you have encountered. I predict there
will be others if you go down this path. You are making life much harder
for yourself than you need to. Just keep everything together and you'll
be fine. If that's really impossible, use symlinks (or junction points
on Windows) to achieve the same effect.

cheers

andrew


From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Andrew Dunstan <andrew(at)dunslane(dot)net>
Cc: BRUSSER Michael <Michael(dot)BRUSSER(at)3ds(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: initdb failure with Postgres 8.4.4
Date: 2010-12-13 18:02:29
Message-ID: 24189.1292263349@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> On 12/13/2010 12:16 PM, BRUSSER Michael wrote:
>> Would it be a completely crazy idea if I try to modify Postgres to look at some env. var
>> (similar to PGDATA) and if exists and path is valid look there for the timezone files?

> Yes, that's only the first problem you have encountered. I predict there
> will be others if you go down this path.

As far as the timezone data files go, you might have use for the
--with-system-tzdata configure option. Note that that is not an
"environment variable"; it requires a hard coded path to the system
copy of the Olsen timezone database. The expectation is that any
system-maintained copy will have a very well-defined location for
those files on any given platform, so there is no need for runtime
tweaking. You would use this only on platforms where you expect
that normal system maintenance procedures will keep the TZ files more
up-to-date than the ones included with Postgres.

I agree with Andrew's point in general. Trying to customize the
relative layout of the various Postgres-installed files is an
exercise in pointless pain.

regards, tom lane