Re: initdb fails on Centos 5.4 x64

Hi,

i've tried to install 8.4.3 for about two days, but i don't get it.

8.1.x has been installed before and was runnig but never used. I've removed it with yum and wiped /var/lib/pgsl.

-=>> service postgresql initdb
Initializing database: [FAILED]

-=>> cat /var/lib/pgsql/pgstartup.log
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale en_US.UTF-8.
The default database encoding has accordingly been set to UTF8.
The default text search configuration will be set to "english".

fixing permissions on existing directory /var/lib/pgsql/data ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 24MB
creating configuration files ... ok
creating template1 database in /var/lib/pgsql/data/base/1 ... ok
initializing pg_authid ... ok
initializing dependencies ... ok
creating system views ... ok
loading system objects' descriptions ... ok
creating conversions ... FATAL: could not load library "/usr/lib64/pgsql/utf8_and_shift_jis_2004.so": /usr/lib64/pgsql/utf8_and_shift_jis_2004.so: failed to map segment from shared object: Cannot allocate memory
STATEMENT: CREATE OR REPLACE FUNCTION shift_jis_2004_to_utf8 (INTEGER, INTEGER, CSTRING, INTERNAL, INTEGER) RETURNS VOID AS '$libdir/utf8_and_shift_jis_2004', 'shift_jis_2004_to_utf8' LANGUAGE C STRICT;

child process exited with exit code 1
initdb: removing contents of data directory "/var/lib/pgsql/data"

-=>> rpm -qa | grep postgre
postgresql-libs-8.1.18-2.el5_4.1
postgresql-libs-8.4.3-2PGDG.el5
postgresql-devel-8.4.3-2PGDG.el5
compat-postgresql-libs-4-1PGDG.rhel5
postgresql-server-8.4.3-2PGDG.el5
postgresql-plperl-8.4.3-2PGDG.el5
postgresql-8.4.3-2PGDG.el5
postgresql-contrib-8.4.3-2PGDG.el5

-=>> ldd /usr/lib64/pgsql/utf8_and_shift_jis_2004.so
libc.so.6 => /lib64/libc.so.6 (0x00002aad6e656000)
/lib64/ld-linux-x86-64.so.2 (0x0000003777a00000)

The server is running with 8 gigs ram, so i think there should be enough mem?

Thanks,

Valentin

valentin(dot)hocher(at)kabelbw(dot)de (Valentin Hocher) writes:
> creating conversions ... FATAL: could not load library "/usr/lib64/pgsql/utf8_and_shift_jis_2004.so": /usr/lib64/pgsql/utf8_and_shift_jis_2004.so: failed to map segment from shared object: Cannot allocate memory

Hmm, there was something similar reported last month, but that person
was completely unhelpful at tracking down the problem. Could we see
the output of "ldd /usr/lib64/pgsql/utf8_and_shift_jis_2004.so"
and "readelf -d /usr/lib64/pgsql/utf8_and_shift_jis_2004.so" ?

regards, tom lane

valentin(dot)hocher(at)kabelbw(dot)de (Valentin Hocher) writes:
> creating conversions ... FATAL: could not load library "/usr/lib64/pgsql/utf8_and_shift_jis_2004.so": /usr/lib64/pgsql/utf8_and_shift_jis_2004.so: failed to map segment from shared object: Cannot allocate memory

FWIW, I tried to duplicate this on a RHEL5-U4 installation, without success:
initdb runs fine for me. I assume you're using the RPMs from
http://yum.pgsqlrpms.org/8.4/redhat/rhel-5-x86_64/
? If so, it seems like this must be some problem in CentOS' version of
the distro. What I find installed on Red Hat's machine is
kernel-2.6.18-164.el5
glibc-2.5-42
(and a boatload of other stuff of course, but those seem like the
packages most likely to be involved). Maybe CentOS 5.4 is a bit
different?

regards, tom lane

> > The solution is very simple and can be done in the cPanel configuration, just disabled "Shell Fork Bomb Protection" in the security center. That's all. The ulimit restrictions are removed!
>
> Huh, that's interesting. With a name like that, I'd have thought it
> would set limits on number of processes, not virtual memory size.
> What ulimit settings did it change exactly?

"Shell Fork Bomb Protection"
Description:
Fork Bomb Protection will prevent users with terminal access (ssh/telnet) from using up all the resources on the server. Unchecked resource allocation can potentially lead to a server crash.
It is recommended that this protection be enabled for servers providing terminal access.

The only thing I discoverd if enabled is the following entry in /etc/profile (if disabled, there is no ulimit set)

#cPanel Added Limit Protections -- BEGIN

#unlimit so we can run the whoami
ulimit -n 4096 -u 14335 -m unlimited -d unlimited -s 8192 -c 1000000 -v unlimited 2>/dev/null

LIMITUSER=$USER
if [ -e "/usr/bin/whoami" ]; then
LIMITUSER=`/usr/bin/whoami`
fi
if [ "$LIMITUSER" != "root" ]; then
ulimit -n 100 -u 20 -m 200000 -d 200000 -s 8192 -c 200000 -v 200000 2>/dev/null
else
ulimit -n 4096 -u 14335 -m unlimited -d unlimited -s 8192 -c 1000000 -v unlimited 2>/dev/null
fi
#cPanel Added Limit Protections -- END

valentin(dot)hocher(at)kabelbw(dot)de (Valentin Hocher) writes:
> [ cPanel's "Shell Fork Bomb Protection" actually does this: ]
> ulimit -n 100 -u 20 -m 200000 -d 200000 -s 8192 -c 200000 -v 200000 2>/dev/null

Just to annotate that: some experimentation I did confirms that on
RHEL5 x86_64, PG 8.4.3 falls over with the mentioned error when run
under ulimit -v in the vicinity of 200000 (ie 200MB). It's kind of
surprising that initdb eats that much virtual memory space, although
certainly loading all the encoding translation libraries simultaneously
is a bit of a stress test. But the actual memory footprint is surely a
lot less than that. Apparently there is a good deal of inefficiency in
address-space consumption when loading a bunch of .so's on this
platform. I'd be interested to know if people can reproduce similar
problems on other Linux variants.

regards, tom lane

On 11/05/10 08:04, Tom Lane wrote:
> valentin(dot)hocher(at)kabelbw(dot)de (Valentin Hocher) writes:
>> [ cPanel's "Shell Fork Bomb Protection" actually does this: ]
>> ulimit -n 100 -u 20 -m 200000 -d 200000 -s 8192 -c 200000 -v 200000 2>/dev/null
>
> Just to annotate that: some experimentation I did confirms that on
> RHEL5 x86_64, PG 8.4.3 falls over with the mentioned error when run
> under ulimit -v in the vicinity of 200000 (ie 200MB). It's kind of
> surprising that initdb eats that much virtual memory space, although
> certainly loading all the encoding translation libraries simultaneously
> is a bit of a stress test. But the actual memory footprint is surely a
> lot less than that. Apparently there is a good deal of inefficiency in
> address-space consumption when loading a bunch of .so's on this
> platform. I'd be interested to know if people can reproduce similar
> problems on other Linux variants.

I wouldn't be surprised if prelinking turned out to be the culprit
there. If it's loading shared libraries to pre-selected address ranges
that're globally unique across the system, it might much some
significant address space.

... though come to think of it, each should be an individual mapping
(right?) so it shouldn't really matter where in virtual memory they're
located, just their size. Scratch that idea?

--
Craig Ringer

Tech-related writing: http://soapyfrogs.blogspot.com/

On Mon, May 10, 2010 at 18:04, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> valentin(dot)hocher(at)kabelbw(dot)de (Valentin Hocher) writes:
>> [ cPanel's "Shell Fork Bomb Protection" actually does this: ]
>>         ulimit -n 100 -u 20 -m 200000 -d 200000 -s 8192 -c 200000 -v 200000 2>/dev/null
>
> Just to annotate that: some experimentation I did confirms that on
> RHEL5 x86_64, PG 8.4.3 falls over with the mentioned error when run
> under ulimit -v in the vicinity of 200000 (ie 200MB).
> ...
> I'd be interested to know if people can reproduce similar
> problems on other Linux variants.

FWIW on arch linux x86_64 breaks ~130000 and 32bit ~55000.