Lists: | pgsql-hackers |
---|
From: | "Walter Cruz" <walter(dot)php(at)gmail(dot)com> |
---|---|
To: | "PostgreSQL Hackers" <pgsql-hackers(at)postgresql(dot)org> |
Subject: | weird - invalid string enlargement request size |
Date: | 2007-12-04 17:18:36 |
Message-ID: | 32cabba0712040918x48546a71vd5db5dad05c98f56@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-hackers |
A simple query:
select prec.bl_considerada_prioritaria
AS "A proposta é considerada prioritária por Conselho Municipal ou Estadual"
from consultaprevia2008.tab_urb_asse_prec prec
The result: ERROR: invalid string enlargement request size 1073741823
But.. If i replace the accented char, leaving the query like:
select prec.bl_considerada_prioritaria
AS "A proposta e considerada prioritaria por Conselho Municipal ou Estadual"
from consultaprevia2008.tab_urb_asse_prec prec
More weird yet: This is part of a large query. I have the keyword
"as", using accented chars in some other places, and works ok.
My pg version:
"PostgreSQL 8.2.5 on x86_64-unknown-linux-gnu, compiled by GCC gcc
(GCC) 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)"
This is a restored backup from 8.1 . In 8.1 it works ok.
[]'s
- Walter
From: | Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org> |
---|---|
To: | Walter Cruz <walter(dot)php(at)gmail(dot)com> |
Cc: | PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: weird - invalid string enlargement request size |
Date: | 2007-12-04 17:46:38 |
Message-ID: | 20071204174638.GE4727@alvh.no-ip.org |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-hackers |
Walter Cruz escribió:
> A simple query:
>
> select prec.bl_considerada_prioritaria
> AS "A proposta é considerada prioritária por Conselho Municipal ou Estadual"
> from consultaprevia2008.tab_urb_asse_prec prec
>
> The result: ERROR: invalid string enlargement request size 1073741823
Weird. Probably encoding mismatch. What are your locale and encoding
settings, and what encoding is the actual client? And what client is
it? Easiest way to show server settings is:
select name, setting
from pg_settings
where name ~~ any ( array['lc_%', '%_encoding']);
--
Alvaro Herrera http://www.advogato.org/person/alvherre
"La Primavera ha venido. Nadie sabe como ha sido" (A. Machado)
From: | "Walter Cruz" <walter(dot)php(at)gmail(dot)com> |
---|---|
To: | "Alvaro Herrera" <alvherre(at)alvh(dot)no-ip(dot)org> |
Cc: | "PostgreSQL Hackers" <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: weird - invalid string enlargement request size |
Date: | 2007-12-04 18:31:49 |
Message-ID: | 32cabba0712041031s3bc750f7lb8bf9b15c6bf42bf@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-hackers |
Yes, encoding mismatch [:)]
"client_encoding";"UNICODE"
"lc_collate";"pt_BR.UTF-8"
"lc_ctype";"pt_BR.UTF-8"
"lc_messages";"pt_BR.UTF_8"
"lc_monetary";"pt_BR.UTF_8"
"lc_numeric";"pt_BR.UTF_8"
"lc_time";"pt_BR.UTF_8"
"server_encoding";"LATIN1"
I'm with some troubles to set the encoding In my PHP program, but this
is not hackers issue :)
I posted on hackers cause I think that was a bug, os something.
[]'s
- Walter
On Dec 4, 2007 3:46 PM, Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org> wrote:
> Walter Cruz escribió:
> > A simple query:
> >
> > select prec.bl_considerada_prioritaria
> > AS "A proposta é considerada prioritária por Conselho Municipal ou Estadual"
> > from consultaprevia2008.tab_urb_asse_prec prec
> >
> > The result: ERROR: invalid string enlargement request size 1073741823
>
> Weird. Probably encoding mismatch. What are your locale and encoding
> settings, and what encoding is the actual client? And what client is
> it? Easiest way to show server settings is:
>
> select name, setting
> from pg_settings
> where name ~~ any ( array['lc_%', '%_encoding']);
>
> --
> Alvaro Herrera http://www.advogato.org/person/alvherre
> "La Primavera ha venido. Nadie sabe como ha sido" (A. Machado)
>
From: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
---|---|
To: | "Walter Cruz" <walter(dot)php(at)gmail(dot)com> |
Cc: | "Alvaro Herrera" <alvherre(at)alvh(dot)no-ip(dot)org>, "PostgreSQL Hackers" <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: weird - invalid string enlargement request size |
Date: | 2007-12-04 21:19:47 |
Message-ID: | 5849.1196803187@sss.pgh.pa.us |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-hackers |
"Walter Cruz" <walter(dot)php(at)gmail(dot)com> writes:
> I posted on hackers cause I think that was a bug, os something.
Yeah, I think so too. Can you extract a reproducible test case?
regards, tom lane
From: | "Walter Cruz" <walter(dot)php(at)gmail(dot)com> |
---|---|
To: | "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
Cc: | "Alvaro Herrera" <alvherre(at)alvh(dot)no-ip(dot)org>, "PostgreSQL Hackers" <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: weird - invalid string enlargement request size |
Date: | 2007-12-05 16:01:40 |
Message-ID: | 32cabba0712050801j47ec4618t5a0dbdff51b77962@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-hackers |
On Dec 4, 2007 7:19 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> "Walter Cruz" <walter(dot)php(at)gmail(dot)com> writes:
> > I posted on hackers cause I think that was a bug, os something.
>
> Yeah, I think so too. Can you extract a reproducible test case?
I'm trying to extract that case!
[]'s
- Walter
From: | "Walter Cruz" <walter(dot)php(at)gmail(dot)com> |
---|---|
To: | "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
Cc: | "Alvaro Herrera" <alvherre(at)alvh(dot)no-ip(dot)org>, "PostgreSQL Hackers" <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: weird - invalid string enlargement request size |
Date: | 2007-12-05 16:17:11 |
Message-ID: | 32cabba0712050817y72486bf6qa25b3de40451046d@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-hackers |
Yes, I did.
What have I done.
My Linux version: Debian Etch.
My Postgres version:
"PostgreSQL 8.2.5 on x86_64-unknown-linux-gnu, compiled by GCC gcc
(GCC) 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)"
It was compiled by myself.
My initdb was:
initdb -E LATIN1 --locale=pt_BR
By that initdb, the $LANG of the system was pt_BR.UTF-8 .
The pt_BR was not even enabled.
A simple query that shows the problem:
select true AS "áaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
(it's needed this exact number of chars, with one of them accented.)
The output of the SQL of Alvaro in this server is:
"client_encoding";"UNICODE"
"lc_collate";"pt_BR.UTF-8"
"lc_ctype";"pt_BR.UTF-8"
"lc_messages";"pt_BR"
"lc_monetary";"pt_BR"
"lc_numeric";"pt_BR"
"lc_time";"pt_BR"
"server_encoding";"LATIN1"
(I have enabled the pt_BR locale before the tests, but lc_collate and
lc_type are still pt_BR.UTF-8, due to initdb).
Just after that, I changed my $LANG to pt_BR, did a initdb.
The query
select true AS "áaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa";
returns:
template1-# ;
NOTICE: identifier
"áaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
will be truncated to
"áaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
áaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
On this second initdb:
template1=# select name, setting
template1-# from pg_settings
template1-# where name ~~ any ( array['lc_%', '%_encoding']);
name | setting
-----------------+---------
client_encoding | LATIN1
lc_collate | pt_BR
lc_ctype | pt_BR
lc_messages | pt_BR
lc_monetary | pt_BR
lc_numeric | pt_BR
lc_time | pt_BR
server_encoding | LATIN1
Is that information enough? I can give you more details, if needed.
[]'s
- Walter
From: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
---|---|
To: | "Walter Cruz" <walter(dot)php(at)gmail(dot)com> |
Cc: | "Alvaro Herrera" <alvherre(at)alvh(dot)no-ip(dot)org>, "PostgreSQL Hackers" <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: weird - invalid string enlargement request size |
Date: | 2007-12-06 06:15:38 |
Message-ID: | 4622.1196921738@sss.pgh.pa.us |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-hackers |
"Walter Cruz" <walter(dot)php(at)gmail(dot)com> writes:
> My initdb was:
> initdb -E LATIN1 --locale=pt_BR
> By that initdb, the $LANG of the system was pt_BR.UTF-8 .
> A simple query that shows the problem:
> select true AS "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
OK, I was able to reproduce this here:
postgres=# select true AS "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
;
ERROR: invalid string enlargement request size 1073741823
What is happening is that vsnprintf() is failing because it doesn't like
its input data. elog.c's EVALUATE_MESSAGE macro thinks that the only
possible reason for failure is that the output buffer isn't large
enough, so it enlarges and tries again, leading soon to the palloc error
displayed above.
Possibly we should put some effort into producing a more useful error
message here, but I'm reluctant to fool with it, because our historical
experience is that vsnprintf's behavior just isn't very consistent
across platforms.
In any case, the bottom-line reason why you're having a problem is that
the database encoding (LATIN1) is inconsistent with the encoding
specified by the server's locale (UTF-8), thus allowing vsnprintf to see
what it thinks is bad data.
PG 8.3 contains defenses to prevent that sort of inconsistency, and
I think that's probably all we need to do about this.
regards, tom lane