Re: foreign table: WARNING: problem in alloc set ExecutorState detected write past chunk end in block ...

While trying to test out the "EXPLAIN and nfiltered, take two" patch
with its test file "explaintesti", I noticed I was getting a warning
message like:

WARNING: problem in alloc set ExecutorState: detected write past
chunk end in block 0x101134e00, chunk 0x101134f38

I got the same warning on unpatched git head as well. I was able to
reproduce the problem with the attached input.sql file, along with a
dummy input file, dummy_passwd, for the foreign table to read from. I
get the above warning along with:

ERROR: missing data for column "pass"
CONTEXT: COPY passwd, line 1: "##"
STATEMENT: SELECT * FROM passwd WHERE uid > 4;

The error message is not surprising, since I am feeding the foreign
table improperly-formatted lines. The warning is what concerns me;
from the comments in aset.c and the warning message text itself, I
gather this is not expected. I tested on OS X 10.6, with
--enable-cassert --enable-debug.

Josh

Josh Kupershmidt <schmiddy(at)gmail(dot)com> writes:
> While trying to test out the "EXPLAIN and nfiltered, take two" patch
> with its test file "explaintesti", I noticed I was getting a warning
> message like:

> WARNING: problem in alloc set ExecutorState: detected write past
> chunk end in block 0x101134e00, chunk 0x101134f38

> I got the same warning on unpatched git head as well.

Can't reproduce that here, on either 32-bit or 64-bit hardware.
However, this sort of issue is often exceedingly sensitive to
environment and build options. What's your platform, what did you use
for configure options, etc?

[ thinks for a bit... ] It could easily also depend on the precise
lengths of strings being pushed around. What's the exact length of
the foreign table pathname you actually used? And is that the exact
dummy file you used?

regards, tom lane

On Tue, Sep 13, 2011 at 6:29 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Josh Kupershmidt <schmiddy(at)gmail(dot)com> writes:
>> While trying to test out the "EXPLAIN and nfiltered, take two" patch
>> with its test file "explaintesti", I noticed I was getting a warning
>> message like:
>
>>  WARNING:  problem in alloc set ExecutorState: detected write past
>> chunk end in block 0x101134e00, chunk 0x101134f38
>
>> I got the same warning on unpatched git head as well.
>
> Can't reproduce that here, on either 32-bit or 64-bit hardware.
> However, this sort of issue is often exceedingly sensitive to
> environment and build options.  What's your platform, what did you use
> for configure options, etc?

I got the warning on OS X 10.6 64-bit. The only configure options I
used were --prefix=/path/ --enable-cassert --enable-debug.

> [ thinks for a bit... ]  It could easily also depend on the precise
> lengths of strings being pushed around.  What's the exact length of
> the foreign table pathname you actually used?  And is that the exact
> dummy file you used?

I saw the warning first with my actual /etc/passwd file, which had
those comment lines to trigger the error, while trying Marko's
explaintesti sample file[1]. Then I reproduced the problem on git head
with that dummy file at location
"/Users/josh/pg_permanent/warning_alloc/dummy_passwd".

FWIW, I can't reproduce the problem on a 64-bit Ubuntu 11.04 machine either.

Josh

--
[1] http://archives.postgresql.org/pgsql-hackers/2011-09/msg00434.php

Josh Kupershmidt <schmiddy(at)gmail(dot)com> writes:
> On Tue, Sep 13, 2011 at 6:29 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> Can't reproduce that here, on either 32-bit or 64-bit hardware.
>> However, this sort of issue is often exceedingly sensitive to
>> environment and build options. What's your platform, what did you use
>> for configure options, etc?

> I got the warning on OS X 10.6 64-bit. The only configure options I
> used were --prefix=/path/ --enable-cassert --enable-debug.

Hmm. I tried on 64-bit Lion (10.7.1) and had no luck.

Please try setting a gdb breakpoint on the warning elog (it's in aset.c)
and looking to see what's the contents of the complained-of chunk. That
might give us a hint anyway ...

regards, tom lane

On Tue, Sep 13, 2011 at 7:38 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Hmm.  I tried on 64-bit Lion (10.7.1) and had no luck.
>
> Please try setting a gdb breakpoint on the warning elog (it's in aset.c)
> and looking to see what's the contents of the complained-of chunk.  That
> might give us a hint anyway ...

D'oh. Looks like this was a false alarm; I think I neglected to
recompile file_fdw.so to correspond to the server I was running. At
least, the warning has gone away upon a rebuild of the file_fdw
module. Sorry for the noise.

Josh