Re: BufFileWrite across MAX_PHYSICAL_FILESIZE boundary

Just noticed buffile.c:292 doesn't look quite right where
BufFileDumpBuffer calls FileWrite:
bytestowrite = FileWrite(thisfile, file->buffer, bytestowrite);
It looks as though it would write the same data each time the
loop is iterated. Would this be better?
bytestowrite = FileWrite(thisfile, file->buffer + wpos, bytestowrite);

"Kurt Harriman" <kharriman(at)greenplum(dot)com> writes:
> Just noticed buffile.c:292 doesn't look quite right where
> BufFileDumpBuffer calls FileWrite:
> bytestowrite = FileWrite(thisfile, file->buffer, bytestowrite);
> It looks as though it would write the same data each time the
> loop is iterated. Would this be better?
> bytestowrite = FileWrite(thisfile, file->buffer + wpos, bytestowrite);

Yeah, I think you're right. We've probably not seen this because in
most usages the buffer is always block-aligned, but it looks possible
for tuplestore.c to get out of alignment if its concurrent write/read
feature is exercised just so. Do you want to try demonstrating the bug
with a smaller-than-normal setting of MAX_PHYSICAL_FILESIZE?

regards, tom lane

This has been saved for the 8.4 release:

http://momjian.postgresql.org/cgi-bin/pgpatches_hold

---------------------------------------------------------------------------

Tom Lane wrote:
> "Kurt Harriman" <kharriman(at)greenplum(dot)com> writes:
> > Just noticed buffile.c:292 doesn't look quite right where
> > BufFileDumpBuffer calls FileWrite:
> > bytestowrite = FileWrite(thisfile, file->buffer, bytestowrite);
> > It looks as though it would write the same data each time the
> > loop is iterated. Would this be better?
> > bytestowrite = FileWrite(thisfile, file->buffer + wpos, bytestowrite);
>
> Yeah, I think you're right. We've probably not seen this because in
> most usages the buffer is always block-aligned, but it looks possible
> for tuplestore.c to get out of alignment if its concurrent write/read
> feature is exercised just so. Do you want to try demonstrating the bug
> with a smaller-than-normal setting of MAX_PHYSICAL_FILESIZE?
>
> regards, tom lane
>
> ---------------------------(end of broadcast)---------------------------
> TIP 5: don't forget to increase your free space map settings

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Bruce Momjian wrote:
>
> This has been saved for the 8.4 release:
>
> http://momjian.postgresql.org/cgi-bin/pgpatches_hold

Huh, no, this is a bug and should be fixed right away.

> ---------------------------------------------------------------------------
>
> Tom Lane wrote:
> > "Kurt Harriman" <kharriman(at)greenplum(dot)com> writes:
> > > Just noticed buffile.c:292 doesn't look quite right where
> > > BufFileDumpBuffer calls FileWrite:
> > > bytestowrite = FileWrite(thisfile, file->buffer, bytestowrite);
> > > It looks as though it would write the same data each time the
> > > loop is iterated. Would this be better?
> > > bytestowrite = FileWrite(thisfile, file->buffer + wpos, bytestowrite);
> >
> > Yeah, I think you're right. We've probably not seen this because in
> > most usages the buffer is always block-aligned, but it looks possible
> > for tuplestore.c to get out of alignment if its concurrent write/read
> > feature is exercised just so. Do you want to try demonstrating the bug
> > with a smaller-than-normal setting of MAX_PHYSICAL_FILESIZE?

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Alvaro Herrera wrote:
> Bruce Momjian wrote:
> >
> > This has been saved for the 8.4 release:
> >
> > http://momjian.postgresql.org/cgi-bin/pgpatches_hold
>
> Huh, no, this is a bug and should be fixed right away.

OK, moved to the 8.3 patch queue.

---------------------------------------------------------------------------

>
> > ---------------------------------------------------------------------------
> >
> > Tom Lane wrote:
> > > "Kurt Harriman" <kharriman(at)greenplum(dot)com> writes:
> > > > Just noticed buffile.c:292 doesn't look quite right where
> > > > BufFileDumpBuffer calls FileWrite:
> > > > bytestowrite = FileWrite(thisfile, file->buffer, bytestowrite);
> > > > It looks as though it would write the same data each time the
> > > > loop is iterated. Would this be better?
> > > > bytestowrite = FileWrite(thisfile, file->buffer + wpos, bytestowrite);
> > >
> > > Yeah, I think you're right. We've probably not seen this because in
> > > most usages the buffer is always block-aligned, but it looks possible
> > > for tuplestore.c to get out of alignment if its concurrent write/read
> > > feature is exercised just so. Do you want to try demonstrating the bug
> > > with a smaller-than-normal setting of MAX_PHYSICAL_FILESIZE?
>
>
> --
> Alvaro Herrera http://www.CommandPrompt.com/
> The PostgreSQL Company - Command Prompt, Inc.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

I wrote:
> "Kurt Harriman" <kharriman(at)greenplum(dot)com> writes:
>> Just noticed buffile.c:292 doesn't look quite right where
>> BufFileDumpBuffer calls FileWrite:
>> bytestowrite = FileWrite(thisfile, file->buffer, bytestowrite);
>> It looks as though it would write the same data each time the
>> loop is iterated. Would this be better?
>> bytestowrite = FileWrite(thisfile, file->buffer + wpos, bytestowrite);

> Yeah, I think you're right.

FYI, I was able to exercise the bug by changing RELSEG_SIZE to 2 within
buffile.c and then running this script in the regression database:

set work_mem = '64kB';
begin;
declare c scroll cursor for select * from tenk1 a join tenk1 b using(two);
fetch 1 from c;
fetch 100 from c;
fetch backward 10 from c;
fetch 100 from c;
fetch backward 10 from c;
fetch 100 from c;
fetch backward 100 from c;
commit;

It doesn't crash with the fix applied. Thanks for spotting it.

regards, tom lane