Re: The number of character limitation of custom script on pgbench

Hi all,

The function of custom script of pgbench allows only BUFSIZ
(i.g.,1024byte) or less as length of a SQL.
I think that when we want to bench mark with long SQL then it will difficult.
At that time even pgbench doesn't return ERROR. It will try to do
query with the broken SQL.
And user can not know why function of custom script is not work fine.
It look as just error of SQL to user.

So I'm thinking following solution.
(1) to increase buffer size
(2) to change to variable buffer size
(3) to return ERROR with information

Thought?

Regards,

-------
Sawada Masahiko

On 11/13/13, 6:18 AM, Sawada Masahiko wrote:
> Hi all,
>
> The function of custom script of pgbench allows only BUFSIZ
> (i.g.,1024byte) or less as length of a SQL.
> I think that when we want to bench mark with long SQL then it will difficult.
> At that time even pgbench doesn't return ERROR. It will try to do
> query with the broken SQL.
> And user can not know why function of custom script is not work fine.
> It look as just error of SQL to user.
>
> So I'm thinking following solution.
> (1) to increase buffer size
> (2) to change to variable buffer size
> (3) to return ERROR with information

I'd go for #2. But at least an error.

Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> On 11/13/13, 6:18 AM, Sawada Masahiko wrote:
>> The function of custom script of pgbench allows only BUFSIZ
>> (i.g.,1024byte) or less as length of a SQL.
>> So I'm thinking following solution.
>> (1) to increase buffer size
>> (2) to change to variable buffer size
>> (3) to return ERROR with information

> I'd go for #2. But at least an error.

#2 definitely. I've run into this limitation myself recently, and
so have other people. It's time to fix it.

regards, tom lane

On Wed, Nov 13, 2013 at 10:27 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
>> On 11/13/13, 6:18 AM, Sawada Masahiko wrote:
>>> The function of custom script of pgbench allows only BUFSIZ
>>> (i.g.,1024byte) or less as length of a SQL.
>>> So I'm thinking following solution.
>>> (1) to increase buffer size
>>> (2) to change to variable buffer size
>>> (3) to return ERROR with information
>
>> I'd go for #2. But at least an error.
>
> #2 definitely. I've run into this limitation myself recently, and
> so have other people. It's time to fix it.
>

Yes, I also think #2 is good.
I will implement the patch.

Regards,

-------
Sawada Masahiko

Sawada Masahiko escribió:

> Yes, I also think #2 is good.
> I will implement the patch.

I remember this was recently discussed in the spanish list. Please see
this email:

http://archives.postgresql.org/message-id/48589.192.168.207.54.1382570043.squirrel@webmail.etecsa.cu

--
Álvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On Wed, Nov 13, 2013 at 10:27 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
>> On 11/13/13, 6:18 AM, Sawada Masahiko wrote:
>>> The function of custom script of pgbench allows only BUFSIZ
>>> (i.g.,1024byte) or less as length of a SQL.
>>> So I'm thinking following solution.
>>> (1) to increase buffer size
>>> (2) to change to variable buffer size
>>> (3) to return ERROR with information
>
>> I'd go for #2. But at least an error.
>
> #2 definitely. I've run into this limitation myself recently, and
> so have other people. It's time to fix it.
>

I attached the patch which solves this problem, and have submitted to CF3.
I changed how to get the SQL from custom script file.

Regards,

-------
Sawada Masahiko

Sawada Masahiko <sawada(dot)mshk(at)gmail(dot)com> writes:
> I attached the patch which solves this problem, and have submitted to CF3.
> I changed how to get the SQL from custom script file.

This needed a bit of work:

- Use of strncat didn't seem particularly safe, or efficient. I changed
it to explicitly account for space consumption in the result buffer.
- It leaked the buffer space used. While this likely doesn't matter for
foreseeable usage, it seemed worth fixing.
- Didn't do the right thing for a file not ending with a newline.
- Didn't follow project code layout standards.

I've committed the attached revised version.

regards, tom lane