Let psql process files with > 4,294,967,295 lines

Folks,

I just ran across an issue where in psql, people can get the line
number in the file so long as it is under 2^32-1 lines long, but once
it gets larger than that, it's hosed.

This patch changes the data type from unsigned int to unsigned long
long, which is probably not the correct thing in order to get 64-bit
arithmetic, but I figure it's good enough to get a discussion started.

Cheers,
D
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
phone: +1 415 235 3778 AIM: dfetter666
Skype: davidfetter

Remember to vote!

David Fetter wrote:

Hi,

> I just ran across an issue where in psql, people can get the line
> number in the file so long as it is under 2^32-1 lines long, but once
> it gets larger than that, it's hosed.
>
> This patch changes the data type from unsigned int to unsigned long
> long, which is probably not the correct thing in order to get 64-bit
> arithmetic, but I figure it's good enough to get a discussion started.

The only thing I can tell you is that you should use INT64_FORMAT
instead of %lld.

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

Alvaro Herrera <alvherre(at)commandprompt(dot)com> writes:
> David Fetter wrote:
>> This patch changes the data type from unsigned int to unsigned long
>> long, which is probably not the correct thing in order to get 64-bit
>> arithmetic, but I figure it's good enough to get a discussion started.

> The only thing I can tell you is that you should use INT64_FORMAT
> instead of %lld.

And the datatype should be declared int64, not "long long" which doesn't
exist everywhere.

Actually you probably want uint64 and UINT64_FORMAT...

regards, tom lane

On Sun, Jul 30, 2006 at 05:40:16PM -0400, Tom Lane wrote:
> Alvaro Herrera <alvherre(at)commandprompt(dot)com> writes:
> > David Fetter wrote:
> >> This patch changes the data type from unsigned int to unsigned
> >> long long, which is probably not the correct thing in order to
> >> get 64-bit arithmetic, but I figure it's good enough to get a
> >> discussion started.
>
> > The only thing I can tell you is that you should use INT64_FORMAT
> > instead of %lld.
>
> And the datatype should be declared int64, not "long long" which
> doesn't exist everywhere.
>
> Actually you probably want uint64 and UINT64_FORMAT...
>
> regards, tom lane

I think this fixes it, but I'm unsure how to test it. Two of the
methods mentioned in IRC, attaching with gdb and setting to a value >
2^32, and setting it directly in some code, seem like OK approaches.

Cheers,
D
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
phone: +1 415 235 3778 AIM: dfetter666
Skype: davidfetter

Remember to vote!

David Fetter <david(at)fetter(dot)org> writes:
> + #include "pg_config.h"

You should not need that. All PG code assumes that c.h and its
inclusions have already been read.

regards, tom lane

[ Tom's include adjustment added.]

Your patch has been added to the PostgreSQL unapplied patches list at:

http://momjian.postgresql.org/cgi-bin/pgpatches

It will be applied as soon as one of the PostgreSQL committers reviews
and approves it.

---------------------------------------------------------------------------

David Fetter wrote:
> On Sun, Jul 30, 2006 at 05:40:16PM -0400, Tom Lane wrote:
> > Alvaro Herrera <alvherre(at)commandprompt(dot)com> writes:
> > > David Fetter wrote:
> > >> This patch changes the data type from unsigned int to unsigned
> > >> long long, which is probably not the correct thing in order to
> > >> get 64-bit arithmetic, but I figure it's good enough to get a
> > >> discussion started.
> >
> > > The only thing I can tell you is that you should use INT64_FORMAT
> > > instead of %lld.
> >
> > And the datatype should be declared int64, not "long long" which
> > doesn't exist everywhere.
> >
> > Actually you probably want uint64 and UINT64_FORMAT...
> >
> > regards, tom lane
>
> I think this fixes it, but I'm unsure how to test it. Two of the
> methods mentioned in IRC, attaching with gdb and setting to a value >
> 2^32, and setting it directly in some code, seem like OK approaches.
>
> Cheers,
> D
> --
> David Fetter <david(at)fetter(dot)org> http://fetter.org/
> phone: +1 415 235 3778 AIM: dfetter666
> Skype: davidfetter
>
> Remember to vote!

[ Attachment, skipping... ]

>
> ---------------------------(end of broadcast)---------------------------
> TIP 9: In versions below 8.0, the planner will ignore your desire to
> choose an index scan if your joining column's datatypes do not
> match

--
Bruce Momjian bruce(at)momjian(dot)us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Patch applied. Thanks. Unnecessary #include file removed.

---------------------------------------------------------------------------

David Fetter wrote:
> On Sun, Jul 30, 2006 at 05:40:16PM -0400, Tom Lane wrote:
> > Alvaro Herrera <alvherre(at)commandprompt(dot)com> writes:
> > > David Fetter wrote:
> > >> This patch changes the data type from unsigned int to unsigned
> > >> long long, which is probably not the correct thing in order to
> > >> get 64-bit arithmetic, but I figure it's good enough to get a
> > >> discussion started.
> >
> > > The only thing I can tell you is that you should use INT64_FORMAT
> > > instead of %lld.
> >
> > And the datatype should be declared int64, not "long long" which
> > doesn't exist everywhere.
> >
> > Actually you probably want uint64 and UINT64_FORMAT...
> >
> > regards, tom lane
>
> I think this fixes it, but I'm unsure how to test it. Two of the
> methods mentioned in IRC, attaching with gdb and setting to a value >
> 2^32, and setting it directly in some code, seem like OK approaches.
>
> Cheers,
> D
> --
> David Fetter <david(at)fetter(dot)org> http://fetter.org/
> phone: +1 415 235 3778 AIM: dfetter666
> Skype: davidfetter
>
> Remember to vote!

[ Attachment, skipping... ]

>
> ---------------------------(end of broadcast)---------------------------
> TIP 9: In versions below 8.0, the planner will ignore your desire to
> choose an index scan if your joining column's datatypes do not
> match

--
Bruce Momjian bruce(at)momjian(dot)us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +