From: | Féliciano Matias <feliciano(dot)matias(at)free(dot)fr> |
---|---|
To: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
Cc: | "pgsql-general(at)postgresql(dot)org" <pgsql-general(at)postgresql(dot)org> |
Subject: | Re: psql and bytea |
Date: | 2003-05-16 04:56:02 |
Message-ID: | 1053060938.4573.53.camel@one.myworld |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-general |
Le jeu 15/05/2003 à 16:47, Tom Lane a écrit :
> =?ISO-8859-1?Q?F=E9liciano?= Matias <feliciano(dot)matias(at)free(dot)fr> writes:
> > one=3D> -- this is not the original size (20M : 20971520).
> > I don't have any problems with smaller files (10Mo is always fine).
> > All of this is used with postgresql 7.3.2 shipped with Red Hat Linux 9 .
>
> Hmm. There used to be some off-by-one type bugs in psql's
> variable-substitution code, but those were fixed long before 7.3.2.
> In any case, it's hard to see why such a problem would only arise
> when you got past 10Mb string lengths.
>
More then 10Mb string lengths :
> > $ tobytea < data | wc
> > 0 81574 74379444 (71 Mo)
> I couldn't duplicate the problem here,
It seems it's a bug in glibc-2.3.2-27.9.i686.rpm coming with RH9.
> so I'm going to suggest that
> maybe you have a hardware problem? Perhaps there's a flaky RAM chip in
> an area of memory that doesn't get used until you push up the size
> of psql quite a bit. It'd be worth running memtest86 for awhile to
> check.
The hardware is ok.
I found the problem. It's bug in sprintf()
===============================================
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#define SIZE (1024*70000)
int main(void) {
char * s = malloc(SIZE) ;
char * d = malloc(SIZE) ;
memset(s, 'a', SIZE-1) ;
d[SIZE-1] = '\0' ;
sprintf(d,"%s",s) ;
printf("%zi\n", strlen(d)) ;
return 0 ;
}
===============================================
$ ./a.out
67108863 (2^26-1)
My libc have a problem. I will fill a bug report to
http://bugzilla.redhat.com/ .
I apply this patch to the postgresql source :
===============================================
diff -urN postgresql-7.3.2.orig/src/bin/psql/mainloop.c postgresql-7.3.2/src/bin/psql/mainloop.c
--- postgresql-7.3.2.orig/src/bin/psql/mainloop.c 2002-10-13 01:09:34.000000000 +0200
+++ postgresql-7.3.2/src/bin/psql/mainloop.c 2003-05-16 04:46:03.000000000 +0200
@@ -389,8 +389,9 @@
exit(EXIT_FAILURE);
}
- sprintf(new, "%.*s%s%s", i, line, value,
- &line[i + thislen + in_length]);
+ sprintf(new, "%.*s", i, line) ;
+ strcat(&new[i], value) ;
+ strcat(&new[i+out_length], &line[i + thislen + in_length]) ;
free(line);
line = new;
===============================================
Also, i want to know if someone is interesting by tools such as
(to|from)bytea. I can put this little toys (with some enhancement.
"--help" :-) ) in contrib and update the man page of psql to show how
to use this tools to copy the content of a file (or stream) into a
field.
Since Postgresql have binary string, it is interesting and better than
\lo_* for little files (this use more memory than \lo_*).
By the way, i can add an option to psql that toggle the output of
carrier return at the end of each records. Perhaps this can be add to
\t.
>
> regards, tom lane
>
--
Féliciano Matias <feliciano(dot)matias(at)free(dot)fr>
From | Date | Subject | |
---|---|---|---|
Next Message | Wayne Armstrong | 2003-05-16 05:09:28 | Re: psql and bytea |
Previous Message | Robert Abbate | 2003-05-16 03:57:11 | Re: using 7.2 & 7.3 together gives problems.... |