Re: bytea on windows perl client

Hi,

I have some code which runs fine on linux. It simply pulls a png from a bytea
column and displays it.

Now I need to get it running on windows. We have installed ActiveState perl
with the DBI module and downloaded and installed the DBD-Pg binary.
Connections to the database are working fine, but the png file is getting
corrupted.

Anybody know the work-around?

- James

James Orr <james(at)lrgmail(dot)com> writes:
> I have some code which runs fine on linux. It simply pulls a png from a bytea
> column and displays it.

> Now I need to get it running on windows. We have installed ActiveState perl
> with the DBI module and downloaded and installed the DBD-Pg binary.
> Connections to the database are working fine, but the png file is getting
> corrupted.

> Anybody know the work-around?

I'd bet money that the problem is a newline conversion issue --- you
could check by noticing whether the PNG signature (first 8 bytes of the
file) comes back correctly or not. The signature is designed to get
changed if it's fed through any of the standard kinds of newline
conversion (\n -> \r\n, for example).

Not that this gives any immediate suggestion for a fix, but
understanding what is happening is always a good first step.

regards, tom lane

Hi Tom,

On Wednesday 17 July 2002 12:08 pm, Tom Lane wrote:
> James Orr <james(at)lrgmail(dot)com> writes:
> > I have some code which runs fine on linux. It simply pulls a png from a
> > bytea column and displays it.
> >
> > Now I need to get it running on windows. We have installed ActiveState
> > perl with the DBI module and downloaded and installed the DBD-Pg binary.
> > Connections to the database are working fine, but the png file is getting
> > corrupted.
> >
> > Anybody know the work-around?
>
> I'd bet money that the problem is a newline conversion issue --- you
> could check by noticing whether the PNG signature (first 8 bytes of the
> file) comes back correctly or not. The signature is designed to get
> changed if it's fed through any of the standard kinds of newline
> conversion (\n -> \r\n, for example).

OK, if I load the output into vi. The one from linux contains a bunch of
control characters etc. The one from windows seems to be all text like so
...

\211PNG\015\012\000 and so on.

Here is my actual perl script:

#! /usr/bin/perl -w

use DBI;
require 'common.pl';

my $dbh; initdb(\$dbh);

# Get the logos
my $image_sth = $dbh->prepare("SELECT logo FROM logos ORDER BY random() LIMIT
1");
$image_sth->execute();
my $image = $image_sth->fetchall_arrayref->[0]->[0];
$image_sth->finish();

$dbh->disconnect();

print "Content-type: image/png\n\n";
print $image;

exit;

James Orr wrote:
> OK, if I load the output into vi. The one from linux contains a
bunch > of control characters etc. The one from windows seems to be all
text
> like so
> ...
>
> \211PNG\015\012\000 and so on.

This is the escaped output form of bytea. It seems that on linux the
perl script is either using a binary cursor, or automagically unescaping
the output, while on windows it isn't. I'm not sure why that would
happen, but hopefully one of the perl gurus hanging around will chime in.

Joe

On Wednesday, July 17, 2002, at 10:06 AM, Joe Conway wrote:

> James Orr wrote:
> > OK, if I load the output into vi. The one from linux contains a
> bunch > of control characters etc. The one from windows seems to be
> all text
> > like so
> > ...
> >
>> \211PNG\015\012\000 and so on.
>
> This is the escaped output form of bytea. It seems that on linux the
> perl script is either using a binary cursor, or automagically
> unescaping the output, while on windows it isn't. I'm not sure why that
> would happen, but hopefully one of the perl gurus hanging around will
> chime in.
>
> Joe

James,

This may be a long shot, but try to binmode your output stream:

http://www.perldoc.com/perl5.6/pod/func/binmode.html

Let us know how it works.

--Jeremy

On Wednesday 17 July 2002 01:17 pm, Jeremy Buchmann wrote:
> >> \211PNG\015\012\000 and so on.
> >
> > This is the escaped output form of bytea. It seems that on linux the
> > perl script is either using a binary cursor, or automagically
> > unescaping the output, while on windows it isn't. I'm not sure why that
> > would happen, but hopefully one of the perl gurus hanging around will
> > chime in.
> >
> > Joe
>
> James,
>
> This may be a long shot, but try to binmode your output stream:
>
> Let us know how it works.

I did this ...

print "Content-type: image/png\n\n";
binmode STDOUT;
print $image;

Didn't work.

- James

James Orr wrote:
> On Wednesday 17 July 2002 01:17 pm, Jeremy Buchmann wrote:
>
>>>>\211PNG\015\012\000 and so on.
>>>
>>>This is the escaped output form of bytea. It seems that on linux the
>>>perl script is either using a binary cursor, or automagically
>>>unescaping the output, while on windows it isn't. I'm not sure why that
>>>would happen, but hopefully one of the perl gurus hanging around will
>>>chime in.
>>>
>>>Joe
>>
>>James,
>>
>>This may be a long shot, but try to binmode your output stream:
>>
>>Let us know how it works.
>
>
> I did this ...
>
> print "Content-type: image/png\n\n";
> binmode STDOUT;
> print $image;
>
> Didn't work.

Does perl (or one of the available libraries) have a function like PHP's
stripcslashes()? See:
http://www.php.net/manual/en/function.stripcslashes.php

If so, you should be able to do something like:

print "Content-type: image/png\n\n";
print stripcslashes($image);

HTH,

Joe

On Wednesday 17 July 2002 13:06, Joe Conway wrote:
> > \211PNG\015\012\000 and so on.
>
> This is the escaped output form of bytea. It seems that on linux the
> perl script is either using a binary cursor, or automagically unescaping
> the output, while on windows it isn't. I'm not sure why that would
> happen, but hopefully one of the perl gurus hanging around will chime in.
>
> Joe

OK, I really need a solution to this quite quickly. It's a simple enough
script (posted earlier) so if this is something I cannot do with DBD::Pg on a
windows platform I can do it with something else. If that's the case what's
the best thing to do it with? PHP? I don't have control of the IIS server
they are running this on and I doubt very much they have PHP running on it.

Is the only way to connect to postgres with ASP via the ODBC driver? How will
that handle the bytea field?

- James

On Thursday, July 18, 2002, at 10:01 AM, James Orr wrote:
>
> OK, I really need a solution to this quite quickly. It's a simple
> enough
> script (posted earlier) so if this is something I cannot do with
> DBD::Pg on a
> windows platform I can do it with something else. If that's the case
> what's
> the best thing to do it with? PHP? I don't have control of the IIS
> server
> they are running this on and I doubt very much they have PHP running on
> it.
>
> Is the only way to connect to postgres with ASP via the ODBC driver?
> How will
> that handle the bytea field?
>

If the script is really, really simple, you *could* write it in (deep
breath) C...though
I've never tried to do anything complicated with Pg in C, so I don't
know how
well it will work.

--Jeremy

James Orr wrote:
> On Wednesday 17 July 2002 13:06, Joe Conway wrote:
>
>>>\211PNG\015\012\000 and so on.
>>
>>This is the escaped output form of bytea. It seems that on linux the
>>perl script is either using a binary cursor, or automagically unescaping
>>the output, while on windows it isn't. I'm not sure why that would
>>happen, but hopefully one of the perl gurus hanging around will chime in.
>>
>>Joe
>
>
> OK, I really need a solution to this quite quickly. It's a simple enough
> script (posted earlier) so if this is something I cannot do with DBD::Pg on a
> windows platform I can do it with something else. If that's the case what's
> the best thing to do it with? PHP? I don't have control of the IIS server
> they are running this on and I doubt very much they have PHP running on it.
>
> Is the only way to connect to postgres with ASP via the ODBC driver? How will
> that handle the bytea field?

I really have to believe a relatively simple solution exists in perl --
I'm just not familiar enough with perl to help. Did you see my post on
PHP's stripcslashes()? I'd look for the equiv function in perl.

The ASP/ODBC approach will have the same issue because the standard
output format of bytea is octal escaping for all non-printable characters.

A quick search on google finds:
Data: Numbers
Why isn't my octal data interpreted correctly?

Perl only understands octal and hex numbers as such when they
occur as literals in your program. If they are read in from
somewhere and assigned, no automatic conversion takes place.
You must explicitly use oct() or hex() if you want the values
converted. oct() interprets both hex ("0x350") numbers and
octal ones ("0350" or even without the leading "0", like
"377"), while hex() only converts hexadecimal ones, with or
without a leading "0x", like "0x255", "3A", "ff", or
"deadbeef".

This problem shows up most often when people try using
chmod(), mkdir(), umask(), or sysopen(), which all want
permissions in octal.

chmod(644, $file); # WRONG -- perl -w catches this
chmod(0644, $file); # right

So maybe you could replace all single "\" characters (but *not* "\\")
with "0" and then use oct()?

FWIW, here is the C code for the PHP stripcslashes() function. Maybe you
can convert this to a perl function and use it.

/* {{{ php_stripcslashes
*/
PHPAPI void php_stripcslashes(char *str, int *len)
{
char *source, *target, *end;
int nlen = *len, i;
char numtmp[4];

for (source=str, end=str+nlen, target=str; source<end; source++) {
if (*source == '\\' && source+1<end) {
source++;
switch (*source) {
case 'n': *target++='\n'; nlen--; break;
case 'r': *target++='\r'; nlen--; break;
case 'a': *target++='\a'; nlen--; break;
case 't': *target++='\t'; nlen--; break;
case 'v': *target++='\v'; nlen--; break;
case 'b': *target++='\b'; nlen--; break;
case 'f': *target++='\f'; nlen--; break;
case '\\': *target++='\\'; nlen--; break;
case 'x': if (source+1<end && isxdigit((int)(*(source+1)))) {
numtmp[0] = *++source;
if (source+1<end && isxdigit((int)(*(source+1)))) {
numtmp[1] = *++source;
numtmp[2] = '\0';
nlen-=3;
} else {
numtmp[1] = '\0';
nlen-=2;
}
*target++=(char)strtol(numtmp, NULL, 16);
break;
}
/* break is left intentionally */
default: i=0;
while (source<end && *source>='0' && *source<='7' && i<3) {
numtmp[i++] = *source++;
}
if (i) {
numtmp[i]='\0';
*target++=(char)strtol(numtmp, NULL, 8);
nlen-=i;
source--;
} else {
*target++=*source;
nlen--;
}
}
} else {
*target++=*source;
}
}

if(nlen != 0) {
*target='\0';
}

*len = nlen;
}
/* }}} */

HTH,

Joe