Re: Opening stdin/stdout in binary mode on Windows

Hi, I have a dump that was generated with pg_dump in 8.0-beta5 and I'm
trying to restore with pg_restore in 8.0-RC3.

The pg_dump command was: # pg_dump -Ft -o -b database
The pg_restore command was: # pg_restore -Ft -d database

When I execute pg_restore I get this error:
pg_restore: [archiver] unsupported version (1.13) in file header

Dumps created in beta5 are incompatible with rc3 ?
Any suggestion for how can I restore this dump ?

Thank for any help.
Cesar

Cesar Schneider <cesar(at)hzta(dot)com(dot)br> writes:
> When I execute pg_restore I get this error:
> pg_restore: [archiver] unsupported version (1.13) in file header

There is no 1.13; you have a corrupt dump file. Personally I'd bet
money that you ran it through a Windows newline conversion (LF to CR/LF).

regards, tom lane

I am having the same problem:
System: the released Postgres 8.0, winXP
Install performed an initDB. Tryed all types of backup/restore
combinations, see below:

Using:
pg_dump --format=t --blobs myDB > DBFile
pg_restore --create -dbname=crm DBFile

Resulted in:
pg_restore: [archiver] unsupported version (1.13) in file header

NOTE: File was created from pg_dump
and then tried as an input to pg_restore. File was not viewed or
modified
in any way.

*************
Using:
pg_dump --format=t --blobs myDB > DBFile

Resulted in:
pg_dump: [custom archiver] WARNING: ftell mismatch with expected
position -- ftell used

************
Using:
pg_dump --format=t --blobs myDB > DBFile

Resulted in:
pg_dump: large-object output is not supported for plain-text dump files
pg_dump: <Use a different output format.>
***********

Tom Lane wrote:
> Cesar Schneider <cesar(at)hzta(dot)com(dot)br> writes:
> > When I execute pg_restore I get this error:
> > pg_restore: [archiver] unsupported version (1.13) in file header
>
> There is no 1.13; you have a corrupt dump file. Personally I'd bet
> money that you ran it through a Windows newline conversion (LF to
CR/LF).
>
> regards, tom lane
>
> ---------------------------(end of
broadcast)---------------------------
> TIP 5: Have you checked our extensive FAQ?
>
> http://www.postgresql.org/docs/faqs/FAQ.html

I'm still having problems to restore a database.

The dump command used was:

# pg_dump -Ft -b -o database > database.dump

This dump was created in Postgres 8.0beta5 (Windows).

When I try to restore this file in Postgres 8.0-rc3 (Linux) I get this
error:

# pg_restore -Ft -d database database.dump

Using a dump file generated in pgadmin3, works fine... but pgadmin
exports some strange things that are not related to the database.

What can be wrong ?

Cesar

On Mon, 2005-01-17 at 18:42, Tom Lane wrote:
> Cesar Schneider <cesar(at)hzta(dot)com(dot)br> writes:
> > but is not a plain text dump, so I don't think that is a CRFL problem.
>
> Exactly, but you did something to it with a program that thought it was
> plain text. The actual current version number is 1.10. I think it is
> pretty suggestive that LF == 10 decimal and CR == 13 decimal.
>
> regards, tom lane
>

More Info:
If I perform a backup of my database with PGAdmin3. I can restore
the database via the command line pg_restore. It seems as if the
backup created with pg_restore when issued from a winXP command prompt
does not create a file that the pg_restore can read... At least for the
file types Compress and Tar. Which I need to use since my database
contains blobs.

One other item: Issuing an pg_restore -C -d myDB myDB.backup
Did not create the database myDB and complained that the database did
not exist. I had to first create the database and then the pg_restore
functioned correctly (without -C option).

"Niederland" <niederland(at)gmail(dot)com> writes:
> System: the released Postgres 8.0, winXP

> Using:
> pg_dump --format=t --blobs myDB > DBFile
> pg_restore --create -dbname=crm DBFile

> Resulted in:
> pg_restore: [archiver] unsupported version (1.13) in file header

Come to think of it, I'll bet that you cannot use "> DBFile" on Windows
because it ends up opening the archive file in text instead of binary
mode. SetOutput() in pg_backup_archiver.c tries to work around this by
doing

fn = fileno(stdout);
AH->OF = fdopen(dup(fn), PG_BINARY_W);

but it wouldn't surprise me in the least to learn that that doesn't work
on Windows.

Does it work if you use
pg_dump --format=t --blobs -f DBFile myDB
? Can anyone on pgsql-hackers-win32 think of a way around this?

regards, tom lane

Tom Lane wrote:
> "Niederland" <niederland(at)gmail(dot)com> writes:
> > System: the released Postgres 8.0, winXP
>
> > Using:
> > pg_dump --format=t --blobs myDB > DBFile
> > pg_restore --create -dbname=crm DBFile
>
> > Resulted in:
> > pg_restore: [archiver] unsupported version (1.13) in file header
>
> Come to think of it, I'll bet that you cannot use "> DBFile" on Windows
> because it ends up opening the archive file in text instead of binary
> mode. SetOutput() in pg_backup_archiver.c tries to work around this by
> doing
>
> fn = fileno(stdout);
> AH->OF = fdopen(dup(fn), PG_BINARY_W);
>
> but it wouldn't surprise me in the least to learn that that doesn't work
> on Windows.
>
> Does it work if you use
> pg_dump --format=t --blobs -f DBFile myDB
> ? Can anyone on pgsql-hackers-win32 think of a way around this?

I never considered the distinction of binary/text mode on ">" but I can
see it could certainly be an issue.

Do we have to start suggesting "-f" for all scripts just so Win32 is OK?

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

That worked thanks. Just can not use the "> DBFile" on windows.

I wrote:
>> pg_restore: [archiver] unsupported version (1.13) in file header

> Come to think of it, I'll bet that you cannot use "> DBFile" on Windows
> because it ends up opening the archive file in text instead of binary
> mode. SetOutput() in pg_backup_archiver.c tries to work around this by
> doing

> fn = fileno(stdout);
> AH->OF = fdopen(dup(fn), PG_BINARY_W);

> but it wouldn't surprise me in the least to learn that that doesn't work
> on Windows.

We now have confirmation that indeed this is the source of the pg_dump
misbehavior. Can anyone suggest a way that does work to do binary I/O
on stdout? What about reading from stdin?

If we can't fix this it will require some serious uglification of the
pg_dump/pg_restore docs for Windows ...

regards, tom lane

I wrote:
> Can anyone suggest a way that does work to do binary I/O
> on stdout? What about reading from stdin?

After a bit of googling I came up with the attached patch. I am not in
a position to test it, however. Can anyone try it out?

regards, tom lane

*** src/bin/pg_dump/pg_backup_archiver.c~ Tue Jan 25 17:44:31 2005
--- src/bin/pg_dump/pg_backup_archiver.c Wed Jan 26 13:34:33 2005
***************
*** 29,34 ****
--- 29,38 ----
#include <ctype.h>
#include <unistd.h>

+ #ifdef WIN32
+ #include <io.h>
+ #endif
+
#include "pqexpbuffer.h"
#include "libpq/libpq-fs.h"

***************
*** 1709,1714 ****
--- 1713,1731 ----
AH->gzOut = 0;
AH->OF = stdout;

+ /*
+ * On Windows, we need to use binary mode to read/write non-text archive
+ * formats. Force stdin/stdout into binary mode in case that is what
+ * we are using.
+ */
+ #ifdef WIN32
+ if (fmt != archNull)
+ {
+ setmode(fileno(stdout), O_BINARY);
+ setmode(fileno(stdin), O_BINARY);
+ }
+ #endif
+
#if 0
write_msg(modulename, "archive format is %d\n", fmt);
#endif

Well, that is a nifty trick. Great to have that solved with no
documentation changes required.

Without this fix we were going to have tons of headaches.

---------------------------------------------------------------------------

Tom Lane wrote:
> I wrote:
> > Can anyone suggest a way that does work to do binary I/O
> > on stdout? What about reading from stdin?
>
> After a bit of googling I came up with the attached patch. I am not in
> a position to test it, however. Can anyone try it out?
>
> regards, tom lane
>
> *** src/bin/pg_dump/pg_backup_archiver.c~ Tue Jan 25 17:44:31 2005
> --- src/bin/pg_dump/pg_backup_archiver.c Wed Jan 26 13:34:33 2005
> ***************
> *** 29,34 ****
> --- 29,38 ----
> #include <ctype.h>
> #include <unistd.h>
>
> + #ifdef WIN32
> + #include <io.h>
> + #endif
> +
> #include "pqexpbuffer.h"
> #include "libpq/libpq-fs.h"
>
> ***************
> *** 1709,1714 ****
> --- 1713,1731 ----
> AH->gzOut = 0;
> AH->OF = stdout;
>
> + /*
> + * On Windows, we need to use binary mode to read/write non-text archive
> + * formats. Force stdin/stdout into binary mode in case that is what
> + * we are using.
> + */
> + #ifdef WIN32
> + if (fmt != archNull)
> + {
> + setmode(fileno(stdout), O_BINARY);
> + setmode(fileno(stdin), O_BINARY);
> + }
> + #endif
> +
> #if 0
> write_msg(modulename, "archive format is %d\n", fmt);
> #endif
>
> ---------------------------(end of broadcast)---------------------------
> TIP 3: if posting/reading through Usenet, please send an appropriate
> subscribe-nomail command to majordomo(at)postgresql(dot)org so that your
> message can get through to the mailing list cleanly
>

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073