Re: Practical maximums (was Re: PostgreSQL theoretical

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jeff Davis wrote:
> On Mon, 2006-07-31 at 09:53 -0500, Ron Johnson wrote:
>
>>> The evasive answer is that you probably don't run regular full pg_dump
>>> on such databases.
>> Hmmm.
>>
>
> You might want to use PITR for incremental backup or maintain a standby
> system using Slony-I ( www.slony.info ).

We want PITR, Incremental & Full.

Currently do:
Full - every other day, directly to 4 tape drives.
Incremental - on "the other" days, to 1 tape drive.
PITR - every 8 hours, to disk.

>>>> Are there any plans of making a multi-threaded, or even
>>>> multi-process pg_dump?
>>> What do you hope to accomplish by that? pg_dump is not CPU bound.
>> Write to multiple tape drives at the same time, thereby reducing the
>> total wall time of the backup process.
>
> pg_dump just produces output. You could pretty easily stripe that output
> across multiple devices just by using some scripts. Just make sure to
> write a script that can reconstruct the data again when you need to
> restore.

But doesn't that mean that pg_dump must dump to disk?

With a *big* database, that's a whole lot of extra kit (not just
spindles) to buy.

> You don't need multi-threaded pg_dump, you just need to use a
> script that produces multiple output streams. Multi-threaded design is
> only useful for CPU-bound applications.
>
> Doing full backups of that much data is always a challenge, and I don't
> think PostgreSQL has limitations that another database doesn't.

I dunno about that. With Rdb/VMS it is trivial to backup a database
directly to multiple tape drives. The Rdb backup utility has an
algorithm with determines which tablespace data is copied to which
tape drive.

- --
Ron Johnson, Jr.
Jefferson LA USA

Is "common sense" really valid?
For example, it is "common sense" to white-power racists that
whites are superior to blacks, and that those with brown skins
are mud people.
However, that "common sense" is obviously wrong.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFE14u3S9HxQb37XmcRAj8xAKC6n4OmHBLeGkGoMz58RFY3FIWf0wCeIlRU
Ott3Uj7/0rpdG7Yb4o+7HPY=
=TdPt
-----END PGP SIGNATURE-----