Re: version upgrade

Lists: pgsql-hackers
From: Josh Berkus <josh(at)agliodbs(dot)com>
To: Andrew Rawnsley <ronz(at)ravensfield(dot)com>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: version upgrade
Date: 2004-08-31 22:16:55
Message-ID: 200408311516.55143.josh@agliodbs.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Andrew,

> If I were loony enough to want to make an attempt at a version updater
> (i.e. migrate a
> 7.4 database to 8.0 without an initdb), any suggestions on where to
> poke first? Does a
> catalog/list of system catalog changes exist anywhere? Any really gross
> problems immediately
> present themselves? Is dusting off pg_upgrade a good place to start, or
> is that a dead end?

Join the Slony project? Seriously, this is one of the uses of slony. All
you'd need would be a script that would:

1) Install PG 8.0 to an alternate directory;
2) Start 8.0;
3) install Slony on both instances (the 7.4 and the 8.0);
4) make 7.4 the "master" and start replicating
5) when 8.0 is caught up, stop 7.4 and promote it to Master
6) turn off Slony.

--
--Josh

Josh Berkus
Aglio Database Solutions
San Francisco


From: "Marc G(dot) Fournier" <scrappy(at)postgresql(dot)org>
To: Josh Berkus <josh(at)agliodbs(dot)com>
Cc: Andrew Rawnsley <ronz(at)ravensfield(dot)com>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: version upgrade
Date: 2004-08-31 22:23:47
Message-ID: 20040831192101.L31538@ganymede.hub.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Tue, 31 Aug 2004, Josh Berkus wrote:

> Andrew,
>
>> If I were loony enough to want to make an attempt at a version updater
>> (i.e. migrate a
>> 7.4 database to 8.0 without an initdb), any suggestions on where to
>> poke first? Does a
>> catalog/list of system catalog changes exist anywhere? Any really gross
>> problems immediately
>> present themselves? Is dusting off pg_upgrade a good place to start, or
>> is that a dead end?
>
> Join the Slony project? Seriously, this is one of the uses of slony. All
> you'd need would be a script that would:
>
> 1) Install PG 8.0 to an alternate directory;
> 2) Start 8.0;
> 3) install Slony on both instances (the 7.4 and the 8.0);
> 4) make 7.4 the "master" and start replicating
> 5) when 8.0 is caught up, stop 7.4 and promote it to Master
> 6) turn off Slony.

Slony is not an upgrade utility, and falls short in one big case ..
literally .. a very large database with limited cash resources to
duplicate it (as far as hardware is concerned). In small shops, or those
with 'free budget', Slony is perfect ... but if you are in an organization
where getting money is like pulling teeth, picking up a new server "just
to do an upgrade" can prove to be difficult ...

----
Marc G. Fournier Hub.Org Networking Services (http://www.hub.org)
Email: scrappy(at)hub(dot)org Yahoo!: yscrappy ICQ: 7615664


From: Josh Berkus <josh(at)agliodbs(dot)com>
To: "Marc G(dot) Fournier" <scrappy(at)postgresql(dot)org>
Cc: Andrew Rawnsley <ronz(at)ravensfield(dot)com>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: version upgrade
Date: 2004-08-31 22:30:07
Message-ID: 200408311530.07635.josh@agliodbs.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Marc,

> Slony is not an upgrade utility, and falls short in one big case ..
> literally .. a very large database with limited cash resources to
> duplicate it (as far as hardware is concerned). In small shops, or those
> with 'free budget', Slony is perfect ... but if you are in an organization
> where getting money is like pulling teeth, picking up a new server "just
> to do an upgrade" can prove to be difficult ...

Huh? You can replicate onto the same server. Kicks your performance in
the teeth but it works fine. Heck, I did it on my laptop as a demo.

--
--Josh

Josh Berkus
Aglio Database Solutions
San Francisco


From: Andrew Rawnsley <ronz(at)ravensfield(dot)com>
To: PostgreSQL-development Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: version upgrade
Date: 2004-09-01 01:38:14
Message-ID: 97A68B1C-FBB7-11D8-8A87-000A95C1D2E6@ravensfield.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers


On Aug 31, 2004, at 6:23 PM, Marc G. Fournier wrote:

> On Tue, 31 Aug 2004, Josh Berkus wrote:
>
>> Andrew,
>>
>>> If I were loony enough to want to make an attempt at a version
>>> updater
>>> (i.e. migrate a
>>> 7.4 database to 8.0 without an initdb), any suggestions on where to
>>> poke first? Does a
>>> catalog/list of system catalog changes exist anywhere? Any really
>>> gross
>>> problems immediately
>>> present themselves? Is dusting off pg_upgrade a good place to start,
>>> or
>>> is that a dead end?
>>
>> Join the Slony project? Seriously, this is one of the uses of
>> slony. All
>> you'd need would be a script that would:
>>

I thought of this quite a bit when I was working over eRServer a while
back.

Its _better_ than a dump and restore, since you can keep the master up
while the
'upgrade' is happening. But Mark is right - it can be quite
problematic from an equivalent
resource point of view. An in-place system (even a faux setup like
pg_upgrade) would be
easier to deal with in many situations.

In the end, using a replication system OR a working pg_upgrade is still
a pretty creaky
workaround. Having to do either tends to lob about 15 pounds of nails
into the gears when
trying to develop a business case about upgrading (Doesn't necessarily
stop it dead, but
does get everyone's attention...). The day when a dump/restore is not
necessary is
the day all of us are hoping for.

>> 1) Install PG 8.0 to an alternate directory;
>> 2) Start 8.0;
>> 3) install Slony on both instances (the 7.4 and the 8.0);
>> 4) make 7.4 the "master" and start replicating
>> 5) when 8.0 is caught up, stop 7.4 and promote it to Master
>> 6) turn off Slony.
>
> Slony is not an upgrade utility, and falls short in one big case ..
> literally .. a very large database with limited cash resources to
> duplicate it (as far as hardware is concerned). In small shops, or
> those with 'free budget', Slony is perfect ... but if you are in an
> organization where getting money is like pulling teeth, picking up a
> new server "just to do an upgrade" can prove to be difficult ...
>
> ----
> Marc G. Fournier Hub.Org Networking Services
> (http://www.hub.org)
> Email: scrappy(at)hub(dot)org Yahoo!: yscrappy ICQ:
> 7615664
>
> ---------------------------(end of
> broadcast)---------------------------
> TIP 2: you can get off all lists at once with the unregister command
> (send "unregister YourEmailAddressHere" to majordomo(at)postgresql(dot)org)
>
--------------------

Andrew Rawnsley
President
The Ravensfield Digital Resource Group, Ltd.
(740) 587-0114
www.ravensfield.com


From: Jan Wieck <JanWieck(at)Yahoo(dot)com>
To: Andrew Rawnsley <ronz(at)ravensfield(dot)com>
Cc: PostgreSQL-development Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: version upgrade
Date: 2004-09-01 03:35:18
Message-ID: 41354376.2020107@Yahoo.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 8/31/2004 9:38 PM, Andrew Rawnsley wrote:

> On Aug 31, 2004, at 6:23 PM, Marc G. Fournier wrote:
>
>> On Tue, 31 Aug 2004, Josh Berkus wrote:
>>
>>> Andrew,
>>>
>>>> If I were loony enough to want to make an attempt at a version
>>>> updater
>>>> (i.e. migrate a
>>>> 7.4 database to 8.0 without an initdb), any suggestions on where to
>>>> poke first? Does a
>>>> catalog/list of system catalog changes exist anywhere? Any really
>>>> gross
>>>> problems immediately
>>>> present themselves? Is dusting off pg_upgrade a good place to start,
>>>> or
>>>> is that a dead end?
>>>
>>> Join the Slony project? Seriously, this is one of the uses of
>>> slony. All
>>> you'd need would be a script that would:
>>>
>
> I thought of this quite a bit when I was working over eRServer a while
> back.
>
> Its _better_ than a dump and restore, since you can keep the master up
> while the
> 'upgrade' is happening. But Mark is right - it can be quite
> problematic from an equivalent
> resource point of view. An in-place system (even a faux setup like
> pg_upgrade) would be
> easier to deal with in many situations.

There is something that you will not (or only under severe risk) get
with an in-place upgrade system. The ability to downgrade back in the
case, your QA missed a few gotchas. The application might not instantly
eat the data, but it might start to sputter and hobble here and there.

With the Slony system, you not only switch over to the new version. But
you keep the old system as a slave. That means that if you discover 4
hours after the upgrade that the new version bails out with errors on a
lot of queries from the application, you have the chance to switch back
to the old version and have lost no single committed transaction.

Jan

>
> In the end, using a replication system OR a working pg_upgrade is still
> a pretty creaky
> workaround. Having to do either tends to lob about 15 pounds of nails
> into the gears when
> trying to develop a business case about upgrading (Doesn't necessarily
> stop it dead, but
> does get everyone's attention...). The day when a dump/restore is not
> necessary is
> the day all of us are hoping for.
>
>
>>> 1) Install PG 8.0 to an alternate directory;
>>> 2) Start 8.0;
>>> 3) install Slony on both instances (the 7.4 and the 8.0);
>>> 4) make 7.4 the "master" and start replicating
>>> 5) when 8.0 is caught up, stop 7.4 and promote it to Master
>>> 6) turn off Slony.
>>
>> Slony is not an upgrade utility, and falls short in one big case ..
>> literally .. a very large database with limited cash resources to
>> duplicate it (as far as hardware is concerned). In small shops, or
>> those with 'free budget', Slony is perfect ... but if you are in an
>> organization where getting money is like pulling teeth, picking up a
>> new server "just to do an upgrade" can prove to be difficult ...
>>
>> ----
>> Marc G. Fournier Hub.Org Networking Services
>> (http://www.hub.org)
>> Email: scrappy(at)hub(dot)org Yahoo!: yscrappy ICQ:
>> 7615664
>>
>> ---------------------------(end of
>> broadcast)---------------------------
>> TIP 2: you can get off all lists at once with the unregister command
>> (send "unregister YourEmailAddressHere" to majordomo(at)postgresql(dot)org)
>>
> --------------------
>
> Andrew Rawnsley
> President
> The Ravensfield Digital Resource Group, Ltd.
> (740) 587-0114
> www.ravensfield.com
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 8: explain analyze is your friend

--
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#================================================== JanWieck(at)Yahoo(dot)com #


From: Andrew Rawnsley <ronz(at)ravensfield(dot)com>
To: PostgreSQL-development Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: version upgrade
Date: 2004-09-01 10:53:25
Message-ID: 261F2BAC-FC05-11D8-8A87-000A95C1D2E6@ravensfield.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers


On Aug 31, 2004, at 11:35 PM, Jan Wieck wrote:

> On 8/31/2004 9:38 PM, Andrew Rawnsley wrote:
>
>> On Aug 31, 2004, at 6:23 PM, Marc G. Fournier wrote:
>>> On Tue, 31 Aug 2004, Josh Berkus wrote:
>>>
>>>> Andrew,
>>>>
>>>>> If I were loony enough to want to make an attempt at a version
>>>>> updater
>>>>> (i.e. migrate a
>>>>> 7.4 database to 8.0 without an initdb), any suggestions on where to
>>>>> poke first? Does a
>>>>> catalog/list of system catalog changes exist anywhere? Any really
>>>>> gross
>>>>> problems immediately
>>>>> present themselves? Is dusting off pg_upgrade a good place to
>>>>> start, or
>>>>> is that a dead end?
>>>>
>>>> Join the Slony project? Seriously, this is one of the uses of
>>>> slony. All
>>>> you'd need would be a script that would:
>>>>
>> I thought of this quite a bit when I was working over eRServer a
>> while back.
>> Its _better_ than a dump and restore, since you can keep the master
>> up while the
>> 'upgrade' is happening. But Mark is right - it can be quite
>> problematic from an equivalent
>> resource point of view. An in-place system (even a faux setup like
>> pg_upgrade) would be
>> easier to deal with in many situations.
>
> There is something that you will not (or only under severe risk) get
> with an in-place upgrade system. The ability to downgrade back in the
> case, your QA missed a few gotchas. The application might not
> instantly eat the data, but it might start to sputter and hobble here
> and there.
>
> With the Slony system, you not only switch over to the new version.
> But you keep the old system as a slave. That means that if you
> discover 4 hours after the upgrade that the new version bails out with
> errors on a lot of queries from the application, you have the chance
> to switch back to the old version and have lost no single committed
> transaction.
>
>

What, you don't like living out on the edge? :)

Doing an upgrade via replication is a great way to do it, if you have
the resources available to do so, no argument there.

> Jan
>
>> In the end, using a replication system OR a working pg_upgrade is
>> still a pretty creaky
>> workaround. Having to do either tends to lob about 15 pounds of nails
>> into the gears when
>> trying to develop a business case about upgrading (Doesn't
>> necessarily stop it dead, but
>> does get everyone's attention...). The day when a dump/restore is not
>> necessary is
>> the day all of us are hoping for.
>>>> 1) Install PG 8.0 to an alternate directory;
>>>> 2) Start 8.0;
>>>> 3) install Slony on both instances (the 7.4 and the 8.0);
>>>> 4) make 7.4 the "master" and start replicating
>>>> 5) when 8.0 is caught up, stop 7.4 and promote it to Master
>>>> 6) turn off Slony.
>>>
>>> Slony is not an upgrade utility, and falls short in one big case ..
>>> literally .. a very large database with limited cash resources to
>>> duplicate it (as far as hardware is concerned). In small shops, or
>>> those with 'free budget', Slony is perfect ... but if you are in an
>>> organization where getting money is like pulling teeth, picking up a
>>> new server "just to do an upgrade" can prove to be difficult ...

In many cases the mere idea of doing an upgrade proves to be difficult,
before you even get to what upgrade procedure to use or whether you
need hardware or not. Add in either of those two issues and people
start to quiver and shake.

>>>
>>> ----
>>> Marc G. Fournier Hub.Org Networking Services
>>> (http://www.hub.org)
>>> Email: scrappy(at)hub(dot)org Yahoo!: yscrappy ICQ:
>>> 7615664
>>>
>>> ---------------------------(end of
>>> broadcast)---------------------------
>>> TIP 2: you can get off all lists at once with the unregister command
>>> (send "unregister YourEmailAddressHere" to
>>> majordomo(at)postgresql(dot)org)
>>>
>> --------------------
>> Andrew Rawnsley
>> President
>> The Ravensfield Digital Resource Group, Ltd.
>> (740) 587-0114
>> www.ravensfield.com
>> ---------------------------(end of
>> broadcast)---------------------------
>> TIP 8: explain analyze is your friend
>
>
> --
> #======================================================================
> #
> # It's easier to get forgiveness for being wrong than for being right.
> #
> # Let's break this rule - forgive me.
> #
> #================================================== JanWieck(at)Yahoo(dot)com
> #
>
> ---------------------------(end of
> broadcast)---------------------------
> TIP 5: Have you checked our extensive FAQ?
>
> http://www.postgresql.org/docs/faqs/FAQ.html
>
--------------------

Andrew Rawnsley
President
The Ravensfield Digital Resource Group, Ltd.
(740) 587-0114
www.ravensfield.com


From: Jeff <threshar(at)torgo(dot)978(dot)org>
To: josh(at)agliodbs(dot)com
Cc: "Marc G(dot) Fournier" <scrappy(at)postgresql(dot)org>, pgsql-hackers(at)postgresql(dot)org, Andrew Rawnsley <ronz(at)ravensfield(dot)com>
Subject: Re: version upgrade
Date: 2004-09-01 13:47:02
Message-ID: 674CF086-FC1D-11D8-AFE7-000393D1F76E@torgo.978.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers


On Aug 31, 2004, at 6:30 PM, Josh Berkus wrote:

> Huh? You can replicate onto the same server. Kicks your
> performance in
> the teeth but it works fine. Heck, I did it on my laptop as a demo.
>

Doesn't work If you have say, a 100GB db and only 50GB free space.
Not nearly enough to duplicate. But plenty of breathing room for normal
operation.

Various db's support in place upgrades. and I'm thankful I tried
Informix's out on a test db first because it simply scribbled over all
the data instead of upgrading. Support told me that can happen
sometimes. COOL HUH?

--
Jeff Trout <jeff(at)jefftrout(dot)com>
http://www.jefftrout.com/
http://www.stuarthamm.net/


From: Joe Conway <mail(at)joeconway(dot)com>
To: Jeff <threshar(at)torgo(dot)978(dot)org>
Cc: josh(at)agliodbs(dot)com, "Marc G(dot) Fournier" <scrappy(at)postgresql(dot)org>, pgsql-hackers(at)postgresql(dot)org, Andrew Rawnsley <ronz(at)ravensfield(dot)com>
Subject: Re: version upgrade
Date: 2004-09-01 14:29:05
Message-ID: 4135DCB1.2080605@joeconway.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Jeff wrote:
>
> On Aug 31, 2004, at 6:30 PM, Josh Berkus wrote:
>> Huh? You can replicate onto the same server. Kicks your
>> performance in
>> the teeth but it works fine. Heck, I did it on my laptop as a demo.
>
> Doesn't work If you have say, a 100GB db and only 50GB free space.
> Not nearly enough to duplicate. But plenty of breathing room for normal
> operation.
>
> Various db's support in place upgrades. and I'm thankful I tried
> Informix's out on a test db first because it simply scribbled over all
> the data instead of upgrading. Support told me that can happen
> sometimes. COOL HUH?

I think that's an incredibly important point, i.e., even if you want to
do an "in place" upgrade, you ought to be testing it out first on a
*full* copy of your production database. IMHO, anything less than a full
test is playing fast-and-loose with your data. This in turn implies that
you need enough space for a full replica anyway, so why not use slony?

Joe


From: Jan Wieck <JanWieck(at)Yahoo(dot)com>
To: Joe Conway <mail(at)joeconway(dot)com>
Cc: Jeff <threshar(at)torgo(dot)978(dot)org>, josh(at)agliodbs(dot)com, "Marc G(dot) Fournier" <scrappy(at)postgresql(dot)org>, pgsql-hackers(at)postgresql(dot)org, Andrew Rawnsley <ronz(at)ravensfield(dot)com>
Subject: Re: version upgrade
Date: 2004-09-01 14:41:42
Message-ID: 4135DFA6.8060009@Yahoo.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 9/1/2004 10:29 AM, Joe Conway wrote:

> Jeff wrote:
>>
>> On Aug 31, 2004, at 6:30 PM, Josh Berkus wrote:
>>> Huh? You can replicate onto the same server. Kicks your
>>> performance in
>>> the teeth but it works fine. Heck, I did it on my laptop as a demo.
>>
>> Doesn't work If you have say, a 100GB db and only 50GB free space.
>> Not nearly enough to duplicate. But plenty of breathing room for normal
>> operation.
>>
>> Various db's support in place upgrades. and I'm thankful I tried
>> Informix's out on a test db first because it simply scribbled over all
>> the data instead of upgrading. Support told me that can happen
>> sometimes. COOL HUH?
>
> I think that's an incredibly important point, i.e., even if you want to
> do an "in place" upgrade, you ought to be testing it out first on a
> *full* copy of your production database. IMHO, anything less than a full
> test is playing fast-and-loose with your data. This in turn implies that
> you need enough space for a full replica anyway, so why not use slony?

Which is another point I was about to ask. How do these people, running
those huge and horribly important databases, ever test a single
application change? Or any schema changes for that matter. Do they
really type "psql -c 'alter table ...' proddb" and believe they are
professional users because they know what they are doing?

And don't tell me "we have a backup, so we could ...". That would mean
that you can afford the downtime in the first place.

Jan

--
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#================================================== JanWieck(at)Yahoo(dot)com #


From: Hannu Krosing <hannu(at)tm(dot)ee>
To: josh(at)agliodbs(dot)com
Cc: "Marc G(dot) Fournier" <scrappy(at)postgresql(dot)org>, Andrew Rawnsley <ronz(at)ravensfield(dot)com>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: version upgrade
Date: 2004-09-01 15:34:32
Message-ID: 1094052872.2824.2.camel@fuji.krosing.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On K, 2004-09-01 at 01:30, Josh Berkus wrote:
> Marc,
>
> > Slony is not an upgrade utility, and falls short in one big case ..
> > literally .. a very large database with limited cash resources to
> > duplicate it (as far as hardware is concerned). In small shops, or those
> > with 'free budget', Slony is perfect ... but if you are in an organization
> > where getting money is like pulling teeth, picking up a new server "just
> > to do an upgrade" can prove to be difficult ...
>
> Huh? You can replicate onto the same server. Kicks your performance in
> the teeth but it works fine.

The kick could be minimized if slony did drop the pk index for the time
of initial COPY (IIRC load would be an order of magnitude faster for
LOAD+CREATE INDEX than the other way round (CREATE INDEX+LOAD))

> Heck, I did it on my laptop as a demo.

--------------
Hannu


From: "Marc G(dot) Fournier" <scrappy(at)postgresql(dot)org>
To: Jan Wieck <JanWieck(at)Yahoo(dot)com>
Cc: Joe Conway <mail(at)joeconway(dot)com>, Jeff <threshar(at)torgo(dot)978(dot)org>, josh(at)agliodbs(dot)com, "Marc G(dot) Fournier" <scrappy(at)postgresql(dot)org>, pgsql-hackers(at)postgresql(dot)org, Andrew Rawnsley <ronz(at)ravensfield(dot)com>
Subject: Re: version upgrade
Date: 2004-09-01 15:34:44
Message-ID: 20040901122908.F47186@ganymede.hub.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Wed, 1 Sep 2004, Jan Wieck wrote:

> On 9/1/2004 10:29 AM, Joe Conway wrote:
>
>> Jeff wrote:
>>>
>>> On Aug 31, 2004, at 6:30 PM, Josh Berkus wrote:
>>>> Huh? You can replicate onto the same server. Kicks your performance
>>>> in
>>>> the teeth but it works fine. Heck, I did it on my laptop as a demo.
>>>
>>> Doesn't work If you have say, a 100GB db and only 50GB free space.
>>> Not nearly enough to duplicate. But plenty of breathing room for normal
>>> operation.
>>>
>>> Various db's support in place upgrades. and I'm thankful I tried
>>> Informix's out on a test db first because it simply scribbled over all the
>>> data instead of upgrading. Support told me that can happen sometimes.
>>> COOL HUH?
>>
>> I think that's an incredibly important point, i.e., even if you want to do
>> an "in place" upgrade, you ought to be testing it out first on a *full*
>> copy of your production database. IMHO, anything less than a full test is
>> playing fast-and-loose with your data. This in turn implies that you need
>> enough space for a full replica anyway, so why not use slony?
>
> Which is another point I was about to ask. How do these people, running
> those huge and horribly important databases, ever test a single
> application change? Or any schema changes for that matter. Do they
> really type "psql -c 'alter table ...' proddb" and believe they are
> professional users because they know what they are doing?

You are assuming that they ever make changes ... :)

Please note that nobody is slamming Slony as *an* upgrade option ... what
we are slamming is that everyone seems to be touting it as *the* upgrade
option ... they seem to be ignoring the fact that nobody everyone *has*
the resources required to use a replication (any replication) option as an
upgrade option ...

God, how many ppl are running applications out there that have never had
an upgrade in 5 years, because the company that first created it is no
longer in business ... yet the application *still* does exactly what the
business wants/needs?

Now, granted, that 5 year old application would probably break if its
database were upgraded ... the point I'm trying to make is that the
data format, and application, would be the static component ... the
backend would be the only thing that changes ... *and* ... there have been
several 'new releases' of PostgreSQL that have required *zero* changes at
the application level in order to work, just requiring a dump/reload due
to changes in the database ...

----
Marc G. Fournier Hub.Org Networking Services (http://www.hub.org)
Email: scrappy(at)hub(dot)org Yahoo!: yscrappy ICQ: 7615664


From: Josh Berkus <josh(at)agliodbs(dot)com>
To: Jeff <threshar(at)torgo(dot)978(dot)org>
Cc: "Marc G(dot) Fournier" <scrappy(at)postgresql(dot)org>, pgsql-hackers(at)postgresql(dot)org, Andrew Rawnsley <ronz(at)ravensfield(dot)com>
Subject: Re: version upgrade
Date: 2004-09-01 16:19:52
Message-ID: 200409010919.52245.josh@agliodbs.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Folks,

> Doesn't work If you have say, a 100GB db and only 50GB free space.
> Not nearly enough to duplicate. But plenty of breathing room for normal
> operation.

From my perspective, anyone who is running a 100GB, can't-be-down-for-a-day
database and does not have more than 100GB free and/or a hot swap server has
some *serious* priority problems. I really don't think it's the job of the
Project to make up for people's bad server room planning. Even Microsoft,
paragon of dumb-user-software, requires the SQL Server upgrade process to
make a complete duplicate copy of the database.

If somebody thinks that they can code an in-place upgrade not involving
replication and switchover, then great! Go for it. But the Slony method
would be much, much easier to code and could even be done around the time of
the 8.0 release, which I don't think you can say for anything more hard-core.

--
Josh Berkus
Aglio Database Solutions
San Francisco


From: David Fetter <david(at)fetter(dot)org>
To: PG Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: version upgrade
Date: 2004-09-01 16:38:50
Message-ID: 20040901163850.GC32358@fetter.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Wed, Sep 01, 2004 at 09:47:02AM -0400, Jeff wrote:
>
> On Aug 31, 2004, at 6:30 PM, Josh Berkus wrote:
>
> >Huh? You can replicate onto the same server. Kicks your
> >performance in the teeth but it works fine. Heck, I did it on my
> >laptop as a demo.
>
> Doesn't work If you have say, a 100GB db and only 50GB free space.
> Not nearly enough to duplicate. But plenty of breathing room for
> normal operation.

There is a technical term for outfits that have uptime requirements
but can't (or won't) allocate resources for testing upgrades. It's a
short Saxon-derived word equivalent to "known biblically."

Cheers,
D
--
David Fetter david(at)fetter(dot)org http://fetter.org/
phone: +1 510 893 6100 mobile: +1 415 235 3778

Remember to vote!


From: Jeff <threshar(at)torgo(dot)978(dot)org>
To: Josh Berkus <josh(at)agliodbs(dot)com>
Cc: "Marc G(dot) Fournier" <scrappy(at)postgresql(dot)org>, pgsql-hackers(at)postgresql(dot)org, Andrew Rawnsley <ronz(at)ravensfield(dot)com>
Subject: Re: version upgrade
Date: 2004-09-01 17:04:18
Message-ID: F60B7771-FC38-11D8-AFE7-000393D1F76E@torgo.978.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers


On Sep 1, 2004, at 12:19 PM, Josh Berkus wrote:

> From my perspective, anyone who is running a 100GB,
> can't-be-down-for-a-day
> database and does not have more than 100GB free and/or a hot swap
> server has
> some *serious* priority problems.

Well, 100GB maybe excessive for this example. but I'm sure there are
plenty of running-on-a-shoe-string shops that don't have DB x 2 space
avail. Then again, those places are very likely the ones who will not
be upgrading.

but this isn't a problem specific to PG.. all db's suffer from it.. and
slony so far seems to provide the easiest, safest path for a PG
upgrade... in my case the problem is I do have another server with
plenty of room, but it doesn't have much CPU or RAM and cannot handle
the volume of live traffic the master gets. So I have no choice but to
plead my case and ask for some downtime. oh well.

--
Jeff Trout <jeff(at)jefftrout(dot)com>
http://www.jefftrout.com/
http://www.stuarthamm.net/


From: Gaetano Mendola <mendola(at)bigfoot(dot)com>
To: Jan Wieck <JanWieck(at)Yahoo(dot)com>
Subject: Re: version upgrade
Date: 2004-09-02 01:02:00
Message-ID: 41367108.4070701@bigfoot.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Jan Wieck wrote:

> Which is another point I was about to ask. How do these people, running
> those huge and horribly important databases, ever test a single
> application change? Or any schema changes for that matter. Do they
> really type "psql -c 'alter table ...' proddb" and believe they are
> professional users because they know what they are doing?

I do alter table, but of course before to do it, I run my regression test
on a database with almost no data inside. Each stored procedure is tested
in order to execute each execution path.
In 3 years working in this way I had no a singol failure after an alter
schema operation.

Regards
Gaetano Mendola


From: Jan Wieck <JanWieck(at)Yahoo(dot)com>
To: Gaetano Mendola <mendola(at)bigfoot(dot)com>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: version upgrade
Date: 2004-09-02 20:16:46
Message-ID: 41377FAE.8030701@Yahoo.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 9/1/2004 9:02 PM, Gaetano Mendola wrote:

> Jan Wieck wrote:
>
>
>> Which is another point I was about to ask. How do these people, running
>> those huge and horribly important databases, ever test a single
>> application change? Or any schema changes for that matter. Do they
>> really type "psql -c 'alter table ...' proddb" and believe they are
>> professional users because they know what they are doing?
>
> I do alter table, but of course before to do it, I run my regression test
> on a database with almost no data inside. Each stored procedure is tested
> in order to execute each execution path.
> In 3 years working in this way I had no a singol failure after an alter
> schema operation.

If it is possible to define a representative but smaller dataset for
test purposes, that is certainly doable. Some systems are just too
complex to do this. SAP for example recommends a 4 stage deployment
scenario in case you do your own application development in R/3 systems.
You would have one or more development systems, that deliver their
changes into test systems with small and not necessarily representative
data. If all tests there succeed, the software is transported into the
integration test system, which is basically a copy of the production
system with full data. Only if that transport and the following tests
succeed, you transport exactly the same set of programs and catalog
changes into the production system. Otherwise you reset the integration
test system back to be a copy of the production system.

There are a lot of possible levels between playing russian roulette with
your data and being paranoid. If a corrupted database can cause the
company to go under, some prefer paranoid.

Jan

--
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#================================================== JanWieck(at)Yahoo(dot)com #


From: Gaetano Mendola <mendola(at)bigfoot(dot)com>
To: Jan Wieck <JanWieck(at)Yahoo(dot)com>
Subject: Re: version upgrade
Date: 2004-09-02 20:47:16
Message-ID: 413786D4.40709@bigfoot.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Jan Wieck wrote:
> On 9/1/2004 9:02 PM, Gaetano Mendola wrote:
>
>> Jan Wieck wrote:
>>
>>
>>> Which is another point I was about to ask. How do these people,
>>> running those huge and horribly important databases, ever test a
>>> single application change? Or any schema changes for that matter. Do
>>> they really type "psql -c 'alter table ...' proddb" and believe they
>>> are professional users because they know what they are doing?
>>
>>
>> I do alter table, but of course before to do it, I run my regression test
>> on a database with almost no data inside. Each stored procedure is tested
>> in order to execute each execution path.
>> In 3 years working in this way I had no a singol failure after an alter
>> schema operation.
>
>
> If it is possible to define a representative but smaller dataset for
> test purposes, that is certainly doable. Some systems are just too
> complex to do this. SAP for example recommends a 4 stage deployment
> scenario in case you do your own application development in R/3 systems.
> You would have one or more development systems, that deliver their
> changes into test systems with small and not necessarily representative
> data. If all tests there succeed, the software is transported into the
> integration test system, which is basically a copy of the production
> system with full data. Only if that transport and the following tests
> succeed, you transport exactly the same set of programs and catalog
> changes into the production system. Otherwise you reset the integration
> test system back to be a copy of the production system.
>
> There are a lot of possible levels between playing russian roulette with
> your data and being paranoid. If a corrupted database can cause the
> company to go under, some prefer paranoid.

Paranoid means also don't trust in new hw without have test it for a while.
How ever if I leave unchanged all interfaces and all my test cases are
continuously passing ( 3200 different test in my case ) all the night long
I'm quite sure that the schema change will not hurt nothing. However I have to
say that add come column, with default value and a check on it is no to
doable with very bigtables. Fortunately with the 8.0 you can do these tasks
in one shot.

Regards
Gaetano Mendola


From: Rod Taylor <pg(at)rbt(dot)ca>
To: Gaetano Mendola <mendola(at)bigfoot(dot)com>
Cc: PostgreSQL Development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: version upgrade
Date: 2004-09-03 05:29:22
Message-ID: 1094189361.25996.394.camel@jester
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

> I'm quite sure that the schema change will not hurt nothing. However I have to
> say that add come column, with default value and a check on it is no to
> doable with very bigtables. Fortunately with the 8.0 you can do these tasks
> in one shot.

I've got a few 30GB tables anxiously awaiting that single alter command
that will alter in all the stuff that should have been added a long time
ago.