pg_upgrade -j broken on Windows

Lists: pgsql-hackers
From: Bruce Momjian <bruce(at)momjian(dot)us>
To: PostgreSQL-development <pgsql-hackers(at)postgreSQL(dot)org>
Subject: pg_upgrade -j broken on Windows
Date: 2013-07-25 14:57:28
Message-ID: 20130725145728.GG16202@momjian.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Everyone should be aware that the 9.3 pg_upgrade -j/--jobs option on
Windows is currently broken, and hopefully will be fixed by the next
beta.

Someone at PGDay UK told me they were getting pg_upgrade -j crashes on
Windows. Andrew Dunstan was able to reproduce the crash, and that has
been fixed, but there is still a race condition that I am trying to
diagnose.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +


From: Bruce Momjian <bruce(at)momjian(dot)us>
To: PostgreSQL-development <pgsql-hackers(at)postgreSQL(dot)org>
Cc: Andrew Dunstan <andrew(at)dunslane(dot)net>
Subject: Re: pg_upgrade -j broken on Windows
Date: 2013-07-26 17:27:34
Message-ID: 20130726172734.GA27611@momjian.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Thu, Jul 25, 2013 at 10:57:28AM -0400, Bruce Momjian wrote:
> Everyone should be aware that the 9.3 pg_upgrade -j/--jobs option on
> Windows is currently broken, and hopefully will be fixed by the next
> beta.
>
> Someone at PGDay UK told me they were getting pg_upgrade -j crashes on
> Windows. Andrew Dunstan was able to reproduce the crash, and that has
> been fixed, but there is still a race condition that I am trying to
> diagnose.

After three days of testing and creating a Windows MinGW build
environment (with Andrew's help), I have come up with the attached patch
which fixes the pg_upgrade -j race condition on Windows. In summary,
creating a file with fopen() from a non-primary thread and then calling
system() soon after can result in a file-in-use error. The solution was
to issue the fopen() after system() in such cases.

This would be applied to head and 9.3.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +


From: Bruce Momjian <bruce(at)momjian(dot)us>
To: PostgreSQL-development <pgsql-hackers(at)postgreSQL(dot)org>
Cc: Andrew Dunstan <andrew(at)dunslane(dot)net>
Subject: Re: pg_upgrade -j broken on Windows
Date: 2013-07-26 17:31:45
Message-ID: 20130726173145.GA16219@momjian.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Fri, Jul 26, 2013 at 01:27:34PM -0400, Bruce Momjian wrote:
> On Thu, Jul 25, 2013 at 10:57:28AM -0400, Bruce Momjian wrote:
> > Everyone should be aware that the 9.3 pg_upgrade -j/--jobs option on
> > Windows is currently broken, and hopefully will be fixed by the next
> > beta.
> >
> > Someone at PGDay UK told me they were getting pg_upgrade -j crashes on
> > Windows. Andrew Dunstan was able to reproduce the crash, and that has
> > been fixed, but there is still a race condition that I am trying to
> > diagnose.
>
> After three days of testing and creating a Windows MinGW build
> environment (with Andrew's help), I have come up with the attached patch
> which fixes the pg_upgrade -j race condition on Windows. In summary,
> creating a file with fopen() from a non-primary thread and then calling
> system() soon after can result in a file-in-use error. The solution was
> to issue the fopen() after system() in such cases.
>
> This would be applied to head and 9.3.

Sorry, patch attached.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

Attachment Content-Type Size
pg_upgrade.diff text/x-diff 4.0 KB

From: Bruce Momjian <bruce(at)momjian(dot)us>
To: PostgreSQL-development <pgsql-hackers(at)postgreSQL(dot)org>
Cc: Andrew Dunstan <andrew(at)dunslane(dot)net>
Subject: Re: pg_upgrade -j broken on Windows
Date: 2013-07-27 19:01:14
Message-ID: 20130727190114.GB3106@momjian.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Fri, Jul 26, 2013 at 01:31:45PM -0400, Bruce Momjian wrote:
> On Fri, Jul 26, 2013 at 01:27:34PM -0400, Bruce Momjian wrote:
> > On Thu, Jul 25, 2013 at 10:57:28AM -0400, Bruce Momjian wrote:
> > > Everyone should be aware that the 9.3 pg_upgrade -j/--jobs option on
> > > Windows is currently broken, and hopefully will be fixed by the next
> > > beta.
> > >
> > > Someone at PGDay UK told me they were getting pg_upgrade -j crashes on
> > > Windows. Andrew Dunstan was able to reproduce the crash, and that has
> > > been fixed, but there is still a race condition that I am trying to
> > > diagnose.
> >
> > After three days of testing and creating a Windows MinGW build
> > environment (with Andrew's help), I have come up with the attached patch
> > which fixes the pg_upgrade -j race condition on Windows. In summary,
> > creating a file with fopen() from a non-primary thread and then calling
> > system() soon after can result in a file-in-use error. The solution was
> > to issue the fopen() after system() in such cases.
> >
> > This would be applied to head and 9.3.
>
> Sorry, patch attached.

Applied, and backpatched to 9.3.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +