Re: Further pg_upgrade analysis for many tables

On Mon, Nov 12, 2012 at 03:59:27PM -0500, Bruce Momjian wrote:
> > The second approach would be to simply try to copy the fsm, vm, and
> > extent files, and ignore any ENOEXIST errors. This allows code
> > simplification. The downside is that it doesn't pull all files with
> > matching prefixes --- it requires pg_upgrade to _know_ what suffixes
> > might exist in that directory. Second, it assumes there can be no
> > number gaps in the file extent numbering (is that safe?).
> >
> > I need recommendations on which direction to persue; this would only be
> > for 9.3.
>
> I went with the second idea, patch attached. Here are the times:
>
> ---------- 9.2 ---------- ------------ 9.3 --------
> -- normal -- -- bin-up -- -- normal -- -- bin-up -- pg_upgrade
> dump rest dump rest dump rest dump rest git patch
> 1 0.12 0.06 0.12 0.06 0.11 0.07 0.11 0.07 11.11 11.02
> 1000 7.22 2.40 4.74 2.78 2.20 2.43 4.04 2.86 19.60 19.25
> 2000 5.67 5.10 8.82 5.57 4.50 4.97 8.07 5.69 30.55 26.67
> 4000 13.34 11.13 25.16 12.52 8.95 11.24 16.75 12.16 60.70 52.31
> 8000 29.12 25.98 59.60 28.08 16.68 24.02 30.63 27.08 123.05 102.78
> 16000 87.36 53.16 189.38 62.72 31.38 55.37 61.55 62.66 365.71 286.00
>
> You can see a significant speedup with those loops removed. The 16k
> case is improved, but still not linear. The 16k dump/restore scale
> looks fine, so it must be something in pg_upgrade, or in the kernel.

It is possible that the poor 16k pg_upgrade value is caused by the poor
9.2 binary-upgrade number (189.38). Perhaps I need to hack up
pg_upgrade to allow a 9.3 to 9.3 upgrade to test this.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +