Re: 9.3: load path to mitigate load penalty for checksums

From: Greg Stark <stark(at)mit(dot)edu>
To: Merlin Moncure <mmoncure(at)gmail(dot)com>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Robert Haas <robertmhaas(at)gmail(dot)com>, Jeff Davis <pgsql(at)j-davis(dot)com>, Noah Misch <noah(at)leadboat(dot)com>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: 9.3: load path to mitigate load penalty for checksums
Date: 2012-06-12 16:50:17
Message-ID: CAM-w4HNXYZq6xNFu9O9bqNfLLX0wk4qwnqZnxX57MsHVDK+WLw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Tue, Jun 12, 2012 at 5:41 PM, Merlin Moncure <mmoncure(at)gmail(dot)com> wrote:
> Also it's not safe to assume that insertion heavy clients
> can be migrated to COPY.  For example, JDBC bulk loading AFAIK does
> not use COPY and even if it did wouldn't be able to decorate the
> command for a long time for most production workloads

A non-mvcc table would only be useful if you're loading the data all
at once and don't plan to load any more. So that wouldn't even be
attempting to cover all "insertion heavy clients". Just specific use
cases.

I had in mind to implement something like this for archived data such
as old partitions in partitioned table. I think data loaded in a copy
that you know is never going to be modified would be a reasonable
other case for the same codepath.

Note that this particular use case is covered by FDW as well. If the
source data is in a dense enough format you could save even the load
step by reading directly from the source. I don't think this
eliminates the idea of having a denser read-only format though.
Postgres has lots of other features like indexing and different data
representations that might be useful and would be missing from, say, a
csv source file.

--
greg

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2012-06-12 17:05:18 Re: /proc/self/oom_adj is deprecated in newer Linux kernels
Previous Message Robert Haas 2012-06-12 16:44:38 Re: /proc/self/oom_adj is deprecated in newer Linux kernels