Re: Parallell Optimizer

On 06/13/2013 02:22 AM, Tatsuo Ishii wrote:
>> On Jun 12, 2013 2:02 AM, "Tatsuo Ishii" <ishii(at)postgresql(dot)org> wrote:
>>> No, I'm not talking about conflict resolution.
>>>
>>> From http://www.cs.cmu.edu/~natassa/courses/15-823/F02/papers/replication.pdf:
>>> ----------------------------------------------
>>> Eager or Lazy Replication?
>>> Eager replication:
>>> keep all replicas synchronized by updating all
>>> replicas in a single transaction
>>>
>>> Lazy replication:
>>> asynchronously propagate replica updates to
>>> other nodes after replicating transaction commits
>>> ----------------------------------------------
>>>
>>> Parallel query execution needs to assume that each node synchronized
>>> in a commit, otherwise the summary of each query result executed on
>>> each node is meaningless.
>> As far as I can see the lazy-eager terminology is based on a
>> multi-master configuration and doesn't really apply for PostgreSQL
>> streaming replication.
>>
>> Parallel query execution doesn't require commits to synchronize all
>> nodes. Parallel execution needs consistent snapshots across all nodes.
>> In effect this means that nodes need to agree on commit ordering,
>> either total order or a partial order that accounts for causality.
>> Most applications also want the guarantee that once they receive
>> commit confirmation, next snapshot they take will consider their
>> transaction as committed.
>>
>> Coincidentally getting cluster wide consistent snapshots and delaying
>> until some specific point in commit ordering is almost trivial to
>> solve with Commit Sequence Number based snapshot scheme that I
>> proposed.
> Can you elaborate more on this? Suppose streaming replication primary
> commits xid = X at time Y. Later on a standy receives WAL including tx
> X and commit it at time Y + 3 seconds. How can a parallel query
> execution (which uses snapshot including X) on the standby be delayed
> until Y + 3 seconds?
I do not think that CSN's change anything basic here, as CSN's
are still local to each node.

What you need is ability to ask for each node to wait until XID
is replicated to it.

Unless you have some central XID/Snapshot source, there is
no global absolute XID order. That is there may be a transaction
which is committed on node A and not yet on node B and at the
same time a transaction which is committed on node B and not
yet on node A.

So to get consistent snapshot "after X is committed" in multimaster
you need some coordination and possibly compromises w.r.t. "single
point in time"

Time in multimaster replication is relativistic, that is the order
of events may depend on where the observer is :)

--
Hannu Krosing
PostgreSQL Consultant
Performance, Scalability and High Availability
2ndQuadrant Nordic OÜ