Re: Merge a sharded master into a single read-only slave

From: Francisco Olarte <folarte(at)peoplecall(dot)com>
To: Sébastien Lorion <sl(at)thestrangefactory(dot)com>
Cc: Keith Fiske <keith(at)omniti(dot)com>, Kevin Goess <kgoess(at)bepress(dot)com>, PostgreSQL <pgsql-general(at)postgresql(dot)org>
Subject: Re: Merge a sharded master into a single read-only slave
Date: 2014-06-05 16:55:41
Message-ID: CA+bJJbxkJDMh+vKofFvQ07MLJLtDySiXqDTVuzVLv=Pi_BQoSQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Hi Sébastien:

On Thu, Jun 5, 2014 at 5:41 PM, Sébastien Lorion
<sl(at)thestrangefactory(dot)com> wrote:

> .... Correct me if I am wrong, but will it not also suffer the same
> limitation as any statement based replication, namely that the "merged"
> slave will have to sustain the same write load as all shards combined ?

I cannot tell you the exact mimeo behaviour, but if you incremental
replication using an id/timestamp by >pulling< changes from the
masters, you will normally batch them and insert all the changes to
the slaves in a single transaction, which leads to less load as many
times your limit is in transaction rate, not record rate. (i.e., every
5 minutes you query for all the tuples changed, and insert/update them
all in one go ) ( Also, if tuples are updated many times between
sweeps the slave will get only one )

Francisco Olarte.

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Dev Kumkar 2014-06-05 17:30:05 Re: Heartbleed Impact
Previous Message Carlos Carcamo 2014-06-05 16:45:57 Re: help with a procedure