Re: Merge a sharded master into a single read-only slave

On Thu, Jun 5, 2014 at 12:55 PM, Francisco Olarte <folarte(at)peoplecall(dot)com>
wrote:

> Hi Sébastien:
>
> On Thu, Jun 5, 2014 at 5:41 PM, Sébastien Lorion
> <sl(at)thestrangefactory(dot)com> wrote:
>
> > .... Correct me if I am wrong, but will it not also suffer the same
> > limitation as any statement based replication, namely that the "merged"
> > slave will have to sustain the same write load as all shards combined ?
>
> I cannot tell you the exact mimeo behaviour, but if you incremental
> replication using an id/timestamp by >pulling< changes from the
> masters, you will normally batch them and insert all the changes to
> the slaves in a single transaction, which leads to less load as many
> times your limit is in transaction rate, not record rate. (i.e., every
> 5 minutes you query for all the tuples changed, and insert/update them
> all in one go ) ( Also, if tuples are updated many times between
> sweeps the slave will get only one )
>
> Francisco Olarte.
>

​You are right, requesting changes at fixed time intervals would certainly
help reduce the load. I will have to test and see if a good balance can be
achieved between not having stale data for too long and keeping up with
writes.

Sébastien