Re: Proposal: Commit timestamp

On 2/7/2007 2:15 PM, Richard Troy wrote:
>> Jan Wieck wrote:
>> > Are we still discussing if the Postgres backend may provide support for
>> > a commit timestamp, that follows the rules for Lamport timestamps in a
>> > multi-node cluster?
>
> ...I thought you said in this thread that you haven't and weren't going to
> work on any kind of logical proof of it's correctness, saw no value in
> prototyping your way to a clear (convincing) argument, and were
> withdrawing the proposal [...]

I said I don't have any such documents. I was asked to continue this
discussion in order to find people willing to help discover potential
problems. I am prepared to continue this development isolated, although
I wouldn't like to.

The PostgreSQL developers community used to be good at throwing out
ideas, brainstorming about the possibilities, adding more to them and
coming up with very unique and flexible solutions. I am a little
disappointed that much of that got lost over the years and please
forgive me if I sound a little grumpy over that. The statement to
withdraw the proposal was certainly premature - consider it not
withdrawn at this time. However, comparing what used to be our process
to what I see today, I must say that something like TOAST would never
have happened. It was the result of a global brainstorming, that I
simply translated into C code. Many details and features of the
implementation are purely mine, but the really big sparks, that got it
to what it is, I'm not claiming for myself. Most importantly, "give me
proof of concept before we can talk about changing backend code" was not
part of the process at all. We were pretty eager to change things back
then, when we needed to get better in almost every way possible ... are
we so good at replication that we need to be conservative in that
respect now? We are certainly good at some things and have to be
conservative with respect to them, but replication in my not so very
humble opinion isn't one of them.

I do understand that we have a codebase used in production these days.
And because of that we have to maintain code and syntax stability to a
degree, we didn't have back in the glory days of introducing EXCEPT and
INTERCEPT (who's first incarnation was committed to the code base while
completely destroying my entire work of fixing the rewriter). Maybe we
need to introduce something entirely different, like the concept of an
experimental feature. Something that we add to the code but that is
explicitly flagged as not final, not stable, not guaranteed to stay or
work in this or any other form. This requires that the feature has very
limited interference with other parts of the system, like (or especially
like) the query parser. If it turns out to be a problem in x.y.0, it
will be backed out and gone in x.y.1. Or in a different way, like we
create an experimental CVS branch off of every major release. That way,
developers can easier share experimental code and if things settle
there, they will be considered to be adopted into HEAD.

> Like Markus, I would like to see the various replication efforts merged as
> best they can be because even if the majority of users don't use a little
> bit of everything, surely the more interesting cases would like to and the
> entire community is better served if the various "solutions" are in
> harmony.

No doubt about that and I was the one organizing the Afilias sponsored
meeting in Toronto back then, where my reversed Postgres-R idea was
taken apart because it won't scale due to the gigantic amount of
synchronized group communication it would require. Again, it might be
that experimental features will cause more of the efforts to converge by
using the same base as a compromise instead of having each and every
support feature being designed completely independent.

I still have a hard time understanding why someone would object to
adding a feature, however useless it might seem to them, as long as it
doesn't cost them anything. Admitted, any feature causes maintenance
costs on the side of the PostgreSQL development community (mainly those,
who actually contribute and maintain the code - fortunately that is a
finite number - everyone please ask themselves if they are part of
that). But aside from that, would anyone, who is questioning the commit
timestamp as I proposed it, likewise vehemently object to yet another
procedural language, or adding another log tuning switch? I don't think
so. As long as it doesn't cost you unless you turn it on, why would you
even care if it serves my purpose or not? The thing that kicked off this
emotional spin was that multimaster replication is what so many people
want, but nobody has a universal solution for. Everyone wants to see
"their" problem solved "as well", or the solution isn't good. Tell you
what, I can live with my problem solved even if it doesn't solve yours.
Can you tell me what I have to modify in order to solve your problem as
well, or are you asking me to not implement anything unless "I" find a
way to solve everyones problems in one, big, universal solution?

Jan

--
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#================================================== JanWieck(at)Yahoo(dot)com #