Re: removing old ports and architectures

Hi,

As discussed in 20130926225545(dot)GB26663(at)awork2(dot)anarazel(dot)de and on quite
some other occasions there's quite some scalability improvements we
could make if we had cross platform support for atomic
operations. Providing that is a fair bit of work for every
architecture/compiler/OS, so I think it's a reasonable thing to remove
unused things first.
Alternatively we can maintain parallel code paths for atomic ops
supporting platforms and for those without, but that seems like a bad
idea from a complexity and testing perspective given that all even
remotely current platforms should support them.

I think we should remove support the following ports:
- IRIX
- UnixWare
- Tru64

Neither of those are relevant.

I think we should remove support for the following architectures:
- VAX
- univel (s_lock support remaining)
- sinix (s_lock support remaining)
- sun3 (I think it's just s_lock support remaining)
- natsemi 32k
- superH
- ALPHA (big pain in the ass to get right, nobody uses it anymore)
- m86k (doesn't have a useable CAS on later iterations like coldfire)
- M32R (no userspace CAS afaics)
- mips for anything but gcc > 4.4, using gcc's atomics support
- s390 for anything but gcc > 4.4, using gcc's atomics support
- 32bit/<v9 sparc (doesn't have proper atomics, old)

Possibly:
- all mips
- PA-RISC. I think Tom was the remaining user there? Maybe just !gcc.

Any arguments against?

Last round of discussion of removing dead ports:
1335292179(dot)13481(dot)4(dot)camel(at)vanquo(dot)pezone(dot)net

Discusses state of of spinlocks and barriers on various platforms:
20130920151110(dot)GA8508(at)awork2(dot)anarazel(dot)de

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On Sat, Oct 12, 2013 at 5:46 PM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> I think we should remove support the following ports:
> - IRIX
> - UnixWare
> - Tru64
>
> Neither of those are relevant.

Seems reasonable.

> I think we should remove support for the following architectures:
> - VAX

Agreed.

> - univel (s_lock support remaining)
> - sinix (s_lock support remaining)
> - sun3 (I think it's just s_lock support remaining)
> - natsemi 32k

I don't know enough about these, which doesn't bode well for them.

> - superH

SuperH isn't dead, but it is only used for very small embedded
systems, I think (mostly microcontrollers). So maybe.

> - ALPHA (big pain in the ass to get right, nobody uses it anymore)

Yes, for many years now ALPHA has only been useful as a way of
illustrating how bad it's possible for CPU memory operation reordering
considerations to get. So I quite agree.

> - m86k (doesn't have a useable CAS on later iterations like coldfire)

It does seem like Motorola 68k is vanishingly close to dead.

> - M32R (no userspace CAS afaics)
> - mips for anything but gcc > 4.4, using gcc's atomics support
> - s390 for anything but gcc > 4.4, using gcc's atomics support
> - 32bit/<v9 sparc (doesn't have proper atomics, old)

Not so sure about these.

> Possibly:
> - all mips
> - PA-RISC. I think Tom was the remaining user there? Maybe just !gcc.

I think we should think hard about removing support for MIPS. A lot of
Chinese chip manufacturers have licensed MIPS technology in just the
last couple of years, so there is plenty of it out there; I'd be
slightly concerned that the proposed restrictions on MIPS would be
onerous. Much of this is the kind of hardware that a person might
plausibly want to run Postgres on.

--
Peter Geoghegan

Andres Freund <andres(at)2ndquadrant(dot)com> writes:
> I think we should remove support for the following architectures:
> - superH

This one was contributed just a year or two ago, if memory serves,
which suggests that somebody out there cares about it. OTOH, if
they still care, we could insist they provide whatever atomic ops
we want to depend on.

> - PA-RISC. I think Tom was the remaining user there? Maybe just !gcc.

Until pretty recently, there was a PA-RISC machine (not mine) in the
buildfarm. I don't see it in the list today though. In any case,
HP's compiler has always been a PITA, so no objection to requiring gcc
for this platform.

regards, tom lane

On 2013-10-12 18:35:00 -0700, Peter Geoghegan wrote:
> Not so sure about these.
> > - M32R (no userspace CAS afaics)

I don't think M32R will hurt us/anybody much.

> > - 32bit/<v9 sparc (doesn't have proper atomics, old)

Sparc v9 is from 1995, so I think not supporting it anymore is
fair. It's afaics not supported by sun studio's intrics either.

> > - mips for anything but gcc > 4.4, using gcc's atomics support

The reason I'd like to de-support mips for older GCCs is that writing
assembler for them isn't trivial enough to do it blindly and I've had -
for other stuff - difficulties getting my hand on them. GCC provides all
the atomics for mips since 4.2, so we can just rely on that.

> > - s390 for anything but gcc > 4.4, using gcc's atomics support

Easier to write assembler, but still untestable and even harder to get
access on.
I think 4.2 should be fine as well.

> I think we should think hard about removing support for MIPS. A lot of
> Chinese chip manufacturers have licensed MIPS technology in just the
> last couple of years, so there is plenty of it out there; I'd be
> slightly concerned that the proposed restrictions on MIPS would be
> onerous. Much of this is the kind of hardware that a person might
> plausibly want to run Postgres on.

That's a fair point. But all of them will use gcc, right? I've
previously thought we'd need 4.4 because there's an incompatibility
between 4.3 and 4.4 but I think it won't touch us, so 4.2 which added
atomics for mips seems fine. Given there's no buildfarm animal and
there's lots of variety out there that seems like a fair amount of
support.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On 2013-10-13 11:34:42 +0200, Tom Lane wrote:
> Andres Freund <andres(at)2ndquadrant(dot)com> writes:
> > I think we should remove support for the following architectures:
> > - superH
>
> This one was contributed just a year or two ago, if memory serves,
> which suggests that somebody out there cares about it. OTOH, if
> they still care, we could insist they provide whatever atomic ops
> we want to depend on.

It was 2009 - aac3c301b5e8178841e5749b3657c1a639ba06c1 . I haven't yet
verified if gcc's atomics support is acceptable for the platform
(checkout is running for the last 3h...).
If it's supported, falling back to that seems easy enough.

> > - PA-RISC. I think Tom was the remaining user there? Maybe just !gcc.
>
> Until pretty recently, there was a PA-RISC machine (not mine) in the
> buildfarm. I don't see it in the list today though. In any case,
> HP's compiler has always been a PITA, so no objection to requiring gcc
> for this platform.

The reason I'd like to generally get rid of PA-RISC is that it's the
only platform that doesn't seem to have any form of compare and
swap. GCC should provide fallbacks - with some warnings - using
spinlocks instead but I am afraid people will start doing things like
atomic operations in signal handlers that won't be noticed and will be a
PITA to debug.

Having read a fair amount of assembler looking at this I have to say,
anybody thinking LL/SC architectures are neat...

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On 2013-10-13 14:08:59 +0200, Andres Freund wrote:
> On 2013-10-13 11:34:42 +0200, Tom Lane wrote:
> > Andres Freund <andres(at)2ndquadrant(dot)com> writes:
> > > I think we should remove support for the following architectures:
> > > - superH
> >
> > This one was contributed just a year or two ago, if memory serves,
> > which suggests that somebody out there cares about it. OTOH, if
> > they still care, we could insist they provide whatever atomic ops
> > we want to depend on.
>
> It was 2009 - aac3c301b5e8178841e5749b3657c1a639ba06c1 . I haven't yet
> verified if gcc's atomics support is acceptable for the platform
> (checkout is running for the last 3h...).
> If it's supported, falling back to that seems easy enough.

Ok, so the git checkout has finally
finished. http://gcc.gnu.org/git/?p=gcc.git;a=blob;f=gcc/config/sh/sync.md;hb=HEAD
makes for an interesting read...

In short: On newer sh variants (SH4) true hardware support is used, on
older ones the linux provides helpers. The variant has to be selected by
gcc parameters unfortunately, but I'd guess that's ok (e.g. SH4
-matomic-model=hard-llcs ).

> > > - PA-RISC. I think Tom was the remaining user there? Maybe just !gcc.
> >
> > Until pretty recently, there was a PA-RISC machine (not mine) in the
> > buildfarm. I don't see it in the list today though. In any case,
> > HP's compiler has always been a PITA, so no objection to requiring gcc
> > for this platform.
>
> The reason I'd like to generally get rid of PA-RISC is that it's the
> only platform that doesn't seem to have any form of compare and
> swap. GCC should provide fallbacks - with some warnings - using
> spinlocks instead

Hm. I had misunderstood the documentation around this a bit - source
code still is the only reasonable documentation :(.
PA-RISC is only supported on linux using kernel helpers (which switch
off interrupts for the duration of the operation...).

> but I am afraid people will start doing things like
> atomic operations in signal handlers that won't be noticed and will be a
> PITA to debug.

That's already a possibility with our memory barriers :(.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

Andres Freund <andres(at)2ndquadrant(dot)com> writes:
> That's a fair point. But all of them will use gcc, right? I've
> previously thought we'd need 4.4 because there's an incompatibility
> between 4.3 and 4.4 but I think it won't touch us, so 4.2 which added
> atomics for mips seems fine. Given there's no buildfarm animal and
> there's lots of variety out there that seems like a fair amount of
> support.

FWIW, I wouldn't have the slightest bit of difficulty with setting a
project policy that if you want some platform to be supported, you
must provide a buildfarm animal running on it.

More to the point for this specific case, it seems like our process
ought to be
(1) select a preferably-small set of gcc atomic intrinsics that we
want to use.
(2) provide macro wrappers for these so that substituting other
implementations isn't too tough.
(3) for any platform where people don't want to use recent gcc,
insist they provide substitute implementations of the macros.

I don't think the core project has to be responsible for implementing
(3), except maybe on very-mainstream arches such as x86. We can
adopt a self-help attitude for everything else.

But ... having said all that, it would be nice to see some proof of
significant performance benefits that we're going to get from kicking
those non-mainstream arches to the curb. I'm not enamored of removing
real functionality now for vague promises of benefits later.

regards, tom lane

On 2013-10-13 16:56:12 +0200, Tom Lane wrote:
> Andres Freund <andres(at)2ndquadrant(dot)com> writes:
> > That's a fair point. But all of them will use gcc, right? I've
> > previously thought we'd need 4.4 because there's an incompatibility
> > between 4.3 and 4.4 but I think it won't touch us, so 4.2 which added
> > atomics for mips seems fine. Given there's no buildfarm animal and
> > there's lots of variety out there that seems like a fair amount of
> > support.

> FWIW, I wouldn't have the slightest bit of difficulty with setting a
> project policy that if you want some platform to be supported, you
> must provide a buildfarm animal running on it.

> More to the point for this specific case, it seems like our process
> ought to be
> (1) select a preferably-small set of gcc atomic intrinsics that we
> want to use.
> (2) provide macro wrappers for these so that substituting other
> implementations isn't too tough.
> (3) for any platform where people don't want to use recent gcc,
> insist they provide substitute implementations of the macros.
>
> I don't think the core project has to be responsible for implementing
> (3), except maybe on very-mainstream arches such as x86. We can
> adopt a self-help attitude for everything else.

Sounds fair to me.

The question about platforms that simply cannot provide such atomics
like PA-RISC, which afaics is the only one, remains tho. I am not sure
we really want to provide codepaths that are only going to be tested
there.

I do plan to propose a set of macros for this once we know what cases we
need to support.

> But ... having said all that, it would be nice to see some proof of
> significant performance benefits that we're going to get from kicking
> those non-mainstream arches to the curb. I'm not enamored of removing
> real functionality now for vague promises of benefits later.

The reason I am bringing this up is that I'd like lift the patch in
http://archives.postgresql.org/message-id/20130926225545.GB26663%40awork2.anarazel.de
from a purely POC state to something committable. The numbers in it seem
to justify causing some pain.
Once the infrastructure is there we can easily get some really nice
further benefits by changing buffer pinning - that's one of the most
expensive things wrt. scalability after the above patch.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

Andres Freund <andres(at)2ndquadrant(dot)com> writes:
> The question about platforms that simply cannot provide such atomics
> like PA-RISC, which afaics is the only one, remains tho. I am not sure
> we really want to provide codepaths that are only going to be tested
> there.

PA-RISC is a dead architecture. According to wikipedia, HP hasn't sold
any such machines since 2008, and won't support them beyond 2013. If
that really is the only case we're worried about supporting, it's an
easy decision.

What worries me more is that you mentioned several cases where the gcc
atomics exist but need kernel support. I have to think that a trap
to the kernel would make the operation so expensive as to be a serious
performance loss, not gain. So it seems to me that platforms like that
are essentially being kicked to the curb if we make this change, even
if they theoretically could still work. Are there any that we really
care about?

regards, tom lane

On 2013-10-13 20:39:21 +0200, Tom Lane wrote:
> Andres Freund <andres(at)2ndquadrant(dot)com> writes:
> > The question about platforms that simply cannot provide such atomics
> > like PA-RISC, which afaics is the only one, remains tho. I am not sure
> > we really want to provide codepaths that are only going to be tested
> > there.
>
> PA-RISC is a dead architecture. According to wikipedia, HP hasn't sold
> any such machines since 2008, and won't support them beyond 2013. If
> that really is the only case we're worried about supporting, it's an
> easy decision.

Great.

> What worries me more is that you mentioned several cases where the gcc
> atomics exist but need kernel support. I have to think that a trap
> to the kernel would make the operation so expensive as to be a serious
> performance loss, not gain. So it seems to me that platforms like that
> are essentially being kicked to the curb if we make this change, even
> if they theoretically could still work.

I think it's not that bad on most - they all seem to use some more
lightweight trap mechanisms that basically just change protection and
stops interrupts while doing the math. No chance of scheduling and such.

> Are there any that we really care about?

I don't think so. At least not if we're restricting ourselves to 32bit
cmpxchg/xadd which I think will be enough for the first rounds of
improvement.

It's:
- PA-RISC
- sparc before ultrasparcs (1995)
- Multi-CPU/core SuperH before SH4 (uni SH2 has some cute interrupt
handling tricks, that do not require a trap)
- arm before v6

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On Sun, Oct 13, 2013 at 5:34 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> - PA-RISC. I think Tom was the remaining user there? Maybe just !gcc.
>
> Until pretty recently, there was a PA-RISC machine (not mine) in the
> buildfarm. I don't see it in the list today though. In any case,
> HP's compiler has always been a PITA, so no objection to requiring gcc
> for this platform.

I object to requiring gcc on that platform; we've had recent interest
in aCC-compiled builds on that platform.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 2013-10-14 09:40:10 -0400, Robert Haas wrote:
> On Sun, Oct 13, 2013 at 5:34 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> >> - PA-RISC. I think Tom was the remaining user there? Maybe just !gcc.
> >
> > Until pretty recently, there was a PA-RISC machine (not mine) in the
> > buildfarm. I don't see it in the list today though. In any case,
> > HP's compiler has always been a PITA, so no objection to requiring gcc
> > for this platform.
>
> I object to requiring gcc on that platform; we've had recent interest
> in aCC-compiled builds on that platform.

On PA-RISC or Itanic?

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On Mon, Oct 14, 2013 at 9:41 AM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> On 2013-10-14 09:40:10 -0400, Robert Haas wrote:
>> On Sun, Oct 13, 2013 at 5:34 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> >> - PA-RISC. I think Tom was the remaining user there? Maybe just !gcc.
>> >
>> > Until pretty recently, there was a PA-RISC machine (not mine) in the
>> > buildfarm. I don't see it in the list today though. In any case,
>> > HP's compiler has always been a PITA, so no objection to requiring gcc
>> > for this platform.
>>
>> I object to requiring gcc on that platform; we've had recent interest
>> in aCC-compiled builds on that platform.
>
> On PA-RISC or Itanic?

Oh, sorry. I think it was Itanium.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 2013-10-14 09:42:46 -0400, Robert Haas wrote:
> On Mon, Oct 14, 2013 at 9:41 AM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> > On 2013-10-14 09:40:10 -0400, Robert Haas wrote:
> >> On Sun, Oct 13, 2013 at 5:34 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> >> >> - PA-RISC. I think Tom was the remaining user there? Maybe just !gcc.
> >> >
> >> > Until pretty recently, there was a PA-RISC machine (not mine) in the
> >> > buildfarm. I don't see it in the list today though. In any case,
> >> > HP's compiler has always been a PITA, so no objection to requiring gcc
> >> > for this platform.
> >>
> >> I object to requiring gcc on that platform; we've had recent interest
> >> in aCC-compiled builds on that platform.
> >
> > On PA-RISC or Itanic?
>
> Oh, sorry. I think it was Itanium.

Good. Not that Itanium is nice, but it certainly supports all required
atomic ops.

I think somebody with access to that will have to figure out what
intrinsics are provided then - I haven't found reference documentation
for aCC's ia64/sys/inline.h.
I think Tom's stance that people who want a platform to be supported
need to help is reasonable. It's really hard to develop not entirely
trivial code for platforms you don't have access to and that don't even
have a buildfarm member.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On 2013-10-13 16:56:12 +0200, Tom Lane wrote:
> More to the point for this specific case, it seems like our process
> ought to be
> (1) select a preferably-small set of gcc atomic intrinsics that we
> want to use.

I suggest:
* pg_atomic_load_u32(uint32 *)
* uint32 pg_atomic_store_u32(uint32 *)
* uint32 pg_atomic_exchange_u32(uint32 *ptr, uint32 val)
* bool pg_atomic_compare_exchange_u32(uint32 *ptr, uint32 *expected, uint32 newval)
* uint32 pg_atomic_fetch_add_u32(uint32 *ptr, uint32 add)
* uint32 pg_atomic_fetch_sub_u32(uint32 *ptr, uint32 add)
* uint32 pg_atomic_fetch_and_u32(uint32 *ptr, uint32 add)
* uint32 pg_atomic_fetch_or_u32(uint32 *ptr, uint32 add)
* u64 variants of the above
* bool pg_atomic_test_set(void *ptr)
* void pg_atomic_clear(void *ptr)

Ontop of that we can generically implement:
* pg_atomic_add_until_u32(uint32 *ptr, uint32 val, uint32 limit)
* pg_atomic_(add|sub|and|or)_fetch_u32()
* u64 variants of the above

We might also want to provide a generic implementation of the math
operations based on pg_atomic_compare_exchange() to make it easier to
bring up a new architecture.

I think we should leave 64bit support optional for now.

Opinions?

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On Tue, Oct 15, 2013 at 8:33 AM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> On 2013-10-13 16:56:12 +0200, Tom Lane wrote:
>> More to the point for this specific case, it seems like our process
>> ought to be
>> (1) select a preferably-small set of gcc atomic intrinsics that we
>> want to use.
>
> I suggest:
> * pg_atomic_load_u32(uint32 *)
> * uint32 pg_atomic_store_u32(uint32 *)

We currently assume simple assignment suffices for this.

> * uint32 pg_atomic_exchange_u32(uint32 *ptr, uint32 val)
> * bool pg_atomic_compare_exchange_u32(uint32 *ptr, uint32 *expected, uint32 newval)
> * uint32 pg_atomic_fetch_add_u32(uint32 *ptr, uint32 add)
> * uint32 pg_atomic_fetch_sub_u32(uint32 *ptr, uint32 add)
> * uint32 pg_atomic_fetch_and_u32(uint32 *ptr, uint32 add)
> * uint32 pg_atomic_fetch_or_u32(uint32 *ptr, uint32 add)

Do we really need all of those? Compare-and-swap is clearly needed,
and fetch-and-add is also very useful. I'm not sure about the rest.
Not that I necessarily object to having them, but it will be a lot
more work.

> * u64 variants of the above
> * bool pg_atomic_test_set(void *ptr)
> * void pg_atomic_clear(void *ptr)

What are the intended semantics here?

> Ontop of that we can generically implement:
> * pg_atomic_add_until_u32(uint32 *ptr, uint32 val, uint32 limit)
> * pg_atomic_(add|sub|and|or)_fetch_u32()
> * u64 variants of the above

Are these really generally needed?

> We might also want to provide a generic implementation of the math
> operations based on pg_atomic_compare_exchange() to make it easier to
> bring up a new architecture.

+1.

I have a related problem, which is that some code I'm currently
working on vis-a-vis parallelism can run lock-free on platforms with
atomic 8 bit assignment but needs a spinlock or two elsewhere. So I'd
want to use pg_atomic_store_u64(), but I'd also need a clean way to
test, at compile time, whether it's available.

More general question: how do we move the ball down the field in this
area? I'm willing to do some of the legwork, but I doubt I can do all
of it, and I really think we need to make some progress here soon, as
it seems that you and I and Ants are all running into the same
problems in slightly different ways. What's our strategy for getting
something done here?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 2013-10-16 12:26:28 -0400, Robert Haas wrote:
> On Tue, Oct 15, 2013 at 8:33 AM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> > On 2013-10-13 16:56:12 +0200, Tom Lane wrote:
> >> More to the point for this specific case, it seems like our process
> >> ought to be
> >> (1) select a preferably-small set of gcc atomic intrinsics that we
> >> want to use.
> >
> > I suggest:
> > * pg_atomic_load_u32(uint32 *)
> > * uint32 pg_atomic_store_u32(uint32 *)
>
> We currently assume simple assignment suffices for this.

Partially that only works because we sprinkle 'volatile's on lots of
places. That can actually hurt performance... it'd usually be something
like:
#define pg_atomic_load(uint32) (*(volatile uint32 *)(atomic))

Even if not needed in some places because a variable is already
volatile, it's very helpful in pointing out what happens and where you
need to be careful.

> > * uint32 pg_atomic_exchange_u32(uint32 *ptr, uint32 val)
> > * bool pg_atomic_compare_exchange_u32(uint32 *ptr, uint32 *expected, uint32 newval)
> > * uint32 pg_atomic_fetch_add_u32(uint32 *ptr, uint32 add)
> > * uint32 pg_atomic_fetch_sub_u32(uint32 *ptr, uint32 add)
> > * uint32 pg_atomic_fetch_and_u32(uint32 *ptr, uint32 add)
> > * uint32 pg_atomic_fetch_or_u32(uint32 *ptr, uint32 add)
>
> Do we really need all of those? Compare-and-swap is clearly needed,
> and fetch-and-add is also very useful. I'm not sure about the rest.
> Not that I necessarily object to having them, but it will be a lot
> more work.

Well, _sub can clearly be implemented with _add generically. I find code
using _sub much easier to read than _add(-whatever), but that's maybe
just me.

But _and, _or are really useful because they can be used to atomically
clear and set flag bits.

>
> > * u64 variants of the above
> > * bool pg_atomic_test_set(void *ptr)
> > * void pg_atomic_clear(void *ptr)
>
> What are the intended semantics here?

It's basically TAS() without defining whether setting a value that needs
to be set is a 1 or a 0. Gcc provides this and I think we should make
our spinlock implementation use it if there is no special cased
implementation available.

> > Ontop of that we can generically implement:
> > * pg_atomic_add_until_u32(uint32 *ptr, uint32 val, uint32 limit)
> > * pg_atomic_(add|sub|and|or)_fetch_u32()
> > * u64 variants of the above
>
> Are these really generally needed?

_add_until() is very useful for implementing thinks like usagecount
where you don't want to increase values too high.

The lwlock scaling thing needs the add_fetch variant because we need to
know what the lockcount is *after* we've registered. I think lots of
lockless algorithm have similar requirements.

Since those are either wrappers around fetch_op or compare_swap and thus
can be implemented generically I don't really see a problem with
providing them.

> I have a related problem, which is that some code I'm currently
> working on vis-a-vis parallelism can run lock-free on platforms with
> atomic 8 bit assignment but needs a spinlock or two elsewhere. So I'd
> want to use pg_atomic_store_u64(), but I'd also need a clean way to
> test, at compile time, whether it's available.

Yes, definitely. There should be a couple of #defines that declare
whether non-prerequisite options are supported. I'd guess we want at least:
* 8byte math
* 16byte compare_and_swap

> More general question: how do we move the ball down the field in this
> area? I'm willing to do some of the legwork, but I doubt I can do all
> of it, and I really think we need to make some progress here soon, as
> it seems that you and I and Ants are all running into the same
> problems in slightly different ways. What's our strategy for getting
> something done here?

That's a pretty good question.

I am willing to write the gcc implementation, plus the generic wrappers
and provide the infrastructure to override it per-platform. I won't be
able to do anything about non-linux, non-gcc platforms in that timeframe
though.

I was thinking of something like:
include/storage/atomic.h
include/storage/atomic-gcc.h
include/storage/atomic-msvc.h
include/storage/atomic-acc-ia64.h
where atomic.h first has a list of #ifdefs including the specialized
files and then lots of #ifndef providing generic variants if not
already provided by the platorm specific file.

We could provide not only per-compiler files but also compiler
independent files for some arches so we could e.g. define the
restrictions for arm once. I think whether that's useful will be visible
when writing the stuff.

Makes sense?

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On Sat, Oct 12, 2013 at 8:46 PM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> I think we should remove support the following ports:
> - IRIX
> - UnixWare
> - Tru64

According to http://en.wikipedia.org/wiki/IRIX, IRIX has been
officially retired. The last release of IRIX was in 2006 and support
will end in December of 2013. Therefore, it will be unsupported by
the time PostgreSQL 9.4 is released.

According to http://en.wikipedia.org/wiki/UnixWare, UnixWare is not
dead, although there have been no new releases in 5 years.

According to http://en.wikipedia.org/wiki/Tru64_UNIX, Tru64 has been
officially retired. Support ended in December, 2012. This seems safe
to remove.

So I vote for removing IRIX and Tru64 immediately, but I'm a little
more hesitant about shooting UnixWare, since it's technically still
supported.

> Neither of those are relevant.
>
> I think we should remove support for the following architectures:
> - VAX

According to http://en.wikipedia.org/wiki/VAX#History, all
manufacturing of VAX computers ceased in 2005, but according to
http://en.wikipedia.org/wiki/OpenVMS#Major_release_timeline, OpenVMS
is still releasing new versions. I'm not sure what to make of that.

> - univel (s_lock support remaining)

We removed the univel port in 9.2 and didn't get any complaints, but
the s_lock support is still used by the SCO and UnixWare ports, except
under GCC.

> - sinix (s_lock support remaining)

This seems to be quite thoroughly dead. The best information I can
find indicates that development ended in 2002 and support in 2008. I
think we can remove this.

> - sun3 (I think it's just s_lock support remaining)
> - natsemi 32k

Both of these are so old I can hardly find any information on them.
Seems clear to nuke these.

> - superH

Support for spinlocks on SuperH was only recently added, in 9.0. I
don't think we can assume that no one cares any more.

> - ALPHA (big pain in the ass to get right, nobody uses it anymore)

It seems somehow a shame to let this one go, but I agree it's a big
pain in the ass to get it right.

> - m86k (doesn't have a useable CAS on later iterations like coldfire)

I don't think we can desupport it just because it doesn't have CAS.
CAS is very useful and I think we should start using it, but I think
we should anticipate a --disable-cas or --disable-atomics option and
regularly test that our code works without it. IOW, we can rely on it
as an optimization, but not for correctness. Eventually, we can
probably desupport all platforms that don't have CAS, but I see that
as something that's probably 5 or 10 years away, not something we can
do tomorrow.

I'm also not sure that this is dead enough to kill.

> - M32R (no userspace CAS afaics)

Support was added for this in 8.3 and it doesn't seem to be particularly dead.

> - mips for anything but gcc > 4.4, using gcc's atomics support
> - s390 for anything but gcc > 4.4, using gcc's atomics support

I'm not clearly how broadly this would sweep, but MIPS doesn't appear dead.

> - 32bit/<v9 sparc (doesn't have proper atomics, old)

Seems fine to remove this.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 10/16/2013 07:04 PM, Robert Haas wrote:
> On Sat, Oct 12, 2013 at 8:46 PM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
>> I think we should remove support the following ports:
>> - IRIX
>> - UnixWare
>> - Tru64
>
> According to http://en.wikipedia.org/wiki/IRIX, IRIX has been
> officially retired. The last release of IRIX was in 2006 and support
> will end in December of 2013. Therefore, it will be unsupported by
> the time PostgreSQL 9.4 is released.
>
> According to http://en.wikipedia.org/wiki/UnixWare, UnixWare is not
> dead, although there have been no new releases in 5 years.
>
> According to http://en.wikipedia.org/wiki/Tru64_UNIX, Tru64 has been
> officially retired. Support ended in December, 2012. This seems safe
> to remove.
>
> So I vote for removing IRIX and Tru64 immediately, but I'm a little
> more hesitant about shooting UnixWare, since it's technically still
> supported.
>
>> Neither of those are relevant.

agreed

>>
>> I think we should remove support for the following architectures:
>> - VAX
>
> According to http://en.wikipedia.org/wiki/VAX#History, all
> manufacturing of VAX computers ceased in 2005, but according to
> http://en.wikipedia.org/wiki/OpenVMS#Major_release_timeline, OpenVMS
> is still releasing new versions. I'm not sure what to make of that.

VAX is also an officially supported OpenBSD port (see
http://www.openbsd.org/vax.html)

Stefan

On 2013-10-16 13:04:23 -0400, Robert Haas wrote:
> So I vote for removing IRIX and Tru64 immediately, but I'm a little
> more hesitant about shooting UnixWare, since it's technically still
> supported.

I think if somebody wants to have it supported they need to provide a
buildfarm member and probably a bit of help. Doing any change in this
area for a platform that nobody has access to in any way seems
pointless because it will be broken anyway.

> > I think we should remove support for the following architectures:
> > - VAX
>
> According to http://en.wikipedia.org/wiki/VAX#History, all
> manufacturing of VAX computers ceased in 2005, but according to
> http://en.wikipedia.org/wiki/OpenVMS#Major_release_timeline, OpenVMS
> is still releasing new versions. I'm not sure what to make of that.

And the last revisions are from 2000 even. I don't think anybody will
actually run a new postgres installation on it.
I also doubt our current version actually compiles there.

> > - superH
>
> Support for spinlocks on SuperH was only recently added, in 9.0. I
> don't think we can assume that no one cares any more.

Yes, I since revised my opinion somewhere downthread. It's pretty much
linux and gcc only, so it's really not that problematic.

Note that our current implementation is broken on many older SuperH (<
SH4) CPUs since their tas isn't safe...

> > - m86k (doesn't have a useable CAS on later iterations like coldfire)

> I don't think we can desupport it just because it doesn't have CAS.
> CAS is very useful and I think we should start using it, but I think
> we should anticipate a --disable-cas or --disable-atomics option and
> regularly test that our code works without it. IOW, we can rely on it
> as an optimization, but not for correctness. Eventually, we can
> probably desupport all platforms that don't have CAS, but I see that
> as something that's probably 5 or 10 years away, not something we can
> do tomorrow.

I think that will result in loads of barely tested duplicative code. If
there were any even remotely popular platforms requiring this, ok. But
unless I miss something there really isn't.

We're talking about CPUs with mostly less than 100MHZ here, mostly with
directly soldered RAM in the one digit MB range. I really don't think
there's a usecase for running PG on them. And I doubt it still works on
many of the architectures we advocate.

> I'm also not sure that this is dead enough to kill.
>
> > - M32R (no userspace CAS afaics)
>
> Support was added for this in 8.3 and it doesn't seem to be particularly dead.

It's not? All I've read seems to point into a different direction.

The newest supported CPU seems to be
http://www.renesas.com/products/mpumcu/m32r/m32r_ecu/32196/index.jsp
sporting a whopping 1024Kb of programmable RAM and only single precision
FPU.

> > - mips for anything but gcc > 4.4, using gcc's atomics support
> > - s390 for anything but gcc > 4.4, using gcc's atomics support
>
> I'm not clearly how broadly this would sweep, but MIPS doesn't appear dead.

Downthread I noticed it's gcc 4.2 not 4.4. There's some API change in
gcc's atomics support in 4.4 which is why I thought of 4.4 but it
shouldn't affect us after looking in more detail.
Mips seems to only be used with gcc these days, so I think that's ok.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

>> - sinix (s_lock support remaining)
>> - sun3 (I think it's just s_lock support remaining)
>> - natsemi 32k

Patch removing spinlock support for these three ports is attached.
This is not to say we couldn't remove more later, but these seem to be
the three spinlock implementations that are most sincerely dead.
Absent objections, I'll apply this tomorrow.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 2013-10-16 13:55:20 -0400, Robert Haas wrote:
> >> - sinix (s_lock support remaining)
> >> - sun3 (I think it's just s_lock support remaining)
> >> - natsemi 32k
>
> Patch removing spinlock support for these three ports is attached.
> This is not to say we couldn't remove more later, but these seem to be
> the three spinlock implementations that are most sincerely dead.
> Absent objections, I'll apply this tomorrow.

Looks good to me.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On 2013-10-16 13:04:23 -0400, Robert Haas wrote:
> > - m86k (doesn't have a useable CAS on later iterations like coldfire)
>
> I don't think we can desupport it just because it doesn't have CAS.

Btw, if necessary we could easily support the pre coldfire
variants. Note that e.g. debian doesn't support coldfire either. Well,
the unofficial m68k port after it has been dropped from core debian in
*2007*.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On Wed, Oct 16, 2013 at 1:52 PM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> On 2013-10-16 13:04:23 -0400, Robert Haas wrote:
>> So I vote for removing IRIX and Tru64 immediately, but I'm a little
>> more hesitant about shooting UnixWare, since it's technically still
>> supported.
>
> I think if somebody wants to have it supported they need to provide a
> buildfarm member and probably a bit of help. Doing any change in this
> area for a platform that nobody has access to in any way seems
> pointless because it will be broken anyway.

I don't agree with that policy. Sure, 97% of our users are probably
running Linux, Windows, MacOS X, or one of the fairly-popular BSD
variants. But I think a part of the appeal of PostgreSQL is that it is
cross-platform, and doesn't require a lot of special hardware support,
and I'm loathe to give that up. It's one thing to say, gee, we don't
know whether this is actually going to compile and work on your
platform, because it's not tested. It's quite something else to say,
anything that's not on this short list of supported platforms has zero
chance of working without jumping through major hoops.

Commit b8ed4cc9627de437e5eafdb81631a0d0f063abb3, from April of this
year, updated our support for --disable-spinlocks. Spinlocks are a
far more basic facility than the sorts of advanced primitives we're
talking about requiring here. If we're going to support compiling
under --disable-spinlocks, then we certainly can't rely on this more
advanced stuff always being present. Even if we get rid of that, it's
throwing up one more barrier in front of people who want to get
PostgreSQL running on a non-mainstream architecture. I've spent
enough time over the years working on odd hardware to have sympathy
for people who want to compile stuff there and have it work, or at
least work after some non-huge amount of hacking.

>> I don't think we can desupport it just because it doesn't have CAS.
>> CAS is very useful and I think we should start using it, but I think
>> we should anticipate a --disable-cas or --disable-atomics option and
>> regularly test that our code works without it. IOW, we can rely on it
>> as an optimization, but not for correctness. Eventually, we can
>> probably desupport all platforms that don't have CAS, but I see that
>> as something that's probably 5 or 10 years away, not something we can
>> do tomorrow.
>
> I think that will result in loads of barely tested duplicative code. If
> there were any even remotely popular platforms requiring this, ok. But
> unless I miss something there really isn't.
>
> We're talking about CPUs with mostly less than 100MHZ here, mostly with
> directly soldered RAM in the one digit MB range. I really don't think
> there's a usecase for running PG on them. And I doubt it still works on
> many of the architectures we advocate.

If stuff is completely ancient and obsolete, I think it's fine to kill
it; it probably doesn't work anyway, and nobody's likely to try. But
I think a lot of the stuff you're talking about is not in that
category.

>> I'm also not sure that this is dead enough to kill.
>>
>> > - M32R (no userspace CAS afaics)
>>
>> Support was added for this in 8.3 and it doesn't seem to be particularly dead.
>
> It's not? All I've read seems to point into a different direction.
>
> The newest supported CPU seems to be
> http://www.renesas.com/products/mpumcu/m32r/m32r_ecu/32196/index.jsp
> sporting a whopping 1024Kb of programmable RAM and only single precision
> FPU.

Well, so, they're an embedded chip. Yes, it's low spec. But then
why'd somebody bother adding spinlock support for it in 2007? It's
not as if that was a high-end chip *then* either.

It's hard to say where to draw the line here. I don't want the
illusion of support for platforms that don't in fact have a prayer of
working to prevent us from making needed improvements. On the other
hand, I also don't want to blithely rip out support for architectures
that people may well still be using and where, in some cases, people
have gone out of their way to add that support. If we get to the
point where some relatively-obscure architecture is the only thing
standing between us and improvement X, then we can weigh those things
against each other and decide. But I don't really want to go rip out
support for half-a-dozen semi-supported architectures without some
clear goal in mind. That just doesn't seem friendly.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 2013-10-16 14:20:58 -0400, Robert Haas wrote:
> > I think if somebody wants to have it supported they need to provide a
> > buildfarm member and probably a bit of help. Doing any change in this
> > area for a platform that nobody has access to in any way seems
> > pointless because it will be broken anyway.
>
> I don't agree with that policy. Sure, 97% of our users are probably
> running Linux, Windows, MacOS X, or one of the fairly-popular BSD
> variants. But I think a part of the appeal of PostgreSQL is that it is
> cross-platform, and doesn't require a lot of special hardware support,
> and I'm loathe to give that up. It's one thing to say, gee, we don't
> know whether this is actually going to compile and work on your
> platform, because it's not tested. It's quite something else to say,
> anything that's not on this short list of supported platforms has zero
> chance of working without jumping through major hoops.

Well, I am not advocating to break platforms just because they are not
on the buildfarm, but I don't see how we can introduce new facilities
that require platform support if we don't have any way to test them on
some platforms.

> Commit b8ed4cc9627de437e5eafdb81631a0d0f063abb3, from April of this
> year, updated our support for --disable-spinlocks. Spinlocks are a
> far more basic facility than the sorts of advanced primitives we're
> talking about requiring here. If we're going to support compiling
> under --disable-spinlocks, then we certainly can't rely on this more
> advanced stuff always being present. Even if we get rid of that, it's
> throwing up one more barrier in front of people who want to get
> PostgreSQL running on a non-mainstream architecture. I've spent
> enough time over the years working on odd hardware to have sympathy
> for people who want to compile stuff there and have it work, or at
> least work after some non-huge amount of hacking.

I think by using architecture independent, compiler provided intrinsics
we take care of that for most non-mainstream platforms. Just about
everything even remotely new that is/will be out there is/will be using gcc
or something compatible to it (e.g. llvm). Those provide the __sync_*
intrinsics we'd wrap.
That's incidentally why I am advocating including pg_atomic_clear and
pg_atomic_test_set in the set of supported atomics. With those porting
to a new platform will often require exactly zero changes since
spinlocks will just use them.

> > We're talking about CPUs with mostly less than 100MHZ here, mostly with
> > directly soldered RAM in the one digit MB range. I really don't think
> > there's a usecase for running PG on them. And I doubt it still works on
> > many of the architectures we advocate.
>
> If stuff is completely ancient and obsolete, I think it's fine to kill
> it; it probably doesn't work anyway, and nobody's likely to try. But
> I think a lot of the stuff you're talking about is not in that
> category.

Which? We only seem to disagree about M32R and m68k, right? I've
recanted mips and superh days ago ;).
If you find alpha important, that's fine, that's supported by gcc. I
just doubt we'll get it even remotely right without tests...

> >> I'm also not sure that this is dead enough to kill.
> >>
> >> > - M32R (no userspace CAS afaics)
> >>
> >> Support was added for this in 8.3 and it doesn't seem to be particularly dead.
> >
> > It's not? All I've read seems to point into a different direction.
> >
> > The newest supported CPU seems to be
> > http://www.renesas.com/products/mpumcu/m32r/m32r_ecu/32196/index.jsp
> > sporting a whopping 1024Kb of programmable RAM and only single precision
> > FPU.
>
> Well, so, they're an embedded chip. Yes, it's low spec. But then
> why'd somebody bother adding spinlock support for it in 2007? It's
> not as if that was a high-end chip *then* either.

Well, 2007 postgres was still smaller. I think these days the postgres
binary alone wouldn't fit onto the executable memory of such devices. A pretty
minimal, stripped postgres (CFLAGS=-Os --without-zlib
--without-readline) weighs 4.5MB these days. 8.3 still was 2.6MB.

Not to even talk about our runtime memory requirements which since have
gone up noticeably.

> It's hard to say where to draw the line here. I don't want the
> illusion of support for platforms that don't in fact have a prayer of
> working to prevent us from making needed improvements. On the other
> hand, I also don't want to blithely rip out support for architectures
> that people may well still be using and where, in some cases, people
> have gone out of their way to add that support. If we get to the
> point where some relatively-obscure architecture is the only thing
> standing between us and improvement X, then we can weigh those things
> against each other and decide. But I don't really want to go rip out
> support for half-a-dozen semi-supported architectures without some
> clear goal in mind. That just doesn't seem friendly.

Well, I only started to look at this somewhat seriously because more and
more people in the last year or so, both onlist and towards 2ndq were
complaining about massive spinlock contention (up to 97% spent in
s_lock) on somewhat bigger machines. The primary offender in many
workloads is the lwlock internal spinlock, quite often for LW_SHARED
acquisition where we shouldn't need to block.

That triggered developing the wait-free LW_SHARED patch
http://www.postgresql.org/message-id/20130926225545.GB26663@awork2.anarazel.de
which indeed shows quite some promise. Unfortunately it pretty
fundamentally requires compare_swap and fetch_add. Now we could just
implement lwlocks in two pretty much independent ways, but that seems to
be pretty bad from a maintainability POV.

The next big thing after that is getting rid of spinlocks around buffer
pinning (and by that in buffer hdr locking). That would end up in quite
some #ifdef's sprinkled around.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On Sat, Oct 12, 2013 at 06:35:00PM -0700, Peter Geoghegan wrote:
> > - ALPHA (big pain in the ass to get right, nobody uses it anymore)
>
> Yes, for many years now ALPHA has only been useful as a way of
> illustrating how bad it's possible for CPU memory operation reordering
> considerations to get. So I quite agree.

Are there any optimizations we have avoided, or 'volatile' designations
added, only for Alpha? Could we improve other things if Alpha support
was dropped?

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +

On 2013-10-16 15:49:54 -0400, Bruce Momjian wrote:
> On Sat, Oct 12, 2013 at 06:35:00PM -0700, Peter Geoghegan wrote:
> > > - ALPHA (big pain in the ass to get right, nobody uses it anymore)
> >
> > Yes, for many years now ALPHA has only been useful as a way of
> > illustrating how bad it's possible for CPU memory operation reordering
> > considerations to get. So I quite agree.
>
> Are there any optimizations we have avoided, or 'volatile' designations
> added, only for Alpha?

I am somewhat sure that some of the code we added in the last years
isn't actually correct for alpha (and others actually). It's just that
nobody actually runs on alpha anymore, so nobody notices.

> Could we improve other things if Alpha support was dropped?

I think the major thing is that if we're going to add more algorithms
that use less locks - which we'll have to, otherwise our scalability
will get more and more problematic - we'll have to adhere to the
weakest cache coherency model we support. And at least I am not
intelligent/experienced enough to blindly write correct code for Alpha.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On Wed, Oct 16, 2013 at 3:26 PM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
>> I don't agree with that policy. Sure, 97% of our users are probably
>> running Linux, Windows, MacOS X, or one of the fairly-popular BSD
>> variants. But I think a part of the appeal of PostgreSQL is that it is
>> cross-platform, and doesn't require a lot of special hardware support,
>> and I'm loathe to give that up. It's one thing to say, gee, we don't
>> know whether this is actually going to compile and work on your
>> platform, because it's not tested. It's quite something else to say,
>> anything that's not on this short list of supported platforms has zero
>> chance of working without jumping through major hoops.
>
> Well, I am not advocating to break platforms just because they are not
> on the buildfarm, but I don't see how we can introduce new facilities
> that require platform support if we don't have any way to test them on
> some platforms.

Well, on that we agree. But my answer to that is - any facilities
that require additional platform support had better be optional. If
we can't do it without raising the bar for platform support, then we
don't do it.

> I think by using architecture independent, compiler provided intrinsics
> we take care of that for most non-mainstream platforms. Just about
> everything even remotely new that is/will be out there is/will be using gcc
> or something compatible to it (e.g. llvm). Those provide the __sync_*
> intrinsics we'd wrap.
> That's incidentally why I am advocating including pg_atomic_clear and
> pg_atomic_test_set in the set of supported atomics. With those porting
> to a new platform will often require exactly zero changes since
> spinlocks will just use them.

I grant that helps a great deal. The fact that gcc has added those
atomics makes this a lot more feasible to contemplate that it would be
otherwise. Now, given some of our other experiences with gcc, I am
slightly worried that there may be bugs. But if that turns out to be
the case we can figure out what to do about it. It's very nice to at
least have a place to start.

On the other hand, I'm not convinced that we don't need to give any
thought to UNIX vendors that are still pushing their proprietary
compilers. Many of the old players are dead, but IBM's ICC and HP's
aCC definitely aren't, and I wouldn't be surprised to find one or two
other big ones as well, plus maybe half-a-dozen others that are
clinging to life.

> Which? We only seem to disagree about M32R and m68k, right? I've
> recanted mips and superh days ago ;).
> If you find alpha important, that's fine, that's supported by gcc. I
> just doubt we'll get it even remotely right without tests...

As far as spinlock support goes, you proposed removing VAX, univel,
sinix, sun3, natsemi 32k, superH, ALPHA, m86k, M32R, mips non-GCC,
s390 non-GCC, and 32bit/<v9 sparc. I agreed unequivocally with 4 of
those (though I don't see the code you're talking about wrt/32bit<v9
sparc), and you withdrew 2 of those suggestions, so I think there are
6 that are still in doubt: vax, univel, ALPHA, m32r, m68k, and s390
non-GCC.

>> It's hard to say where to draw the line here. I don't want the
>> illusion of support for platforms that don't in fact have a prayer of
>> working to prevent us from making needed improvements. On the other
>> hand, I also don't want to blithely rip out support for architectures
>> that people may well still be using and where, in some cases, people
>> have gone out of their way to add that support. If we get to the
>> point where some relatively-obscure architecture is the only thing
>> standing between us and improvement X, then we can weigh those things
>> against each other and decide. But I don't really want to go rip out
>> support for half-a-dozen semi-supported architectures without some
>> clear goal in mind. That just doesn't seem friendly.
>
> Well, I only started to look at this somewhat seriously because more and
> more people in the last year or so, both onlist and towards 2ndq were
> complaining about massive spinlock contention (up to 97% spent in
> s_lock) on somewhat bigger machines. The primary offender in many
> workloads is the lwlock internal spinlock, quite often for LW_SHARED
> acquisition where we shouldn't need to block.
>
> That triggered developing the wait-free LW_SHARED patch
> http://www.postgresql.org/message-id/20130926225545.GB26663@awork2.anarazel.de
> which indeed shows quite some promise. Unfortunately it pretty
> fundamentally requires compare_swap and fetch_add. Now we could just
> implement lwlocks in two pretty much independent ways, but that seems to
> be pretty bad from a maintainability POV.
>
> The next big thing after that is getting rid of spinlocks around buffer
> pinning (and by that in buffer hdr locking). That would end up in quite
> some #ifdef's sprinkled around.

I do understand that it's going to be painful to carry multiple
implementations of some of these facilities. But at least from where
I sit I'm not sure we really have a choice. If we have a
--disable-atomics option, there's no reason we can't have regular
buildfarm coverage of that code; broken lwlocks tend to make things
die pretty horribly, so I'm relatively confident bugs will show up.

Obviously, #ifdefs scattered all over the backend is not going to be
good. We'll need to find ways to isolate the code that does
platform-specific things, but we have been relatively successful at
doing that in the past and I see no particular reason to be
pessimistic about the future.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 2013-10-16 16:10:18 -0400, Robert Haas wrote:
> On the other hand, I'm not convinced that we don't need to give any
> thought to UNIX vendors that are still pushing their proprietary
> compilers. Many of the old players are dead, but IBM's ICC and HP's
> aCC definitely aren't, and I wouldn't be surprised to find one or two
> other big ones as well, plus maybe half-a-dozen others that are
> clinging to life.

acc provides intrinsics for IA64, so that's easily supported. For IBM,
do you mean XLC? If so, that provides intrinsics as well. So does sun
studio.
ICC as in Intel's compiler provides intrinsics as well. In fact, that's
where gcc cribbed most of it's API from.

> > Which? We only seem to disagree about M32R and m68k, right? I've
> > recanted mips and superh days ago ;).
> > If you find alpha important, that's fine, that's supported by gcc. I
> > just doubt we'll get it even remotely right without tests...
>
> As far as spinlock support goes, you proposed removing VAX, univel,
> sinix, sun3, natsemi 32k, superH, ALPHA, m86k, M32R, mips non-GCC,
> s390 non-GCC, and 32bit/<v9 sparc. I agreed unequivocally with 4 of
> those (though I don't see the code you're talking about wrt/32bit<v9
> sparc), and you withdrew 2 of those suggestions, so I think there are
> 6 that are still in doubt: vax, univel, ALPHA, m32r, m68k, and s390
> non-GCC.

Well, you didn't sound like you deemed vax and alpha to be that
important and I was only talking about architectures, not OSs...

But anyway hardware architecture wise, all but m32r, m68k and pa-risc
have the required hardware support for atomic add and cmpxchg.

univel/unixware: Supports only x86 (these days at least), so writing the
required assembly is trivial if it comes to that. Testing on the other hand...

s390, s390x: We only support linux anyway, so I don't see the
restriction to gcc being problematic.

alpha: We use gcc inline assembly currently, so it's only gcc again. It
is supported by gcc's intrinsics. We can easily support it if we trust
ourselves to understand the cache coherency.

mips: Besides gcc we only support IRIX, since you voted to remove that,
the restriction to gcc doesn't cost anything.

m68k: Only coldfire chips don't have working CAS. All support atomic add.
m32r: Doesn't have CAS, doesn't support atomic add.
pa-risc: Doesn't have CAS, doesn't support atomic add.

> I do understand that it's going to be painful to carry multiple
> implementations of some of these facilities. But at least from where
> I sit I'm not sure we really have a choice. If we have a
> --disable-atomics option, there's no reason we can't have regular
> buildfarm coverage of that code; broken lwlocks tend to make things
> die pretty horribly, so I'm relatively confident bugs will show up.

I am pretty sure lots of that code will only be noticeably under
noticeable concurrency. And we don't exercise that all that much in the
regression tets.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On 2013-10-16 16:10:18 -0400, Robert Haas wrote:
> (though I don't see the code you're talking about wrt/32bit<v9
> sparc)

< v9 sparc doesn't support compare-and-swap like operations, that's the
background.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On Wed, Oct 16, 2013 at 10:04:29PM +0200, Andres Freund wrote:
> On 2013-10-16 15:49:54 -0400, Bruce Momjian wrote:
> > On Sat, Oct 12, 2013 at 06:35:00PM -0700, Peter Geoghegan wrote:
> > > > - ALPHA (big pain in the ass to get right, nobody uses it anymore)
> > >
> > > Yes, for many years now ALPHA has only been useful as a way of
> > > illustrating how bad it's possible for CPU memory operation reordering
> > > considerations to get. So I quite agree.
> >
> > Are there any optimizations we have avoided, or 'volatile' designations
> > added, only for Alpha?
>
> I am somewhat sure that some of the code we added in the last years
> isn't actually correct for alpha (and others actually). It's just that
> nobody actually runs on alpha anymore, so nobody notices.
>
> > Could we improve other things if Alpha support was dropped?
>
> I think the major thing is that if we're going to add more algorithms
> that use less locks - which we'll have to, otherwise our scalability
> will get more and more problematic - we'll have to adhere to the
> weakest cache coherency model we support. And at least I am not
> intelligent/experienced enough to blindly write correct code for Alpha.

Removing support for alpha is a different animal compared to removing support
for non-gcc MIPS and most of the others in your list. A hacker wishing to
restore support for another MIPS compiler would fill in the assembly code
blanks, probably using code right out of an architecture manual. A hacker
wishing to restore support for alpha would find himself auditing every
lock-impoverished algorithm in the backend. At the same time, as you suggest,
the benefit to general PostgreSQL development from dropping alpha is greater
in the same way. Dropping non-gcc MIPS reduces effort to complete the initial
patch that adds the atomic primitives; dropping alpha reduces effort to
implement every future algorithm that uses barriers. I do think dropping
support for alpha is the right decision. That's a firm and likely one-way
downgrade of support, much like we've done for compilers with no 64-bit type.

Concerning cases like non-gcc MIPS, I'll mostly echo Tom's comments[1]. I'm
comfortable with the project saying "We've added atomics for the architectures
we have. Help us with the rest!" It's reasonable to introduce an improvement
that entails architecture-dependent code without requiring that the initial
patch and initial author address every interesting target. The bar to restore
"support", even years later, will be pretty low. In that light, I don't favor
ripping out longstanding architecture-specific code for borderline platforms.
Doing so raises the bar for restoring support, without proximate benefit.

[1] http://www.postgresql.org/message-id/4694.1381676172@sss.pgh.pa.us

--
Noah Misch
EnterpriseDB http://www.enterprisedb.com

On Thu, Oct 17, 2013 at 12:22 AM, Noah Misch <noah(at)leadboat(dot)com> wrote:
> Removing support for alpha is a different animal compared to removing support
> for non-gcc MIPS and most of the others in your list. A hacker wishing to
> restore support for another MIPS compiler would fill in the assembly code
> blanks, probably using code right out of an architecture manual. A hacker
> wishing to restore support for alpha would find himself auditing every
> lock-impoverished algorithm in the backend.

I had much the same thought last night. So I reverse my vote on
Alpha: let's drop it. I had thought that perhaps there'd be some
value in keeping it to force ourselves to consider what will happen
under the weakest generally-understood memory model, but in fact
that's probably a doomed effort without having the hardware available
to test the code. As you say, any future atomics support for such a
platform will be a major undertaking.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Wed, Oct 16, 2013 at 1:25 PM, Stefan Kaltenbrunner
<stefan(at)kaltenbrunner(dot)cc> wrote:
> On 10/16/2013 07:04 PM, Robert Haas wrote:
>> On Sat, Oct 12, 2013 at 8:46 PM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
>>> I think we should remove support the following ports:
>>> - IRIX
>>> - UnixWare
>>> - Tru64
>>
>> According to http://en.wikipedia.org/wiki/IRIX, IRIX has been
>> officially retired. The last release of IRIX was in 2006 and support
>> will end in December of 2013. Therefore, it will be unsupported by
>> the time PostgreSQL 9.4 is released.
>>
>> According to http://en.wikipedia.org/wiki/UnixWare, UnixWare is not
>> dead, although there have been no new releases in 5 years.
>>
>> According to http://en.wikipedia.org/wiki/Tru64_UNIX, Tru64 has been
>> officially retired. Support ended in December, 2012. This seems safe
>> to remove.
>>
>> So I vote for removing IRIX and Tru64 immediately, but I'm a little
>> more hesitant about shooting UnixWare, since it's technically still
>> supported.
>>
>>> Neither of those are relevant.
>
> agreed

The attached patch, which I propose to apply relatively soon if nobody
objects, removes the IRIX port.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 10/17/13 12:45 PM, Robert Haas wrote:
> The attached patch, which I propose to apply relatively soon if nobody
> objects, removes the IRIX port.

+1

On Thu, Oct 17, 2013 at 5:41 PM, Peter Eisentraut <peter_e(at)gmx(dot)net> wrote:
> On 10/17/13 12:45 PM, Robert Haas wrote:
>> The attached patch, which I propose to apply relatively soon if nobody
>> objects, removes the IRIX port.
>
> +1

Done. And here's a patch for removing the alpha architecture and
Tru64 UNIX (aka OSF/1) which runs on that architecture, per discussion
upthread. Barring objections, I'll apply this next week.

On a related note, I think we should update the paragaraph in
installation.sgml that begins "In general, PostgreSQL can be expected
to work on these CPU architectures". Any architecture that doesn't
have a buildfarm animal should be relegated to the second sentence,
which reads "Code support exists for ... but these architectures are
not known to have been tested recently." Similarly, I think the
following paragraph should be revised so that only operating systems
for which we have current buildfarm support are considered fully
supported. Others should be relegated to a sentence later in the
paragraph that says something like "code support exists but not tested
recently" or "expected to work but not tested regularly".

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Just to be pedantic, commit message shows
"support for Tru64 ended in 201."

I think you mean 2012.

On 18/10/2013 13:41, "Robert Haas" <robertmhaas(at)gmail(dot)com> wrote:

>On Thu, Oct 17, 2013 at 5:41 PM, Peter Eisentraut <peter_e(at)gmx(dot)net> wrote:
>> On 10/17/13 12:45 PM, Robert Haas wrote:
>>> The attached patch, which I propose to apply relatively soon if nobody
>>> objects, removes the IRIX port.
>>
>> +1
>
>Done. And here's a patch for removing the alpha architecture and
>Tru64 UNIX (aka OSF/1) which runs on that architecture, per discussion
>upthread. Barring objections, I'll apply this next week.
>
>On a related note, I think we should update the paragaraph in
>installation.sgml that begins "In general, PostgreSQL can be expected
>to work on these CPU architectures". Any architecture that doesn't
>have a buildfarm animal should be relegated to the second sentence,
>which reads "Code support exists for ... but these architectures are
>not known to have been tested recently." Similarly, I think the
>following paragraph should be revised so that only operating systems
>for which we have current buildfarm support are considered fully
>supported. Others should be relegated to a sentence later in the
>paragraph that says something like "code support exists but not tested
>recently" or "expected to work but not tested regularly".
>
>--
>Robert Haas
>EnterpriseDB: http://www.enterprisedb.com
>The Enterprise PostgreSQL Company
>
>--
>Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org)
>To make changes to your subscription:
>http://www.postgresql.org/mailpref/pgsql-hackers

On Fri, Oct 18, 2013 at 9:39 AM, Tim Kane <tim(dot)kane(at)gmail(dot)com> wrote:
> Just to be pedantic, commit message shows
> "support for Tru64 ended in 201."
>
> I think you mean 2012.

Duh, I'm a dork. Thanks.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 10/18/2013 02:41 PM, Robert Haas wrote:
> On Thu, Oct 17, 2013 at 5:41 PM, Peter Eisentraut <peter_e(at)gmx(dot)net> wrote:
>> On 10/17/13 12:45 PM, Robert Haas wrote:
>>> The attached patch, which I propose to apply relatively soon if nobody
>>> objects, removes the IRIX port.
>>
>> +1
>
> Done. And here's a patch for removing the alpha architecture and
> Tru64 UNIX (aka OSF/1) which runs on that architecture, per discussion
> upthread. Barring objections, I'll apply this next week.

hmm there are still some operating systems that "officially" support the
alpha architecture which will likely result in problems for their ports.
One example is OpenBSD both the current version (5.3) as well as the
upcoming release do fully support alpha and have binary packages and
source "ports" for postgresql and afaik they have no intention to stop
supporting that plattform.

>
> On a related note, I think we should update the paragaraph in
> installation.sgml that begins "In general, PostgreSQL can be expected
> to work on these CPU architectures". Any architecture that doesn't
> have a buildfarm animal should be relegated to the second sentence,
> which reads "Code support exists for ... but these architectures are
> not known to have been tested recently." Similarly, I think the
> following paragraph should be revised so that only operating systems
> for which we have current buildfarm support are considered fully
> supported. Others should be relegated to a sentence later in the
> paragraph that says something like "code support exists but not tested
> recently" or "expected to work but not tested regularly".

seems like an improvement to me.

Stefan

On 2013-10-18 18:24:58 +0200, Stefan Kaltenbrunner wrote:
> On 10/18/2013 02:41 PM, Robert Haas wrote:
> > On Thu, Oct 17, 2013 at 5:41 PM, Peter Eisentraut <peter_e(at)gmx(dot)net> wrote:
> >> On 10/17/13 12:45 PM, Robert Haas wrote:
> >>> The attached patch, which I propose to apply relatively soon if nobody
> >>> objects, removes the IRIX port.
> >>
> >> +1
> >
> > Done. And here's a patch for removing the alpha architecture and
> > Tru64 UNIX (aka OSF/1) which runs on that architecture, per discussion
> > upthread. Barring objections, I'll apply this next week.
>
> hmm there are still some operating systems that "officially" support the
> alpha architecture which will likely result in problems for their ports.
> One example is OpenBSD both the current version (5.3) as well as the
> upcoming release do fully support alpha and have binary packages and
> source "ports" for postgresql and afaik they have no intention to stop
> supporting that plattform.

Hm. If you read their status page (which I think you linked to before):
http://openbsd.org/alpha.html you can find stuff like X11 not
working. So I don't see that forcing us to much.
Note also that we already don't support all openbsd platforms.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On 10/18/2013 06:29 PM, Andres Freund wrote:
> On 2013-10-18 18:24:58 +0200, Stefan Kaltenbrunner wrote:
>> On 10/18/2013 02:41 PM, Robert Haas wrote:
>>> On Thu, Oct 17, 2013 at 5:41 PM, Peter Eisentraut <peter_e(at)gmx(dot)net> wrote:
>>>> On 10/17/13 12:45 PM, Robert Haas wrote:
>>>>> The attached patch, which I propose to apply relatively soon if nobody
>>>>> objects, removes the IRIX port.
>>>>
>>>> +1
>>>
>>> Done. And here's a patch for removing the alpha architecture and
>>> Tru64 UNIX (aka OSF/1) which runs on that architecture, per discussion
>>> upthread. Barring objections, I'll apply this next week.
>>
>> hmm there are still some operating systems that "officially" support the
>> alpha architecture which will likely result in problems for their ports.
>> One example is OpenBSD both the current version (5.3) as well as the
>> upcoming release do fully support alpha and have binary packages and
>> source "ports" for postgresql and afaik they have no intention to stop
>> supporting that plattform.
>
> Hm. If you read their status page (which I think you linked to before):
> http://openbsd.org/alpha.html you can find stuff like X11 not
> working. So I don't see that forcing us to much.

not sure that page is acurate and not sure how relevant X11 support is
for postgresql :)

Anyway they do currently have packages (9.2 in -stable and 9.3 in
-current) available and I think we should consider packagers here as
well - I personally don't have any particular need for alpha but it is
clearly not as dead as some of the others we are discussing.

> Note also that we already don't support all openbsd platforms.

sure - but does that also mean we should desupport without at least
considering it?

Stefan

On Thu, Oct 17, 2013 at 3:10 PM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> On Thu, Oct 17, 2013 at 12:22 AM, Noah Misch <noah(at)leadboat(dot)com> wrote:
>> Removing support for alpha is a different animal compared to removing support
>> for non-gcc MIPS and most of the others in your list. A hacker wishing to
>> restore support for another MIPS compiler would fill in the assembly code
>> blanks, probably using code right out of an architecture manual. A hacker
>> wishing to restore support for alpha would find himself auditing every
>> lock-impoverished algorithm in the backend.
>
> I had much the same thought last night. So I reverse my vote on
> Alpha: let's drop it. I had thought that perhaps there'd be some
> value in keeping it to force ourselves to consider what will happen
> under the weakest generally-understood memory model, but in fact
> that's probably a doomed effort without having the hardware available
> to test the code. As you say, any future atomics support for such a
> platform will be a major undertaking.

FWIW, I think that if we approach coding lock free algorithms
correctly - i.e. "which memory barriers can we avoid while being
safe", instead of "which memory barriers we need to add to become
safe" - then supporting Alpha isn't a huge amount of extra work. We
only need a couple of extra barriers after atomic reads where I think
we should have a comment anyway explaining that we don't need a read
barrier because a data dependency provides ordering.

In general I agree that we are unlikely to provide a bug free result
without a build farm animal, so I'm ±0 on removing support. We can try
to support, but we are unlikely to succeed. I also find it unlikely
that anyone will create a new architecture with a similarly loose
memory model. The experience with Alpha and other microprocessors
shows that the extra hardware needed for fast and strong memory
ordering guarantees more than pays for itself in performance.

Regards,
Ants Aasma

On Fri, Oct 18, 2013 at 9:55 AM, Ants Aasma <ants(at)cybertec(dot)at> wrote:
> FWIW, I think that if we approach coding lock free algorithms
> correctly - i.e. "which memory barriers can we avoid while being
> safe", instead of "which memory barriers we need to add to become
> safe" - then supporting Alpha isn't a huge amount of extra work.

Alpha is completely irrelevant, so I would not like to expend the
tiniest effort on supporting it. If there is someone using a very much
legacy architecture like this, I doubt that even they will appreciate
the ability to upgrade to the latest major version.

--
Peter Geoghegan

On 2013-10-18 18:36:03 +0200, Stefan Kaltenbrunner wrote:
> On 10/18/2013 06:29 PM, Andres Freund wrote:
> > On 2013-10-18 18:24:58 +0200, Stefan Kaltenbrunner wrote:
> >> hmm there are still some operating systems that "officially" support the
> >> alpha architecture which will likely result in problems for their ports.
> >> One example is OpenBSD both the current version (5.3) as well as the
> >> upcoming release do fully support alpha and have binary packages and
> >> source "ports" for postgresql and afaik they have no intention to stop
> >> supporting that plattform.
> >
> > Hm. If you read their status page (which I think you linked to before):
> > http://openbsd.org/alpha.html you can find stuff like X11 not
> > working. So I don't see that forcing us to much.
>
> not sure that page is acurate and not sure how relevant X11 support is
> for postgresql :)

Only in as much as it's a major piece of software missing. So removing
pg isn't exactly a shocking thing.

> Anyway they do currently have packages (9.2 in -stable and 9.3 in
> -current) available and I think we should consider packagers here as
> well - I personally don't have any particular need for alpha but it is
> clearly not as dead as some of the others we are discussing.
>
>
> > Note also that we already don't support all openbsd platforms.
>
> sure - but does that also mean we should desupport without at least
> considering it?

We should consider it, yes. But I don't see it counting much in this
case. Multiplatform OSs like (free|open|net)bsd and linux will mostly be
the last to drop support for a platform. It will very rarely be the
applications first.
And openbsd still supporting it doesn't change the fact that there
hasn't been new hardware for 10 years by the time 9.4 is going to be
released.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On Fri, Oct 18, 2013 at 8:04 PM, Peter Geoghegan <pg(at)heroku(dot)com> wrote:
> On Fri, Oct 18, 2013 at 9:55 AM, Ants Aasma <ants(at)cybertec(dot)at> wrote:
>> FWIW, I think that if we approach coding lock free algorithms
>> correctly - i.e. "which memory barriers can we avoid while being
>> safe", instead of "which memory barriers we need to add to become
>> safe" - then supporting Alpha isn't a huge amount of extra work.
>
> Alpha is completely irrelevant, so I would not like to expend the
> tiniest effort on supporting it. If there is someone using a very much
> legacy architecture like this, I doubt that even they will appreciate
> the ability to upgrade to the latest major version.

It's mostly irrelevant and I wouldn't shed a tear for Alpha support,
but I'd like to point out that it's a whole lot less irrelevant than
some of the architectures being discussed here. The latest Alpha
machines were sold only 6 years ago and supported up to 512GB of
memory with 64 1.3 GHz cores, something that can run a very reasonable
database load even today.

Regards,
Ants Aasma

On 10/17/2013 09:45 AM, Robert Haas wrote:
>>> According to http://en.wikipedia.org/wiki/UnixWare, UnixWare is not
>>> dead, although there have been no new releases in 5 years.

Gee, I wonder why?

I'll point out that SCO laid off all of its packagers three or four
years ago. So nobody is packaging PostgreSQL for Unixware anymore. In
general, I think we can reasonably expect that anyone still using
Unixware isn't upgrading PostgreSQL, so if there's any effort at all in
maintaining the port, we should dump it. The only reason we *have* the
port in the first place is that SCO created it and used to maintain it.

Oh, and +1 to dumping all of those other ports (IRIX, Tru64, Alpha),
especially Alpha which I've been told is a maintenance issue.

If we're concerned that there are users out there, the answer is to do
some blogging and see if anyone speaks up. I'll post something.

--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com