Re: Git-master regression failure

Hi All.

I just subscribed to RRReviewers (that should be pronounce with a nice
rolling r-r-reviewers, right?)

As part of my getting up to speed, I tried to build and run test on the
current master 073d7cb513f5de44530f4bdbaaa4b5d4cce5f984

Basically I did:
1) Clone into new dir
2) ./configure --enable-debug --enable-cassert --with-pgport=5499
--prefix=$(realpath ../root)
3) make -j4
4) maje -j4 check

And expecting in 4) to get "all test passed", I was surprised to see
that an index-test failed.

I run (Gentoo) Linux x86_64, gcc-4.7.3.

Any ideas what might have happened?

Svenne

Svenne Krap <svenne(dot)lists(at)krap(dot)dk> wrote:

> current master 073d7cb513f5de44530f4bdbaaa4b5d4cce5f984

> I was surprised to see that an index-test failed.

It works for me.  Could you paste or attach some detail?

--
Kevin Grittner
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 18-06-2013 18:40, Svenne Krap wrote:
> Any ideas what might have happened?

After doing some more digging...

My laptop (which runs PostgreSQL 9.2.4 on x86_64-pc-linux-gnu, compiled
by x86_64-pc-linux-gnu-gcc (Gentoo 4.7.3 p1.0, pie-0.5.5) 4.7.3,
64-bit) also returns "99", if I

- - run the CREATE TABLE tenk1 (from the git-master)
- - load data from tenk.data (from git-master)
- - run the "offending part" of the create_index.sql (also from git-master):

The offending offending_part is:
CREATE TABLE dupindexcols
AS
SELECT unique1 as id, stringu2::text as f1 FROM
tenk1;
CREATE INDEX dupindexcols_i ON dupindexcols (f1, id, f1
text_pattern_ops);
ANALYZE
dupindexcols;

EXPLAIN (COSTS
OFF)

SELECT count(*) FROM
dupindexcols
WHERE f1 > 'WA' and id < 1000 and f1 ~<~
'YX';
SELECT count(*) FROM
dupindexcols
WHERE f1 > 'WA' and id < 1000 and f1 ~<~ 'YX';

As I have no real idea of what "~<~" is for an operator (I have looked
it up as scalarltjoinsel), but I cannot find any semantics for it in the
docs*... So I have no way of manually checking the expected result.

*=The term ~<~ is not exactly google-friendly and the docs site's search
also returns empty...

Anyone has any idea what to look after next?

Svenne
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.20 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQI5BAEBCAAjBQJRwKTQHBpodHRwOi8vc3Zlbm5lLmRrL3BncC9wb2xpY3kACgkQ
/zLSj+olL/KGHQ//d5WM2gi4yIaMCpaij1+PdCzBwyMwz0E3Ys4mlj1x7x22v8ty
6Gs6HZlofuuUdBkhDvRYYvbJ8asdi51KbjVMbrBkR49txiM00cWLtT74e9mFdTaA
f5lOeUmfyqy7O9jlyFvKaC8hVRe7yQrbRK+b2sOBIq2TchVkvjT/AvvDcuPWCKD7
6FgHOtFF4Ae4yqC6foylfBQ59TpOYHBd+ohl38egtDr87tXSXcuIU1bNZeJfqcPM
rs02h8L/kmWrx32TmYKLUw6PdluPEJ2WB5Gcsd8Va5p7ai3+rbqzzJGw9ZOFkPv4
9ruTqjTV0l7xhwuztHpSCTiyOoV0dIw13USTO7D/rAseqIUgb0mQE4cXVQW9GVT1
WjCqnaqRwdgYnvke7jtOfX2AeiZjluZfF1e2B2rVMfDan1A5PxjLmxpzH7fYs0bs
RDEnOx9bcVUYH6xedVyYHbSmx7GXyzu0xfADsWgsYNclv91cZ5pz9+fwJv8ceNNz
ckOcA4DlKeo1fw0idvbEUtVWf/tO8H6r0heNuMgR5M6qOtlbsLsdsJvOIUTzGn7W
oxikvMAKbAX0Hl4arQSQE5Og7HcLsEB6YaMp0bpj55Y4prr/fhzJHKL4ioCI6NOH
gLknu+IPDyJiNez4QqaqAITDtxo+68euv5uUE+O55o9ssBEWoTX0bz8trM8=
=2IBo
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 18-06-2013 20:17, Kevin Grittner wrote:
>
>> I was surprised to see that an index-test failed.
>
> It works for me. Could you paste or attach some detail?
>
>
Gladly, if you tell me what would be relevant to attach :)

I am brand new to the postgresql source code and hence have no real idea
how to catch it..

Svenne
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.20 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQI5BAEBCAAjBQJRwKjRHBpodHRwOi8vc3Zlbm5lLmRrL3BncC9wb2xpY3kACgkQ
/zLSj+olL/LIOQ/9E7PsAosZbQemCKeHj3/61kM1m3WMD9YtjZE4OdAJjEM+YCpE
HkMheIpsIBo9rV1mH0mNt/rog0GZupGA9xWo+OsRPbQ3bd089ny++eSmGaJ4+nb3
cZiJpGizgQ+8xrMFLtkyt72WFUWashhb6IdokC1WeSrOWOcXGad+Sl48nxkODetR
WJHuq0cdzLkivLSXBE3I5a7h28QWAwUHRlOUFNLaVTbVtPiv9Rx8YAnzYCvr3HC1
y+tdrHxONjoM4moLJH8qRQs6nMTCq+3mJgaKjW8FPCf5is5IMZLoACvjf0TOZm9o
NnGv/R0TwLVjA/w44qMNJ4kXoX49IDX1IaYBvOraAjjWr+ggC2bmXwPZWqEHAZTG
OCLD/t7DONW7uQEXEZSyur1N3CIZ231jl/ufYSefXaV81mtBVF6w7EVLi56UsC1A
C626RY0r/87P9Y0avG1oh3ba6hOZhcv2R8GZZKkO7LNGUMSbIm90+FcV3bv/YSKi
H5T3nYM0GWJ6kbqt+Mynqy9Q2N/33/rcpAL3ut0wIRkGALuIrb5zowi52y95+zAX
ePpMFbM8I8nHuM0ouvcABHXtr6r5m6c1qwfJNbNQuSFz1T3s4/SY2W1IsinLSSKT
/bqxP4qJxONDrIv9W4ZpIkxLYyfTHTuD3jP7GzndKcGHtHzMrTqxf1RGBs0=
=hhm0
-----END PGP SIGNATURE-----

Svenne Krap <svenne(dot)lists(at)krap(dot)dk> wrote:
> On 18-06-2013 20:17, Kevin Grittner wrote:
>>
>>> I was surprised to see that an index-test failed.
>>
>> It works for me.  Could you paste or attach some detail?
>
> Gladly, if you tell me what would be relevant to attach :)
>
> I am brand new to the postgresql source code and hence have no
> real idea how to catch it..

Apologies; I somehow missed the file attached to your initial post.
That's the sort of thing I was looking for.

Having reviewed that, the source code comments indicate it is for
"character-by-character (not collation order) comparison operators
for character types".  Perhaps your collation is having an impact
regardless of that.  Could you show us the values of your settings
related to locale?

--
Kevin Grittner
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 18-06-2013 20:48, Kevin Grittner wrote:
>
> Apologies; I somehow missed the file attached to your initial post.
> That's the sort of thing I was looking for.
Aplogy accepted... :)
> Having reviewed that, the source code comments indicate it is for
> "character-by-character (not collation order) comparison operators
> for character types". Perhaps your collation is having an impact
> regardless of that. Could you show us the values of your settings
> related to locale?

I am not entirely sure what you mean by settings related to locale ...
but here is the system locale output and the definition of my main
instance (the 9.2 one I wrote about in the second test)...I don't know
if/how to extract the same information from the temp instance made by
"make check".

Btw. could you point a newbie to where you found the function (so that I
know that for later)?

If you need other data, please be more specific about how I get them :)

# locale

LANG=da_DK.UTF8

LC_CTYPE="da_DK.UTF8"

LC_NUMERIC="da_DK.UTF8"

LC_TIME="da_DK.UTF8"

LC_COLLATE="da_DK.UTF8"

LC_MONETARY="da_DK.UTF8"

LC_MESSAGES=en_US.UTF8

LC_PAPER="da_DK.UTF8"

LC_NAME="da_DK.UTF8"

LC_ADDRESS="da_DK.UTF8"

LC_TELEPHONE="da_DK.UTF8"

LC_MEASUREMENT="da_DK.UTF8"

LC_IDENTIFICATION="da_DK.UTF8"

LC_ALL=

(sk(at)[local]:5432) [sk] > \l

List of databases

Name | Owner | Encoding | Collate | Ctype | Access
privileges

-
-------------+----------+----------+------------+------------+-----------------------

postgres | postgres | UTF8 | da_DK.UTF8 | da_DK.UTF8 |

root | root | UTF8 | da_DK.UTF8 | da_DK.UTF8 |

sk | sk | UTF8 | da_DK.UTF8 | da_DK.UTF8 |

template0 | postgres | UTF8 | da_DK.UTF8 | da_DK.UTF8 |
=c/postgres +

| | | | |
postgres=CTc/postgres

template1 | postgres | UTF8 | da_DK.UTF8 | da_DK.UTF8 |
=c/postgres +

| | | | |
postgres=CTc/postgres

(5 rows)

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.20 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQI5BAEBCAAjBQJRwK87HBpodHRwOi8vc3Zlbm5lLmRrL3BncC9wb2xpY3kACgkQ
/zLSj+olL/JVBQ//UQfke7jc2mcaTV9lvdU8b5B/SnGzsn8d2M2mDPYjcgjmuAmT
pacRemmiVf8w6FTKlmz+A7faYVMuFSjJXi25OzvWu8on+zFtstdHWBbMNYI1CuBP
itSqs+JVYF8FvkAa+iC9KilV1McRWCOxnlztuN7F29lzpnrcWsGmvhiYL3qVt3TT
jyJNTdVIOFGta4f4Kzkq9ZqH6ry42UrXRFa1pFRxfY42zX7LV3BsJuEMQeHnRYjz
fkcivuwJJHdn9l6GGZbF4+s227SFOO98GiPah10n5GKzy2saAMntos1p15l5Qy/c
TaHE5cp0fttaQMbQEMWB3fY6r3NETxnsDyh6z7EaLV/WHhmzjrUGDp2gCgWbVs5C
nvoFL9SkcfzZsuWvy21zmIHOvzAUMm9p57/i5REWqWc2ZH5giLKC71m7skiu/utF
9fmpNEh1U+DG3/VnJjcIyfNq4y3pmkNN063/aLKX21vvBHJ9/oqgL7czYRCBZH6L
ufNL0ACZ/zeDTL16K58/gMhf7Si9jAZq7PcobIgSJHKo++7DevgUiSRmbxQ6jV6X
ZXseeQJT7Dcf3yVNqNkPJApSiCbR4e0wbgbSPXXkJEjZpo+k3wo3gZI6fEULifdh
i/hcLpq+nPrME2l9gcneJ8gNHpBOK4AyfBvfhxOIV35pCpYrBvk3toKPUEc=
=bUcn
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 18-06-2013 21:04, Svenne Krap wrote:
> (sk(at)[local]:5432) [sk] > \l
>
> List of databases
>
> Name | Owner | Encoding | Collate | Ctype | Access
> privileges
>
> -

Arghh... crappy mailer... I have the information attached here instead...

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.20 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQI5BAEBCAAjBQJRwLACHBpodHRwOi8vc3Zlbm5lLmRrL3BncC9wb2xpY3kACgkQ
/zLSj+olL/LJQw//avuPNPvG9sTy+itx9pmdRBHrdWCER4TXyKu15/0cphigMCHo
mLNuLfcdwEbuppaqZgrEKP9GicppExmnGHGdgwn60CQiavki9HZpxuH+FX/+DRI6
IEM7Ml6PZw+8g+Uvh26vprgBhFw2B2xxx9uDEN5YEmpjJPqgxh8SXIs7Sy14FmEw
uivU+YVLdoqLGUfSO3kng/ypTwqWC7vdycvZM6DtJdJLeZf+444kpX4TlVAncYtG
LKRm7ym+9sUY5Tkq/cKNS/hM93fnrGGyD0bg6GtTHUKq0S0Cw79FcvCgWBNu0fOe
QU6AtNIslvpxarECN2fkJ02S8pf0lnrDm8l6ZExO3BgY/Fo0wSPo7rhtdFEgPB6d
uPy4YL9ezRfY7gkXWEgIX1zNA0h6vVcNi5OqbyC5WHenvI9mQPiwgwvbXTqCQmEF
SKC6PoBOaJXaCajyXGVSjk+0IC/QYSMsqoWe1CcySFahfYFxKdHbkImQ02N+PB3x
9ABBuG2QUHTmh09MsCtr/l+McqlLVCPRngV5wYz5Z7guKu1lS6yzaAj6VHxVaw1A
wCmVtJcCFNv/WhbarLOJOMsPjrQ5EXIOuM+k61nznooaqnBtChQTT1/ddzx+G3vU
0lX59b7tAM9+z2q+VCj57L6DgspCODMwHucv2MYkInS++ta0QV0U5ebzw/s=
=/PiO
-----END PGP SIGNATURE-----

On Tue, Jun 18, 2013 at 11:20 AM, Svenne Krap <svenne(dot)lists(at)krap(dot)dk> wrote:

>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
>
>
> On 18-06-2013 18:40, Svenne Krap wrote:
> > Any ideas what might have happened?
>
> After doing some more digging...
>
> My laptop (which runs PostgreSQL 9.2.4 on x86_64-pc-linux-gnu, compiled
> by x86_64-pc-linux-gnu-gcc (Gentoo 4.7.3 p1.0, pie-0.5.5) 4.7.3,
> 64-bit) also returns "99", if I
>
> - - run the CREATE TABLE tenk1 (from the git-master)
> - - load data from tenk.data (from git-master)
> - - run the "offending part" of the create_index.sql (also from
> git-master):
>

But 9.2.4 does pass "make check", and only fails if you reproduce those
things manually?

If so, I'm guessing that you have some language/locale settings that "make
check" neutralizes in 9.2.4, but that neutralization is broken in HEAD.

> As I have no real idea of what "~<~" is for an operator (I have looked
> it up as scalarltjoinsel), but I cannot find any semantics for it in the
> docs*... So I have no way of manually checking the expected result.
>

Yes, it does seem to be entirely undocumented. Using:
git grep '~<~', I found the code comment "character-by-character (not
collation order) comparison operators for character types"

Anyway, if REL9_2_4 passes make check, but 073d7cb513f5de44530f fails, then
you could use "git bisect" to find the exact commit that broke things.

Cheers,

Jeff

Jeff Janes escribió:
> On Tue, Jun 18, 2013 at 11:20 AM, Svenne Krap <svenne(dot)lists(at)krap(dot)dk> wrote:

> > As I have no real idea of what "~<~" is for an operator (I have looked
> > it up as scalarltjoinsel), but I cannot find any semantics for it in the
> > docs*... So I have no way of manually checking the expected result.
>
> Yes, it does seem to be entirely undocumented. Using:
> git grep '~<~', I found the code comment "character-by-character (not
> collation order) comparison operators for character types"

To look up an operator you can search in pg_operator.h (where you'd also
see the DESCR() line nearby containing a description) or the pg_operator
catalog. The pg_operator row contains a reference to the pg_proc entry
that implements the operator. The pg_proc row, in turn, refers to a
(typically) C-language function that implements the function. Normally
looking at the function and surrounding code you can figure out what the
operator is about.

In this case you should probably be able to find the operator referenced
in pg_amop as well, as part of the "*_pattern_ops" opclasses.

--
Álvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

Svenne Krap <svenne(dot)lists(at)krap(dot)dk> wrote:

> I have the information attached here instead...

I find it suspicious that the test is using an index which sorts
first by the "f1" column, then later by "f1 text_pattern_ops"
column.  I'm not 100% sure whether the test is bad or you have
found a bug, although I suspect the latter.  The actual result
should not depend on the index definition; the index should only
affect performance and possibly the order of results where order is
not specified.

--
Kevin Grittner
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 18-06-2013 21:14, Jeff Janes wrote:
>
> But 9.2.4 does pass "make check", and only fails if you reproduce
those things manually?
>
No, I was lazy and used the (distribution-installed) 9.2....

I have tried "make check" on REL_9_2_4 and that fails to (same sole
failure)...

> If so, I'm guessing that you have some language/locale settings that "make check" neutralizes in
9.2.4, but that neutralization is broken in HEAD.
>
Nope, just never ran "make check" on it...
>
>
> As I have no real idea of what "~<~" is for an operator (I have looked
> it up as scalarltjoinsel), but I cannot find any semantics for it
in the
> docs*... So I have no way of manually checking the expected result.
>
>
>
> Yes, it does seem to be entirely undocumented. Using:
> git grep '~<~', I found the code comment "character-by-character (not
collation order) comparison operators for character types"
>
> Anyway, if REL9_2_4 passes make check, but 073d7cb513f5de44530f fails,
then you could use "git bisect" to find the exact commit that broke things.
>
It does not..

I will dig futher and get back...

Svenne
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.20 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQI5BAEBCAAjBQJRwLfnHBpodHRwOi8vc3Zlbm5lLmRrL3BncC9wb2xpY3kACgkQ
/zLSj+olL/J4oA/7Blq1eQarny6VVu7UUgtN+CWvy3zoiE36AlE/vEvzb4aXWP4c
5Mq8z3vkHrjKCdaioGYLrkEE2PTom+pBV5cK+cT67TAN1J/OJErtRuKhVLk2Bd8I
ry/gDgF8dvM0Sx03eN52ViCOjFUmksk+HXOSk8z0aPBdanbOdIQoSC25XJ2Yye8K
DP+yY6h9+PxMomMzaz3aK0MechZaJyvbLpStjJEd/nuAiDIymzpzxJfyjpn/jKra
6GVdRfQCMkqiBy5E4YFgxDkGfDJLFtwMJPMw8OAe+Zhgp8YLPSms58bwtdmwcPYD
NSfowZa/yaKpLP3k6J/2JVToa+JBPY/vYGlxdU+h15VmsDV8kxCWnRwxC/gYYg9l
7CqWPZskd9ZAonfb781JxoiVmoGbGzFCLE1wlPVazCZsAzQcvVTJD8yl/Ibv8PKv
7LjxuItm3iW8GAp93x7+MGmxKx/YKVu8lfAFTw2/X/G2MfoiJHmAUaklqdATerXl
xarQOL0UbiRGQNYSBHjKVxZHQrgAShPS2O2cSw6MijaDsr4US+pcY+S0F0KmtnD/
mobI3vSFFsZxnaIUAL4ExQuyF0PgGm7Ce4jmQxsUx7Ph/ud6AZVLhKStYBcwChPH
W+4Laq2fNB6/deH81x+x+j0QSk1oyvSVG+73/bmURWSP9WuhikYJ5kqbOdk=
=siUv
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 18-06-2013 21:41, Svenne Krap wrote:
>
>
>
> I will dig futher and get back...

The regression test was added in 9.2, the earliest interesting commit is
d6d5f67b5b98b1685f9158e9d00a726afb2ae789,
where Tom Lane changes the definition to the current.

It still fails (which suggests that it has always and will always fail
on my setup....)

I am happy to run whatever relevant tests you can dream up, but I am
fresh out of ideas :)

Svenne
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.20 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQI5BAEBCAAjBQJRwLo5HBpodHRwOi8vc3Zlbm5lLmRrL3BncC9wb2xpY3kACgkQ
/zLSj+olL/JH5w/+PRU8Ghr56RuOcybp3S86Ig8sw4y56Xu8pb4RMHXNTJcL1zk0
fj0NRr+0sCAR9JXIS1mGHnNZuLpdppLklb0Zi02JeNbXelnY34QSiqgn3c9ikBIZ
NTq63MVHj2bOC2YkpJIjCCZI7adjyOYm0Fosm453VV6lSh7PooNLUfqTlGA9lHKo
lItndci7ei+JTCFr2G3zv2zshwNW87+nd9oAzf+n0Hz3djswrJWvdby56UtTb5S0
6ayYH14BnF0UU2C1Ko5/JSBgDyzT/kkPyn/YEplcHkO3yYKXgaYdanTyQEgYbL5I
kSebe6eh271IY4W7wt+Bb5202HDOU6d3ikCJykE8G9BHw77exUbk27BJWipcUNL0
63IERJAleJ29yOqVZggZ0p3hf7H34lFqLKCTi7+p4mP1ZLlMlaMzbTGMdVJuc21A
UyqQBY7nv9m7gIiDxXEQaoFbycn7Lb14xYgUOYIYGmnTblg4A5oUREXkMa08vWZg
yq+oWOmusmbB3EET0jvQCQe9ves9J4Pa+5Axry2c4tbPQD2PH25FZGyAeCwJLon9
ZmMdFf3uNp91f+YW/BPDgXxi9A+fn1twI0ldMUYuX6MTopWvMrVHzpqQ3MJxPKLb
RdWgKHWXZnu0ygw1F/AqpEgx2Y/SAyyuOId1cmamG8BXJsRZVgezqK+roww=
=TzBa
-----END PGP SIGNATURE-----

Svenne Krap <svenne(dot)lists(at)krap(dot)dk> wrote:

> I am happy to run whatever relevant tests you can dream up, but I am
> fresh out of ideas :)

psql regression
begin;
drop index dupindexcols_i;
SELECT count(*) FROM dupindexcols
  WHERE f1 > 'WA' and id < 1000 and f1 ~<~ 'YX';
rollback;
select f1 from dupindexcols where f1 like 'W%' ORDER BY f1;

What are the results?

--
Kevin Grittner
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Tue, Jun 18, 2013 at 12:51 PM, Svenne Krap <svenne(dot)lists(at)krap(dot)dk> wrote:

>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> On 18-06-2013 21:41, Svenne Krap wrote:
> >
> >
> >
> > I will dig futher and get back...
>
> The regression test was added in 9.2, the earliest interesting commit is
> d6d5f67b5b98b1685f9158e9d00a726afb2ae789,
> where Tom Lane changes the definition to the current.
>

I get it back to e2c2c2e8b1df7dfdb01e7, where the ability to have one index
with the same column twice appears.

The problem is the f1 > 'WA' part of the query. In Danish, apparently 'AA'
> 'WA', so two more rows show up.

SELECT f1 FROM
dupindexcols
WHERE f1 > 'WA' and id < 1000 and f1 ~<~ 'YX' except
SELECT f1 FROM
dupindexcols
WHERE f1 ~>~ 'WA' and id < 1000 and f1 ~<~ 'YX';

f1
--------
AANAAA
AAMAAA

I don't know how important it is to the community for make check to pass
under every possible LANG setting, or the best way to go about fixing it if
it is important.

Cheers,

Jeff

Jeff Janes <jeff(dot)janes(at)gmail(dot)com> wrote:

> The problem is the f1 > 'WA' part of the query.  In Danish,
> apparently 'AA' > 'WA', so two more rows show up.

Thanks -- I didn't have the right locale installed, and wasn't
quite sure what package to install to get it.

So, the test is bad, rather than there being a production bug.

> I don't know how important it is to the community for make check
> to pass under every possible LANG setting, or the best way to go
> about fixing it if it is important.

We should probably tweak the test to not use a range which fails in
any known locale.

--
Kevin Grittner
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 18-06-2013 22:18, Jeff Janes wrote:
> In Danish, apparently 'AA' > 'WA', so two more rows show up.

Yes of course....

We have three extra vowels following Z (namely Æ, Ø and Å) and for
keyboard missing those essential keys we have an official alternate way
to write them as AE , OE and AA.

Which of course means that AA is larger than any other letter ;)

Nice find :)

Svenne
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.20 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQI5BAEBCAAjBQJRwUjzHBpodHRwOi8vc3Zlbm5lLmRrL3BncC9wb2xpY3kACgkQ
/zLSj+olL/LfqxAAqZ6Qc9uC7lZ/gE/ZT5RjMqEudQvHvRxGiUQLcLdif/n2novf
yf3Phe7gJOMPgt7V47dT5Wu/l3/PunV6Yo2Q1Ifg+XsrvUXC4CDMLo/RDP97EUwZ
B96o1UH2Tx1+GnaJrZkcjq5V8y+4X5QJ6+vP11SEUyuMl7Ist2EegMet55N3ndTe
W/mjZr5uTpmEyBr1aMJo6RJ3T1gxPLelc2b4aA5jnQlO7RbXlCgtWUQzQ/Q/dBWb
kuE0MEd9CSR4kuo9u6yImVKUMUdX+NGgOeD6D36UtS4pdvnWOI+gX95Mze4AfiY/
ZPt6mCv0afBfQW/uioctHDLScY9v6cT58dkzrLjWyuNd6CfvqBiWfXBygwUclDZK
cWpuszwIjVaOnHiDlHK8sg7yXU+FkhV5V8PXz6KBulCPxmR+Z8oI0AKiU2MhRIe7
mQhgDHgHT6nBpqWGlmX3VAjiTL6+/DZT34TnxNqdLDtWKrHUK+2KzZ/kYJ97zwo4
C78UX6F+1/6JxOkMVjm77M1ZQx306pJC8YGS3+7/fRh1EZd3LQXPaOXCU2EtM6UL
sBaafDB5yMEZsdNw3DKOLnbg/bTq9MPa2uBZdEJ1w8LsvNrSeOohTXZFn2PL2zoJ
L8JnJcAndM3SJG+UO1K1ALJhZlD0oRovYnzolAlKCXavNTn+zBd2uyNYSIQ=
=qC25
-----END PGP SIGNATURE-----

Svenne Krap <svenne(dot)lists(at)krap(dot)dk> wrote:
> On 18-06-2013 22:18, Jeff Janes wrote:

>>   In Danish, apparently 'AA' > 'WA', so two more rows show up.
>
> Yes of course....
>
> We have three extra vowels following Z (namely Æ, Ø and Å) and
> for keyboard missing those essential keys we have an official
> alternate way to write them as AE , OE and AA.
>
> Which of course means that AA is larger than any other letter ;)

Does anyone object to the attached change, so that regression tests
pass when run in a Danish locale?  I think it should be
back-patched to 9.2, where the test was introduced.

--
Kevin Grittner
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 2013-06-19 06:18:20 -0700, Kevin Grittner wrote:
> Svenne Krap <svenne(dot)lists(at)krap(dot)dk> wrote:
> > On 18-06-2013 22:18, Jeff Janes wrote:
>
> >>   In Danish, apparently 'AA' > 'WA', so two more rows show up.
> >
> > Yes of course....
> >
> > We have three extra vowels following Z (namely Æ, Ø and Å) and
> > for keyboard missing those essential keys we have an official
> > alternate way to write them as AE , OE and AA.
> >
> > Which of course means that AA is larger than any other letter ;)
>
> Does anyone object to the attached change, so that regression tests
> pass when run in a Danish locale?  I think it should be
> back-patched to 9.2, where the test was introduced.

Don't we actually run make check/standard pg_regress with an enforced C
locale? In which case this would imply some bigger problem we probably
don't want to hide.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On 2013-06-19 15:23:16 +0200, Andres Freund wrote:
> On 2013-06-19 06:18:20 -0700, Kevin Grittner wrote:
> > Svenne Krap <svenne(dot)lists(at)krap(dot)dk> wrote:
> > > On 18-06-2013 22:18, Jeff Janes wrote:
> >
> > >>   In Danish, apparently 'AA' > 'WA', so two more rows show up.
> > >
> > > Yes of course....
> > >
> > > We have three extra vowels following Z (namely Æ, Ø and Å) and
> > > for keyboard missing those essential keys we have an official
> > > alternate way to write them as AE , OE and AA.
> > >
> > > Which of course means that AA is larger than any other letter ;)
> >
> > Does anyone object to the attached change, so that regression tests
> > pass when run in a Danish locale?  I think it should be
> > back-patched to 9.2, where the test was introduced.
>
> Don't we actually run make check/standard pg_regress with an enforced C
> locale? In which case this would imply some bigger problem we probably
> don't want to hide.

Misremembered, we only do that optionally. So yes, seems sensible.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On 6/19/13 9:18 AM, Kevin Grittner wrote:
> Svenne Krap <svenne(dot)lists(at)krap(dot)dk> wrote:
>> On 18-06-2013 22:18, Jeff Janes wrote:
>
>>> In Danish, apparently 'AA' > 'WA', so two more rows show up.
>>
>> Yes of course....
>>
>> We have three extra vowels following Z (namely Æ, Ø and Å) and
>> for keyboard missing those essential keys we have an official
>> alternate way to write them as AE , OE and AA.
>>
>> Which of course means that AA is larger than any other letter ;)
>
> Does anyone object to the attached change, so that regression tests
> pass when run in a Danish locale? I think it should be
> back-patched to 9.2, where the test was introduced.

Yes, that should be fixed. I wouldn't put in the comment, though. A
few releases ago, I fixed a number of other "Danish" issues, so adding
this comment would give the impression that this the only place.

Peter Eisentraut <peter_e(at)gmx(dot)net> wrote:
> On 6/19/13 9:18 AM, Kevin Grittner wrote:

>> Does anyone object to the attached change, so that regression tests
>> pass when run in a Danish locale?  I think it should be
>> back-patched to 9.2, where the test was introduced.
>
> Yes, that should be fixed.  I wouldn't put in the comment, though.  A
> few releases ago, I fixed a number of other "Danish" issues, so adding
> this comment would give the impression that this the only place.

OK, pushed without the comment.

--
Kevin Grittner
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 19-06-2013 17:41, Kevin Grittner wrote:
>
> OK, pushed without the comment.
>
Works like a charm :)

Svenne
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.20 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQI5BAEBCAAjBQJRwhpmHBpodHRwOi8vc3Zlbm5lLmRrL3BncC9wb2xpY3kACgkQ
/zLSj+olL/LBzRAAje+BsZ6eGbNEXwPWPob0sJwAD+51UNT24DMHCLAg4T38Bheo
HaT+xSQm3hlIIjL/JF5BLiRFG4Fz1+GGOGbykKRW1Fm+VWzw6934lajbh/Z7wqzO
T+n0XXwhbMnA4O++FvIJ67C7iazsNbAobOmqwjPe+6D6ID8VR9rNB/NEwelueR3L
dVmVrSg1Mc8DVHS2h7Aq4bJaeCJHOQstB89+AEa14cwsJ25PiQFcdbUWsNNqbnoE
TN3jQlXZLAl/qCzdKQ7FRAsi7Yk1r9uU7g1HNSY7ggh+W5KmRNqX99LS4v5mO6rf
XquzXNf6nzl3ngfucB+60C2NloqieYKGVbzDh7LjCThbv284/fdlW+Wu80vYGcto
TWIb5Ku4fbKP0Hmcuby1VlcYyZhyiBa76zV7JF5AtMe9yfCpOdmyAUdHDQUq6T6c
/BVV/bCVsDAhJN/AJQC2itNnt4G+Bw62YEQCwuw1cNT5BUzjf0mbcer5kRchkNr3
lBdTEp4HBzElBDnCY7oSre+aSKomgNZwyJsps2aLuyz8nTj4WxpDQTDo1m7Kyi7K
dFGRZLJRy5FS0xNrVtBW1cfoYy8B2GM9cmcZru5bFo2XAJfJaSSl4EZLdlIk5k66
k+VkwS5XoIl7oR/qT1fSwERM8TfDS2An2rr+jvVAATm2QQsKL42AN7aSROk=
=2COi
-----END PGP SIGNATURE-----

On Wed, Jun 19, 2013 at 8:41 AM, Kevin Grittner <kgrittn(at)ymail(dot)com> wrote:

> Peter Eisentraut <peter_e(at)gmx(dot)net> wrote:
> > On 6/19/13 9:18 AM, Kevin Grittner wrote:
>
> >> Does anyone object to the attached change, so that regression tests
> >> pass when run in a Danish locale? I think it should be
> >> back-patched to 9.2, where the test was introduced.
> >
> > Yes, that should be fixed. I wouldn't put in the comment, though. A
> > few releases ago, I fixed a number of other "Danish" issues, so adding
> > this comment would give the impression that this the only place.
>
> OK, pushed without the comment.
>

I had started this and let it run overnight:

for LANG in `locale -a`; do make check >& /dev/null ; echo $? $LANG; done

Of the 735 language/locales/encodings, I got 93 failures. After your
commit I re-tested just the failures, and it fixed 25 of them.

Of the ones I looked at, most of the problems are in create_index, some in
matview as well.

Lithuanian has Y coming between I and J. Estonian has Z between S and T.
Norwegian seems to treat V and W as being equal except to break
suffix-ties.

Is there an infrastructure to use a different expected file depending on
the LANG used?

Cheers,

Jeff

Jeff Janes <jeff(dot)janes(at)gmail(dot)com> wrote:
> Kevin Grittner <kgrittn(at)ymail(dot)com> wrote:
>> Peter Eisentraut <peter_e(at)gmx(dot)net> wrote:
>>> On 6/19/13 9:18 AM, Kevin Grittner wrote:
>>
>>
>>>> Does anyone object to the attached change, so that regression tests
>>>> pass when run in a Danish locale?  I think it should be
>>>> back-patched to 9.2, where the test was introduced.
>>>
>>> Yes, that should be fixed.  I wouldn't put in the comment, though.  A
>>> few releases ago, I fixed a number of other "Danish" issues, so adding
>>> this comment would give the impression that this the only place.
>>
>> OK, pushed without the comment.
>
> I had started this and let it run overnight:
>
> for LANG in `locale -a`; do make check >& /dev/null ; echo $? $LANG; done
>
> Of the 735 language/locales/encodings, I got 93 failures.

Ouch!

> After your commit I re-tested just the failures, and it fixed 25
> of them.

That's more than I would have guessed.  Cool.

> Of the ones I looked at, most of the problems are in
> create_index, some in matview as well.

So of the 68 remaining locales which fail, most are due to a couple
scripts.

> Lithuanian has Y coming between I and J.
> Estonian has Z between S and T.
> Norwegian seems to treat V and W as being equal except to break suffix-ties.

> Is there an infrastructure to use a different expected file
> depending on the LANG used?

Well, any one test can have alternative "expected" scripts; but in
previous discussions we decided that that facility should not be
used for locale issues.  It would tend to get into multiplicative
permutations with other reasons to have alternatives.  What we have
done is to try to create tests that don't hit those edge conditions
when we know of them.

Could you share your detailed information on the remaining failures?

--
Kevin Grittner
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Wed, 2013-06-19 at 14:50 -0700, Jeff Janes wrote:
> Is there an infrastructure to use a different expected file depending
> on the LANG used?

Not really. A couple of years ago I did the same exercise you just did,
and we just fixed most of what was reasonable to fix by adjusting the
test cases.