Re: UTF-8 and LIKE vs =

From: Tatsuo Ishii <t-ishii(at)sra(dot)co(dot)jp>
To: barwick(at)gmail(dot)com
Cc: twanger(at)bluetwanger(dot)de, david(at)kineticode(dot)com, pgsql-general(at)postgresql(dot)org
Subject: Re: UTF-8 and LIKE vs =
Date: 2004-08-24 00:22:28
Message-ID: 20040824.092228.15271660.t-ishii@sra.co.jp
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

> >
> > В Пнд, 23.08.2004, в 23:04, David Wheeler пишет:
> > > On Aug 23, 2004, at 1:58 PM, Ian Barwick wrote:
> > >
> > > > er, the characters in "name" don't seem to match the characters in the
> > > > query - '국방비' vs. '북한의' - does that have any bearing?
> > >
> > > Yes, it means that = is doing the wrong thing!!
> >
> > The collation rules of your (and my) locale say that these strings are
> > the same:
> >
> > [markus(at)teetnang markus]$ cat > t
> > 국방비
> > 북한의
> > [markus(at)teetnang markus]$ uniq t
> > 국방비
> > [markus(at)teetnang markus]$
>
> wild speculation in need of a Korean speaker, but:
>
> ian(at)linux:~/tmp> cat j.txt
> テスト
> 환경설
> 전검색
> 웹문서
> 국방비
> 북한의
> てすと
> ian(at)linux:~/tmp> uniq j.txt
> テスト
> 환경설
> てすと
>
> All but the first and last lines are random Korean (Hangul)
> characters. Evidently our respective locales think all Hangul strings
> of the same length are identical, which is very probably not the
> case...

Locales for multibyte encodings are often broken on many platforms. I
see identical things with Japanese on Red Hat. This is one of the
reason why I tell Japanese PostgreSQL users not to enable locale while
initdb...
--
Tatsuo Ishii

In response to

Browse pgsql-general by date

  From Date Subject
Next Message Marc G. Fournier 2004-08-24 00:33:22 Re: Unsupported 3rd-party solutions (Was: Few questions
Previous Message David Wheeler 2004-08-24 00:13:03 Re: UTF-8 and LIKE vs =