Re: Careful PL/Perl Release Not Required

On Feb 11, 2011, at 10:01 AM, Alex Hunsaker wrote:

>> That *looks* like it is decoding the input string, which it is, but
>> actually that will double utf8 encode your string. It does not seem to
>> in this case because we are dealing with all ascii input. The trick
>> here is its also telling perl to decode/treat the *output* string as
>> utf8.
>
> Urp, this is a bit of a fib. The problem is actual in plperl not perl
> persay. Pre 9.1 we always fetched perls internal string *ignoring* the
> utf8 flag. So if you had octets that were utf8 things would work.

In 9.0 in a utf-8 database, the utf8 flag is turned on.

> The
> utf8::decode($_[0]); uri_unescape($_[0]); happened to make the return
> string internally be utf8 and so it would only return 1 char. Thats
> what the op wanted and why it seemed to fix his problem. But thats
> actually a bug! utf8::decode($_[0]) should not have changed anything
> at all on the output side. It should still have returned 2 characters
> instead of 1.

I don't understand where the bug is. If a string is encoded in utf-8 Perl will not treat it as such unless the utf-8 flag is set.

Best,

David