Encoding issues in console and eventlog on win32

From: Itagaki Takahiro <itagaki(dot)takahiro(at)oss(dot)ntt(dot)co(dot)jp>
To: pgsql-hackers(at)postgresql(dot)org
Subject: Encoding issues in console and eventlog on win32
Date: 2009-08-17 01:53:46
Message-ID: 20090817101222.9A23.52131E4D@oss.ntt.co.jp
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

We can choose different encodings from platform-dependent one
for database, but postgres writes serverlogs in the database encoding.
As the result, serverlogs are filled with broken characters.

The problem could occur on all platforms, however, there is a solution
for win32. Since Windows supports wide characters to write logs, we can
convert log texts => UTF-8 => UTF-16 and pass them to WriteConsoleW()
and ReportEventW().

Especially in Japan, encoding troubles on Windows are unavoidable
because postgres doesn't support Shift-JIS for database encoding,
that is the native encoding for Windows Japanese edition.

If we also want to support the same functionality on non-win32 platform,
we might need non-throwable version of pg_do_encoding_conversion():

log_message_to_write = pg_do_encoding_conversion_nothrow(
log_message_in_database_encoding,
GetDatabaseEncoding() /* as src_encoding */,
GetPlatformEncoding() /* as dst_encoding */)

and pass the result to stderr and syslog. But it requires major rewrites
of conversion functions, so I'd like to submit a solution only for win32
for now. Also, the issue is not so serious on non-win32 platforms because
we can choose UTF-8 or EUC_* on those platforms.

Regards,
---
ITAGAKI Takahiro
NTT Open Source Software Center

Attachment Content-Type Size
eventlog-20090817.patch application/octet-stream 6.7 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Greg Stark 2009-08-17 02:43:47 Re: opportunistic tuple freezing
Previous Message Jeff Davis 2009-08-17 01:32:39 opportunistic tuple freezing