Re: [SQL] Question regarding multibyte.

[Charley Tiggs <lists(at)tiggs(dot)net>]

Karthikeyan Sundaram wrote:
> Hi,
>
>   I am new to postgres.  I asked a question regarding multibyte 
> display.  I got only one response.
> Hence, I am re-iterating the question again to a larger audience.
>
> We are using 8.2 release of postgres.  Recently we converted our 
> database to multibyte on our dev machine. we want to test the following.
>
>   1) How will I insert multibyte from our php?  Do we need to use any 
> special encoding?
>   2) I am using psql command line to get the data into a flat file and 
> push to our data warehouse
>       We use IBM redbrick warehouse.
>   3) If I have multibyte, I know the output from the psql will scatter 
> to many lines.  How can we avoid that.  What is the sql command ? I need 
> 1 record per row to supply to our warehouse.
>
>    Can somebody help me?
>
> Regards
> skarthi

http://www.php.net/mbstring
http://www.php.net/iconv

Those are the two pages that will give you more information about your 
needs.  But these aren't the only factors when sending data collected 
via php to postgres.  There's also your webserver and the user's browser 
to account for, both of which handle data according to the defaults of 
the browser.

My solution to this problem was to standardize on UTF-8 and then use 
mbstring and iconv to convert everything to UTF-8.  It's still not 
perfect as there are still characters that aren't converted properly. 
But by and large, it's extremely rare considering the amount of data 
we're dealing with.

We also chose to use PHP to pull the data out of db and write to file. 
By doing this, we were able to make sure the data was encoded correctly.

Charley