topic Re: SIM Relay Response data truncates at non-ASCII characters in Integration and Testing

SIM Relay Response data truncates at non-ASCII characters

bobbymray — Tue, 12 Mar 2013 21:34:50 GMT

I've been using SIM with Relay Response for over a year without any problems. I update a table with customer info from the response. Recently I noticed that if a response field contains non-ASCII characters, the data is truncated right before the non-ASCII character, so if the data is 'Rhône-Alpes', only 'Rh' gets put in the table.

Actually, I'm not sure if the Relay Response is complete. Perhaps it doesn't contain the non-ASCII characters.

The table columns use utf8 for the character set, and I'm using array(PDO::MYSQL_ATTR_INIT_COMMAND => 'SET NAMES utf8') in my PHP PDO database connection.

I've tested the table update with several form posts that includes lots of characters from all over the Unicode table. It works fine.

Should I be encoding or decoding the Relay Response POST data? I'm stuck.

Thanks.

Re: SIM Relay Response data truncates at non-ASCII characters

RaynorC1emen7 — Tue, 12 Mar 2013 23:54:54 GMT

Can't help you with the database question.

But for the relay response, I try it with C# as x_first_name and as a merchant define field by sending it to

https://developer.authorize.net/tools/paramdump/index.php

and it return the data correctly.

Re: SIM Relay Response data truncates at non-ASCII characters

bobbymray — Thu, 14 Mar 2013 00:05:49 GMT

Thank you.

Neither C nor # are outside the range of the 128 ASCII characters, so your example hasn't quite tested for the problem I'm having.

Are the values displayed by the paramdump tool received as POST values?

Is it possible, for whatever reason, that the POST values prepared for the Relay Response are mangled if they contain characters from the upper range of Unicode? Sorry, my knowledge of character encoding is negligible, so I can't ask the question with any more precision than that.

Re: SIM Relay Response data truncates at non-ASCII characters

RaynorC1emen7 — Thu, 14 Mar 2013 00:10:13 GMT

Neither C nor # are outside the range of the 128 ASCII characters, so your example hasn't quite tested for the problem I'm having.

But it show that data wasn't truncates.

Are the values displayed by the paramdump tool received as POST values?

Yes.

Re: SIM Relay Response data truncates at non-ASCII characters

bobbymray — Thu, 14 Mar 2013 00:22:58 GMT

The data I receive is fine. It is not truncated unless there is a character with, for instance, a diacritic, such as ö, or ê; something above the Basic Latin character set, in which case, yes, the value is truncated at that character.

Re: SIM Relay Response data truncates at non-ASCII characters

RaynorC1emen7 — Thu, 14 Mar 2013 00:45:45 GMT

Since you said the data receive is working. So the problem is with your database and not relay response?

Re: SIM Relay Response data truncates at non-ASCII characters

bobbymray — Thu, 14 Mar 2013 00:54:04 GMT

I'm not sure if the data I'm receiving is fine all of the time. I am not 100% sure.

I don't believe the problem is with the database. The table columns use an appropriate character set, AFAIK, which is utf8.

What I did in my original post, in addition to describing the problem I'm having, is ask for advice regarding decoding the POST values.

So, to reiterate, does anyone have any advice regarding whether or not Relay Response values, which may include upper Unicode characters, should be decoded, and if so, how? I'm using PHP 5.2.

Thank you.

Re: SIM Relay Response data truncates at non-ASCII characters

bobbymray — Fri, 15 Mar 2013 21:56:58 GMT

PHP's utf8_encode function is what I was looking for.

Thank you.

Re: SIM Relay Response data truncates at non-ASCII characters

n_fatima — Wed, 06 May 2026 17:29:15 GMT

This looks like a classic UTF-8 encoding issue where the Relay Response gets truncated when it encounters non-ASCII characters like ô in Rhône-Alpes.

Since ASCII characters work fine but non-ASCII characters get cut off, it suggests the data is being mishandled before it reaches MySQL, possibly during transport or parsing of the response.

Even if your database and PDO are correctly set to UTF-8, the issue might already exist in the raw POST data. I would suggest logging the raw input before processing to confirm whether the truncation happens in transit or at insert time. Also make sure all headers and source data are consistently UTF-8.

I’ve seen similar issues when multi-byte characters are not preserved in intermediate layers.
I’ve also been working on Unicode and encoding tools that might help in such cases: https://banglaconvertertool.com/