topic Re: JSON API Response ERROR in Integration and Testing

JSON API Response ERROR

jima — Fri, 11 Aug 2017 22:28:10 GMT

While whipping up some code to directly use the API, I spent some time sussing out why the JSON response was crashing things. Turns out it's because it is returning a UTF-8+BOM character encoded string, which it definitely should not. As written in the RFC:

"Implementations MUST NOT add a byte order mark to the beginning of a JSON text"
(https://tools.ietf.org/html/rfc7159#section-8.1)

When coming here to make this post, I looked around in the forums and (almost) got excited when I saw this thread:
https://community.developer.authorize.net/t5/News-and-Announcements/Important-JSON-API-Response-Change/m-p/55370#M185

But that excitement quickly vanished when I read that it was decided to NOT implement the fix. I would strongly encourage you to revert that decision, and actually DO implement the prescribed fix as your current implementation is not RFC compliant.

Making the decision to NOT adhere to RFC standards just doesn't make sense, and costs developers untold (combined) hours of unnecessary troubleshooting time. PLEASE re-discuss this decision internally, and (hopefully) come down on the side to have your API conform to the RFC standards.

For those that have come across this headache, here's a workaround for it, that *shouldn't* need to be done, but will work until A.net decides to make the change away from UTF-8+BOM.

$tmp_bom = pack('H*','EFBBBF');
$result = preg_replace("/^$tmp_bom/", '', $result);
unset($tmp_bom);
$result = json_decode($result);

Re: JSON API Response ERROR

Aaron — Fri, 25 Aug 2017 23:56:22 GMT

Hi Jim,

What can I say? I think you've personally hit on every single one of the existing pain points for people developing to our spec. I can't comment on the BOM issue and whether that would be fixed, but I can say I'm sorry you've been having so much trouble.

I do really appreciate you taking the time to document it here for any else that may be afflicted. If I remember correctly, you're in Utah, right? If you ever want to head down to our office in Lehi where I'm located, lunch is on me.

Re: JSON API Response ERROR

winternet — Thu, 21 Sep 2017 11:04:27 GMT

Could we at least get a reason for this highly irrational decision announced on https://community.developer.authorize.net/t5/News-and-Announcements/Important-JSON-API-Response-Change/m-p/55370#M185 ?!

Re: JSON API Response ERROR

sameer — Thu, 21 Sep 2017 11:58:29 GMT

hi @jima

I think API returning json string with a special character i.e.  ... at the start of string .

Response could be like this

{"messages":{"resultCode":"Error","message":[{"code":"E00003","text":"The 'AnetApi/xml/v1/schema/AnetApiSchema.xsd:expirationDate' element is invalid - The value XXXXX is invalid according to its datatype 'String' - The actual length is less than the MinLength value."}]}}

 will be hidden as browser detect it as HTML entity....

please use inspect element and find the issue with the returning string ....

I think thats json_decode() returning NULL ...

I Hope it will help ...

Re: JSON API Response ERROR

Aaron — Thu, 21 Sep 2017 15:02:58 GMT

@winternet wrote:

Could we at least get a reason for this highly irrational decision announced on https://community.developer.authorize.net/t5/News-and-Announcements/Important-JSON-API-Response-Change/m-p/55370#M185 ?!

Usage of the JSON API was too great, and it was determined that there was too high of a likelihood of breaking people's existing implementations in production. It's not that we thought people were explicitly depending on the BOM, but rather that their workarounds could cause their response parsing to break if the BOM stopped appearing.

Everything's a tradeoff. With any web service or API, there's a constant struggle between making improvements/fixes and maintaining compatibility, and we generally have to go for the latter.

There's a lot of lessons we've learned here. The most obvious is that if it had been done fully to spec in the initial release, we wouldn't have this problem.

Another lesson: If we had made it clearer to developers that the JSON API was beta when it was first announced, we'd probably feel more comfortable making a potentially breaking change like this. We asked developers to let us know if they were using the JSON API specifically so that we could inform them of changes. However, when it came time to discuss actually making a change, the actual usage of the API was many orders of magnitude higher than what would be coming from just the ones who had notified us. Maybe a little of that is on developers for not following our instructions, but I think at least some of that is on us for not making it clearer that there might be changes or actually gating the API to those who've registered.

At that point, we could attempt to correlate the login IDs actually and contact the merchants directly to say "hey, we're about to make a potentially breaking change to your business", but that's a tough thing to do. Remember these are the login IDs of the merchants, not specifically the developers that created the integration. The best we can do in a situation like that is to say "contact whoever built your Authorize.Net integration and tell them to check on this". That's a tough sell, and no matter how we word it, not everyone will get the message.

That's one other benefit to registering for a Solution ID if you're developing an integration to Authorize.Net. If every developer writing something to our API used a Solution ID, we'd at least be able to know from every request who to contact in the case of trouble. Heck, we could have built the API to require a Solution ID in every request, and make every developer register to build an integration. That would give us a good list of developer contacts, but at the expense of making it a lot harder for a developer to get started. Again, everything's a tradeoff.

This is perhaps more detail than you asked for, and more than I would usually share, but I feel strongly that without a good explanation, it does seem like an irrational decision. Hopefully, what I've shared will lead you to conclude that even if you would have chosen a different tradeoff, the decision was at least not made irrationally. I wasn't here when that decision was made, but if I was, I suspect I would have been on the losing side of that argument. But, I'm confident that all of the ramifications were considered very thoroughly. Again, there's room to argue that this was the wrong decision, but it was at least very carefully considered.

Hopefully this goes without saying, but if not, I'll say it explicitly: There will not be any weird BOM in the wrong place in the next version of our API.

Re: JSON API Response ERROR

jima — Thu, 21 Sep 2017 15:15:47 GMT

Thanks for that reply Aaron.

I can certainly see the concern behind making a "breaking change" to the API response, but (IMO) if it is a change that complies to a universal standard, I think it would be widely understood.

A quick an easy solution would be to offer an argument, to the API call, that would allow the the return to give either version of the return, depending on the argument. Best of both worlds there.

new API parameter: returnWithBOM
default value: TRUE

Doing that would keep existing (patched) systems running with the BOM, but would also allow for standards compliance, should the developer choose to pass in a value of "FALSE". Everyone happy as clams :D

Re: JSON API Response ERROR

madcolor — Thu, 21 Feb 2019 01:21:43 GMT

Wow.. I just wasted a half-day on this. Go ahead and call me inexperienced, but dealing with this exact issue had me ready to sign into Bellevue. I'm working out of a word-press class, on a remote server, so debugging was painful to say the least. Not seeing hidden character(s) when outputting the response to a browser had me completely baffled. Thanks to OP (jima) here, and no thanks to Auth.net for chosing not to follow the standard.

Re: JSON API Response ERROR

Langworth — Fri, 22 Oct 2021 06:35:05 GMT

Cloud Storage uses the standard HTTP error reporting format for the JSON API. Successful requests return HTTP status codes in the 2xx range. Failed requests return status codes in the 4xx and 5xx ranges.

myccpay