topic Re: Intermittent Invalid OTS Token Errors (E00114) in Integration and Testing

Intermittent Invalid OTS Token Errors (E00114)

DoomHamster — Mon, 03 Apr 2017 19:30:51 GMT

We have started to get intermittent "Invalid OTS Token" errors today when creating a new user profile. Most of our attempts to create a user profile work fine and, prior to today, we recieved an occaisional error every couple of weeks. We have found that if a customer gets this error (nine so far today) and they refresh the checkout page to initiate a new transaction, their next attempt works fine.

Is anyone else seeing an uptick in these errors today? Any good way to mitigate?

Re: Intermittent Invalid OTS Token Errors (E00114)

RichardH — Tue, 04 Apr 2017 17:29:30 GMT

Hello @DoomHamster

Are you still experiencing this issue? If for your production account, I would suggest contacting customer support by phone at 877-447-3938 to report the issue. They are more familiar with production issues and can route your question appropriately. Feel free to ask for escalation.

Richard

Re: Intermittent Invalid OTS Token Errors (E00114)

DoomHamster — Tue, 04 Apr 2017 17:51:06 GMT

They are the ones that recommended I post here. :)

Edit: in fact, we have contacted them twice via chat and once via phone and, in each case, they referred us to this forum or the email contact form (which we have also submitted).

To answer your question, yes, we were recieving this error on our production account until we disabled the feature temporarily at 10 AM. We have double checked our configurations but, honestly, that seems like a non-starter since the issue is intermittent and the majority of the requests are successful. We have had our network guys look into the possibility of latency issues but they have seen nothing to indicate a network related issue.

Re: Intermittent Invalid OTS Token Errors (E00114)

RichardH — Tue, 04 Apr 2017 17:50:17 GMT

@DoomHamster Hmm... If you were given a case number, I can ask the team to review.

Re: Intermittent Invalid OTS Token Errors (E00114)

DoomHamster — Tue, 04 Apr 2017 17:59:55 GMT

We were never given a case number.

In each case they said that this is clearly an issue better suited to the developers and refered us to you. We were also told that this error should not occur at all unless we are in sandbox mode which we clearly aren't since most of our requests are successful and result in credit cards being saved which we can then see in our live CIM.

Re: Intermittent Invalid OTS Token Errors (E00114)

RichardH — Tue, 04 Apr 2017 18:02:28 GMT

@DoomHamster Would you mind sending me a private message with your gateway id, that will allow us to investigate further.

Richard

Re: Intermittent Invalid OTS Token Errors (E00114)

DoomHamster — Tue, 04 Apr 2017 18:05:50 GMT

Done!

Re: Intermittent Invalid OTS Token Errors (E00114)

dbroome — Wed, 05 Apr 2017 12:56:36 GMT

We have also started seeing this error. It has happened 127 times in the past 24 hours. Is there a solution for it yet?

Re: Intermittent Invalid OTS Token Errors (E00114)

DoomHamster — Wed, 05 Apr 2017 13:17:16 GMT

I'm both sorry to hear that and glad to know we aren't alone at the same time!

Re: Intermittent Invalid OTS Token Errors (E00114)

Lilith — Wed, 05 Apr 2017 15:07:56 GMT

@DoomHamster, you imply that you saw an uptick in errors starting April 3 -- is that correct?

@dbroome, when did you see these errors start?

Re: Intermittent Invalid OTS Token Errors (E00114)

dbroome — Wed, 05 Apr 2017 15:20:24 GMT

I had a tech support person mention they started seeing an increase yesterday morning, so I turned on a higher level of logging to trap details. Unfortunately, I can't go back any further to gauge a trend.

Re: Intermittent Invalid OTS Token Errors (E00114)

DoomHamster — Wed, 05 Apr 2017 15:40:46 GMT

@Lilith Yes.

Prior to the 3rd we had four errors on 03/13, two on 03/28, and one on 04/02. These didn't raise any red flags. Then, on 04/03, we recieved 17 between 08:56:09.333 and 21:30:05.677 MDT. The errors then stopped until the next morning (04/04) at 09:50:47.900 after which we temporarily disabled the feature.

Oddly, we did have one more error yesterday at 13:34:50.307...four hours after disabling the feature. The only way I can explain this one is someone sitting on the checkout page in a background tab after having loaded it before we removed the checkbox and coming back to it later to finish up. Unfortunately, the try/catch block for this block of code doesn't log the specific checkout transaction ID as it should so I can't confirm that.

Edit: I should probably mention that we implemented the feature on 02/02/2017 so we had no issues for the first 5.5 weeks and we have not changed our implementation since launch.

Re: Intermittent Invalid OTS Token Errors (E00114)

Lilith — Wed, 05 Apr 2017 16:59:14 GMT

@dbroome -- thanks. If you could message me your Production Gateway ID or API Login ID, I can ensure you're added to the internal ticket we're opening for this.

Re: Intermittent Invalid OTS Token Errors (E00114)

AgileWorkshop — Wed, 05 Apr 2017 17:06:05 GMT

Hello

We've just started using your product and are suffering the same problem. The same nonce is always returned by accept.js, and attempting an authCaptureTransaction request returns "Invalid OTS Token."

Please advise

Thanks

Re: Intermittent Invalid OTS Token Errors (E00114)

Lilith — Wed, 05 Apr 2017 18:12:33 GMT

To everyone reporting this issue:

If you've seen a spike in E00114s starting around April 1 or shortly thereafter, please send me a private message with your Production Gateway ID number, or API Login ID. I can then add you to the existing ticket. Thank you.

@DoomHamster -- I believe you gave yours to @RichardH already, and I have that.

@dbroome and @AgileWorkshop -- please send me your Gateway ID or API Login ID.

Thanks.

Re: Intermittent Invalid OTS Token Errors (E00114)

dbroome — Wed, 05 Apr 2017 18:17:10 GMT

I have sent you a PM with gateway id. To followup, the failure rate today is approaching 25%. If you can escalate this, please do. I have varified that the returned nonce is not the same between calls, but have not yet identified a pattern as to when it happens.

Re: Intermittent Invalid OTS Token Errors (E00114)

dbroome — Wed, 05 Apr 2017 20:48:09 GMT

Haven't received the error since 1:52 central. I was getting it once every 5 minutes or so.

Re: Intermittent Invalid OTS Token Errors (E00114)

Lilith — Wed, 05 Apr 2017 20:51:06 GMT

We believe we found and fixed the issue, and our own monitoring shows a drastic decrease in E00114 errors.

@dbroome -- thank you for confirming you see improvement.

@DoomHamster and @AgileWorkshop -- thank you for your details. Could you please confirm if you see improvement?

Re: Intermittent Invalid OTS Token Errors (E00114)

DoomHamster — Thu, 06 Apr 2017 14:19:57 GMT

@Lilith We re-enabled the feature early this morning. So far so good.

Re: Intermittent Invalid OTS Token Errors (E00114)

Lilith — Thu, 06 Apr 2017 16:21:20 GMT

Thanks, @DoomHamster. We'll keep an eye on the situation on this end, but if you see this error recur, please let us know when it starts and we'll dig deeper.