Degraded tagging performance and delayed internal email notifications

Resolved
Resolved
After 2 weeks, 2 days, 5 hours, and 30 minutes

We've fully resolved the issue on the May 13, but forgot to change the status—sorry about that!

Avatar for
{:closed=>"Closed", :complete=>"Complete", :false_alarm=>"False Alarm", :identified=>"Identified", :investigating=>"Investigating", :open=>"Open", :recovering=>"Recovering", :resolved=>"Resolved", :scheduled=>"Scheduled", :underway=>"Underway"}
After 5 hours and 43 minutes

We are working to clear the backlog of orders without tags or notification emails sent. We will update when the issue is fully resolved.

Avatar for
Recovering
After 2 hours and 43 minutes

We've fixed the core issue, and are waiting for things to recover.

Avatar for
Identified
After 2 hours and 29 minutes

We've confirmed there is a problem, we're working to resolve it.

Avatar for
Recovering
After 1 hour and 12 minutes

We've fixed the core issue, and are waiting for things to recover.

Avatar for
Identified
After 10 minutes

We've confirmed there is a problem, we're working to resolve it. The issue is isolated to our US-2 orders servers and appears to be affecting more stores that initially expected. This does not affect your ability to receive orders and all tags/notifications will be queued and sent as soon as we have resolved the issue.

Avatar for
Investigating

We are investigating sporadic reports of delays to order tagging and delayed delivery of email notifications on a handful of stores on our US-2 servers. We will update this post with our progress as we discover the core issue.

Avatar for
Began at:

Affected components
  • Store Pickup + Delivery
    • Orders (US)