Thursday March 28, 2024

Issue summary:
On March 28, 2024 at 10:47 AM PDT until 11:20 AM PDT, a small number of users experienced trouble using workflows, connecting or signing into Slack, and creating accounts.

A recent change to a database service that's responsible for processing user activity in Slack had an influx of unexpected traffic. This traffic caused an increased number of error rates and failures which prevented users from accessing certain features.

When our internal health tools alerted us of the influx of errors, we paused any further code changes in an effort to improve database processing.

During our investigation, it was discovered that as a result of the unexpected traffic, the database service automatically timed out after about 30 minutes. This timeout allowed the database to return to normal and process new requests without causing errors.

We moved forward with a continued period of investigation and monitored to ensure impact did not reoccur.

3:42 PM PST

Customers should no longer be experiencing any trouble using workflows. Apologies for the trouble today and thank you for your patience.

7:28 PM PST

While the majority of users should no longer be experiencing issues, our investigation of the root cause is still ongoing. We’ll be back with an update once we determine the issue is fully resolved.

2:34 PM PST

We’re still looking into the cause of these issues and we’ll provide new information as soon as it’s available.

12:10 PM PST

We’re seeing signs of improvement, but we’re still monitoring the situation. We’ll confirm once these issues are fully resolved or provided another update in 30 minutes.

11:40 AM PST

We’re receiving reports around trouble signing in and using workflows. Slack is down completely for some users.

We’re on the case and will provide another update once we have more details to share. We’re sorry for the disruption.

11:15 AM PST