Monday January 6, 2025

Incident

Scheduled workflows not triggering

Summary

On January 6, from 3:23 AM PST to 4:50 PM PST scheduled workflows were not triggering correctly for some users.

We identified that a recent update to our internal infrastructure had added unexpected load to the backend database that processes workflow triggers, which was causing scheduled workflows to fail.

To mitigate further issues, we cleared these failed triggers from our systems. However, this inadvertently caused some workflows to re-trigger outside their scheduled time on customer’s workspaces.

Once these failed triggers had been fully removed, we confirmed that scheduled workflows were working as expected again.

12:24 AM PST

Resolved

We rolled out a fix resolving the issue with the subset of affected workflows yesterday. This fix created a one-time side effect that caused impacted workflows to run unexpectedly. We don’t anticipate this occurring again and apologize for any confusion.

Workflow functionality has been fully restored. If you still experience trouble, please write in to feedback@slack.com. 

10:54 AM PST

Update

We previously implemented a fix that improved error rates and resolved the initial problem. However, we continue monitoring our health metrics as we work to resolve an underlying issue with affected workflows.

The affected workflows with an "On a schedule" trigger that were set to run on January 6, 2025 may still not execute at their designated times. To resolve this, we're repairing the backend job that's responsible for handling re-scheduled workflow triggers, restoring their functionality. All other workflows should run as normal.

We'll be back with an update once we've resolved the impact to the affected workflows.

12:53 PM PST

Resolved

We have identified the cause of the problem and implemented additional capacity to manage the load. We believe these changes have addressed the issue, but we will continue monitoring the situation.

We apologize for the disruption to your work. If you have any further issues with scheduled workflows, please contact us at feedback@slack.com.


6:10 PM PST

Update

We are continuing to investigate this issue, but currently do not have any new information to share. We will provide further information as soon as we have updates to share with you. Thank you for your continued patience.

12:56 PM PST

Update

We are continuing to investigate this issue, but we don’t have any new information to share at this time. We’ll be back when we have some new information to share with you. Thank you for your patience! 

11:38 AM PST

Update

We are continuing to investigate this issue, but we don’t have any new information to share at this time. We’ll be back with another update in 30 minutes.

10:37 AM PST

Update

We have no additional news to share just yet, but we’re focused on getting things back to normal as quickly as we can. 

Thank you for your patience as we continue to work towards a fix. 

9:59 AM PST

Update

We’re still actively investigating the issue with scheduled workflows failing to trigger, but we don’t have any new information to share at this time. 

Thank you for your patience and we’ll provide new information as soon as it’s available.

8:15 AM PST

Investigating

Some users may be experiencing issues with scheduled workflows not triggering. We're currently investigating the issue and we'll be back with an update as soon as we have more information.

We're very sorry for the inconvenience. 

7:11 AM PST

Features affected

Workflows

Status

Incident

Incident

Some users may be experiencing issues with their sidebar not being badged for new DMs and channel notifications.

Resolved

We have deployed a fix to resolve the badging issues and your notifications should now be functioning as expected. If you have any further trouble, please write to us at feedback@slack.com. 


Thank you for sticking with us while we worked on this. We very much appreciate your patience and understanding.

9:09 PM PST

Update

Our efforts to fully resolve this issue is still ongoing. While we haven't resolved this yet, we will provide further updates directly through support tickets with impacted users.

If you’re experiencing this behavior, please reach out to us at feedback@slack.com. We apologize for the trouble and appreciate your patience as we sort this out.

12:58 PM PST

Cause Identified

We've identified the cause of the badging issues with older DMs and are working on a fix.

There are additional behaviors related to the badging issues with these DMs that we are investigating and working to restore full functionality. Other notifications, such as app icon badges, sounds, and banners, may still be functioning properly. We appreciate your patience and will share updates as our work continues.

3:43 PM PST

Update

We believe we may have isolated the cause of problem and we’re focused on getting Slack working again as soon as possible. We'll continue to surface any new updates as they come in. 

11:54 AM PST

Update

Our investigation is still ongoing, we'll be back with another update once we have news to share. We appreciate your continued patience as we sort this out.

3:33 PM PST

Update

We are continuing to investigate this issue, but we don’t have any new information to share at this time. We’ll be back when we have some new information to share with you. Thank you for your patience! 

12:45 PM PST

Investigating

Some users may be experiencing issues with their sidebars not being badged/bolded for new DMs and channel notifications. 

We’ll be back with another update as soon as we have more information to share. 


9:48 AM PST

Features affected

Notifications

Status

Incident

Incident

Some users cannot switch between workspaces

Summary

On January 6, 2025 from 12:14 AM PST to 11:09 AM PST, some customers were unable to switch between workspaces on desktop.

We identified that an error had been introduced via a routine code change that prevented an upgrade prompt from closing as expected. As the upgrade modal retained focus while open, this prevented users with a workspace on the free plan from switching between workspaces.

We reverted the code change and the issue was resolved for all impacted users.

9:07 PM PST

Resolved

We have identified the root cause of the modal not closing correctly and have reverted the modal to prevent this from impacting user's desktop and web clients going forward. If you’re still encountering any trouble, please reload Slack using Command + Shift + R (Mac) or Ctrl + Shift + R (Windows/Linux). Thank you for your patience while we got to the bottom of this issue.

11:54 AM PST

Investigating

Some users who have a free workspace on their desktop or web browser Slack clients are unable to switch between Workspaces because of a modal not closing correctly. While this only affects free workspaces, it can impact any Slack user who has a free workspace added to their Slack client. Users can use a mobile app as a workaround and refreshing the impacted Slack client will sometimes allow users to switch workspaces. Our team is actively working to resolve this issue as soon as possible. We appreciate your continued patience and will be back with an update once we have more news to share.

11:30 AM PST

Features affected

Connectivity

Status

Incident