# Pandium Status - Incident History

## Incident / 30156997  
**Date:** May 16, 2026  
**12:50 EDT**  
**Status:** Resolved  
This incident has been resolved and we are continuing to monitor. We apologize for the inconvenience.

**12:40 EDT**  
**Status:** Identified  
Our US environments are experiencing instability stemming from a maintenance task by an upstream provider. We are working to mitigate and will update shortly.

## Incident / 30156328  
**Date:** May 16, 2026  
**12:04 EDT**  
**Status:** Resolved  
This issue has been resolved and we are continuing to monitor. Apologies for the inconvenience.

**11:55 EDT**  
**Status:** Update  
A fix has been implemented and we are monitoring.

**11:30 EDT**  
**Status:** Identified  
The issue has been identified and we are working on a fix.

**11:28 EDT**  
**Status:** Investigating  
We are currently investigating an issue and will update when we know more.

## Incident / 28144951  
**Date:** January 22, 2026  
**16:17 EST**  
**Status:** Resolved  
This incident has been resolved.

**13:53 EST**  
**Status:** Monitoring  
A fix has been implemented and we are monitoring to assure peak efficiency.

**12:47 EST**  
**Status:** Update  
We are continuing to monitor recovery and will update as soon as we have further details.

**11:13 EST**  
**Status:** Update  
We are continuing to monitor the fix as we recover.

**10:35 EST**  
**Status:** Update  
We are currently implementing a fix for the issue.

**09:59 EST**  
**Status:** Identified  
An issue has been identified and we are working on a fix.

**09:51 EST**  
**Status:** Investigating  
We are currently investigating an incident affecting our EU environments. Logging into the Integration Hub and runs are affected. We are actively investigating and will update when we know more.

## Incident / 25455888  
**Date:** June 12, 2025  
**17:21 EDT**  
**Status:** Resolved  
This issue has been resolved and all services are caught up and fully operational. We will continue to monitor to ensure stability.

**16:50 EDT**  
**Status:** Monitoring  
All systems are back online and we are monitoring the recovery of the backlog of jobs.

**15:43 EDT**  
**Status:** Update  
EU environments have recovered and we are continuing to monitor. US environments remain affected by the outage.

**15:20 EDT**  
**Status:** Update  
Runs are firing with delays in our EU environment. We are monitoring.

**14:37 EDT**  
**Status:** Update  
The issue with our cloud hosting provider is ongoing. We are continuing to monitor.

**14:13 EDT**  
**Status:** Identified  
All services are inaccessible or responding slowly due to an issue with our cloud hosting provider. We will update as soon as we know more.

## Incident / 24412125  
**Date:** March 21, 2025  
**10:24 EDT**  
**Status:** Resolved  
This issue has been resolved and runs are processing normally.

**10:00 EDT**  
**Status:** Identified  
We have identified the issue and deployed a fix.

**09:36 EDT**  
**Status:** Investigating  
Some runs in the US region are firing with delays. We are investigating.

## Incident / 22739043  
**Date:** November 1, 2024  
**19:13 EDT**  
**Status:** Resolved  
This issue has been resolved.

**19:09 EDT**  
**Status:** Monitoring  
A fix has been implemented and we are monitoring recovery.

**19:03 EDT**  
**Status:** Identified  
The issue has been identified and a fix is being implemented.

**19:02 EDT**  
**Status:** Update  
The issue with runs has been identified and we are implementing a fix. No other services are affected.

**18:49 EDT**  
**Status:** Investigating  
Runs for some US-Based accounts are processing with delays. We are investigating.

## Incident / 22355522  
**Date:** October 1, 2024  
**15:30 EDT**  
**Status:** Resolved  
This incident has been resolved.

**15:25 EDT**  
**Status:** Investigating  
We are currently investigating an issue in which some runs are firing with delays.

## Incident / 22305663  
**Date:** September 27, 2024  
**10:47 EDT**  
**Status:** Resolved  
This incident has been resolved.

**09:35 EDT**  
**Status:** Identified  
The issue has been identified and we are working on a fix.

**09:15 EDT**  
**Status:** Investigating  
We are investigating.

## Incident / 22295737  
**Date:** September 26, 2024  
**15:52 EDT**  
**Status:** Resolved  
This incident has been resolved.

**15:12 EDT**  
**Status:** Monitoring  
A fix has been implemented and we are monitoring.

**13:35 EDT**  
**Status:** Identified  
Runs are starting with delays as we investigate the underlying cause.

## Incident / 22241204  
**Date:** September 21, 2024  
**21:35 EDT**  
**Status:** Resolved  
The underlying service issue causing instability has been resolved and runs are firing. We are continuing to monitor for the next few hours and will open a new incident if necessary.

**21:09 EDT**  
**Status:** Monitoring  
A fix has been implemented and runs are recovering. We are closely monitoring and will update when fully resolved.

**19:35 EDT**  
**Status:** Update  
The Pandium Integration Hub is fully operational however jobs are still running with delays. We will provide an update as soon as possible.

**16:03 EDT**  
**Status:** Update  
We are continuing to work on a fix for this issue.

**14:36 EDT**  
**Status:** Identified  
We have identified that there is an issue with our underlying cloud hosting provider and we are working with them to implement a fix. They have escalated this issue and we will provide an update as soon as possible.

**13:06 EDT**  
**Status:** Monitoring  
A fix has been implemented and we are monitoring recovery.

**11:45 EDT**  
**Status:** Investigating  
After recovery, we are experiencing a different issue with our underlying platform. We are investigating.

**10:37 EDT**  
**Status:** Update  
We are continuing to monitor for any further issues.

**10:09 EDT**  
**Status:** Monitoring  
The released fix was effective and we are monitoring platform recovery.

**10:05 EDT**  
**Status:** Identified  
The issue has been identified and a fix has been released.

**09:35 EDT**  
**Status:** Update  
We have identified the issue and are investigating a fix.

**09:02 EDT**  
**Status:** Update  
We are continuing to investigate this issue.

**09:02 EDT**  
**Status:** Investigating  
Some runs are firing slowly due to an intermittent issue with token management. We are investigating.

## Incident / 21520297  
**Date:** July 25, 2024  
**02:39 EDT**  
**Status:** Resolved  
Will continuing to monitoring. All systems green.

**02:27 EDT**  
**Status:** Monitoring  
We have increased replicas and are seeing a reduction in error counts. Monitoring.

**02:12 EDT**  
**Status:** Investigating  
API responses exceeding error limits. We are investigating.

## Incident / 21408463  
**Date:** July 14, 2024  
**17:42 EDT**  
**Status:** Resolved  
Scheduled run runs have caught up, and we have not experienced any more issues with control plane.

**13:37 EDT**  
**Status:** Monitoring  
A fix has been implemented. We are now monitoring recovery.

**12:35 EDT**  
**Status:** Update  
The underlying issue has been identified. Ectd cluster of the control plane is having a fit. Working on mitigation.

**11:53 EDT**  
**Status:** Identified  
Issue is with the control plane of underlying hosting provider. We are working with them on mitigation.

**03:06 EDT**  
**Status:** Investigating  
Underlying Control Plane under high load. Investigating. Run and API are running slow.

## Incident / 19631163  
**Date:** January 6, 2024  
**08:30 EST**  
**Status:** Resolved  
Runs were failing across the platform due to a service that was stuck in a terminating state.

## Incident / 19128521  
**Date:** November 15, 2023  
**11:00 EST**  
**Status:** Resolved  
This incident has been resolved and we're monitoring to ensure performance.

**10:59 EST**  
**Status:** Update  
Activity reports are slow to load via the UI. There is no issue with runs or integrations.

**10:45 EST**  
**Status:** Identified  
Runs are currently processing slower than normal. We are currently addressing the issue and will update shortly.

## Incident / 18125518  
**Date:** August 12, 2023  
**10:31 EDT**  
**Status:** Resolved  
The underlying issue that caused webhook runs to be queued is resolved.

New manual, scheduled and webhook runs are all functioning normally while queued webhook payloads are being processed.

**09:46 EDT**  
**Status:** Monitoring  
While a core system restarted, runs were saved and queued, including webhook payloads. They are currently processing and we expect this issue to resolve shortly. We are continuing to monitor.

## Incident / 17454425  
**Date:** June 2, 2023  
**11:42 EDT**  
**Status:** Resolved  
This incident has been resolved.

**11:38 EDT**  
**Status:** Monitoring  
We are monitoring a fix.

**11:31 EDT**  
**Status:** Identified  
We have identified an issue and are implementing a fix.

**11:27 EDT**  
**Status:** Update  
Some front-ends are unavailable and we are investigating. Runs are not affected.

**11:25 EDT**  
**Status:** Investigating  
We are investigating this issue.

## Incident / 17354820  
**Date:** May 24, 2023  
**13:32 EDT**  
**Status:** Resolved  
This issue has been resolved.

**13:26 EDT**  
**Status:** Monitoring  
A fix has been implemented and we are monitoring.

**13:24 EDT**  
**Status:** Identified  
We have identified an issue we believe is the cause for the API instability.

**13:22 EDT**  
**Status:** Update  
Update: All jobs are continuing to run.

**13:19 EDT**  
**Status:** Investigating  
We are experiencing intermittent issues with our APIs. We are investigating the cause.

## Incident / 17352254  
**Date:** May 24, 2023  
**09:29 EDT**  
**Status:** Resolved  
This issue is resolved.

**09:21 EDT**  
**Status:** Monitoring  
A fix has been implemented and we are monitoring.

**09:20 EDT**  
**Status:** Investigating  
We are currently investigating this issue. Runs are not affected.

## Incident / 16955192  
**Date:** April 24, 2023  
**15:32 EDT**  
**Status:** Resolved  
This incident has been resolved.

**15:17 EDT**  
**Status:** Monitoring  
The API is recovering and we will continue to monitor.

**15:16 EDT**  
**Status:** Identified  
Some pages may be slow to load. We are working to recover.

## Incident / 16867194  
**Date:** April 14, 2023  
**13:54 EDT**  
**Status:** Resolved  
This issue has been resolved.

**13:53 EDT**  
**Status:** Monitoring  
We identified an issue where some sites were inaccessible. We implemented a fix and are monitoring.

## Incident / 16179289  
**Date:** February 17, 2023  
**12:09 EST**  
**Status:** Resolved  
This issue has been resolved.

**12:08 EST**  
**Status:** Update  
We are continuing to investigate this issue.

**12:07 EST**  
**Status:** Investigating  
Some instances of the admin dash and marketplace are unavailable. We are investigating this issue.

## Incident / 15357089  
**Date:** December 17, 2022  
**11:23 EST**  
**Status:** Completed  
Maintenance is complete! Have a good weekend y'all!

**11:20 EST**  
**Status:** Verifying  
Verification is currently underway for the maintenance items.

**09:00 EST**  
**Status:** In progress  
Scheduled maintenance is currently in progress. We will provide updates as necessary.

**15:50 EST**  
**Status:** Scheduled  
We will be doing scheduled upgrades in production between 9am-1pm EST on Saturday 12/17. During this time, scheduled jobs will continue to run, however there may be intermittent slowness in the UI. If this occurs, we expect this to resolve in under 1 minute.

## Incident / 11897990  
**Date:** October 12, 2022  
**13:56 EDT**  
**Status:** Resolved  
This incident has been resolved.

**12:20 EDT**  
**Status:** Monitoring  
After an earlier incident, we are continuing to monitor as we are experiencing intermittent API slowness.

## Incident / 11893655  
**Date:** October 12, 2022  
**12:10 EDT**  
**Status:** Resolved  
This issue is resolved.

**11:47 EDT**  
**Status:** Update  
We are continuing to monitor for any further issues.

**11:28 EDT**  
**Status:** Monitoring  
All front-ends have recovered. We are continuing to monitor.

**11:19 EDT**  
**Status:** Identified  
We have identified the issue and have implemented a fix. APIs are recovering.

**10:55 EDT**  
**Status:** Investigating  
We have identified an issue with a core service and are investigating. Scheduled jobs are continuing to run. We will update when we have more information.

## Incident / 11089378  
**Date:** September 9, 2022  
**10:34 EDT**  
**Status:** Resolved  
This issue has been resolved. Apologies for the inconvenience.

**10:33 EDT**  
**Status:** Monitoring  
A fix has been implemented and we are monitoring.

**10:32 EDT**  
**Status:** Update  
We have identified the issue and are deploying a fix.

**10:28 EDT**  
**Status:** Investigating  
The in-app marketplace is unavailable for some customers. We are actively investigating. Runs are continuing but the UI is unavailable.