Sunshine Conversations integration is currently experiencing issues

Incident Report for Ultimate AI's System Uptime

Postmortem

On October 1, 2024, One of our bots experienced a significant spike in chat messages. The incident began at approximately 20:50, leading to operational challenges that peaked around 21:25. As chat volumes increased, our system encountered rate limiting from the Sunshine Conversations API, which was not handled gracefully. This resulted in CPU overutilization across multiple pods, degrading performance. Oncall team, after being notified, updated the system’s maximum resources, ultimately stabilising the platform.

Posted Oct 02, 2024 - 15:17 CEST

Resolved

This incident has been resolved.
Posted Oct 02, 2024 - 00:06 CEST

Monitoring

A fix has been implemented and we are monitoring the results.
Posted Oct 01, 2024 - 23:47 CEST

Identified

The issue has been identified and a fix is being implemented.
Posted Oct 01, 2024 - 23:33 CEST

Update

We are continuing to investigate this issue.
Posted Oct 01, 2024 - 23:06 CEST

Update

We are continuing to investigate this issue.
Posted Oct 01, 2024 - 22:25 CEST

Investigating

Our integration with Sunshine Conversations is experiencing issues. We are currently investigating.
Posted Oct 01, 2024 - 22:20 CEST
This incident affected: Chat integrations (Sunshine).