Luware - All Clusters - Elevated Graph Response Times – Incident details

All Clusters - Elevated Graph Response Times

Resolved
Degraded performance
Started 5 months agoLasted 49 hours 24 minutes

Affected

Nimbus

Degraded performance from 9:33 AM to 10:57 AM, Operational from 7:35 AM to 10:57 AM

Nimbus Australia 01

Degraded performance from 9:33 AM to 10:57 AM

Microsoft Service Health

Degraded performance from 9:33 AM to 10:57 AM

Nimbus Germany 01

Degraded performance from 9:33 AM to 10:57 AM

Microsoft Service Health

Degraded performance from 9:33 AM to 10:57 AM

Nimbus Germany 02

Degraded performance from 9:33 AM to 10:57 AM

Updates
  • Resolved
    UTC
    Resolved

    Microsoft has completed the rollback. Continued monitoring by Luware confirms that the issue has not reoccurred.

  • Update
    UTC
    Update

    Microsoft is in the ongoing process of rolling back changes. Luware Engineers are working closely with Microsoft's support team to ensure a full resolution and will share any related news as they become available.

  • Update
    UTC
    Update

    Microsoft has reported implementing changes to mitigate the root cause. Nimbus telemetry shows that the situation has improved but is not yet fully back to normal. There has been no impact to the running of Nimbus components, but customers may notice intermittent degraded performance in areas which rely on Microsoft Graph. Luware Engineers are tracking the situation closely and we will share an update when we have further news to share.

  • Monitoring
    UTC
    Monitoring

    Microsoft has published a related incident in the Azure Portal under reference TM1286773, related to availability of inbound calls in Teams. Our telemetry reports that Graph responsiveness is returning to normal thresholds and our engineers are continuing to monitor the situation.

  • Investigating
    UTC
    Investigating

    Our engineering team is currently investigating reports from our telemetry of elevated/extended response times with Microsoft's Graph integration across all Nimbus Clusters.