This video walks through an example of using SRE to identify an anomaly, and investigate and manage a web page load time performance issue impacting multiple users. It shows how engineers use SysTrack Reliability Engineering to analyze trends, document findings, monitor changes and close the problem once service has returned to normal.
Video Transcript
On the SysTrack homepage, we see that there is a webpage load time exception anomaly. This indicates that there has been an abnormal increase in webpage load times. The trend chart shows the issue escalating throughout the day. Around 9:30 a.m., approximately 90 devices were affected. By 11:45, that number had grown to 112 systems, followed by a significant spike in exception activity before gradually declining later in the day.
To investigate, we open SysTrack Reliability Engineering, where someone has created a problem record named Web Page Load Time Excessive. This problem serves as the central location for tracking the investigation and resolution efforts. Opening the problem, we can review when it was created, examine the triggering sensor, and see the investigation history.
Notes added by the team indicate that users experienced significant delays accessing Google Drive and Google Docs on several days. Another note shows that the issue was escalated to Google Support, with a support case number recorded for reference.
The affected systems section allows us to drill in and see exactly which devices were impacted. The investigation ultimately determined that the root cause was a back-end service issue within Google's environment. Because the issue was external, no remediation was required on our side.
In other scenarios, we may be able to use Take Action to launch an automated response. Finally, on the Monitor tab, the sensor activation trend confirms that activity has returned to normal levels.
With the issue resolved and no ongoing impact to users, we can update the problem status from In Progress to Resolved.