Preliminary Reason for Outage Summary:
This this is a preliminary RFO summary. An official RFO will be distributed and posted to the CenturyLink Customer Portal following the completion of the full post incident review.
An offending flowspec announcement prevented Border Gateway Protocol (BGP) from establishing correctly, impacting client services.
The IP NOC deployed a configuration change to block the offending flowspec announcement, thus restoring services to a stable state.
On August 30, 2020 10:04 GMT, CenturyLink identified an issue to be affecting users across multiple markets. The IP Network Operations Center (NOC) was engaged, and initial research identified that an offending flowspec announcement prevented Border Gateway Protocol (BGP) from establishing across multiple elements throughout the CenturyLink Network. The IP NOC deployed a global configuration change to block the offending flowspec announcement, which allowed BGP to begin to correctly establish. As the change propagated through the network, the IP NOC observed all associated service affecting alarms clearing and services returning to a stable state.
Many customers impacted by this incident were unable to open a trouble ticket due to the extreme call volumes present at the time of the issue. Additionally, the CenturyLink Customer Portal was also impacted by this incident, preventing customers from opening tickets via the Portal. As such, once the official RFO document is complete, it will be posted to the CenturyLink Customer Portal.