Jim Salter
Yesterday, packages.microsoft.com—the repository from which Microsoft serves software program installers for Linux distributions together with CentOS, Debian, Fedora, OpenSUSE, and extra—went down onerous, and it stayed down for round 18 hours. The outage impacted customers making an attempt to put in .NET Core, Microsoft Groups, Microsoft SQL Server for Linux (sure, that is a factor) and extra—in addition to Azure‘s personal devops pipelines.
We first turned conscious of the issue Wednesday night once we noticed 404 errors within the output of apt replace on an Ubuntu workstation with Microsoft Groups put in. The outage is considerably better-documented at this .NET Core challenge report on Github, with many customers from all around the globe sharing their experiences and theories.
The quick model is that your complete repository cluster that serves all Linux packages for Microsoft was utterly down—issuing a spread of HTTP 404 (content material not discovered) and 500 (Inner Server Error) messages for any URL—for roughly 18 hours. Microsoft engineer Rahul Bhandari confirmed the outage roughly 5 hours after it was initially reported, with a cryptic remark concerning the infrastructure staff “operating into some area points.”
Eighteen hours after the difficulty was detailed, Bhandari stated that the mirrors had been as soon as once more out there—though with quickly degraded efficiency, seemingly as a consequence of chilly caches. On this replace, Bhandari stated that the unique reason for the outage was “a regression in [apt repositories] throughout some characteristic migration work that resulted in these packages turning into unavailable on the mirrors.”
We’re nonetheless ready for a complete incident report, since Bhandari’s standing updates present clues however no actual explanations. The excellent news: We will affirm that packages.microsoft.com is certainly up as soon as once more, and it’s serving packages because it ought to.
