New Update Center (updates.jenkins.io) Architecture in Production: 18 November 2024
Summary (TL;DR)
Note: this is a follow up of the 07 and 08 November 2024 24-hour brownout.
The service https://updates.jenkins.io will switch its implementation to the new system, on Monday 18 November 2024 at 09:00 am UTC.
All Jenkins users are impacted but should not see any functional change.
⚠️ Please check that your organization respects the advertised DNS TTL or you might be stuck in the brownout longer than expected.
What is the "Update Center"?
Jenkins Update Center is a web server at the core of the Jenkins public infrastructure that distributes the plugins, tool installers, and version index to all Jenkins servers worldwide.
From the installation wizard to regular plugin updates, if you run Jenkins, then you use this service under the hood.
Today, it serves around 50 TB of data (outbound bandwidth) each month from a single virtual machine on AWS, which costs around $6,000 per month.
The Jenkins infrastructure team has worked relentlessly over the past years to implement a new sustainable implementation for this service in order to sustain and improve it.
The new Update Center implementation features a highly available system that redirects user requests to a download mirror close to their location.
Additional information is available in the GitHub issue.
What Happens?
After running 6 brownouts (e.g. using the new system in production from 1 to 24 hours), we are confident we ran out of things to break (©Basil Crow).
We’ll simply switch the DNS entry updates.jenkins.io to this new service and continue monitoring it.
In 1 month, if no major issue arises, we’ll start decommissioning the old service on the VM which was used for years.
Please refer to the helpdesk ticket for more information.
About the authors
