3
Microsoft’s Approach to Software Updates
Carlos Picoto, Microsoft Corporation
Carlos Picoto is the general manager for the Microsoft Corporation team that produces software updates for all versions of Windows. He opened by sharing Microsoft’s stated mission, “To empower every person and every organization on the planet to achieve more,” a mission that he allowed can be sometimes challenged by the need to continue supporting legacy software. He offered insights on today’s threat landscape before describing Microsoft’s approach to addressing vulnerabilities and deploying updates.
TODAY’S THREAT LANDSCAPE
The latest version of Microsoft’s operating system, Windows 10, is far more than just software for personal computers, Picoto said. The operating system also runs on diverse products such as Raspberry Pis and augmented reality devices. Reiterating other workshop participants’ characterizations of security breaches as “only a matter of time”—a risk to be managed but perhaps never eliminated—Picoto noted that the inherent vulnerabilities of software are nonetheless often underappreciated by users and organizations.
As operating systems have evolved, Picoto said, one crucial lesson is that new threats demand new defense techniques; older solutions are no longer effective at keeping information secure. Picoto told the story of a recent flight in which Picoto watched the passenger in front of him type in his password to log in even though he could have used Touch ID. What’s worse, the device had such a long timeout between keys that Picoto was able to see every letter of the user’s password.
Consumer inattention to security protections can sometimes provoke changes in vendor behavior and in how security tools are designed. As one example, Picoto explained that consumer behavior is a large part of why Microsoft is moving away from passwords and toward biometric identification. In Windows 10, the company has also experimented with hardware virtualization, which gives separate software components separate trust verification protocols in order to increase security. In addition, he noted, Microsoft features such as Defender Advanced Threat Protection and the Enhanced Mitigation Experience Toolkit enable engineers to detect attacks and begin designing security updates to address them before they become public.
Picking up on a theme that emerged throughout the workshop, Picoto emphasized that the growing landscape of Internet of Things (IoT) devices presents new challenges. Another tension, he noted, is the risk inherent to implementing a security update, which can require system downtime and cost customers productivity.
A NEW MODEL FOR BUILDING AND DELIVERING UPDATES
To address these challenges, Microsoft has created a new product offering: “Windows as a service.” In addition to a different sales model from the old product-based approach, this shift has been reflected in “radical changes” to every aspect of deployment and maintenance, including how the software is built, how it is updated, and how those updates are delivered, improving the company’s ability to keep customers both secure and up to date, Picoto said.
Picoto’s teams are constantly making updates to Windows. Initially, those updates are sent to Microsoft employees, creating a large testing base that helps bring broad validation to the updates. After honing them based on employee feedback, Picoto’s team pilots the updates with the Insider Preview Program, a group of almost 10 million enterprise users. Their feedback is then incorporated into the larger Windows updates. This multistep testing and validation process thus improves the overall quality of software updates made across the board.
This process represents an enormous change for Microsoft. Previously, new Windows versions were released in alpha and beta versions before a major release,
which occurred every few years. Today’s extended pilot period and more frequent deployment of updates enables Microsoft to ensure that each new operating system version is ready for major release while providing new features to customers much faster. However, the fast pace of these releases has been a difficult adjustment for some of Microsoft’s enterprise users, who must accommodate far more frequent updates than in the past.
Microsoft’s new model also addresses the challenge of what Picoto called selective patching, a previous approach that allowed users to pick and choose the different security patches they wanted to install. He said that software updates are developed with the assumption that all the currently recommended patches are in place, and when this isn’t the case, it can cause quality issues. To address this problem, the release of Windows 10 eliminated the option for users to select which patches to install, and instead combined all the available patches into one. All operating system (OS) versions are now structured this way, Picoto noted. Eliminating fragmentation improves the quality and reliability of the OSs for Microsoft, its users, and third-party companies who use the OS as a platform to design applications, said Picoto.
QUALITY UPDATES VERSUS FEATURE UPDATES
Microsoft’s new approach involves two separate types of software updates: quality updates and feature updates. Quality updates, released on “Update Tuesdays” (also known as “Patch Tuesdays”), are minor updates that fix security bugs or other quality
issues. Feature updates occur twice a year and deliver new OS features to customers. Customers can choose whether they want to receive feature updates each time they become available or remain with the same features for 10 years, which is how long Microsoft commits to supporting a given OS version. After experimenting with this update model in Windows 10, Microsoft decided to make it available to customers using both Windows 7 (which most of their enterprise customers are currently running) and Windows 8.
Feature updates are reviewed for 4 to 8 months before being made available to customers, at which point Microsoft releases the software publicly but doesn’t push it to everyone right away. The first customers to use the update are “determined seekers” who are up on the latest tech news and enjoy being the first to use new software, Picoto said. Users with the newest computers and fastest connection speeds are early adopters, and rolling out software in these contexts first allows Microsoft to learn more about potential quality issues that are not related to platform constraints. Eventually, the update is rolled out to all users.
THE STRUGGLE WITH LONGEVITY
Of course, at the same time that these new OS updates are being released, Picoto’s team must also keep up with security updates to the previous Windows versions. It wasn’t until July 2016 that his team stopped producing updates to Windows 2000, for example—which is, remarkably, still in use despite being 17 years old. Microsoft’s default life-span for software is 10 years, although customers often ask for extensions, Picoto said.
Longevity of systems is always a struggle, because customers want consistency and the cost of change can be considerable, particularly in machines built to function for decades. Windows 2003 was a particular challenge, Picoto recalled; when the scheduled end to updates arrived, there were still unpatched security vulnerabilities, but fixing them was too complicated given how old the software was in the first place.
BEHIND THE SCENES ON “UPDATE TUESDAY”
Picoto sketched a typical Update Tuesday workflow, which starts well before the update is released. Researchers from around the globe are constantly discovering and disclosing vulnerabilities, sometimes giving Microsoft a deadline for making a fix. Microsoft developers then create and deploy security updates, sometimes releasing these fixes first
to qualifying organizations as part of the Security Update Validation Program, which provides additional validation.
A common criticism is that fixing a bug takes too long. The reason, Picoto noted, is that the team is not updating one OS but all of them, in order to minimize the impact to all of Microsoft’s customers, regardless of the OS version they might be using. Synchronizing the fixes for all supported versions to arrive at the same time can create challenges.
Of course, releasing updates is not the end of the story for Picoto’s team. The next critical task is to monitor delivery and adoption of the updates. Despite their best efforts at running the most complete tests, problems can still occur, for example, if a user configuration runs both Microsoft and another operating system. If there is a problem, Picoto said it usually comes to Microsoft’s attention on a Wednesday or Thursday, often through Twitter. This cycle continues at a relentless pace; while preparing the next batch of Update Tuesday deployments, Picoto’s team must be prepared to face emergent security issues arising from a previous update that would have to be addressed quickly as well. “You can imagine that with the matrix of browsers and other different operating systems, it gets complex,” he said.
A bright spot among these problems is that as customers move to the most recent software updates, Microsoft has seen a drop in help desk phone calls reporting malware, Picoto said. Drastically simplifying user choice (e.g., by eliminating the ability to choose which pieces of the software to update) has also helped keep customers secure and up to date. Picoto closed with data indicating that the latest Windows software update, released in July 2016, has been installed on 79 percent of personal computers running Windows 10 in the United States. Of those devices, 84 percent are also on the latest Update Tuesday patch.
