5


Cisco’s Approach to Software Updates

Ed Paradise, Cisco Systems, Inc.

Ed Paradise is vice president of engineering for the Security and Trust Organization at Cisco Systems, Inc. He addressed today’s attack environment, Cisco’s response to security vulnerabilities, and key considerations for the network infrastructure across the Internet.

Paradise is responsible for making Cisco products secure and trustworthy. His teams define and maintain Cisco’s Secure Development Lifecycle (CSDL), including its engineering, design, and security update procedures, plus product testing, evaluation, and incorporation of feedback. As a center for security innovation, his teams also develop embedded security technologies, which are deployed broadly across Cisco’s full portfolio. Examples of these technologies, which are mandated by CSDL, include CiscoSSL, trust anchor modules, secure boot, and image signing, as well as many other hardware- and software-based security technologies.

THE ATTACK ENVIRONMENT

Paradise shared projections indicating that by 2020 global Internet Protocol (IP) traffic is expected to double, to 2.3 zettabytes, and that broadband speeds are also expected

to double. Two-thirds of that traffic will be mobile and wireless, and 82 percent of all consumer Internet traffic is projected to be video streaming. This large number of devices and vast amount of bandwidth raises the stakes considerably, Paradise suggested, when considering the prospect of another attack like Mirai.

Paradise described an experiment demonstrating how rapidly and frequently software is contacted and probed. He connected a brand new device to the Internet and tracked what happened. Within 5 seconds, another device made contact. Within 5 minutes, a device from China made contact, and another actor checked if the device could be turned into a botnet and used in a distributed denial-of-service (DDoS) attack. Within 30 minutes, an actor had run a full vulnerability scan on the device. By the end of its first 24 hours connected to the Internet, about 4,000 different machines tried to contact the device.

As his experiment demonstrates, devices and software can be compromised in a matter of seconds. According to Paradise’s research, about 70 percent of contacts are attempts to use the device as a botnet in DDoS attacks. Other key risks include fraud and data theft. “There are a lot of bad things happening, by a lot of bad actors, pretty quickly,” he summarized.

CISCO’S APPROACH TO ADDRESSING VULNERABILITIES

Paradise detailed how Cisco handles security incidents. Vulnerabilities can be identified through a variety of mechanisms, including internal testing and validation, customer notification, or open-source notification. Depending on the vulnerability, it can take minutes or months for Paradise’s team to learn about it. Activating Cisco’s Product Security Incident Response Team Process (PSIRT), the team assesses the scope and scale of the situation and then comes up with a plan for fixing the problem and alerting users. Ultimately, he explained, an update goes out simultaneously to all the affected customers, and the team monitors and incorporates any feedback they receive to improve the process, repeating fixes as necessary until the crisis is over. Prompted by a question from Bob Blakley, CitiGroup, in the discussion, Paradise reiterated Cisco’s policy of providing simultaneous notification to all users, rather than identifying priority users for advance notification.

Eric Grosse, an independent consultant, recalled that in his experience, it was difficult to know which Cisco updates to install because the names were so confusing. Paradise recognized that that was indeed an issue, mostly because of the large number of products that Cisco offers, and he noted that the company is working to beef up

customer service to help customers navigate the product portfolio in order to find needed updates.¹

Steven Lipner, an independent consultant, asked how Cisco handles updates to products that include open-source components. Paradise noted that Cisco does a significant amount of testing on these products, and it also has a robust certification program. For example, Cisco has an automated registration and verification process for certain software that incorporates CiscoSSL (based on OpenSSL with different features and functionalities). The two groups have a strong working relationship for when joint updates are necessary, and Cisco helps OpenSSL by doing daily automated testing.

That partnership is an example of a good relationship with open-source software. However, Paradise noted, there can be problems—for example, when the Heartbleed bug came out. Tens of thousands of products were affected, and because of that large number, it took a long time to complete all the necessary verifications.

Forum Chair Fred Schneider asked Paradise to describe the extent to which Cisco controls its software and how it is used, and the extent to which Cisco is therefore able to mitigate the potential for destabilization. Paradise said that Cisco maintains tight control over the majority of its software—although there are some customers who want to add code to Cisco products—and he underscored that updates are thoroughly tested to prevent destabilization in the context of use that Cisco can control (e.g., not necessarily accounting for any code added on top of a Cisco product).

Paul Kocher, Cryptography Research Division, Rambus, Inc., asked what Cisco might be doing to help provide a safe platform for vulnerable or unmaintained Internet of Things (IoT) devices to continue to operate, as discussed during Nicko van Someren’s presentation (see Chapter 4). Paradise suggested that there may be opportunities to prevent future DDoS attacks by properly securing networks for IoT devices and embedding a “kill switch” that detects when a device is performing a function that it was not built to perform.

Building on this discussion, David Hoffman, Intel Corporation, agreed that IoT devices are vulnerable because they don’t have regular security updates the way that

___________________

¹ See, for example, O. Santos, 2017, “Keeping up with Security Vulnerability Updates,” openVuln API, Cisco Blogs, January 24, http://blogs.cisco.com/security/openvuln-update.

computer operating systems do. But, he said, companies may be prevented from doing security scans on these devices by the Computer Fraud and Abuse Act. He suggested that companies might overcome this limitation by working together to scan devices for security issues or breaches as an added service to companies and individuals, thus increasing transparency and alerting Internet service providers to potentially harmful traffic.

Paradise agreed that these issues present an opportunity for Cisco and other network infrastructure companies to create relationships with IoT providers to improve device security, including a multi-tier “IoT ready” certification program. Using cameras as an example, Paradise said such an arrangement could allow Cisco to offer consumers a list of camera models that work best with a Cisco router and would be supported against future security risks. Choosing to buy a camera not on that list would mean that a customer is taking the risk, knowingly, that the camera may not be protected and could be used to transmit sensitive data or propagate harm through the network.

UNDERSTANDING USER PERSPECTIVES

Zooming out to the technology-using population as a whole, Paradise reiterated the notion, raised elsewhere during the workshop, that most users live “in a world of false confidence” about the true level of their security. A recent Cisco survey revealed that a majority of users have “strong confidence” that, among other things, they can detect security vulnerabilities in advance (51 percent), that they can defend themselves against

such attacks (54 percent), and that they regularly review security policies (56 percent). He noted, however, that these numbers are slowly dropping, as users become less confident with every well-publicized security breach.

Another Cisco research report showed that 92 percent of devices currently connected to the Internet have an average of 26 security vulnerabilities each. This isn’t just old iPods, he reminded the participants: “This is more about devices that are on the Internet as part of the infrastructure.” A whopping 31 percent of connected devices are classified as “end of service,” meaning that manufacturers will no longer support security updates, and the manufacturers are not patching vulnerabilities. “Even if we wanted to,” Paradise explained, “we probably don’t have the capability to service that device any longer.”

After devices enter “end-of-life” status, they are no longer serviced or sold at all. Five percent of devices currently connected are in this category. The main reason people hang on to devices, particularly between end-of-life and end-of-service, is financial: If the device is still working, consumers don’t feel the need to replace it. Convincing these customers of the security risks they are taking can feel like a losing battle.

Richard Danzig, Johns Hopkins University Applied Physics Laboratory, asked about the role of psychology in determining how people perceive security. He pointed to the cognitive psychologists Amos Tversky and Daniel Kahneman (whose collaboration is the subject of the recent Michael Lewis book The Undoing Project²), who study the psychological factors that color people’s perceptions of different types of risky

___________________

² M. Lewis, 2017, The Undoing Project: A Friendship that Changed the World, Norton & Co., New York.

situations. Paradise agreed that psychology is a factor when customers are deciding whether the risks they face are large enough to invest in new equipment. He suggested that the industry could do more in the way of sales, marketing, or consumer education to highlight the security risks customers invite when they continue to rely on end-of-service or end-of-sale equipment. “We haven’t presented the arguments to our customers,” he said.

Paradise suggested that part of the solution is that customers need to be taught better questions to ask about security features when they are buying new equipment. While software companies are motivated to incorporate new features into their software in order to woo customers and grow revenue, unfortunately, “we don’t have a large base of our customers asking for those security features,” he said. Cisco is pursuing research on “the psychology of security” in order to better educate their customers about the security risks they might be taking when they choose not to update or replace their software.