The Art of Software Security Testing: Identifying Software Security Flaws

Chris Wysopal is cofounder and CTO of Veracode, where he is responsible for the software security analysis capabilities of Veracode’s technology. Previously he was vice president of research and development at @stake. As a member of the groundbreaking security research think tank L0pht Heavy Industries, he and his colleagues testified to the U.S. Senate that they could “take down the Internet in 30 minutes.” They were praised as “modern-day Paul Reveres” by the senators for their research and warnings of computer security weaknesses. Wysopal has also testified to the U.S. House of Representatives and has spoken at the Defense Information Systems Agency (DISA), Black Hat, and West Point. He is coauthor of L0phtCrack, the password auditor used by more than 6,000 government, military, and corporate organizations worldwide. He earned his bachelor of science degree in computer and systems engineering from Rensselaer Polytechnic Institute in Troy, New York.

Lucas Nelson is the technical manager for Symantec’s New York region, where he is responsible for all aspects of security consulting services delivery. Within Symantec he also leads the Application Security Center of Excellence, which develops application security practices and guidelines and trains new hires in the methodology of application testing. He has taught a number of classes on both attacking and defending computer systems to several groups, including state governments and large financial institutions. Nelson worked as a developer specializing in security for a number of small startups before joining Symantec/ @stake in 2002. He researched computer security at Purdue University’s CERIAS lab under the guidance of professor Eugene Spafford, graduating with a degree in computer science.

Dino A. Dai Zovi is a principal member of Matasano Security, where he performs ShipSafe product penetration tests for software vendors and DeploySafe third-party software penetration tests for enterprise clients. He specializes in product, application, and operating system penetration testing and has done so in his previous roles at Bloomberg, @stake, and Sandia National Laboratories. He is also a frequent speaker on his computer security research, including presentations at the Black Hat Briefings, IEEE Information Assurance Workshop, Microsoft’s internal Blue Hat Security Briefings, CanSecWest, and DEFCON. He graduated with honors with a bachelor of science in computer science and a minor in mathematics from the University of New Mexico.

Elfriede Dustin is author of Effective Software Testing and lead author of Automated Software Testing and Quality Web Systems, books that have been translated into various languages and that have sold tens of thousands of copies throughout the world. The Automated Testing Lifecycle Methodology (ATLM) described in Automated Software Testing has been implemented in various companies throughout the world. Dustin has written various white papers on software testing. She teaches various testing tutorials and is a frequent speaker at software testing conferences. In support of software test efforts, Dustin has been responsible for implementing automated test and has acted as the lead consultant/manager guiding the implementation of automated and manual software testing efforts. She is cochair of VERIFY, an annual international software testing conference held in the Washington, DC area. Dustin has a bachelor of science in computer science. She has more than 15 years of IT experience and currently works as an independent consultant in the Washington, DC area. You can reach her via her Web site at www.effectivesoftwaretesting.com.

Foreword and Preface

Foreword

Who can argue with testing things before you allow yourself to depend on them? No one can argue. No one will argue. Therefore, if testing is not done, the reasons have to be something other than a reasoned objection to testing. There seem to be exactly three: I can't afford it, I can get along without it, and I don't know how.

• Not being able to afford it—Allowing for economists to disagree over fine points, the cost of anything is the foregone alternative. If you do testing, what didn't you do? If it is to add yet another feature, perhaps you deserve congratulations on choosing a simpler product. Simpler products are in fact easier to test (and for good reason: the chief enemy of security is complexity, and nothing breeds complexity like creeping featuritis). If you didn't do testing, the usual reason given is to "get the product out on time." That reason is insufficient if not petulant. The sort of testing taught in this book is about the future even more than getting the product out on time is about the future. Only CEOs intoxicated on visions of wealth are immune to thinking about the future in ways that preclude testing. Testing is about controlling your future rather than allowing it to control you. Testing accelerates the inevitable future failure of products into the present. When William Gibson famously said, "The future is already here—it's just unevenly distributed," he wasn't thinking of testing as we mean it here. What you explicitly want is to unevenly distribute the future so that your product gets to see its future before your customers (and opponents) do. Since you are reading this paragraph, it's pretty likely you are of a testing frame of mind, so we'll drop the argument and move on.
• Getting along without it—Some products probably don't need much testing. They are not subject to innovation; they're nonperishable commodities, or something equally boring. That's not why we are here. We are here to protect security-sensitive products. Which products are those? A product is security-sensitive if, in its operation, it faces sentient opponents. If the only perils it faces are cluelessness ("Hey, watch this!") or random happenstance (alpha particles), the product may well not be security-sensitive. But with software and networks being as they are, nearly everything is security-sensitive because, if nothing else, every sociopath is your next-door neighbor. The burden of perfection is no longer on the criminal to commit the perfect crime but rather is on the defender to commit the perfect defense. Sure, you can get away with not testing, just as you can get away with never wearing protective gear while you band-saw aluminum, mountain-bike in Moab, or scrub down a P3 containment lab. There's always someone who has gotten away with that and more. That doesn't apply here. Why? Because the more successful and widespread your product is, the more those sociopaths, the more those sentient opponents, will adopt you as a special project. Just ask Microsoft. If you want to get widespread adoption, you will be tested. The only question is "Tested by whom?"
• Not knowing how—And so we come to the purpose of this book. You are ready, willing, and unable. Or you want to make sure that you're as up to date as your opponents. Or you need raw material for even more extreme sports than what is outlined here. You've come to a right place (there is no "the" right place). This is (let's be clear) a very right place. The authors are proven, and the techniques are current. Although techniques in security have the terrible beauty of never being "done," you won't do much better than these. If you can, there is an audience for your book. In the meantime, absorb what Chris Wysopal, Lucas Nelson, Dino Dai Zovi, and Elfriede Dustin have to teach you, and put it into practice. Skill sets like these do not grow on trees, and they don't stand still any more than the opposition stands still.

As you can see from the table of contents, testing is a way of thinking, not a button to press or a budget item to approve. You very nearly have to adopt this way of thinking—and nothing enforces a way of thinking as much as the regular use of tools and techniques that embody it. This is no joke. The outside attacker is skillful and increasingly professional and has tools and thought patterns. Malware—in particular, malware that turns good citizens into unintentionally bad citizens—has made true the long-standing supposition of security geeks: The real threat is the insider.

Question: What is an external attacker's first measure of success? Answer: Gaining an insider's credentials, access, and authority. If that attacker intends to do so by exploiting software the target insider runs, only your design and your testing stand in the way of the attacker's goals. As shown in the following figure, the idea is not "Does the product do what it is supposed to do?" but "Does the product not do what it supposed to not do?" That question is far harder than the quality assurance question because it is inherently open-ended. It cannot be fully handled by development per se. It has to be tested—preferably by informed testers not tangled up with the build process.

That, again, is where this book comes in. It tells you how to exert the kind of expert pressure that does accelerated failure time testing. You learn how to do so efficiently enough to be willing to do the testing and not think that you can get away without it. In other words, this is hunting in the bush. You can learn to do it by yourself, but following an expert tracker is a faster education than learning everything the hard way. Absorb everything that is here, and you'll either be a formidable hunter or you'll be in a position to be a tracker yourself. Remember, all skill is the result of practice. These authors are well practiced; it is your turn, and they have given you a leg up.

Daniel E. Geer, Jr., Sc.D.
24 July 2006

Endnote

1. Herbert H. Thompson and James A. Whittaker. Testing for Software Security. Dr. Dobb's Journal, 27(11): 24–32, November 2002.

Preface

With the growth of threats targeting computer systems, various organizations and institutions are looking for solutions that protect brand equity and customer confidence while minimizing maintenance costs.

The challenge is growing because of the widening scope of threats. Attackers started with UNIX-based Internet services and then moved to Windows-based PCs. Now they are beginning to target Apple MacOS X systems, networked video game consoles, wireless handheld devices, and even cell phones. After a software vulnerability is exposed, it often takes less than a week for hackers to come up with an exploit for it. And although the window of vulnerability has shrunk, it still takes about six weeks before software vendors issue a security patch. 1

Symantec has reported that attackers are moving away from large, multiple-purpose attacks against traditional security devices such as firewalls and routers. At Symantec, 69% of the vulnerabilities reported in the last half of 2005 were in Web applications.

Instead, attackers are focusing their efforts on regional targets, desktops, and Web applications that potentially allow an attacker to steal corporate, personal, financial, or confidential information. A new Symantec Internet Security Threat Report cites the growing trend of attackers using bot 2 networks, targeted attacks on Web applications and Web browsers, and modular malicious code.

Security issues often translate into the loss of revenue and reputation for an organization. They also can result in the loss of market share, which may be vital to the organization's future. The security firms Counterpane Internet Security and MessageLabs estimate that a piece of malware with a modest infection rate could cost a small company $83,000 a year—and may cost a large company $1 million or more. They add that these are direct losses and do not include indirect losses such as losses of reputation. 3

Dave Cullinane, chief information security officer of the financial firm Washington Mutual in Seattle says, "If you have an application exposed to the Internet that will allow people to make money, it will be probed." Cullinane believes the consequences of being breached are not only financial but also damaging to the company's reputation. "The reputation risk can literally put you out of business. Twenty percent to 45% of customers will leave you if you report a security breach." 4 CardSystems, which processes credit card transactions, nearly went out of business because of software that let criminals steal the private financial data of millions of customers. The com-pany that purchased CardSystems was ordered by the FTC to undergo independent audits for the next 20 years.

This book discusses how this type of insecure handling of sensitive data could have been prevented. Specifically, testing for these types of scenarios is covered in Chapters 6 and 7.

Additionally, this CardSystems security disaster could possibly have been prevented if security testing had been part of the company's process and if it had learned how to detect this type of situation. Then, when it was detected, the company could have taken the removal steps discussed. Keep in mind that it is against the VISA PCI regulations to store most secrets at all. This is a credit card security standard and security policy requirement that should have been implemented, as further discussed in Chapter 3.

The threat is enormous, according to Gartner, which says that 70% of business security vulnerabilities are at the application layer. This is compounded by 64% of in-house business software developers admitting that they lack the confidence to write secure applications, according to research done by Microsoft.

Implementing security testing early on can help block a wide spectrum of current and future threats. Protecting the security and integrity of customer data—including, for example, online transactions containing personal information—is critical to maintaining customers' trust.

This book addresses the challenges facing today's software security engineers, software project managers, and other software professionals responsible for their applications' security.

These professionals work to develop and deploy systems. They are under pressure to complete secure development efforts, incorporate upgrades, and maintain secure systems ahead of the competition.

This book's objective is to teach the engineers who build and test software that to protect their customers, they must perform security testing throughout the development process. Unless security is addressed throughout the development lifecycle, software will inevitably have security issues that can lead to the theft of financial information, the loss of data and performance, and the compromise of personal data. In the past, software vendors have been able to get away with not focusing much on security testing. They simply wait for an external security researcher or customer to find a security problem and report it to them. Then they fix the problem and issue a patch. Enterprise customers are becoming overwhelmed by the hundreds of patches they must test and install, sometimes on thousands of machines.

Software developers can prevent the bad publicity and customers' loss of trust by adding security testing to their development processes. Every software publisher has a quality process in which QA professionals create and execute tests to verify the software's functionality. Security testing can extend this process to verify that the software does not contain common classes of security vulnerabilities. In the past, poor-quality software providers have been chided for using their customers as testers. Today, companies that do not perform security testing are putting their customers at risk while they wait for an external person to come forward and tell them about their security problems. Unfortunately, not everyone is honest. Some people who discover security vulnerabilities in software will not tell the software vendor and may instead use the vulnerability to steal information.

In December 2005 criminals in Russia used a vulnerability in Internet Explorer called WMF to compromise unsuspecting Web surfers' machines and install bot software. One of the main reasons that this attack affected thousands of computers was because the vulnerability was discovered as it was being exploited in the wild before Microsoft could produce a fix. The window of vulnerability—the time between a vulnerability's being discovered and the vendor's having a fix available—is the riskiest time for customers who have a piece of vulnerable software. Security testing aims to eliminate the vulnerability. The window of vulnerability can be reduced by responsible disclosure. We can't prevent the window of vulnerability because it was found by "bad guys," but we can prevent the vulnerability itself.

Attacks on the Internet have changed over the last few years to become more criminal and insidious in nature. Three or four years ago the people taking advantage of software vulnerabilities were more interested in making a name for themselves with their peers, by writing and releasing viruses and worms, than in financial gain. The trend today is to use software vulnerabilities to help criminals gather financial, corporate, and government identity information that is then used to commit crimes. When home machines are compromised today, it is likely that they are used for phishing attacks or to relay spam. The software we use has not significantly improved with regards to security quality, yet today's attackers are more motivated than ever.

As a customer, you need to demand that your software suppliers integrate security into the development process. Request to see their process and tools. If the software is guarding financial data or other sensitive information, demand to see third-party validation of the supplier's security quality. Unless the industry voluntarily accepts security quality standards or the government mandates it, it is up to the customer to require proof of a certain level of security diligence. People can't occupy a building until the building inspector confirms that it is up to code. Automobiles and other potentially dangerous products also have requirements that must be independently tested. Today it is up to the software customer to perform these tests—or, more likely, to have a trusted third party perform the testing. Most corporations, however, do very limited third-party application security testing. Previously, this was done in federal organizations, but now it is spreading to larger financial institutions and enterprises. But even there, they don't have enough people adequately trained in black box security testing to be completely effective. Customers are required to do their own security testing (or hire third parties) because the vendors are not doing enough of it themselves.

Computer users accustomed to treating adware and spyware as just low-level annoyances were in for a surprise recently. According to Reuters, 5 a California man was indicted on federal charges of creating a robot-like network of hijacked computers that helped him and two others bring in $100,000 for installing unwanted adware.

The people behind adware always make money from low-level annoyances. However, keylogging spyware is the more insidious threat. Adware is just an annoyance, but hackers can use keylogging spy...

"Sobre este título" puede pertenecer a otra edición de este libro.