To increase the reliability of security testing results, developers are finding that security testing should be a combination of analysis techniques—utilizing source code analysis information to direct a second, more practical approach called dynamic analysis. This enables developers to identify vulnerabilities more accurately and confidently than with either technique individually. This combination approach, known as hybrid analysis, produces the accurate and reliable security information that developers need to assess the security of their code.
The Guessing Game
Source code analysis products use a technique called variable tracing. Developers use these tools to inject test data into the application during security testing to study the software’s potential values and behaviors through the call graphs that represent data flows through the application. By injecting test data this way, the source code analysis product infers what behavior may occur for a certain scenario and variable value–some refer to this technology as an inference engine.
The danger of the source code analysis technique is that it produces inferences, or guesses, as to how the system might behave during run-time and production configuration conditions. Source code analysis can only determine possible security vulnerabilities in the application, which usually results in high false positive rates during security testing.
In the security testing field, trusting the inferred results of source code analysis is analogous to trusting that an application will function according to design when it compiles cleanly. If all code is syntactically and semantically correct, then it will compile. But do you have confidence that it will meet the functional requirements simply because it compiled? Similarly, developers who rely on source code analysis to infer security problems in the application must also perform additional security testing in order to validate the application’s real run-time behavior with respect to the potential vulnerabilities.
Also known as automated penetration or fuzz testing, dynamic analysis occurs when a security tool actively attacks the running application based on thousands of known vulnerabilities and attack patterns. A dynamic analysis tool executes thousands of hack attempts on the application in a matter of minutes, just as a hacker would over days or weeks.
The danger of taking only the dynamic analysis approach is that it can be less thorough than source code analysis because it does not have access to or detailed knowledge of the application source code. Dynamic analysis tools are used during security testing to crawl an application like a Web spider to discover all of its pages and files and then use this site map to direct automated hack attempts.