-----BEGIN PGP SIGNED MESSAGE----- __________________________________________________________ The U.S. Department of Energy Computer Incident Advisory Capability ___ __ __ _ ___ / | /_\ / \___ __|__ / \ \___ __________________________________________________________ INFORMATION BULLETIN HP-UX SYN Flood Vulnerability May 1, 1997 23:00 GMT Number H-50 ______________________________________________________________________________ PROBLEM: A vulnerabilities have been addressed for TCP SYN flooding denial of service PLATFORM: HP-UX 9 and 10 DAMAGE: Networked hosts can be made unavailable. SOLUTION: Apply patches as indicated below. ______________________________________________________________________________ VULNERABILITY Exploit details involving this vulnerability have been made ASSESSMENT: publicly available. ______________________________________________________________________________ [ Start Hewlett-Packard Advisories ] Document ID: HPSBUX9704-060 Date Loaded: 970501 Title: SYN Flooding Security Vulnerability in HP-UX - ------------------------------------------------------------------------- HEWLETT-PACKARD SECURITY BULLETIN: HPSBUX9704-060, 30 April 1997 - ------------------------------------------------------------------------- The information in the following Security Bulletin should be acted upon as soon as possible. Hewlett Packard will not be liable for any consequences to any customer resulting from customer's failure to fully implement instructions in this Security Bulletin as soon as possible. - ------------------------------------------------------------------------- PROBLEM: Vulnerability to 'SYN Flood' denial of service (DOS) attack PLATFORM: HP 9000 Series 700/800s running versions of HP-UX 9.X & 10.X DAMAGE: Potential denial of service for network users. SOLUTION: If protection is needed in your environment; Apply the appropriate patch and enable/tune the defense mechanism. A white paper and tuning script are included within this bulletin to assist with the tuning process. AVAILABILITY: All patches are available now. - ------------------------------------------------------------------------- I. A. Background B. Fixing the problem C. Recommended solution D. Impact of the patch Please refer to the following white paper for this information. The white paper also includes a shell archive containing the script to enable and tune the syn-flood defense mechanism. __________________________________________________________________________ SYN Attack And HP-UX's Solution Rev. 1 1. Introduction This paper explains what a SYN attack is, briefly describes what defenses are available today, and describes the HP-UX solution available today. It is assumed that the reader has a basic knowledge of TCP/IP and Socket. In particular, the reader is expected to know the fields in a IP header and a TCP header, and the handshake in establishing a TCP connection. 2. What is a SYN attack? SYN attack is a denial of service attack in that at least one internet port is blocked from legitimate access. The attacker achieves this by sending enough packets to targeted ports to completely block or severely curtail access to these ports. These packets are legal packets in compliance with TCP/IP protocols, except that they carry faked source addresses. SYN attack is one of the more severe denial of service attacks, since every faked SYN packet can disproportionately consume a system's resources for a disproportional amount of time. A TCP connection establishment process normally takes an exchange of three TCP packets: an initial SYN packet from a client, a SYN-ACK packet from a server, and a SYN-ACK-ACK packet from the client. Since the source address of the attacker's SYN packet is faked, the SYN-ACK-ACK packet will never come. Until the connection establishment process times out, a disproportional amount of system resources are occupied: a slot in the attacked port's listen queue, memory to maintain connection information, and CPU and network bandwidth to retransmit the SYN-ACK packet. A TCP listen port has a finite number of slots in its listen queue and normally that number of slots is relatively small. When an attacker sends enough faked SYN packets, the listen queue can be fully occupied and subsequently deny any legitimate SYN packet from entering into the listen queue. 3. What are the defenses today against a SYN attack? The best defense is to stop it at the source. End systems should not allow unauthorized users or applications to generate any faked SYN packet. Access to raw socket interface should be restricted to trusted users or applications. Routers may provide a second line of defense by screening incoming IP packets to make sure that they are actually coming from valid sources. Certain firewall products today also can filter off faked IP packets. End systems can also provide a last line of defense by accommodating a much larger number of incoming SYN packets and appropriately replacing those half-open connections that have been sitting in the listen queue. 4. HP-UX's solution today HP-UX restricts raw socket access to root. Raw socket is not an officially supported interface for normal users on HP-UX. Applying the appropriate patch (or a superseding patch) from the list below provides defense against SYN attacks that reach the machine. Patch Number Release Hardware Platform - ------------------------------------------------------ PHNE_9525 9.0 s800 PHNE_10864 9.01 s700 PHNE_9100 9.03, 9.05, 9.07 s700 PHNE_9101 9.04 s800 PHNE_9102 10.01 s700 PHNE_9103 10.01 s800 PHNE_9104 10.10 s700 PHNE_9105 10.10 s800 PHNE_9106 10.20 s700 PHNE_9107 10.20 s800 A system wide kernel parameter is provided to set a minimal length for a listen socket queue without requiring programatic change. A replacement algorithm is used to remove a half-open connection from the listen socket queue when the listen socket queue is full. 4.1. Setting up a SYN attack defense on HP-UX There are a couple kernel parameters you will have to set. A shell script called syn_defense may be used to set these kernel parameters: the script will modify both the core image and the kernel file, so the modification takes place immediately, and persists across reboots. A copy of the syn_defense script in the form of a shar file is attached to the end of this paper. 1. hp_syn_protect By default, the SYN attack defense is not turned on. To turn it on, set hp_syn_protect to 1. To turn it off, set hp_syn_protect to 0. As explained in more detail below, turning on SYN attack defense will change the system behavior, and in a stress condition can consume more memory and CPU resources even if the system is not under attack. Because only a very small percentage of HP systems may be at risk of SYN attacks, the SYN attack defense is not turned on by default. 2. so_qlimit_min When enabled, so_qlimit_min specifies the minimum length of a listen socket queue, applications requesting less will be given so_qlimit_min entries. When the socket queue limit is reached, any new incoming TCP connection request will replace one of the pending TCP connections in the socket queue using a HP chosen replacement algorithm. By default, so_qlimit_min is set to 500. This value should comfortably defend against an attacker using a 56K baud modem. Consult the section below for different exposures. 4.2. Determining a right so_qlimit_min value for a system A proper value for so_qlimit_min can be derived from the following formula that calculates the probability of a successful connection establishment while a system is under a SYN attack: P = ((L-1)/L)^(T*R) where P = The probability a valid SYN packet can still be processed and be turned into an established TCP connection while a system is under a SYN attack. L = so_qlimit_min T = Time in seconds that it normally takes between sending the SYN-ACK packet and receiving the SYN-ACK-ACK packet. This can be approximated by the round trip time as reported by the ping command. R = Incoming rate of SYN packets in packets per second during a SYN attack. To come up a number with a high confidence of success, a worse case estimate may be used. For example, the full bandwidth of a dial-up link may be assumed to be utilized by an attacker. The intermediate routers may be assumed not to introduce any delay between packets. With these assumptions, the incoming rate can be derived from the the formula below: R = B/S, where B = Bandwidth in bits/sec, S = SYN packet size in bits = (F + IP header size + TCP header size)*(8 + I) = (F + 20 + 20)*(8 + I), where F = Frame overhead in bytes per packet I = Link overhead in bits per packet byte A formula for so_qlimit_min can be derived from the above probability formula: L = 1/(1 - P^(1/(T*R))) Following is an example showing how to estimate a desired so_qlimit_min value. Suppose a 70% success rate is desired during an attack through a 56K baud SLIP dialup link. In that case, P=.7 B=57344 F=2 I=2 S= (2 + 20 + 20)*(8+2) = 420 R= B/S = 57344/420 = 137 (round up to nearest integer) Let T= 1 sec. L= 1/(1 - .7^(1/137)) = 385 (round up to nearest integer) Note, in SLIP there is 1 END byte in front and 1 END byte at the end of a packet. Since only a ballpark number is needed, it can be assumed that there is no END character or a SLIP ESC character within a SYN packet itself. It is also assumed that 1 START bit and 1 STOP bit is used per packet byte. The round trip time, T, is set to 1 sec. in this calculation. To establish a round trip time for a system, one may identify a farthest node from the system and use the ping command to sample the round trip time between that node and the system. 4.3. What impacts are there to the system if the SYN attack defense is turned on? In general, there should not be any direct noticeable performance impact under normal conditions. However, turning SYN attack defense on will change the system behavior. High connection attempt rates on a listen socket will result in some of the client applications seeing ECONNREFUSED instead of ETIMEOUT. Likewise, more system resources may be held by the application than normal under these circumstances. 4.4. Memory Requirement Amount of memory consumed by faked SYN packets during an attack is proportional to the attack rate. The worst case requirement can be approximated with the formula below: M = 32700 * R where M = memory in bytes, R = Incoming SYN attack rate in packets per second To fully protect against an attacker using a 56K baud modem, approximately 4.3 megabytes of memory should be added to the networking memory pool. 5 Conclusion SYN attack is a sophisticated attack against a system attached to TCP/IP networks. With the technology today, an effective defense should be a multi-layer approach, using strict access control at the source, source screening in the intermediate routers and firewalls, and SYN attack defense solution at the end system. With sufficient memory, HP-UX can provide an effective last line of defense against a SYN attack. #------------------------------ cut here ---------------------------------- # This is a shell archive. Remove anything before this line, # then unpack it by saving it in a file and typing "sh file". # # This archive contains: # syn_defense # # Modification/access file times will be preserved. # Error checking via wc(1) will be performed. # Error checking via sum(1) will be performed. # Files are compressed using compress(1). LANG=""; export LANG PATH=/bin:/usr/bin:$PATH; export PATH if sum -r /dev/null 2>&1 then sumopt='-r' else sumopt='' fi rm -f /tmp/uud$$ (echo "begin 666 /tmp/uud$$\n#;VL*n#6%@x\n \nend" | uudecode) >/dev/null 2>&1 if [ X"`cat /tmp/uud$$ 2>&1`" = Xok ] then unpacker=uudecode else echo Compiling unpacker for non-ascii files pwd=`pwd`; cd /tmp cat >unpack$$.c <<'EOF' #include #define C (*p++ - ' ' & 077) main() { int n; char buf[128], *p, a,b; scanf("begin %o ", &n); gets(buf); if (freopen(buf, "w", stdout) == NULL) { perror(buf); exit(1); } while (gets(p=buf) && (n=C)) { while (n>0) { a = C; if (n-- > 0) putchar(a << 2 | (b=C) >> 4); if (n-- > 0) putchar(b << 4 | (a=C) >> 2); if (n-- > 0) putchar(a << 6 | C); } } exit(0); } EOF cc -o unpack$$ unpack$$.c rm unpack$$.c cd $pwd unpacker=/tmp/unpack$$ fi rm -f /tmp/uud$$ echo x - syn_defense '[compressed]' $unpacker <<'@eof' Attachment Converted: c:\eudora\attach\syn_defense @eof uncompress /tmp/compress$$ mv /tmp/compress$$ syn_defense set `sum $sumopt