(Last Updated On: September 10, 2018)

TCP (Transmission Control Protocol) is a standard that defines how network conversation between two systems is established and maintained to facilitate an exchange of data between applications. Internet Protocol (IP) defines how systems send packets of data to each other.

The TCP States in Linux

Below is a list of TCP connection states that can be viewed using netstat or ss command on Linux.

ESTABLISHED                                                                                                                                    
     The socket has an established connection.                                                                                               
                                                                                                                                                      
SYN_SENT                                                                                                                                       
     The socket is actively attempting to establish a connection.                                                                            
                                                                                                                                                      
SYN_RECV                                                                                                                                       
     A connection request has been received from the network.                                                                                
                                                                                                                                                      
FIN_WAIT1                                                                                                                                      
    The socket is closed, and the connection is shutting down.                                                                              
                                                                                                                                                      
FIN_WAIT2                                                                                                                                      
    Connection is closed, and the socket is waiting for a shutdown
    from the remote end.                                                     
                                                                                                                                                      
TIME_WAIT                                                                                                                                      
    The socket is waiting after close to handle packets still in 
    the network.                                                               
                                                                                                                                                      
CLOSE  The socket is not being used.                                                                                                           
                                                                                                                                                      
CLOSE_WAIT                                                                                                                                     
    The remote end has shut down, waiting for the socket to close.                                                                          
                                                                                                                                                      
LAST_ACK                                                                                                                                       
    The remote end has shut down, and the socket is closed.
     Waiting for acknowledgement.                                                    
                                                                                                                                                      
LISTEN The  socket  is  listening for incoming connections.
        Such sockets are not included in the output unless you 
        specify the --listening    
        (-l) or --all (-a) option.                                                                                                              
                                                                                                                                                      
CLOSING                                                                                                                                        
   Both sockets are shut down but we still don't have all our 
   data sent.                                                                   

UNKNOWN
    The state of the socket is unknown.

For the difference in usage between ss and netstat command, check netstat vs ss usage guide on Linux. Use the command below to check all applications TCP states on your Linux server, it will give you the number of processes in each state.

# netstat -nat | awk '{print $6}' | sort | uniq -c | sort -r
   8959 CLOSE_WAIT
    887 FIN_WAIT2
      6 SYN_RECV
   5597 TIME_WAIT
    472 ESTABLISHED
     24 LISTEN
      1 SYN_SENT
      1 Foreign
      1 FIN_WAIT1
      1 established)
    183 LAST_ACK

To understand the options used in the command, read netstat vs ss usage guide on Linux. You can also get the list of processes in a particular state by piping the output to grep. For example to get processes in CLOSEWAIT state, use   # netstat -apn | grep CLOSE_WAIT You can further filter this output to get process ID of the processes in CLOSEWAIT state.

netstat -apn | grep CLOSE_WAIT | awk '{ print $7 }' | sort | uniq -c | sort -nr

If you want to limit the output to top 10 processes with CLOSE_WAIT TCP connection state, use head

# netstat -apn | grep CLOSE_WAIT | awk '{ print $7 }' | sort | uniq -c | sort -nr | head -n 10
   3856 8166/jsvc.exec
   1783 15643/jsvc.exec
   1313 26749/jsvc.exec
   1203 11450/jsvc.exec
    563 22495/jsvc.exec
    270 6698/jsvc.exec
    229 22625/jsvc.exec
      9 9729/jsvc.exec
      2 32038/httpd
      2 29352/httpd

This shows that the Process with ID  8166 has 3856 CLOSE_WAIT connection states.

If you’re running short of TCP connections or doing troubleshooting, you may need to identify this process with a large number of CLOSE_WAIT connection states. It could mean that the application doesn’t close connections as expected.

# ps 8166
  PID TTY      STAT   TIME COMMAND
 8166 ?        Sl   242:29 jsvc.exec -debug -pidfile /var/run/myapp.pid myapp.jar

I made a simple bash script which uses the commandnetstat to identify count for TCP connection states and the processes with many states in CLOSE_WAIT.

#!/bin/bash
# Script to print Linux TCP connections using netstat
# Guthub: https://github.com/jmutai
#  
#    vvvv vvvv-- the code from above
RED='\033[0;31m'
NC='\033[0m' # No Color
echo ""
echo -en "${RED} ALL TCP Connections Count: ${NC}\n"
netstat -nat | awk '{print $6}' | sort | uniq -c | sort -r
echo ""
echo -en "${RED} Top CLOSE_WAIT state TCP Connections: ${NC}\n"
netstat -apn | grep CLOSE_WAIT | awk '{ print $7 }' | sort | uniq -c | sort -nr | head -n 10

Sample output:

Thanks for your time.