The netstat utility summarizes a variety of characteristics of the networking stack. With netstat you can learn a number of important things. If no other type of data is requested it will report on the state of all active sockets. You can however request the routing table, masquerading table, network interface statistics, and network stack statistics [60].
One of the most common uses of the netstat utility is to determine the state of sockets on a machine. There are many questions that netstat can answer with the right set of options. Here's a list of some of the things different things we can learn.
which services are listening on which sockets
what process (and controlling PID) is listening on a given socket
whether data is waiting to be read on a socket
what connections are currently established to which sockets
By invoking netstat without any options, you are asking for a list of all currently open connections to and from the networking stack on the local machine. This means IP network connections, unix domain sockets, IPX sockets and Appletalk sockets among others. Naturally, we'll skip over the non-IP sockets since this is about IP networking with linux.
Assume the --inet switch in all cases below unless
we are examining a particular higher layer protocol (e.g., TCP with
the --tcp switch or UDP with --udp
switch.
A convenient feature of netstat is its ability to
differentiate between two different sorts of name lookup. Normally
the -n specifies no name lookup, but this is
ambiguous when there are hostnames, port names, and user names.
Fortunately, netstat offers the following options
to differentiate the different forms of lookup and suppress only the
[un-]desired lookup.
--numeric-hosts
--numeric-ports
--numeric-users
The option -n (my favorite), suppress all hostname,
port name and username lookup, and is a synonym for
--numeric. I'll reiterate that hostnames and DNS in
particular can be confusing, or worse, misleading when trying to
diagnose or debug a networking related issue, so it is wise to
suppress hostname lookups in these sorts of situations.
In Example G.11, “Displaying IP socket status with netstat” we will look at netstat's numeric output and then we'll invoke the same command but suppress the host lookups. Though the output is almost the same, a particular situation might call for one or the other invocation.
Example G.11. Displaying IP socket status with netstat
[root@morgan]#netstat --inet -nActive Internet connections (w/o servers) Proto Recv-Q Send-Q Local Address Foreign Address State tcp 0 192 192.168.98.82:22 192.168.99.35:40991 ESTABLISHED tcp 0 0 192.168.98.82:42929 192.168.100.17:993 ESTABLISHED tcp 96 0 127.0.0.1:40863 127.0.0.1:6010 ESTABLISHED tcp 0 0 127.0.0.1:6010 127.0.0.1:40863 ESTABLISHED tcp 0 0 127.0.0.1:38502 127.0.0.1:6010 ESTABLISHED tcp 0 0 127.0.0.1:6010 127.0.0.1:38502 ESTABLISHED tcp 0 0 192.168.98.82:53733 209.10.26.51:80 SYN_SENT tcp 0 0 192.168.98.82:44468 192.168.100.17:993 ESTABLISHED tcp 0 0 192.168.98.82:44320 192.168.100.17:139 TIME_WAIT[root@morgan]#netstat --inet --numeric-hostsActive Internet connections (w/o servers) Proto Recv-Q Send-Q Local Address Foreign Address State tcp 0 0 192.168.98.82:ssh 192.168.99.35:40991 ESTABLISHED tcp 0 0 192.168.98.82:42929 192.168.100.17:imaps ESTABLISHED tcp 0 0 127.0.0.1:40863 127.0.0.:x11-ssh-offset ESTABLISHED tcp 0 0 127.0.0.:x11-ssh-offset 127.0.0.1:40863 ESTABLISHED tcp 0 0 127.0.0.1:38502 127.0.0.:x11-ssh-offset ESTABLISHED tcp 0 0 127.0.0.:x11-ssh-offset 127.0.0.1:38502 ESTABLISHED tcp 0 0 192.168.98.82:53733 209.10.26.51:http SYN_SENT tcp 0 0 192.168.98.82:44468 192.168.100.17:imaps ESTABLISHED tcp 0 0 192.168.98.82:44320 192.168.100:netbios-ssn TIME_WAIT
Each line represents a either the sending or
receiving half of a connection. In the above output on morgan it
appears that there are no connections other than TCP connections.
If you are very familiar with TCP ports and the service associated
with that port, then the first format will suffice in most cases. A
possibly misleading aspect of the latter output is visible in the
connections to and from localhost and the final line.
netstat abbreviates the IP endpoint in order to
reproduce the entire string retrieved from the port lookup
(in /etc/services). Also interestingly, this
line conveys to us (in the first output) that the kernel is
waiting for the remote endpoint to acknowledge the 192 bytes which
are still in the Send-Q buffer.
The first line describes a TCP connection to the IP locally hosted
on morgan's Ethernet interface. The connection was initiated from
an ephemeral port (40991) on tristan to a service running on port
22. The service normally running on this well-known port is sshd,
so we can conclude that somebody on tristan has connected to the
morgan's ssh server.
The second line describes a TCP session open to port 993 on
isolde, which probably means that the user on morgan has an open
connection to an IMAP over SSL server.
The third through the sixth lines can be understood in pairs. By examining the source and destination IP and port pairs, we can see that two different TCP sessions have been established with the source and destination address of 127.0.0.1. For an administrator to publish services on localhost is not at all uncommon. This makes the service harder to abuse from the network. In this case, when we allow the service lookup, the port in question (6010) appears to be used to tunnel forwarded X applications over ssh.
The next line is the first TCP session in our output which is not in a state of ESTABLISHED. Refer to Table G.1, “Possible Session States in netstat output” for a full list of the possible values of the State field in the netstat output. The state SYN_SENT means that an application has made arequest for a TCP session, but has not yet received the return SYN+ACK packet.
The final line of our netstatoutput shows a connection in the TIME_WAIT state, which means that the TCP sessions have been terminated, but the kernel is waiting for any packets which may still be left on the network for this session. It is not at all abnormal for sockets to be in a TIME_WAIT state for a short period of time after a TCP session has ended.
If we needed to know exactly which application owned a particular
network connection, we would use the -p | --program
switch which gives us the PID and process name of the owner process.
If we want to see the unix user and the PID and process we'll add the
-e | --extend switch.
Example G.12. Displaying IP socket status details with netstat
[root@masq-gw]#netstat -p -e --inet --numeric-hostsProto Recv-Q Send-Q Local Address Foreign Address State User Inode PID/Program name tcp 0 0 192.168.100.254:ssh 192.168.100.17:49796 ESTABLISHED root 25453 6326/sshd tcp 0 240 192.168.99.254:ssh 192.168.99.35:42948 ESTABLISHED root 171748 31535/sshd
There doesn't appear to be a large number of connections to and
from the masq-gw host. The two sessions are initiated to the sshd
running on port 22, and the process which owns each socket is a root
process.
Table G.1. Possible Session States in netstat output
| State | Description |
|---|---|
| LISTEN | accepting connections |
| ESTABLISHED | connection up and passing data |
| SYN_SENT | TCP; session has been requested by us; waiting for reply from remote endpoint |
| SYN_RECV | TCP; session has been requested by a remote endpoint for a socket on which we were listening |
| LAST_ACK | TCP; our socket is closed; remote endpoint has also shut down; we are waiting for a final acknowledgement |
| CLOSE_WAIT | TCP; remote endpoint has shut down; the kernel is waiting for the application to close the socket |
| TIME_WAIT | TCP; socket is waiting after closing for any packets left on the network |
| CLOSED | socket is not being used (FIXME. What does mean?) |
| CLOSING | TCP; our socket is shut down; remote endpoint is shut down; not all data has been sent |
| FIN_WAIT1 | TCP; our socket has closed; we are in the process of tearing down the connection |
| FIN_WAIT2 | TCP; the connection has been closed; our socket is waiting for the remote endpoint to shut down |
One of the most common uses of netstat, especially in cross-platform environments is the reporting of the main routing table. On many platforms, netstat -rn is the preferred method of displaying routing information, although linux provides at least two alternatives to this: route and ip route show.
Example G.13. Displaying the main routing table with netstat
[root@morgan]#netstat -rnKernel IP routing table Destination Gateway Genmask Flags MSS Window irtt Iface 192.168.98.0 0.0.0.0 255.255.255.0 U 40 0 0 eth0 127.0.0.0 0.0.0.0 255.0.0.0 U 40 0 0 lo 0.0.0.0 192.168.98.254 0.0.0.0 UG 40 0 0 eth0
This output should look familiar. The routing cache itself may not be as familiar to most, but can also be displayed with netstat. The ouput below is exactly the same as the ouput from route -enC. Refer also to Example D.3, “Viewing the routing cache with route”.
Example G.14. Displaying the routing cache with netstat
[root@tristan]#netstat -rnCKernel IP routing cache Source Destination Gateway Flags MSS Window irtt Iface 194.52.197.133 192.168.99.35 192.168.99.35 l 40 0 0 lo 192.168.99.35 194.52.197.133 192.168.99.254 1500 0 29 eth0 192.168.99.35 192.168.99.254 192.168.99.254 1500 0 0 eth0 192.168.99.254 192.168.99.35 192.168.99.35 il 40 0 0 lo 192.168.99.35 192.168.99.35 192.168.99.35 l 16436 0 0 lo 192.168.99.35 194.52.197.133 192.168.99.254 1500 0 0 eth0 192.168.99.35 192.168.99.254 192.168.99.254 1500 0 0 eth0
Consult Section 1.1, “Displaying the routing table with route” for more detail on reading and interpreting the data in this output. Because this is simply another way of reporting the routing table information, we'll skip over any detailed description.
netstat -i summarizes interface statistics in a terse format. This format
OK! This is strange. netstat -ie looks exactly like ifconfig output. That's weird!
For machines which perform masquerading, typically dual-homed
packet-filtering firewalls like masq-gw a tool to list the current
state of the masquerading table is convenient.
Each masqueraded connection can be described by a tuple of six pieces of data: the source IP and source port, the destination IP and destination port, and the (usually implicit) locally hosted IP and a local port.
FIXME; this command seems to fail on all of the iptables boxen, even
if I'm using the -j MASQUERADE target. I
can use it successfully on ipchains boxen. Anybody have any ideas
or explanation here?
[60]
Additionally, netstat can display multicast
information with the --group switch. I have zero
experience here. Anybody with experience want to write about
this?