TCP/IP basics and practical UNIX network tools
(last time edited: 2021-03-27)
tags: internet, tcp/ip, networks
I'm writing this blog to share and log some basic knowledge.
The Internet protocol suite is the conceptual model and set of communications protocols used in the Internet and similar computer networks. It is commonly known as TCP/IP because the foundational protocols in the suite are the Transmission Control Protocol (TCP) and the Internet Protocol (IP). During its development, versions of it were known as the Department of Defense (DoD) model because the development of the networking method was funded by the United States Department of Defense through DARPA. Its implementation is a protocol stack.
The Transmission Control Protocol and the Internet Protocol is a suite of communication protocols developed for military purposes *ehem and mass surveillance ehem* between 1969 and 1975 then later propagated to the civil and commercial world as we now see. There are other set of communication protocols that didn't prosper but we aren't gonna waste time on that.
The architecture model of TCP/IP consists in four layers. The Link Layer, the Internet Layer, the Transport Layer, and the Application Layer.
At application layer level we use lots of different protocols that play with TCP/IP. This is the first and most user friendly layer. In this post I'll play around with DHCP, TELNET, DNS, Dynamic DNS and more!
At transport layer level we will find protocols such as TCP, UDP and many others. We will stick to those.
At internet layer level we find another set of protocols. We'll take a look at IPv4, IPv6.
At link layer level we only will be messing around with Ethernet protocol. This layer is the physical and logical network component used to connect.
It is not necessary to learn every set of protocols from each layer but it's good to look at the big picture and grasp an idea. The order is important.
The best way to learn about networking is messing around with your own modem and router.
Other useful terms you need to understand are:
Local Area Network (LAN)
Wide Area Network (WAN)
Network Address Translation (NAT)
I'll add some basic explanations of protocols and services that work on top of the TCP/IP suite. Then I'll add some UNIX tools you can play with.
In computer networking, a port is a communication endpoint. At the software level, within an operating system, a port is a logical construct that identifies a specific process or a type of network service. A port is identified for each transport protocol and address combination by a 16-bit unsigned number, known as the port number. The most common transport protocols that use port numbers are the Transmission Control Protocol (TCP) and the User Datagram Protocol (UDP).
Ports are a gateway for transferring messages one way to another and viceversa.
In UNIX systems if you take a look at the
/etc/services file you'll find a list of ports reserved for specific software and protocols. This list of ports in your system is mantained by the IANA. Usually comes installed under a package with the name of iana-etc or just plain iana.
There are common misconceptions about ports. If a program with lots of vulnerabilities is running and listening on specific port, yes, it can be compromised. But life isn't a Hollywood movie. Don't be scared if you're not running a firewall. Remember your program has to be listening.
This is how ports are registered.
System (privileged - well-known) ports: 0 to 1023
User (non-privileged - registered) ports: 1024 to 49151
Ephemeral (dynamic - private) ports: 49152 to 65535
Universal Plug and Play (UPnP) is a set of networking protocols that permits networked devices, such as personal computers, printers, Internet gateways, Wi-Fi access points and mobile devices to seamlessly discover each other's presence on the network and establish functional network services for data sharing, communications, and entertainment. UPnP is intended primarily for residential networks without enterprise-class devices.
UPnP is not a protocol. UPnP is a service your programs use to automatically create port forwarding rules. Usually programs have their own way to communicate to the UPnP service, sometimes they depend of a client such as miniupnpc.
First enable UPnP on your router or modem.
Install miniupnpc in your UNIX system and you can mess with ports. Miniupnpc asks as a client to communicate with UPnP but could also run as as daemon.
If you wanna retrieve all current forwarded ports by UPnP run the next command:
$ upnpc -l
You can also see your public IP by using a binary that comes inside that package.
By the way, I don't recommend using UPnP. Just disable it, it could be very insecure. It's an old technology that could only be secure inside a LAN.
Port forwarding (Virtual servers)
Port forwarding, or tunneling, is the behind-the-scenes process of intercepting data traffic headed for a computer's IP/port combination and redirecting it to a different IP and/or port. A program that's running on the destination computer (host) usually causes the redirection, but sometimes it can also be an intermediate hardware component, such as a router, proxy server or firewall.
So you wanna forward ports manually instead of letting UPnP doing it automatically, huh? Connect to your router web admin and let's add some new rules, they need look something like this.
Service name will be your program's name but it doesn't really matter what you put here.
External port is the port that an outside client will be using along your public IP to connect. The public IP is the unique IP your ISP gives to your router. ISPs tends to just give one public IP.
Internal IP will be your LAN address of your device. If you want to mantain port forwarding rules for X amount of time you don't wanna be looking which LAN address your device has been assigned to by the router DHCP server. You might want to take a look at Address Reservation and add your device's MAC address so it never loses its assigned IP.
Internal port will be the same port as the external port. Not necessary to enter any input.
Protocol can be TCP, UDP or both. Programs can listen to both protocols or just one. It depends on the program. For example most online video games use UDP, connections are lossy. Torrents use TCP, connection needs to be reliable.
I'll show you some basic difference between TCP and UDP. I know this info is easy to find on the Internet, but it's good to have it here too for fast reference.
TCP vs UDP
|Ordered Data Delivery||No Sequence Guarantee|
|Error Free||Error Packets Discarded|
|Flow Control||No Flow Control|
|Relatively Slow||Relatively Fast|
|Point to Point||Supports Multicast|
|Security: TLS/SSL||Security: DTLS|
Port forwarding on a VPN
A common case for port forwarding is for web development and netplay. Especially with programs such as RetroArch using the Libretro.
There are some VPN that don't offer port forwarding.
You should only be using port forwarding on VPN if by some reason the modem/router your ISP gave you is giving you trouble to open ports and you cannot set up modem in bridge mode, etc.
It's very easy to set up a VPN using OpenVPN command-line program. Usually the VPN provider gives you their own configuration to launch OpenVPN to make the process automatic and simple.
I don't really recommend paying for a VPN since they are just bandwidth resellers and snake-oil. They also lie about not preserving logs and who knows their association. Yes, there are multiple uses for privacy (not anonimity), avoiding IP rangebans and such. But if you're truly paranoid your computer shouldn't be connected to the Internet.
If you want privacy avoid VPNs, disconnect your Ethernet cable and return to monkey.
Anyways VPNs work great for port forwarding.
DMZ stands for demilitarized zone.
Don't. Avoid it like the plague. Keep this setting disabled at all costs.
Dynamic DNS or DDNS is not a protocol but a service offered by Internet commercial companies to mantain your dynamic IP being linked to a subdomain.
It's useful when your dynamic IP changes over time and you need to share that address. Most common usage I know of is for homemade game servers.
I have used No-IP before and it works great. TP-Link routers usually comes with DynDNS and No-IP as default providers for this service.
These type of services are offered free, sometimes for more enhanced options you need to pay. It's understandable I guess.
Basically, comfort and easy sharing of your public IP at cost of your privacy and personal security.
I don't recommend using this method.
Access Control setting in routers
If you have ever seen this option in your routers, they are a gift from God itself. Start using it.
They let you block or accept incoming connections from specific MAC addresses.
It's a nice practice to whitelist your LAN devices. All WAN connection requests won't be able to reach you.
UNIX network utils
If you wanna get some basic info of your networks you can use some of these programs I will list below.
Keep in mind these are just a few commands that pops up on my head. There are lots! But for entry level management and analysis these are fine and very useful.
There are multiple ways to check if a program is listening behind a port.
A manual and rude way to check is using the telnet client program. It's part of the GNU network utilities bundle. You can install it by installing
inetutils-telnet in your Linux distribution, be sure to double check, programs are packaged differently depending the distribution. Maybe you can find it under the name of
$ telnet public_ip_here port_here
If it keeps on Trying... and nothing happens, then no program is listening on the other side.
This network auditing little tool can save your life. You can run it on IP or a domain and will scan ports. Nmap requires lots of dependencies to be run despite of its small size. I don't really recommend it, but it's a good tool.
$ nmap ip_here
$ nmap domain_here
$ nmap localhost
You can even specify it to run on specific port, or specific listening protocol.
$ nmap -p port ip_here
Similar tool. Comes in net-tools package. Does not need root permissions. Netstat is a smaller tool than ss.
$ netstat --tcp --udp --listening --program
$ netstat -tulp
Install lsof and you can see ports listening. You can even pipe stdout to grep. Does not need root permissions.
$ lsof | grep retroarch
If you wanna specify a protocol you can do it like this:
$ lsof -i UDP
$ lsof -i TCP
ss is used to dump socket statistics. It allows showing information similar to netstat. It can display more TCP and state information than other tools.
ss comes inside the package iproute2 distributed across all Linux distros. Does not require root permissions.
Usage is similar as other networking tools.
$ ss -tulw
You can use curl to see your current public IP.
$ curl icanhazip.com
This tool will help you find out how your packets are traveling through the network. Does not need root permissions.
It can also help you verifying if you are behind double NAT or not.
$ traceroute randomwebsite.com traceroute to randomwebsite.com (172.207.xx.38), 30 hops max, 60 byte packets 1 192.168.1.1 (192.168.1.1) 0.308 ms 0.479 ms 0.605 ms 2 * * * 3 10.xxx.111.33 (10.xxx.111.33) 18.142 ms 18.213 ms 18.130 ms ...
In the case above shows my local IP as first item. This is where the routing begins. With that info we can detect that I'm using a router connected to a bridged modem.
Otherwise if we connected our system straight to the bridged modem we would get something like this. Of course, all other LAN sockets from the bridged modem won't get Internet connection.
$ traceroute randomwebsite.com traceroute to randomwebsite.com (172.207.xx.38), 30 hops max, 60 byte packets 1 * * * 2 10.101.111.33 (10.101.111.33) 18.632 ms 18.699 ms 13.492 ms 3 10.100.26.14 (10.100.26.14) 19.733 ms 10.100.5.178 (10.100.5.178) 19.607 ms 10.100.26.14 (10.100.26.14) 19.434 ms ...
If you wanna read the incoming packets from an specific port in your server you can use Termshark. It is a terminal based interface for Wireshark.
Example to read connections. Does not need root permissions. This program also has lots of dependencies.
$ tshark -i ethernet-device-here -f "tcp port 55435"
or an example reading SOCKS5 connections on Loopback virtual device.
$ tshark -i lo -f "port 9050"
An alternative to Termshark. Checks if there is incoming traffic with a nice terminal UI. Needs root permissions.
# tcptrack -i ethernet-device-here tcp port 55435
or an example to read SOCKS5 traffic.
# tcptrack -i lo port 9050
Another alternative to check if there is incoming traffic from another system. Needs root permissions.
# tcpdump -n udp port 55435
Install openbsd-netcat to check if specific UDP ports in a system are open and listening. Does not need root permissions.
$ nc -vu 142.32x.23x.41 20595
I truly recommend tldr or its Rust implementation called tealdeer. It's an alternative to man-pages and it will help you save time looking for specific network analysis commands.
Welcome to the Internet.