Structure
Some networking basics:
How to Network's "Network Fundamentals - TCP/IP"
Magda El Zarki's "Introduction to TCP/IP"
Linux Network Interfaces:
Network interface types:
Physical:
systemd's "Predictable Network Interface Names"
Chris Siebenmann's "Understanding something about udev's normal network device names"
Virtual:
- wl*: Wi-Fi.
- ???: Bluetooth.
- ???: NFC.
- e*: Ethernet.
systemd's "Predictable Network Interface Names"
Chris Siebenmann's "Understanding something about udev's normal network device names"
Virtual:
- lo*: loopback.
- tun*: VPN tunnel.
- v*: virtual machine adapter.
- wg*: WireGuard VPN.
ip addr ls -l /sys/class/net/ nmcli d nmcli device show nmcli connection show sudo ethtool eno1 sudo ethtool -i eno1 # show driver info rfkill list nmcli radio networkctl list wpa_cli list_networks iwconfig iwlist iwlist wlp1s0 frequency iw list iwctl station list man bluetoothctl nfctool --list
udev connects interface name to device filename, which maps to device type and major/minor number, which maps inside the kernel to a driver ?
Routing tables:
ip rule # first col = priority; lower num = higher priority ip route show table local ip route show table main ip route show table default # Note: "proto" is just a label for what package installed # this rule, not a filter like "ipproto tcp". # With VPN on, probably main table will have a rule: # default via 10.n.n.n dev tun0 ... # and a low "metric" number, # which means "anything not matched by other rules # goes to the tun0 device". The other rules all should # apply to specific addresses or have a higher "metric". # With VPN off, probably main table will have only a couple # of rules, but the one for public internet is: # default via 192.168.1.1 dev eno1 ...
Changes made via "ip route" or "ip rule" are not persistent; they go away when you reboot. To make persistent changes, you have to edit network config files. Maybe define your own new table (add it to cat /etc/iproute2/rt_tables), populate it with rules via "ip route add", save/load contents via "ip route save" and "ip route restore".
Scott Lowe's "A Quick Introduction to Linux Policy Routing"
Vivek Gite's "Linux Set Up Routing with ip Command"
IPROUTE2 Utility Suite Howto
Paul Gorman's "iproute2 cheat sheet"
A virtual device interface can change the IP address and send the altered request for routing again. Eventually the request should end up at a physical device interface and go out to a network.
To block addresses using ip rule / ip route:
sudo ip route add table main blackhole IPADDRESS sudo ip route add table main blackhole IPADDRESSPREFIX/24 sudo ip rule add to IPADDRESS table main blackhole sudo ip rule add to IPADDRESSPREFIX/24 dport 25-30 ipproto tcp table main blackhole sudo ip rule add from IPADDRESS table main blackhole
IPdeny's "country block downloads"
Ways to implement VPN:
- TCP(?)-level: e.g. OpenVPN. Default device is tun*.
Encrypted tunnel to a VPN server, which does user authentication and then implements a proxy (changing IP address) and goes to internet or LAN. - IP-level: e.g. WireGuard. Default device is wg*.
Key set-up is done in advance, when the connection is first defined. So there is no user authentication to do for each connection; just use the keys. Encrypted tunnel to a VPN server, which implements a proxy (changing IP address) and goes to internet or LAN.
Ways to implement firewall:
- Dynamic code snippets inside kernel: eBPF.
- Kernel modules: e.g. iptables, nftables.
- Routing rules: e.g. ip rule, ip route.
Each kernel module usually has a user-space utility to configure it. Kernel module ip_tables is controlled by utility iptables, kernel module nftables is controlled by utility nft ("sudo nft list ruleset | less").
Application sandboxes (Firejail, AppArmor, SELinux) can disable networking or maybe filter it ?
Linux system control:
- SysV init and service: Older way of doing things. Run-levels for init.
"chkconfig --list"
- systemd: New way of doing things. Targets and run-units.
"pidof systemd"
my "Linux systemd" page
Linux Network control:
Usually a system is running either Network Manager (usually on desktop machines) or systemd-networkd (more common in servers), but not both.
article
Network Manager
"sudo systemctl status NetworkManager.service --full --lines 1000"
"man nmcli"
"man nmtui"
"nmcli connection show"
"nmcli device"
Has new ifup/ifdown scripts.
Schkn's "Network Manager on Linux with Examples"
David Both's "Get started with NetworkManager on Linux"
ArchWiki's "NetworkManager"
GNOME Developer's "NetworkManager"
Chris Siebenmann's "Desktops don't always use NetworkManager's programs"
Network Manager connectivity-checking
"man nmcli"
"man nmtui"
"nmcli connection show"
"nmcli device"
Has new ifup/ifdown scripts.
Schkn's "Network Manager on Linux with Examples"
David Both's "Get started with NetworkManager on Linux"
ArchWiki's "NetworkManager"
GNOME Developer's "NetworkManager"
Chris Siebenmann's "Desktops don't always use NetworkManager's programs"
Network Manager connectivity-checking
Does a ping every 5 minutes ?
Justin Chapin's "How to prevent NetworkManager connectivity checking"
"journalctl --since=today | grep connectivity"
Arch: "cat /etc/NetworkManager/conf.d/20-connectivity.conf"
"cat /etc/NetworkManager/NetworkManager.conf"
"man NetworkManager.conf" and see "Connectivity" section.
Justin Chapin's "How to prevent NetworkManager connectivity checking"
"journalctl --since=today | grep connectivity"
Arch: "cat /etc/NetworkManager/conf.d/20-connectivity.conf"
"cat /etc/NetworkManager/NetworkManager.conf"
"man NetworkManager.conf" and see "Connectivity" section.
systemd-networkd
Alternative to Network Manager ?
"Use systemd-networkd to configure your devices and iwctl to configure your Wi-Fi connections."
"sudo systemctl status systemd-networkd.service --full --lines 1000"
"networkctl list"
"networkctl status"
"cat /etc/systemd/networkd.conf"
"ls /etc/systemd/network"
Derrik Diener's "How to set up systemd-networkd on Linux"
Sahitya Maruvada's "Working with systemd-networkd"
"Use systemd-networkd to configure your devices and iwctl to configure your Wi-Fi connections."
"sudo systemctl status systemd-networkd.service --full --lines 1000"
"networkctl list"
"networkctl status"
"cat /etc/systemd/networkd.conf"
"ls /etc/systemd/network"
Derrik Diener's "How to set up systemd-networkd on Linux"
Sahitya Maruvada's "Working with systemd-networkd"
Debian inherent networking.service
"sudo systemctl status networking.service --full --lines 1000"
"ls /etc/network/interfaces" ?
"ls /etc/sysconfig/network-scripts" ?
"ls /etc/network/interfaces" ?
"ls /etc/sysconfig/network-scripts" ?
ConnMan
Netplan
"netplan is a Ubuntu tool to create a configuration file for networking.
netplan can create either a NetworkManager configuration or a systemd-networkd
configuration file at system boot time."
"ls /etc/netplan"
Egidio Docile's "Netplan network configuration tutorial for beginners"
Linux Shell Tips' "How to Set Static IP Address on Ubuntu Linux"
Canonical's "Netplan reference"
From someone on reddit 2/2021:

I don't think it's aimed at the non-skilled developer. Netplan is the kind of thing aimed for cloud, servers, rpi enthusiasts who want a permanent network configuration to set and forget. It has one advantage compared to the others, and that is that the configuration can exist in /boot. As such, it is possible to edit/adjust the network configuration if you need to adjust it from a separate machine.
For that purpose it does what it's intended to do, although I would argue it's superfluous considering systemd-networkd meets the exact same need as set-forget network configuration that is actually sandboxed. I also think netplan's configuration.yaml is a lot nicer than systemd's units stuff.
"ls /etc/netplan"
Egidio Docile's "Netplan network configuration tutorial for beginners"
Linux Shell Tips' "How to Set Static IP Address on Ubuntu Linux"
Canonical's "Netplan reference"
From someone on reddit 2/2021:
I don't think it's aimed at the non-skilled developer. Netplan is the kind of thing aimed for cloud, servers, rpi enthusiasts who want a permanent network configuration to set and forget. It has one advantage compared to the others, and that is that the configuration can exist in /boot. As such, it is possible to edit/adjust the network configuration if you need to adjust it from a separate machine.
For that purpose it does what it's intended to do, although I would argue it's superfluous considering systemd-networkd meets the exact same need as set-forget network configuration that is actually sandboxed. I also think netplan's configuration.yaml is a lot nicer than systemd's units stuff.
network-scripts
Deprecated.
Uses old ifup/ifdown scripts.
Uses old ifup/ifdown scripts.
Network Services:
Many of these have a component on both the client and server (router) sides.
Services:
- ARP: Handles MAC addresses, knows which port of switch a given device is on.
Linux Journey's "Link Layer"
To see client's ARP cache, do "ip neigh" or "arp -n" or "arp".
To see MAC addresses of client's network interfaces, do "ip link show".
"sudo arp-scan -l" - DHCP: Assigns IP addresses.
Linux Journey's "DHCP Overview"
To see where DHCP server is, on client do "ip route | grep dhcp".
Client's DHCP leases:
"sudo bash; ls -l /var/lib/NetworkManager/*.lease"
"sudo dhclient" - Routing: Routes IP traffic from the LAN switch out to WAN (Internet).
AKA "gateway".
- DNS: Maps domain names to IP addresses.
Normally the router does not do DNS, but your LAN may have a separate DNS server, if you're doing self-hosting or ad-blocking.
On client: bind, dnsmasq, systemd-resolved, or openresolv. Also "cat /etc/hosts" and "cat /etc/resolv.conf". If systemd-resolved, see "ls -l /etc/resolv.conf" to tell which of 4 modes it is running in ? Also do "resolvectl status". Also see /etc/systemd/resolved.conf
[I'd like a command or logging that shows ALL the steps as an address gets resolved, including local config-file and cache lookups, but I can't find one.]
See "DNS" section of "Connection Security and Privacy" page.
How does it all fit together ?
Configuration of my Fedora 34 KDE system
- Network interfaces: lo, eno1, wlp1s0, tun0.
- udev.
- Systemd.
- Network Manager (/etc/NetworkManager/NetworkManager.conf).
- systemd-resolved (/etc/resolv.conf symlinks to /run/systemd/resolve/stub-resolv.conf, also see /etc/systemd/resolved.conf).
- Windscribe VPN connected via OpenVPN.
- Firewall stack: nftables / firewalld / firewall-config.
(iptables is loaded but the tables always just say "accept".)
What happens when an app does a network access ?
To do a network access, an application written in C would:
- Construct an addrinfo struct specifying
port number or service name, and host name or IP address.
Could also specify IPv4 versus IPv6, stream versus datagram, more.
- Call
getaddrinfo() library-function
to fill in address.
I think this involves resolving via tables, services, and DNS. From source, on one path, getaddrinfo() is calling gethostbyname() [source] which maybe is calling getservent() to find a DNS service ? There's an indirection in the middle of the NSS source, making everything hard to follow. - Call
socket() system-call
to get a file descriptor for an open socket.
I think this involves using the routing table and picking a network interface, then choosing a client port on that interface. There may be a stack of interfaces (such as tun0 on top of ether0) ? - Call
connect() system-call,
giving the socket and address. If success,
you have a live connection to the destination.
- Call
send() system-call,
giving socket and a message to send (such
as "GET / HTTP/1.0").
Anthony Critelli's "A beginner's guide to network troubleshooting in Linux"
Ricardo Gerardi's "5 Linux network troubleshooting commands"
This section has been very pragmatic. For more theory/standards info, see:
Ian Shields' "Fundamentals of internet protocols"
Connecting
Connecting Linux and Windows (separate machines):
Could just format a USB drive as NTFS and move it back and forth.
Create Samba file-share on Linux:
Mohd Sohail's "Share Folders On Local Network Between Ubuntu And Windows"
Jonathan Moeller's "Install & Configure Samba On Linux Mint 19"
Ubuntu Tutorials' "Install and Configure Samba"
Stephan Avenwedde's "Share files between Linux and Windows computers"
Share folder from Linux:
In Mint Cinnamon: Nemo-share extension to Nemo file explorer.
In KDE: in Dolphin file explorer, select a folder, right-click, Properties, Share.
Apparently case of letters does not matter.
Leave read-only for everyone. No need to check "Allow Guests".
Set up networking on Linux side:
Make sure gufw/firewall is set to allow incoming connections.
Apparently VPN can be left on.
Do "ip addr | grep 192" to get LAN IP address for Linux system.
Install Samba and run Samba (server ?).
Connect from Windows:
Right-click on "This PC" or "My Computer", select "Add a new connection", type location "\\IPADDRESS\\FOLDERNAME", click Next.
Create Samba file-share on Windows:
Create a file-share on Windows:
In Windows file explorer, select a folder, right-click, Properties, click Sharing, click Share, "Choose people to share with" == "all", click Share. Click Advanced Sharing, enable "Share this folder", set "Share name" to something simple with NO spaces in it, click on Permissions, make sure "All" have "Read" permission.
Start Menu, run Task Manager, click More Details, click Users, see logged-in username.
Windows-R, run Powershell, run "ipconfig" to get LAN IP address.
In Linux:
gufw can have "Incoming" set to "Deny".
VPN can be on.
In browser on Linux, go to address "smb://IPADDRESS/SHARENAME", login with Windows account username and password. DIDN'T WORK
In KDE/Dolphin, on left side click on Network. Double-click on "Add Network Folder". Select "Microsoft Windows network drive". Name == anything, Server == IPADDRESS, folder name == name shared from Windows. Get login dialog, use username and password from Windows.
Get success.
On left side, click on Network, see share's name as one of items in list.
OpenSSH client on Windows, and use ssh to log in, or scp to copy across:
WinSCP's "Installing SFTP/SSH Server on Windows using OpenSSH"
On Linux:
"sudo apt install openssh-server"
"sudo systemctl status sshd --full --lines 1000"
Make sure gufw/firewall is set to allow incoming connections.
Could make a specific rule to allow incoming port 22 from Windows machine's IP address.
Apparently VPN can be left on.
On Windows:
Start Menu, search for "Manage optional features".
See "OpenSSH Client".
Windows-R to open PowerShell.
Do "ssh IPADDRESS".
Get messages about trusting the Linux system, say yes.
Get prompted to login to Linux system, give user password.
Get shell prompt.
Ctrl-D to log out.
Do "scp IPADDRESS:.profile profile"
Get prompted to login to Linux system, give user password.
See new file "profile" appear in Windows dir.
Stephan Avenwedde's "Establish an SSH connection between Windows and Linux" (PuTTY)
ssh-audit
OpenSSH server on Windows, and use scp to copy back and forth:
WinSCP's "Installing SFTP/SSH Server on Windows using OpenSSH"
On Windows:
Start Menu, search for "Manage optional features".
See "OpenSSH Server" ? I don't.
???
Create simple read-only web-server on Linux:
ip addr | grep 192 # get LAN IP address of Linux machine cd DIRTOSHARE python -m SimpleHTTPServer # go to Windows machine, and in browser go to: http://LANIPADDRESS:8000
Various ways:
croc
Seafile (very highly recommended by Noah Chelliah)
Sandra Henry-Stocker's "How to share files between Linux and Windows"
Sandra Henry-Stocker's "Moving files between Unix and Windows systems"
Kristen Waters' "How to Mount SMB or NFS Shares With Ubuntu"
In Linux Mint, Nemo file explorer has a "File / Connect to Server ..." menu item.
Unison
Connecting Linux and Windows (single machine dual-booting):
Mount Linux filesystem while running Windows:
Mount the Windows main partition (NTFS filesystem) for read/write access under Linux:
Windows must be fully shut down, not hibernated, to allow Linux to have read/write access to the Windows partition. If all you want is read-only access in Linux, ignore the rest of this section.
In Windows 10, normally if you select "Start / Shutdown", it hibernates, doesn't fully shut down.
Ways to make Windows fully shut down:
- Turn off "Fast Startup", and now "Start / Shutdown" will do a full shutdown.
- Hold down Shift key while selecting "Start / Shutdown", and it will do a full shutdown.
Chris Hoffman's "How to Mount Your Windows 10 (or 8) System Drive on Linux"
Unix & Linux Stack Exchange's "How to mount the 'D:\' disk of Windows in linux mint?"
community.linuxmint.com's "gnome-disk-utility"
But: Ubuntu 18 / Mint Tara automatically recognizes Windows OS partition in a dual-boot system and mounts it; no package installation or other steps needed. It was read-only in my live session, maybe because I didn't shut down Windows fully.
Dislocker (access BitLocker drive on Linux)
Connecting Two Linux Machines:
- With just an Ethernet cable between them, no network, so no security issues:
- rsync
- Across a network, so security is important:
- Warpinator (available as Flatpak). Machines must be on same LAN.
- Croc. article
- NFS:
- Create simple read-only web-server on source Linux machine:
ip addr | grep 192 # get LAN IP address of source machine cd DIRTOSHARE python -m SimpleHTTPServer # go to destination Linux machine, and in browser go to: http://LANIPADDRESS:8000
- SSH: sftp, rsync, scp, etc over ssh
But I think scp is deprecated, has known security issues that won't be fixed.
Dedoimedo's "How to connect and share data between two Linux systems"
Swapnil Bhartiya's "How to Securely Transfer Files Between Servers with scp"
Linux Shell Tips' "How to Mount a Remote Linux Filesystem Using SSHFS"
- SyncThing
- Seafile (very highly recommended by Noah Chelliah)
- In Linux Mint, Nemo file explorer has a "File / Connect to Server ..." menu item.
- In KDE, application "KDE Connect". Works with computers and smartphones that are running KDE Connect or compatible app. Android, Windows, Linux are supported; iPhone not supported because of licensing.
- In GNOME, application "GSConnect". Can connect to app KDE Connect running on Android phone.
Install openssh-server on the server machine, disable password-only login, set default protocol to ssh, enable compression. Generate ssh keys on the client machine; file id_rsa.pub is the public key and file id_rsa is the private key. - Single machine dual-booting or multi-booting:
Just mount the filesystem from one partition to a mount point in the currently running system ?
Jonathan Moeller's "Install & Configure Samba On Linux Mint 19"
If you want to torrent plus do normal traffic, or have multiple people streaming on your LAN, set QOS using "fireqos". Don't try to do it using "tc".
Troubleshooting
- If you have no network access:
Try "ping 1.1.1.1", then "ping google.com", then "curl google.com".
Nice little script for checking your network connectivity: hakerdefo / check-my-net
Maybe you have duelling firewalls ? You can't run firewalld and ufw/gufw at the same time. firewalld and ufw both use iptables.
"whereis -b -B /usr/bin /usr/sbin -f gufw ufw firewalld"
Some VPN clients also use iptables. "sudo iptables -L -v"
Check status of network interfaces. "ip addr"
Any services failing ? "sudo systemctl list-units --state failed"
- If Wi-Fi says "connected but not secure:
Make sure time on your device is correct / set automatically.
- If your Wi-Fi stops working:
Does same Wi-Fi network still work on other machines in the house ?
Have you turned off Wi-Fi with a switch on the outside of the laptop ?
Have you turned off Wi-Fi in BIOS ?
Have you turned off Wi-Fi in software ? "rfkill"
Does the option to turn on Wi-Fi exist in the Settings ?
Does a Wi-Fi device (probably named "wSOMETHING") exist in output of "ip addr" ?
Does a Wi-Fi device (maybe "Network controller") exist in output of "lspci" ?
Check Wi-Fi device and driver in output of "inxi -n".
Run the "Drivers" GUI app for your distro. Maybe switch between "proprietary driver" and "don't use device". "Don't use device" may be a lie; try that option.
- If you have very slow network access:
Is your primary DNS timing out and falling back to another DNS ?
Turn off VPN if you're using one.
Try ping and traceroute to destination, see if any useful info.
Maybe wrong driver for your network adapter ?
Test through Wi-Fi, Ethernet, and Wi-Fi to phone hotspot.
- If in Network Manager applet in System Tray, all devices and connections disappear:
On CLI, do "killall nm-applet; nohup nm-applet &" or "sudo systemctl restart NetworkManager.service"
ICMP blackhole check (IPv4)
ICMP blackhole check (IPv6)
Testing performance (need a client and a server): iperf3.
Public iPerf3 servers
"iperf3 -c SERVERNAME -p PORTNUM"
Works with VPN on or off.
Android app: "Magic iPerf including iPerf3" by NextDoorDeveloper.
Bluetooth
# in BIOS, check to see that Bluetooth is available and enabled inxi --bluetooth bt-device -l hwinfo --bluetooth sudo rfkill list sudo rfkill unblock bluetooth sudo rfkill unblock 1 # another form of the command sudo rfkill unblock all sudo service bluetooth status sudo dmesg | grep -i blue bluetoothctl # then type help to see commands # Front-end apps: gnome-bluetooth # GNOME 3+ bluedevil # KDE blueman # others
Some of these are old, conflict with each other, may be dangerous:
Ayo Isaiah's "How to Set Up Bluetooth in Linux"
HowtoForge's "How to send sound through Bluetooth on Linux"
Bruce Byfield's "Adding a Bluetooth Speaker to Linux"
Arnab Satapathi's "Linux bluetooth setup with bluez and hcitool"
winterheart / broadcom-bt-firmware
ArchWiki's "Blueman and PulseAudio"
Linux.com's "Understanding Bluetooth Technology for Linux"
From someone on reddit:
"TLP and/or powertop --auto-tune may put the Bluetooth module into power-saving mode; stop that".
From someone on reddit:
If your built-in Bluetooth doesn't work, disable it and buy a usb receiver for $20 or so.
Kevin Norman's "Improving Bluetooth Audio Quality on Ubuntu Linux"
Miscellaneous
A design-level problem in computer networking: error-reporting:
Suppose loading a web page in Firefox browser fails. User sees a blank page or "page couldn't be loaded".
Is it because of settings of uMatrix/uBlockOrigin, Privacy Badger, Canvas-blocker, the browser Containers, the browser settings, the anti-virus, firewall in computer is blocking it, VPN connection is down so VPN "kill switch" is denying, Ethernet or Wi-Fi connection is down, firewall in router is blocking it, router's ISP/WAN connection is down, the ad/site-blocker in the VPN server, the VPN server's IP address is blocked by site, site is down, site is Chrome-only, or what ? In most cases, the user is given no useful information about the failure.
The design problem is that most levels of the network stack don't or can't report that they're blocking something, and why. They just return "fail", or just drop the packets on the floor and let a timeout occur.
A few cases do have error-reporting. DNS lookup failure (site not found). Page URL wrong (404). HTTPS/TLS errors. I think parental controls usually give a "blocked because of parental controls" page.