Structure



Some networking basics:
How to Network's "Network Fundamentals - TCP/IP"
Magda El Zarki's "Introduction to TCP/IP"



Linux Network Interfaces:

+/-
Network interface types:
Physical:
  • wl*: Wi-Fi.
  • ???: Bluetooth.
  • ???: NFC.
  • e*: Ethernet.
systemd.net-naming-scheme
systemd's "Predictable Network Interface Names"
Chris Siebenmann's "Understanding something about udev's normal network device names"


Virtual:
  • lo*: loopback.
  • tun*: VPN tunnel.
  • v*: virtual machine adapter.
  • wg*: WireGuard VPN.


ip addr
ls -l /sys/class/net/
nmcli d
nmcli device show
nmcli connection show

sudo ethtool eno1
sudo ethtool -i eno1    # show driver info

rfkill list
nmcli radio
networkctl list
wpa_cli list_networks
iwconfig
iwlist
iwlist wlp1s0 frequency
iw list
iwctl station list
man bluetoothctl
nfctool --list

udev connects interface name to device filename, which maps to device type and major/minor number, which maps inside the kernel to a driver ?

Routing tables:

ip rule    # first col = priority; lower num = higher priority

ip route show table local
ip route show table main
ip route show table default

# Note: "proto" is just a label for what package installed
# this rule, not a filter like "ipproto tcp".

# With VPN on, probably main table will have a rule:
#    default via 10.n.n.n dev tun0 ...
# and a low "metric" number,
# which means "anything not matched by other rules
# goes to the tun0 device".  The other rules all should
# apply to specific addresses or have a higher "metric".

# With VPN off, probably main table will have only a couple
# of rules, but the one for public internet is:
#    default via 192.168.1.1 dev eno1 ...

Changes made via "ip route" or "ip rule" are not persistent; they go away when you reboot. To make persistent changes, you have to edit network config files. Maybe define your own new table (add it to cat /etc/iproute2/rt_tables), populate it with rules via "ip route add", save/load contents via "ip route save" and "ip route restore".

Scott Lowe's "A Quick Introduction to Linux Policy Routing"
Vivek Gite's "Linux Set Up Routing with ip Command"
IPROUTE2 Utility Suite Howto
Paul Gorman's "iproute2 cheat sheet"

A virtual device interface can change the IP address and send the altered request for routing again. Eventually the request should end up at a physical device interface and go out to a network.


To block addresses using ip rule / ip route:

sudo ip route add table main blackhole IPADDRESS
sudo ip route add table main blackhole IPADDRESSPREFIX/24

sudo ip rule add to IPADDRESS table main blackhole
sudo ip rule add to IPADDRESSPREFIX/24 dport 25-30 ipproto tcp table main blackhole
sudo ip rule add from IPADDRESS table main blackhole


IPdeny's "country block downloads"


Ways to implement VPN:
  • TCP(?)-level: e.g. OpenVPN. Default device is tun*.

    Encrypted tunnel to a VPN server, which does user authentication and then implements a proxy (changing IP address) and goes to internet or LAN.

  • IP-level: e.g. WireGuard. Default device is wg*.

    Key set-up is done in advance, when the connection is first defined. So there is no user authentication to do for each connection; just use the keys. Encrypted tunnel to a VPN server, which implements a proxy (changing IP address) and goes to internet or LAN.

[Where does IPSec/Strongswan fit in this ? IKEv2 ?]


Ways to implement firewall:
  • Dynamic code snippets inside kernel: eBPF.

  • Kernel modules: e.g. iptables, nftables.

  • Routing rules: e.g. ip rule, ip route.

    Each kernel module usually has a user-space utility to configure it. Kernel module ip_tables is controlled by utility iptables, kernel module nftables is controlled by utility nft ("sudo nft list ruleset | less").

Also there are front-end apps: On top of iptables, there are ufw and gufw or firewalld (use one). On top of nftables, there is firewalld. There may be a GUI app on top of that, such as Fedora's firewall-config.

Application sandboxes (Firejail, AppArmor, SELinux) can disable networking or maybe filter it ?




Linux system control:

+/-
  • SysV init and service: Older way of doing things. Run-levels for init.
    "chkconfig --list"


  • systemd: New way of doing things. Targets and run-units.
    "pidof systemd"
    my "Linux systemd" page





Linux Network control:

+/-
Usually a system is running either Network Manager (usually on desktop machines) or systemd-networkd (more common in servers), but not both.
article

Network Manager
"sudo systemctl status NetworkManager.service --full --lines 1000"
"man nmcli"
"man nmtui"
"nmcli connection show"
"nmcli device"
Has new ifup/ifdown scripts.

Schkn's "Network Manager on Linux with Examples"
David Both's "Get started with NetworkManager on Linux"
ArchWiki's "NetworkManager"
GNOME Developer's "NetworkManager"
Chris Siebenmann's "Desktops don't always use NetworkManager's programs"

Network Manager connectivity-checking
Does a ping every 5 minutes ?
Justin Chapin's "How to prevent NetworkManager connectivity checking"

"journalctl --since=today | grep connectivity"
Arch: "cat /etc/NetworkManager/conf.d/20-connectivity.conf"
"cat /etc/NetworkManager/NetworkManager.conf"
"man NetworkManager.conf" and see "Connectivity" section.


systemd-networkd
Alternative to Network Manager ?
"Use systemd-networkd to configure your devices and iwctl to configure your Wi-Fi connections."

"sudo systemctl status systemd-networkd.service --full --lines 1000"
"networkctl list"
"networkctl status"
"cat /etc/systemd/networkd.conf"
"ls /etc/systemd/network"
Derrik Diener's "How to set up systemd-networkd on Linux"
Sahitya Maruvada's "Working with systemd-networkd"

Debian inherent networking.service
"sudo systemctl status networking.service --full --lines 1000"
"ls /etc/network/interfaces" ?
"ls /etc/sysconfig/network-scripts" ?

ConnMan

Netplan
"netplan is a Ubuntu tool to create a configuration file for networking. netplan can create either a NetworkManager configuration or a systemd-networkd configuration file at system boot time."

"ls /etc/netplan"

Egidio Docile's "Netplan network configuration tutorial for beginners"
Linux Shell Tips' "How to Set Static IP Address on Ubuntu Linux"
Canonical's "Netplan reference"
From someone on reddit 2/2021:
+/-
I don't think it's aimed at the non-skilled developer. Netplan is the kind of thing aimed for cloud, servers, rpi enthusiasts who want a permanent network configuration to set and forget. It has one advantage compared to the others, and that is that the configuration can exist in /boot. As such, it is possible to edit/adjust the network configuration if you need to adjust it from a separate machine.

For that purpose it does what it's intended to do, although I would argue it's superfluous considering systemd-networkd meets the exact same need as set-forget network configuration that is actually sandboxed. I also think netplan's configuration.yaml is a lot nicer than systemd's units stuff.


network-scripts
Deprecated.
Uses old ifup/ifdown scripts.




Network Services:

+/-
Many of these have a component on both the client and server (router) sides.

Services:
  • ARP: Handles MAC addresses, knows which port of switch a given device is on.
    Linux Journey's "Link Layer"

    To see client's ARP cache, do "ip neigh" or "arp -n" or "arp".
    To see MAC addresses of client's network interfaces, do "ip link show".
    "sudo arp-scan -l"

  • DHCP: Assigns IP addresses.
    Linux Journey's "DHCP Overview"

    To see where DHCP server is, on client do "ip route | grep dhcp".

    Client's DHCP leases:
    "sudo bash; ls -l /var/lib/NetworkManager/*.lease"
    "sudo dhclient"

  • Routing: Routes IP traffic from the LAN switch out to WAN (Internet). AKA "gateway".

  • DNS: Maps domain names to IP addresses.

    Normally the router does not do DNS, but your LAN may have a separate DNS server, if you're doing self-hosting or ad-blocking.

    On client: bind, dnsmasq, systemd-resolved, or openresolv. Also "cat /etc/hosts" and "cat /etc/resolv.conf". If systemd-resolved, see "ls -l /etc/resolv.conf" to tell which of 4 modes it is running in ? Also do "resolvectl status". Also see /etc/systemd/resolved.conf

    [I'd like a command or logging that shows ALL the steps as an address gets resolved, including local config-file and cache lookups, but I can't find one.]

    See "DNS" section of "Connection Security and Privacy" page.





How does it all fit together ?

+/-
Configuration of my Fedora 34 KDE system
+/-
  • Network interfaces: lo, eno1, wlp1s0, tun0.
  • udev.
  • Systemd.
  • Network Manager (/etc/NetworkManager/NetworkManager.conf).
  • systemd-resolved (/etc/resolv.conf symlinks to /run/systemd/resolve/stub-resolv.conf, also see /etc/systemd/resolved.conf).
  • Windscribe VPN connected via OpenVPN.
  • Firewall stack: nftables / firewalld / firewall-config.
    (iptables is loaded but the tables always just say "accept".)

What happens when an app does a network access ?
+/-
To do a network access, an application written in C would:
  1. Construct an addrinfo struct specifying port number or service name, and host name or IP address. Could also specify IPv4 versus IPv6, stream versus datagram, more.

  2. Call getaddrinfo() library-function to fill in address.

    I think this involves resolving via tables, services, and DNS. From source, on one path, getaddrinfo() is calling gethostbyname() [source] which maybe is calling getservent() to find a DNS service ? There's an indirection in the middle of the NSS source, making everything hard to follow.

  3. Call socket() system-call to get a file descriptor for an open socket.

    I think this involves using the routing table and picking a network interface, then choosing a client port on that interface. There may be a stack of interfaces (such as tun0 on top of ether0) ?

  4. Call connect() system-call, giving the socket and address. If success, you have a live connection to the destination.

  5. Call send() system-call, giving socket and a message to send (such as "GET / HTTP/1.0").

Beej's Guide to Network Programming




Anthony Critelli's "A beginner's guide to network troubleshooting in Linux"
Ricardo Gerardi's "5 Linux network troubleshooting commands"



This section has been very pragmatic. For more theory/standards info, see:
Ian Shields' "Fundamentals of internet protocols"





Connecting



Connecting Linux and Windows (separate machines):

+/-
Could just format a USB drive as NTFS and move it back and forth.

Create Samba file-share on Linux:
+/-
Mohd Sohail's "Share Folders On Local Network Between Ubuntu And Windows"
Jonathan Moeller's "Install & Configure Samba On Linux Mint 19"
Ubuntu Tutorials' "Install and Configure Samba"
Stephan Avenwedde's "Share files between Linux and Windows computers"

Share folder from Linux:
In Mint Cinnamon: Nemo-share extension to Nemo file explorer.
In KDE: in Dolphin file explorer, select a folder, right-click, Properties, Share.
Apparently case of letters does not matter.
Leave read-only for everyone. No need to check "Allow Guests".

Set up networking on Linux side:
Make sure gufw/firewall is set to allow incoming connections.
Apparently VPN can be left on.
Do "ip addr | grep 192" to get LAN IP address for Linux system.
Install Samba and run Samba (server ?).

Connect from Windows:
Right-click on "This PC" or "My Computer", select "Add a new connection", type location "\\IPADDRESS\\FOLDERNAME", click Next.

Create Samba file-share on Windows:
+/-
Create a file-share on Windows:
In Windows file explorer, select a folder, right-click, Properties, click Sharing, click Share, "Choose people to share with" == "all", click Share. Click Advanced Sharing, enable "Share this folder", set "Share name" to something simple with NO spaces in it, click on Permissions, make sure "All" have "Read" permission.
Start Menu, run Task Manager, click More Details, click Users, see logged-in username.
Windows-R, run Powershell, run "ipconfig" to get LAN IP address.

In Linux:
gufw can have "Incoming" set to "Deny".
VPN can be on.

In browser on Linux, go to address "smb://IPADDRESS/SHARENAME", login with Windows account username and password. DIDN'T WORK

In KDE/Dolphin, on left side click on Network. Double-click on "Add Network Folder". Select "Microsoft Windows network drive". Name == anything, Server == IPADDRESS, folder name == name shared from Windows. Get login dialog, use username and password from Windows.
Get success.
On left side, click on Network, see share's name as one of items in list.

OpenSSH client on Windows, and use ssh to log in, or scp to copy across:
+/-
WinSCP's "Installing SFTP/SSH Server on Windows using OpenSSH"

On Linux:
"sudo apt install openssh-server"
"sudo systemctl status sshd --full --lines 1000"
Make sure gufw/firewall is set to allow incoming connections.
Could make a specific rule to allow incoming port 22 from Windows machine's IP address.
Apparently VPN can be left on.

On Windows:
Start Menu, search for "Manage optional features".
See "OpenSSH Client".

Windows-R to open PowerShell.

Do "ssh IPADDRESS".
Get messages about trusting the Linux system, say yes.
Get prompted to login to Linux system, give user password.
Get shell prompt.
Ctrl-D to log out.

Do "scp IPADDRESS:.profile profile"
Get prompted to login to Linux system, give user password.
See new file "profile" appear in Windows dir.

Stephan Avenwedde's "Establish an SSH connection between Windows and Linux" (PuTTY)
ssh-audit

OpenSSH server on Windows, and use scp to copy back and forth:
+/-
WinSCP's "Installing SFTP/SSH Server on Windows using OpenSSH"

On Windows:
Start Menu, search for "Manage optional features".
See "OpenSSH Server" ? I don't.
???

Create simple read-only web-server on Linux:
+/-

ip addr | grep 192		# get LAN IP address of Linux machine
cd DIRTOSHARE
python -m SimpleHTTPServer
# go to Windows machine, and in browser go to:
http://LANIPADDRESS:8000

Various ways:
croc
Seafile (very highly recommended by Noah Chelliah)
Sandra Henry-Stocker's "How to share files between Linux and Windows"
Sandra Henry-Stocker's "Moving files between Unix and Windows systems"
Kristen Waters' "How to Mount SMB or NFS Shares With Ubuntu"
In Linux Mint, Nemo file explorer has a "File / Connect to Server ..." menu item.
Unison



Connecting Linux and Windows (single machine dual-booting):

+/-
Mount Linux filesystem while running Windows:
Mount the Windows main partition (NTFS filesystem) for read/write access under Linux:
+/-
Windows must be fully shut down, not hibernated, to allow Linux to have read/write access to the Windows partition. If all you want is read-only access in Linux, ignore the rest of this section.

In Windows 10, normally if you select "Start / Shutdown", it hibernates, doesn't fully shut down.

Ways to make Windows fully shut down:
  • Turn off "Fast Startup", and now "Start / Shutdown" will do a full shutdown.
  • Hold down Shift key while selecting "Start / Shutdown", and it will do a full shutdown.
I think it's best to leave "Fast Startup" turned off. But Windows will start up slower.

Chris Hoffman's "How to Mount Your Windows 10 (or 8) System Drive on Linux"
Unix & Linux Stack Exchange's "How to mount the 'D:\' disk of Windows in linux mint?"
community.linuxmint.com's "gnome-disk-utility"

But: Ubuntu 18 / Mint Tara automatically recognizes Windows OS partition in a dual-boot system and mounts it; no package installation or other steps needed. It was read-only in my live session, maybe because I didn't shut down Windows fully.

Dislocker (access BitLocker drive on Linux)



Connecting Two Linux Machines:

+/- Alexandru Andrei's "How to Use Netcat to Quickly Transfer Files Between Linux Computers"
Jonathan Moeller's "Install & Configure Samba On Linux Mint 19"



If you want to torrent plus do normal traffic, or have multiple people streaming on your LAN, set QOS using "fireqos". Don't try to do it using "tc".





Troubleshooting



Odd case I ran into: Using an Android phone to provide a Wi-Fi hotspot, setting DNS manually in (Windows) laptop that was connecting through the phone, got "connected, but no internet" on the laptop. Had to set DNS in laptop to "automatic" to get it to work. I don't know if similar could happen with Linux.



ICMP blackhole check (IPv4)
ICMP blackhole check (IPv6)



Testing performance (need a client and a server): iperf3.
Public iPerf3 servers
"iperf3 -c SERVERNAME -p PORTNUM"
Works with VPN on or off.
Android app: "Magic iPerf including iPerf3" by NextDoorDeveloper.





Bluetooth




# in BIOS, check to see that Bluetooth is available and enabled

inxi --bluetooth
bt-device -l
hwinfo --bluetooth

sudo rfkill list
sudo rfkill unblock bluetooth
sudo rfkill unblock 1	# another form of the command
sudo rfkill unblock all

sudo service bluetooth status

sudo dmesg | grep -i blue

bluetoothctl	# then type help to see commands

# Front-end apps:
gnome-bluetooth    # GNOME 3+
bluedevil          # KDE
blueman            # others



Some of these are old, conflict with each other, may be dangerous:
Ayo Isaiah's "How to Set Up Bluetooth in Linux"
HowtoForge's "How to send sound through Bluetooth on Linux"
Bruce Byfield's "Adding a Bluetooth Speaker to Linux"
Arnab Satapathi's "Linux bluetooth setup with bluez and hcitool"
winterheart / broadcom-bt-firmware
ArchWiki's "Blueman and PulseAudio"
Linux.com's "Understanding Bluetooth Technology for Linux"



From someone on reddit:
"TLP and/or powertop --auto-tune may put the Bluetooth module into power-saving mode; stop that".

From someone on reddit:
If your built-in Bluetooth doesn't work, disable it and buy a usb receiver for $20 or so.

Kevin Norman's "Improving Bluetooth Audio Quality on Ubuntu Linux"





Miscellaneous



A design-level problem in computer networking: error-reporting:
+/-
Suppose loading a web page in Firefox browser fails. User sees a blank page or "page couldn't be loaded".

Is it because of settings of uMatrix/uBlockOrigin, Privacy Badger, Canvas-blocker, the browser Containers, the browser settings, the anti-virus, firewall in computer is blocking it, VPN connection is down so VPN "kill switch" is denying, Ethernet or Wi-Fi connection is down, firewall in router is blocking it, router's ISP/WAN connection is down, the ad/site-blocker in the VPN server, the VPN server's IP address is blocked by site, site is down, site is Chrome-only, or what ? In most cases, the user is given no useful information about the failure.

The design problem is that most levels of the network stack don't or can't report that they're blocking something, and why. They just return "fail", or just drop the packets on the floor and let a timeout occur.

A few cases do have error-reporting. DNS lookup failure (site not found). Page URL wrong (404). HTTPS/TLS errors. I think parental controls usually give a "blocked because of parental controls" page.