↓ Skip to main content
  1. Blog/

Hardening the Home Lab, Part 2: Knowing What You've Got

·1838 words·9 mins
Chris Ramsay
Author
Chris Ramsay
Ultra and mountain runner, maker and amateur astronomer
Hardening the Home Lab - This article is part of a series.
Part 2: This Article

In Part 1 I set out the plan: split the network into zones so that a compromised device becomes a dead end. Before any of that can happen, there’s a more basic question. What is actually on the network?

That sounds easy. I set most of it up myself, after all. But a home lab built over a decade has layers: a Pi from one project, an ESP32 from another, a camera someone else looks after, a switch that arrived with a box of other things. Every one of them needs a decision about which zone it belongs in, and a device you’ve forgotten about is exactly the one that never gets patched.

Why an inventory comes first
#

Each later step in this series depends on it:

  • Segmentation needs a target zone for every device. Miss one, and it either breaks when the old network goes away, or quietly stays where it was.
  • Firewall rules need to know who talks to whom. The weather station needs the message broker; does it need anything else?
  • Configuration management and patching need a list of machines to manage. Anything not on the list doesn’t get updated.

Devices hide in different places
#

The lesson that took me longest to learn: no single tool sees everything. Each source of information has a blind spot, and devices live in the gaps.

SourceWhat it seesWhat it misses
DHCP leases (Pi-hole, router)Everything that asked for an addressAnything with a static IP
ARP or ping sweep (nmap -sn)Everything live on the subnet you sweepDevices on another subnet, devices that are switched off
Switch MAC address tableEverything plugged into that switch, whatever its IPWi-Fi devices, devices behind an unmanaged switch elsewhere
Service scan (nmap -sV)What each device is actually runningNothing new: it only describes devices you’ve already found

So the inventory comes from all of them, cross-checked:

flowchart LR
  D["DHCP leases"] --> S[("Inventory
spreadsheet")] A["ARP sweep"] --> S M["Switch MAC table"] --> S V["Service scan"] --> S S --> P["VLAN plan
firewall rules
patch coverage"]

The sweep
#

The core of the scan is nmap. Run as root on the local subnet, its “ping” sweep actually uses ARP, which finds devices even when they ignore ping:

sudo nmap -sn 10.0.10.0/24
Nmap scan report for weather.example.home (10.0.10.23)
Host is up (0.0011s latency).
MAC Address: B8:27:EB:xx:xx:xx (Raspberry Pi Foundation)

The MAC vendor is surprisingly useful. B8:27:EB is an older Raspberry Pi, D8:3A:DD a newer one, and Espressif prefixes mean an ESP32. Even an unnamed device usually gives away what kind of thing it is.

Then, for each live host, a service scan to see what it’s running:

sudo nmap -sV --top-ports 200 10.0.10.23
PORT     STATE SERVICE VERSION
22/tcp   open  ssh     OpenSSH 9.2p1 Debian 2+deb12u3
80/tcp   open  http    nginx 1.22.1
1883/tcp open  mqtt

Turning scans into a spreadsheet
#

Running those by hand gets old quickly, so a script does the discovery for me: estate_scan.py (download it from this post). It combines three of those sources in one run:

  1. An nmap ARP sweep of the subnet, plus a quick SYN scan of a handful of role-hinting ports: SSH, DNS, web, MQTT, Grafana, InfluxDB, the ESPHome dashboard. An open 1883 says “message broker” faster than any amount of guessing.
  2. The DHCP leases from Pi-hole, fetched over SSH. These catch the sleepy devices that didn’t answer the sweep in time.
  3. Reverse DNS names, by pointing nmap at Pi-hole as its DNS server.

It writes a CSV I import into the inventory spreadsheet. The design decision that makes it useful is in how it handles re-runs.

Re-running without losing your work
#

A scan can only fill in some of the columns. The important ones (what a device is for, where it is, which zone it should move to) only I can fill in. So the columns are split in two:

COLS = [
    "ip", "hostname", "mac", "vendor", "category_guess", "open_ports", "ssh",
    # --- planning columns: filled in by you, never overwritten ---
    "friendly_name", "role", "location", "connection",
    "current_vlan", "target_vlan", "semaphore_group", "patchmon", "notes",
    # --- housekeeping ---
    "dns_name", "dhcp_name", "mac_randomised",
    "seen_this_run", "first_seen", "last_seen",
]

Rows are keyed on MAC address rather than IP, because IPs change and MACs (mostly) don’t. On each run, every existing row starts out assumed gone, and only devices that turn up again get their discovery columns refreshed. Planning columns are never touched:

def merge(existing_rows, current, today):
    merged = {key_for(r): r for r in existing_rows}
    for k, row in merged.items():          # assume gone until proven present
        row["seen_this_run"] = "no"
    for k, rec in current.items():
        row = merged.get(k, {c: "" for c in COLS})
        if not row.get("first_seen"):
            row["first_seen"] = today
        for c, v in rec.items():
            if c in COLS and c not in PLANNING:
                # keep a known vendor/name if this run didn't find one
                if v or c in ("open_ports", "ssh", "seen_this_run"):
                    row[c] = v
        ...

So I can scan, fill in roles and target zones, scan again a week later, and nothing I wrote is lost. The seen_this_run column then tells three stories: yes, lease only (it has a DHCP lease but didn’t answer), or no (it was here before and has vanished, which is always worth a look).

Small things that help
#

Guessing what a device is. The MAC vendor does most of the work, with a few extra hints for things the vendor can’t tell you:

def guess_category(rec):
    v, h = rec["vendor"].lower(), rec["hostname"].lower()
    if h.startswith("uk0"):          # my meteor cameras' station-ID hostnames
        return "GMN meteor camera"
    if "raspberry" in v:
        return "Raspberry Pi"
    if "espressif" in v:
        return "ESP32 / IoT"
    ...
    if rec["mac_randomised"] == "yes":
        return "Phone/tablet (random MAC)"

Spotting phones. Modern phones and tablets use a random MAC per network. Those set the “locally administered” bit, which shows up as the second hex digit being 2, 6, A or E. One line flags them, so they don’t get mistaken for mystery hardware:

def mac_is_randomised(mac):
    # Locally-administered bit set => 2nd hex digit is 2, 6, A or E
    return bool(mac) and mac[1].lower() in "26ae"

Running under sudo properly. nmap needs root for ARP and SYN scans, but the SSH connection to Pi-hole should use my credentials, not root’s. The script drops back to the real user for that one command:

if os.environ.get("SUDO_USER"):
    cmd = ["sudo", "-u", os.environ["SUDO_USER"]] + cmd

It also hands the CSV back to that user afterwards, and keeps a .bak of the previous file in case a run goes wrong.

sudo ./estate_scan.py --subnet 10.0.10.0/24 --dns 10.0.10.1

Each run ends with a one-line summary, something like:

13 devices -> network_estate.csv  (up: 11, lease only: 1, not seen: 1)

There’s also an offline mode (--from-xml plus --leases-file) that parses a saved nmap run, which made testing it much easier.

The spreadsheet
#

The scan is the raw material; the spreadsheet is where decisions get made. There’s one row per device, and the columns that matter most are the planning ones a scan can’t fill in (a few columns trimmed here to fit):

iphostnamecategory_guessroleconnectioncurrent_vlantarget_vlannotes
10.0.10.23weatherRaspberry PiWeather stationWi-Fi1 (flat LAN)IoTRead-only root
10.0.10.31soilESP32 / IoTSoil probePoE, sw2 p31 (flat LAN)IoT
10.0.10.12nasDocker hostsw1 p11 (flat LAN)Trusted
10.0.10.140Raspberry PiVisiting fireball camerasw1 p91 (flat LAN)GuestNot mine to manage

That target_vlan column is the VLAN plan in miniature. By the time every row has one, the next stage of the project is mostly a matter of working down the list.

The sheet has grown other tabs since, each one a plan in its own right: one per network zone, an SSH access policy saying which machines may log into which, a Wi-Fi plan, and a migration checklist. It’s the closest thing my network has to documentation.

The device that wasn’t there
#

The most satisfying find came from the switch, not the scan.

When I installed a managed switch, I compared its MAC address table against the inventory. Four MACs didn’t match anything. Three were quickly explained. The fourth had a Raspberry Pi prefix, no DHCP lease, and didn’t show up in any sweep. Something was plugged in, powered on and talking, but invisible.

No lease suggested a static IP. Then I remembered that years ago, before changing broadband provider, the house network had used a completely different address range. A Pi configured back then would still be trying to use it, on a subnet that no longer existed.

You can test that theory without touching the device. A machine with a stale static configuration keeps asking for its old gateway, and those ARP requests reveal its address:

sudo timeout 120 tcpdump -i eth0 -n -e ether src b8:27:eb:xx:xx:xx
... ARP, Request who-has 10.99.0.1 tell 10.99.0.92
                         ^ old gateway  ^ the lost Pi

The device is still on the same physical network, just on a different subnet, so the fix for reaching it is simple: temporarily give a Linux machine a second address in the old range.

sudo ip addr add 10.99.0.250/24 dev eth0     # temporary second address
sudo nmap -sV -sC -O 10.99.0.92              # what is it?
sudo ip addr del 10.99.0.250/24 dev eth0     # tidy up

It turned out to be the camera from an old bird box project. It had been sitting there for years on an operating system several releases out of support: exactly the kind of device Part 1 was worried about. Forgotten, unpatched, and on the same network as everything else.

Names, not numbers
#

The other job at this stage was giving everything a sensible name, because “the thing on .92” isn’t a plan.

  • Use a real domain for internal names. Pi-hole’s DHCP hands out a domain with every lease, and the default is something like lan. Switching it to a subdomain of a domain I own means every device gets a proper, unambiguous name.
  • Fixed addresses live below the DHCP pool. Infrastructure (router, DNS, switches, servers) gets a reserved address in a low range; everything else takes a lease from a pool higher up. You can tell what kind of device something is from its address alone.
  • Name by role and place. Switches are office-sw01 and iot-sw01, not their model numbers. When a switch gets replaced, the name stays and only the hardware changes.

Names pay off later in the series too. Firewall rules, SSH certificates and the Ansible inventory can all refer to weather rather than an IP address, and they stay readable when an address changes.

Keeping it current
#

An inventory is only useful while it’s true. I re-run the scan whenever something changes, and compare the switch MAC tables against the sheet now and then. Anything unexplained gets a row before it gets anything else.

Next time
#

With every device listed and a target zone against each one, Part 3 does the actual carving: VLANs for trusted, IoT and guest devices, across a router and two managed switches, and the router firewall rules that decide what may cross between them.

Hardening the Home Lab - This article is part of a series.
Part 2: This Article

Related