Understanding the pulse

What disk usage, inodes, swap, load and SMART mean – and when a number is worth worrying about.

A device's Pulse tab shows numbers that only help if you know what they measure. This page explains each of them – without an account, so that people who do not (yet) use NODE64 can read up too.

Every value comes from the agent on the device itself. NODE64 measures nothing from the outside and guesses nothing.

Disk usage #

The percentage of a filesystem that is occupied. This always refers to a mount point, not to a physical drive: / and /boot/efi can sit on the same disk and still be filled completely differently.

When it gets tight depends on what runs on the system:

A full / partition brings a Linux system to a standstill. A full data partition usually affects only one service.

Inodes #

A filesystem manages two supplies, not one: space and inodes. An inode is a file's bookkeeping entry – permissions, owner, timestamps, where the data sits. Every file and every directory needs exactly one.

When ext4 is formatted, the number of inodes is fixed once and for all, normally one per 16 KB. Anything that creates many tiny files – session files, mail queues, caches, node_modules – burns through inodes far faster than through space.

The situation you notice this in looks absurd:

$ df -h /var
/dev/sda2  100G  41G  59G  41% /var

$ touch /var/test
touch: cannot touch '/var/test': No space left on device

Fifty-nine gigabytes free, and still nothing works. The answer is in df -i:

$ df -i /var
/dev/sda2  6553600  6553600  0  100% /var

The inodes are exhausted. That is why NODE64 puts the inode share next to disk usage rather than below it: it is a separate question, and it usually only gets asked once it is too late.

The only remedy is deleting files (often in /var/spool, /tmp or old caches) – the inode count of an existing ext4 filesystem cannot be raised afterwards.

Why some filesystems show no inode figure #

btrfs, ZFS and XFS allocate inodes dynamically. They have no fixed ceiling and therefore report zero to the operating system. NODE64 shows no value there instead of "0 % used" – an invented zero would be reassurance without grounds.

vfat, exfat and NTFS do not have the concept at all. There too the field stays empty, with a note explaining why.

An empty inode field is therefore not a measurement error but a property of the filesystem.

Memory and swap #

Memory (RAM) is the fast storage in which running programs work.

On Linux, high RAM usage is normal and intended. The kernel uses free memory as a cache for files; "free" means unused, and unused memory is wasted memory. NODE64 therefore counts the genuinely occupied share, not what free prints on its first line.

Swap is spillover onto disk: when memory runs short, the kernel moves rarely used pages there. That is orders of magnitude slower.

A little swap in use is harmless – the kernel also swaps out when there is no shortage at all, simply because a page has not been touched for days. Persistently high swap usage, on the other hand, means the system is short of memory: the machine becomes noticeably slow without CPU usage explaining it.

If no swap is configured, NODE64 shows no tile for it. "0 %" would suggest a reserve that does not exist.

Load and processor usage #

These are two different things, and they are frequently confused.

Processor usage is the share of time the CPU spent computing – a percentage.

Load average is the average number of processes that are running or waiting for something. On Linux, processes waiting for disk or network count as well. That is why load can be high while the CPU does almost nothing – a classic sign of an overloaded drive.

The decisive point: load has to be divided by the number of cores. A load of 4.0 on a four-core machine means "fully used"; the same 4.0 on one core means "oversubscribed fourfold". NODE64 knows the device's core count and converts the threshold accordingly – which is why the alert threshold for load is given as a percentage, not as an absolute number.

The three figures from uptime are averages over 1, 5 and 15 minutes. NODE64 records the 1-minute load.

Temperature #

The temperature of the mainboard or processor, if the device has sensors. Virtual machines and most containers do not – the tile is absent there.

Usual magnitudes: 30–50 °C at idle, 60–80 °C under load. Modern processors throttle themselves from around 95–100 °C before damage occurs. What should worry you is less a single high reading than a reading that climbs over weeks – usually dust in the heatsink or a fan that is giving up.

SMART: what the drive says about itself #

SMART is the self-monitoring built into hard disks and SSDs. Three counters are the most telling:

Reallocated sectors – sectors the drive found defective and replaced from its spare pool. A value above zero is no cause for panic; a value that grows is. That is exactly why NODE64 shows the history and not only the current figure.

Pending sectors – sectors that failed to read and have not been replaced yet. This is the more serious of the two: there is data here the drive can no longer read reliably.

Wear level / percentage used (SSD only) – how much of the intended write endurance has been consumed. 10 % after two years is unremarkable; 90 % means a replacement should be planned.

One important caveat: SMART does not reliably predict failure. A substantial share of drives fail with no prior warning signs at all. SMART is an early indicator, not a substitute for backups.

The forecast #

Next to a disk usage figure you will sometimes see a sentence like "at this rate, full in 12 to 20 days". That is an estimate from the history so far, not a prediction.

Here is how it is made: NODE64 takes the measurements of the last 30 days and computes the slopes of all pairs of points. The median of those slopes is the estimated growth per day; the lower and upper quartile give the range. This method (due to Theil and Sen) tolerates outliers far better than an ordinary best-fit line – and outliers are the normal case with disk usage, for instance when a backup drops 20 GB overnight and clears it again in the morning.

The estimate appears only if it holds up:

• at least eight measurements spanning at least two days,
• and the middle half of the slopes must agree on the direction.

If usage fluctuates without rising, it says "no clear trend" instead of a number. If the date lies more than a year out, there is likewise no number – by then the machine will have been rebuilt several times, and "full in 1400 days" sounds like knowledge without being any.

An estimate is no substitute for an alert. If you want to be notified, set up a rule under Alerts – a button right next to every usage bar takes you there.

What runs inside this host #

Until then NODE64 measured the host. What ran inside it was known - but not measured. A container eating memory is invisible until the host runs into trouble. By then it is too late for a calm decision.

The pulse now shows every container and every virtual machine with its own numbers: memory, throttled CPU time, processes, OOM kills and stall time. Six kinds are recognised - Docker, Podman, LXC, Proxmox LXC, KVM and systemd-nspawn - through the same kernel files. No cAdvisor, no extra service, no library.

Throttled time is the real find. A container stuck at its CPU limit is slow - and on the host you see nothing of it, because there is still headroom there.

On cgroup v1 exactly one number is missing: stall time per container. The kernel does not keep it there. Every other value is complete, and the line says why the one is missing - instead of showing a zero that would mean "nobody had to wait".

With no memory limit set, the amount is shown instead of a percentage. "80 % of unlimited" would not be a statement.

Thresholds can be set per container: memory near the limit, sustained throttling, OOM kill. The subject is half the rule - a host has twenty containers, and one of them is usually what you mean.

Eight numbers that explain why it feels slow #

Load, CPU, memory, swap, disk usage, temperature, uptime: all numbers about consumption. None of them explains a server that stalls at forty percent CPU because it is waiting on the disk.

All eight are only worth something over time. In the moment you can see them with top too. Three days later - when the question comes why it was slow last Tuesday at three - only if someone wrote it down.

If one of them is missing on your system, it says so. A virtual machine often has no thermal throttle counter; then the line stays empty instead of showing a zero.