Timbre – what is normal on your system

How NODE64 compares your server with itself – and how to trigger an alert yourself, safely.

Every other part of NODE64 checks against a standard that applies everywhere: this version has a hole, this setting is wrong, this framework demands that. Timbre knows only one standard – your server, yesterday.

The name comes from music. Timbre is what distinguishes two instruments playing the same note: same pitch, same volume, and you still hear immediately whether it is a violin or a clarinet. Servers work the same way. Two machines with the same software behave differently, because one runs a shop and the other is a backup target.

So there is no universal normality one could build in. Five hundred changed files are everyday life on a build server and an incident on a mail server.

Where to find it #

On the device, in the Timbre tab. And before that: the device overview puts Timbre on its first line, and every card in the device list carries one sentence about it – "Timbre currently sees nothing unusual" or "Timbre sees 3 unusual traits across 2 areas at once". That is deliberate: reading a finding without knowing what is normal on this system makes it impossible to judge.

Why there is no verdict yet #

That is the most common question, and the answer is unspectacular: Timbre needs a history, and it is only building up.

Every agent run is one measurement. A feature needs at least twenty of them for a verdict – for the counters of the check procedures, eight are enough. Until then it says "no verdict possible yet" with the number of measurements so far next to it.

That is not an all-clear. A zero there would claim that something was checked and nothing found. Grey means: it is collecting, but there is nothing to say yet.

The comparison window counts in runs, not in hours. That is why every piece of evidence states the period the measurements cover – otherwise the same number would mean different things on two plans.

How to read a feature #

A row looks like this:

running services        44     44 instead of the usual 40, from 96 measurements over 4 days

On the left the current value, on the right the evidence: the value that is usual on this device, the number of measurements and the period. That makes every verdict recomputable – and refutable. Which is exactly why there is no overall score: a number from 0 to 100 could be neither recomputed nor refuted.

The reliability next to it answers the question "how seriously should I take this". It grows with the number of measurements and deliberately sits next to every verdict rather than in a footnote.

Does it work on my system? #

A security tool you cannot try out is one you have to take on faith. And nobody believes an alert they have never seen fire.

The following four commands each trigger exactly one immediate indicator. They are undone in seconds, they cause no damage, and they leave nothing behind. Run them on a device whose agent is running – the alert should reach you in under a minute.

1 · Shell under the web server #

The most telling test. It looks exactly like a real web shell – deliberately so, because that is what should be detected.

The user the web server runs as is named differently per distribution:

If you pick the wrong line, you get "unknown user" and nothing else. No alert then does not mean Timbre is broken – only that the command never ran.

2 · Launch from a scratch directory #

cp /bin/true /tmp/timbre-test && /tmp/timbre-test; rm /tmp/timbre-test

Copies a harmless system program to /tmp, runs it and deletes it again. /tmp is world-writable and therefore the first place anything arriving from outside lands.

3 · Reverse connection pattern #

timeout 1 bash -c 'exec 3<>/dev/tcp/192.0.2.1/4444' 2>/dev/null

This command connects nowhere. 192.0.2.1 is in the range explicitly reserved for documentation (RFC 5737) – nothing answers there. What is detected is the shape of the call, not its success.

4 · Mining pattern in the command line #

bash -c 'echo stratum+tcp://pool.example:4444; sleep 1'

Prints a string that looks like the configuration of a cryptocurrency miner. Nothing is computed and nothing is connected.

Run them one at a time, not all at once #

Run the commands one after another, each on its own line. If you chain them, your own shell's command line contains all the patterns at once – and then Timbre reports not four events but eight: the four test programs plus the shell that started them.

That is not a bug but correct: a shell whose command line contains stratum+tcp:// and /dev/tcp/ is exactly what should be detected. It is just confusing while testing.

What happens – and what does not #

The agent establishes these four things itself; it needs no history for them. But it passes no judgement on whether they are unusual – the platform decides that, because only it knows what is normal on this system.

What the agent transmits is the event in its shortest form: 20114:php-fpm>sh. The command line itself never leaves your server – passwords regularly appear in it, and the agent cannot tell when.

If the connection to the platform happens to be down, it writes the observation straight to your system log (journalctl) and reports it as soon as it can.

What Timbre builds its picture from #

Up to phase 70 Timbre mostly counted: how many services, how many ports, how many accounts. That finds a system that changes – and misses a lot that happens within the same numbers. Four dimensions have been added since.

The network #

Since phase 62 NODE64 knows which program talks to which target network on which port. Three quantities now come with it:

The data volume per individual connection is usually missing, and it says so rather than showing a zero. The kernel only counts it when net.netfilter.nf_conntrack_acct is switched on – and that is off by default on every distribution. NODE64 does not switch it on: the agent changes nothing on your system. If you want it:

sysctl -w net.netfilter.nf_conntrack_acct=1

The rare destination port is descriptive, not accusatory. A rare port is a rare port – whether it is a problem is decided by what else stands out at the same time.

The time of day #

A backup service talks to the backup target at three in the morning. If it does so at noon, that is remarkable – and was invisible, because every comparison ran against the average across all hours.

Timbre therefore keeps twenty-four buckets instead of one, one per hour of the day, and compares against what this axis usually does at this hour.

On the Free plan the time-of-day baseline can never reach a verdict, and the arithmetic is shown:

15-minute interval   4 samples per hour and day  →  20 samples after  5 days
60-minute interval   1 sample  per hour and day  →  20 samples after 20 days

Free plan            60-min interval · 7 days retention  →  never meet

Instead of permanently showing you "still collecting", the dashboard says why.

There is no time-of-day baseline for logins, and that stays that way. When someone was on their own server, and for how long, is not a security question – it is attendance monitoring. The difference is who acts: a service backing up at three is a machine with a schedule. A human logging in at three is a human. A test in our build process makes sure nobody changes that by accident.

A sequence across several runs #

A break-in spreads over hours: first the login, then the new account, later the foreign network destination, in the evening the files. Four runs, four separate spikes, none of them enough on its own.

Timbre pulls that together – with order, because the order is the statement:

accounts → network → files     a break-in
files → network → accounts     a restore from backup

The same building blocks, the opposite meaning. Treat them as a set and the two cannot be told apart.

Every program on its own #

Until then a baseline always described the whole system: "this server usually opens 22 connections". That misses the program that changes while the rest of the system covers the number.

Timbre therefore keeps one baseline per program. There is explicitly no list of known software: a self-written service under /opt/firma/server gets the same treatment as nginx – namely none in particular. Every list of known programs is missing exactly what you built yourself.

A program needs eight measurements before anything is judged about it. Something that runs once a week has four after a month – and Timbre does not judge that, it stays quiet.

The shipped state #

Your package manager records the checksum every file had when it shipped. That list sits on your server, costs nothing and went unread until now – on an ordinary Debian it covers over sixty thousand files.

It answers a different question than the integrity check:

The first is the interesting one if a server was already altered at install time: then your own confirmation recorded the alteration as the normal state.

Checked are the monitored paths plus every program with the setuid or setgid bit set – there a change means the most, because it runs with someone else's rights. Not checked is what is allowed to change: configuration files, documentation, and files that do not exist on disk at all.

Without a package manager it says "not applicable" – not "no deviation". That is the same distinction as everywhere in Timbre: not measured is not the same as nothing found. And it takes not a single call to the outside: the list is already on your machine.

Have I seen this situation before? #

Out of all of it, each run produces a row of numbers – the timbre of your system. Two runs with the same row are the same situation, even weeks apart and even when nobody remembers.

That makes a question possible that could not be asked before: do I know this? And if not, NODE64 says so instead of presenting the closest match as a match. Because the closest match is always some match, the difference would otherwise never show.

The row of numbers contains no paths, no program names, no addresses – only dimensionless quantities: how far from the usual, how new, how rare.

What Timbre does not do #

• No signature database in the agent. It carries no list of known malware and downloads none.
• No neural network. Median, spread, percentile – methods you can recompute.
• No countermeasures. Timbre deletes nothing, kills no process, blocks no connection. It detects and explains.
• No file contents, no command lines. Both stay on your server.
• No overall score. There used to be one; it was dropped because nobody could say what 73 meant.
• No verdict without grounds. Where the history is not enough, it says so.

What holds when the connection drops #

The agent keeps running and sends later – but not everything can be answered without the platform. "Works without a backend" would be a headline this table contradicts, and a promise someone believes in an emergency is worse than none.

Why the line says "indicator" and not "finding": A finding is a judgement, and that needs the comparison with what is usual on this system. The agent does not have that comparison – it states that something happened.

Aug 21 12:34:56 server node64-agent[1234]: NODE64 Indikator: shell-unter-webserver

What happens afterwards. If the connection stays away longer than the queue holds, the agent discards the oldest states – and leaves a mark. On the next report it appears in every piece of evidence:

44 instead of the usual 40, from 96 measurements over 4 days
— with a gap of 12 runs that never arrived

Without that mark the series would afterwards look like a gapless measurement. It is not, and a number that looks more precise than it is, is worse than one that names its limits.

When it matters: the express report #

If the process watch detects one of the four immediate indicators, the agent does not wait for the next reporting interval. It brings the next full run forward and marks it as urgent; the platform then evaluates the anomalies first and sends the alert before starting the expensive comparison against the vulnerability database.

kernel event → watch               under 1 ms
collection window                  30 s
run brought forward                ~3 s
upload + priority processing       ~5 s
alert dispatch                     right after
                                   ─────────
instead of up to 15 or 60 minutes: under one minute

So that an attacker cannot set the pace this way, the express report is capped: at most one per five minutes, at most six per hour. What the limits turn away is not lost – it comes with the regular run.

What other installations see #

Next to each area there is now a second statement: how ordinary is what is happening here, on other installations?

locally conspicuous · globally 0.94 similar to pattern P-1827
· seen on 431 independent installations · trend rising

It is deliberately separate. The shared knowledge does not replace this device's baseline — a server without a connection detects exactly as much as one with. What is missing is missing from the context, not from the detection.

And a shared statement alone never creates a finding. A pattern seen on 431 installations is ordinary — more a reason for reassurance than for an alert. Whether something is a problem on your server is decided by your own history.

If the knowledge base knows nothing, nothing is shown. Not "unknown pattern", not "0 installations".

Related pages #

• Understanding findings – what a finding is and what to do with it
• Integrity – the checksums of system files
• What data is collected – complete, feature by feature