THUGS(red) security check

How to run the benchmark

Twenty minutes end to end. Most of that is you watching your SOC to see whether they notice.

download 0.1.1

Tell someone first — then don't tell them everything

The point of the exercise is to see what your SOC detects unprompted. So: get written authorisation from whoever owns the risk in advance, but keep the exact timing away from the analysts on shift if you want an honest measurement.

Have a named contact who can call the test off, and be ready to identify yourself the moment anyone asks. An unexplained "intrusion" that runs into a major-incident process is expensive for everybody.

Afterwards, always disclose. Compare their timeline against the one in your report.

Step by step

  1. Prepare the machine A clean install, or a VM snapshot you can restore. Leave the security tooling exactly as your fleet has it configured — do not add exclusions, and do not disable anything. Measuring your real configuration is the entire point.
  2. Download the client Grab the binary for the platform under test and verify its SHA-256 against the published checksum. One file, no installer, no runtime.
  3. Read what it is about to do --explain prints every command the run would execute, every outbound destination it would contact and every change it would make — and runs nothing. --list-targets prints the destinations on their own. These two are what a security team needs in order to approve a run.
  4. Run it with the consent flag The client refuses to do anything destructive-looking until you have explicitly consented. See below.
  5. Watch it run — and watch your SOC Every check and every trigger prints its result as it completes. The main process supervises each test in its own worker, so when your EDR kills a worker the run continues and records the kill as a detection. Anything that changes machine state writes down how to undo itself before making the change, so a worker that is killed mid-change still gets reverted.
  6. Collect the report The client writes securitycheck-report-<timestamp>.json next to itself and also prints it to stdout. --formats html,markdown,csv,navigator writes the same run as a self-contained HTML report, a summary you can paste into a ticket, a CSV of the results, and an ATT&CK Navigator layer for your own coverage map.
  7. Upload it, or don't Uploading gets you the comparison against everyone else — either by hand with curl, or with --upload at the end of the run. Reading the JSON locally and never sending it is a perfectly valid way to use this tool.
  8. Destroy the machine Wipe it or roll the snapshot back. The client cleans up after itself, but a machine that has just been used to simulate an intrusion is not a machine to hand back to a user.

What a full run measures

Three separate things, kept separate in the report because they answer different questions.

Posture
What is configured: protection products and their health, firewall, disk encryption, patch currency, boot and firmware state, logging and audit configuration, certificate trust, proxy inspection, backup, cloud management, browsers, remote-access paths and installed software. Read-only, and collected even in --safe.
Hardening control checks
A curated set of CIS-aligned controls, each ending in pass, fail, not-applicable or unknown — and unknown is never scored as a failure. They are also mapped onto CIS Controls v8, ISO/IEC 27001:2022, NIS2 Article 21 and SOC 2 identifiers, which is a themed mapping to start a conversation with, not an audit position.
Trigger tests
Simulated attacker behaviour across antivirus signatures, ransomware-like activity, persistence, credential access, process injection, living-off-the-land binaries, defence evasion, discovery, lateral movement, command-and-control, exfiltration, cloud abuse, device control and access to known hacking services. Every test declares the ATT&CK techniques it exercises, and the registry refuses to load one that declares none.

Where the client can read the host's own logs, it also correlates each trigger against them and records how long the first matching entry took to appear. A match is evidence rather than proof — it is matched by time window and indicator — and a test with no matching entry is left out of the timing figures rather than counted as slow.

--list prints every test, --only and --skip select by key or category, and --list-profiles shows the named scenarios. A long run can be paced with --pace, and one interrupted mid-way can be picked up again with --resume.

Uploading a report

Uploads are server-to-server with a bearer token, so nothing depends on a browser session. Create a token under Account → API tokens — it is shown once, at creation.

upload with curl
curl -sS -X POST https://securitycheck.thugs.red/api/v1/reports \
  -H "Authorization: Bearer $SECURITYCHECK_TOKEN" \
  -H "Content-Type: application/json" \
  --data-binary @securitycheck-report-20260813T101500Z.json

The client can also do it for you at the end of a run. It still writes the report to disk first, and it never uploads unless you ask:

upload from the client
SECURITYCHECK_TOKEN=… ./securitycheck --i-understand --upload

Without --upload the client writes the report and stops, so you can read it, keep it, or decide not to share it at all — nothing leaves the test machine unless you send it. The token is read from the environment so it never appears in the process list.

Endpoint
POST https://securitycheck.thugs.red/api/v1/reports
Authentication
Authorization: Bearer <token>
Body
The report JSON exactly as the client wrote it, unmodified.
Maximum size
2 MB per report
Rate limit
20 uploads per hour per account
Duplicates
Re-uploading the same run is idempotent — it will not create a second report.

Reports arrive private. Nothing you upload appears in the public statistics until you change its visibility yourself.

What gets shared

The three visibility levels for an uploaded report.
Visibility Who can read it In public statistics?
private Only you No
organisation Members of your organisation No
public Anyone with the link Yes — this is what makes the comparison possible

The client hashes anything identifying before it ever leaves the machine: hostname, serial numbers and USB serials are SHA-256 truncated, and your IP is stored only as a keyed hash. Aggregates are additionally suppressed until enough separate reports back them, so a public statistic can never describe a single contributor.

The half the client cannot measure

The report tells you what the machine did. It cannot tell you what your organisation did. Keep your own notes against the run timeline:

  • Did an alert fire at all — and how long after the trigger?
  • Did it reach a human, or die in a queue?
  • Was the correct severity assigned? Did major incident management activate?
  • Were the documented contact and escalation paths actually followed?
  • Did anyone isolate the machine, and how long did that take?
  • Was the "user" contacted? Did anyone ask you what you were doing?

Every test in the report carries a timestamp, so you can lay your SOC's timeline directly against the machine's.

Those answers can also travel with the report instead of living in a separate document. --soc-template writes a blank answers file, --soc-questions prints the questions, and --soc-answers attaches a filled-in file to the report on the next run or upload:

capture the response side after the fact
./securitycheck --soc-template soc.json      # write the blank file
# … fill it in once the team has had a chance to react …
./securitycheck --safe --soc-answers soc.json

A question you skip is recorded as unanswered, never as “no”. The difference between “we asked and nobody was contacted” and “we did not ask” is the whole value of the section, and these answers are self-reported — which is exactly why they are kept apart from everything the machine measured.

Get the client

root@thugsred:~$ ./securitycheck --i-understand