Technical Challenges

Performance

Since it's a desktop/CLI app where responsiveness is expected, I optimized for performance earler in the dev process than I normally would. Synchronizing content from Atomic Red Team and MITRE Enterprise ATT&CK would be very slow if MACAT replaced all existing content each time. MACAT extracts various indexes and the large atomic red team repository zip to persist procedural and binary contents in the SQLITE db. MACAT uses these binary files to auto-extract to disk if they're needed to run a particular procedure.

Synchronizing Procedure Content

I route all content handling through a synchronizer (macatable_synchronizer) that diffs against existing data. This pulls each lookup table including platforms, executors, step and argument types, tags, and threat profiles into hash maps once per run. It bulk-fetches procedures that already exist by id and compares each incoming procedure against stored versions, keying on command text so each row gets tagged as new, removed, or unchanged. Unchanged content is left alone so only real differences require I/O against the SQLite db. Procedure changes create a ProcedureMorphAction which are collected and flattened across a batch, and then applied as batched bulk inserts and deletes in a single transaction. First time sync returns early and does a full import of procedure data etc.

The batching makes for better performance than the alternatives for the majority of common use cases. The synchronizer described above batches insert statements. The hash-map lookups on smaller data collections prevent looping queries to the DB, and the diff makes it so re-importing a basically unchanged Atomic Red Team repo does very little writing. Synchronization is mostly fast and idempotent, but the tradeoff is a more complex implementation and dependence on a stable key (I use command strings for this because some of the content doesn't have a natural key). If a procedure has duplicate command keys, it just replaces all the steps.

Synchronizing Binary Data (Files in DB)

The ART payload files are handled similarly to the procedure content sync. The Atomic Red Team repo is downloaded as a large zip of thousands of small files. If I expanded this to disk, Defender/EDR goes apoplectic, so to avoid that I iterate the entries through a streaming extractor with caps on total and expanded size. I compress files with zstd and write into SQLite as a blob. The file row is deliberately split into metadata (path, size, mime, a SHA256) in one table and bytes in another. The split at least theoretically helps with lookups and I/O. New or modified files are detected and inserted or updated by hash matching.

As mentioned above, extraction to disk is intentionally deferred until runtime. For Atomic Red Team content, a procedure's referenced files are extracted only when something actually runs and the on-disk copy is missing, and once they're already on disk, an "already exists" check skips the db check and extract entirely. For MACAT content, file extraction steps are followed explicitly.

Separation & Modularity

I intentionally enforced an organized project structure and kept code cleanly separated for testability and also because a lot of functionality is reused by the various deployment & distribution options or as part of different build steps.

Commands / Service

The UI uses a lightweight controller pattern which calls a command API in the service layer. This command API is shared by both the MCP server and the CLI which makes the CLI very lightweight. This way the UI, CLI, and MCP are mostly just concerned with transmission and dispatch.

Content Management

Modularity of the core crates means it's easy to create small CLI tools that use MACAT functionality. One especially helpful one that I have baked into the pnpm build process is prebuild which uses locally cached copies of MITRE and ART data plus the existing MACAT migrations to build a db that gets embedded in the MACAT portable binaries. This db is extracted on first run.