Skip to main content

​
Overview

Workers are the execution engines of the task management system. Each worker operates as an independent Tokio task that competes for tasks from a shared queue, executes them via the task executor, and reports results back to the pool. The worker pool manages workers.

​
Architecture

A worker is fundamentally an async event loop that processes tasks until shutdown or idle timeout. Each worker has a UUID-based worker ID, holds a reference to the shared task queue and task store, and uses two coordination primitives:
  • Task notification - a shared Notify signals when new tasks are enqueued. Workers wake up and compete to dequeue a task (another worker may grab it first).
  • Shutdown signal - a CancellationToken propagated from the worker pool triggers graceful termination.
The worker’s execution loop uses tokio::select! to multiplex the task notifier, the cancellation token, and (for non-baseline workers) an idle timeout. When notified, the worker attempts to dequeue a task from the shared priority queue. If another worker already grabbed it, the loop continues waiting.

​
Worker types

The worker pool maintains two categories of workers:
  • Baseline workers (min_workers) - always running, no idle timeout. The scheduler spawns these on startup, and they stay alive until shutdown.
  • Demand workers - spawned when all active workers are busy and the pool is below max_workers. These have an idle timeout and exit automatically when they receive no work within that window.

​
Backpressure

Workers signal “currently executing a task” using an atomic busy counter. The worker pool checks this counter on every task enqueue: if all active workers are busy and the pool is below max_workers, the pool spawns a new demand worker. This ensures the pool scales up under load without over-provisioning.

​
Task execution

When a worker picks up a task, it drives the full task lifecycle:
  1. Initializing - load task details, emit TaskStarting event
  2. Preparing resources - acquire a machine from the machine pool (or fast-path for host-only tasks)
  3. Running - transfer sample to guest VM, register guest plugins, execute the sample, then run all plugins with an analysis timeout
  4. Stopping - release the machine, unregister guest plugins
  5. Final state - mark the task as Completed, Failed, TimedOut, or Canceled
Task cancellation is cooperative: each task gets a child CancellationToken that is checked at safe points during execution, allowing in-flight tasks to clean up (release machines, unregister plugins) before returning.

​
Configuration

Configuration options control worker behavior:
  • Maximum and minimum worker counts
  • Idle timeout for demand workers
  • Compatible task types (enabling worker specialization)
  • Execution mode (single or batch processing)
  • Resource limits (memory, CPU, disk, network)
  • Plugin allow/deny lists
See the complete configuration reference.