Technology / Architecture Deep Dive
Architecture

Architecture
Deep Dive

A detailed look at Scalefield Secure's three-zone architecture — from database-level audit collection through centralized processing to compliance visualization.

Three-Zone Architecture

Zone 1: Gather

Secure Gather modules run alongside each monitored database. For PostgreSQL, this means WAL-embedded collection via output plugins. For Oracle, integration with Unified Audit Trail. For SQL Server, DB2, and MongoDB, engine-specific audit connectors. All data is compressed into Apache Parquet format and written to configurable storage backends — local filesystem, Ceph, or S3-compatible object stores.

Architecture zone diagram

Zone 2: Secure Engine

The central processing layer. Receives Parquet files from all Gather nodes, performs aggregation, scheduling, alerting, and reporting. ML algorithms (Isolation Forest, DBSCAN) run here to detect anomalies. The engine maintains the compliance state for your entire database estate and triggers alerts when policy violations occur.

Architecture zone diagram

Zone 3: Visualization

Three distinct interfaces. Administration for system configuration and user management. Management for compliance monitoring, reporting, and audit-ready exports. Operator for day-to-day operational views including geo-mapped host status, patch tracking, and infrastructure health dashboards.

Architecture zone diagram

Data Flow Pipeline

Gather

Secure Gather modules capture SQL statements, execution statistics, authentication events, and schema changes from each monitored database.

Collect & Transform

The Collector process extracts raw audit events and transforms them into compressed Apache Parquet files for efficient storage and transport.

Import & Process

The Secure Engine imports Parquet data, aggregates events across instances, applies retention policies, and generates alerts on policy violations.

Monitor & Report

Dashboards display real-time compliance status, operational intelligence, and audit-ready reports for regulators and internal governance teams.

Network & Storage Requirements

Storage Backends

Local filesystem (PoC), Ceph (on-premise petabyte-scale), S3-compatible object stores (AWS S3, MinIO, Azure Blob, GCS) for cloud.

Network

Gather nodes communicate with the Secure Engine over TLS-encrypted connections. Dashboard access via HTTPS. No inbound internet access required for on-premise deployments.

Scalability

Horizontal scaling via Kubernetes. Add Gather instances per database, scale Engine replicas for processing throughput.

See the Architecture in Action

Ready to see the architecture in action? Request a demo.