Governance · Caching · Replication

Your database survives self-service

Self-service BI fails in one specific way: someone points a fifteen-card dashboard with a thirty-second auto-refresh at a production database and the whole company notices. Qrly answers that with a per-connection query governor, per-user daily budgets, four opt-in cache tiers and, at the far end, full CDC replication of raw source tables. All of it is off until you turn it on.

Layer 1 · Admit

Query governor and budgets

  • Per-connection semaphore sized to max concurrent queries
  • FIFO wait queue capped at max queue size
  • Queue-full error instead of a stampede
  • Wait timeout after the configured number of seconds
  • Per-user daily query budget, atomically reserved
  • Live admin stats and persisted lifecycle events
Layer 2 · Remember

Result caching, T1 and T2

  • T1 in-JVM, encrypted, byte-budgeted, optional columnar store
  • T2 persistent — shared app database or per-connection SQLite WAL
  • Passive, stale-while-revalidate and cron-driven active refresh
  • AES-256-GCM at rest in both tiers
  • SHA-256 key over normalised SQL and sorted parameters
  • A cache failure degrades to a miss, never an error
Layer 3 · Replicate

Materialization and table sync, T3 and T4

  • T3 writes a question's result as a real table you own
  • External BI tools can point straight at that table
  • T4 replicates raw source tables, not question results
  • Full reload, watermark, snapshot-delta and CDC modes
  • Embedded Debezium for Postgres, MySQL/MariaDB, SQL Server
  • Read routing rewrites visual questions to the synced copy
01 / THE PROBLEM

A dashboard is a concurrency amplifier

A dashboard with fifteen cards is fifteen queries. Put it on a wall display with a one-minute auto-refresh, share it with a department, and the arithmetic runs away from you quickly. Nobody involved did anything wrong — they built a dashboard, which is the entire point of the product — but the source database now sees a sustained burst of analytical queries it was never sized for.

The classic outcome is not a slow dashboard. It is connection-pool exhaustion on the application that shares the database, at which point the analytics tool has taken down the system it was supposed to be reporting on. Retrying makes it worse. This is the single most common way self-service BI loses the trust of the team that runs the database.

02 / THE MECHANISM

A semaphore, a queue, and a refusal

Every Qrly connection has a governor: a semaphore sized to that connection's max concurrent queries, fronted by a FIFO wait queue capped at max queue size. Queries that cannot acquire a permit wait in order. Queries that arrive when the queue is already full get an immediate queue-full error, and a query that has waited longer than queue wait timeout is failed rather than left hanging.

The refusal is the feature. A tool that queues without bound eventually delivers every one of those queries to the database, just later and all at once. Qrly bounds the queue and returns an error, so load shedding happens in the BI tier where it is cheap, not in the database where it is not.

03 / VISIBILITY

Persisted events, live stats

Governor lifecycle events are persisted on a background executor, so recording them never sits on the query's own critical path. The admin page shows live per-connection statistics — active, queued, max concurrent, max queue and last wait — with auto-refresh, alongside a recent-events table.

That turns "the dashboard felt slow this morning" into a question with an answer. You can see whether queries were waiting, how long, on which connection, and whether the queue was ever full — before you start blaming the database.

04 / BUDGETS

Per-user daily query budgets

Concurrency limits shape the peak. Budgets shape the total. Each user has an effective daily budget resolved as per-user override → organisation default → 1000, checked and reserved atomically before every query and made race-safe by an upsert on the (user, date) pair — so two browser tabs cannot both spend the last query.

Bytes returned and AI tokens are recorded asynchronously afterwards. Administrators get a budget page, a per-user editor, a reset action and a usage report; users see their own remaining budget on their profile, which stops the whole thing being a mystery when a query is refused.

Execution limit
Default
What it bounds
Query timeout
60 s
How long any single statement may run
Max rows
10,000
Rows materialised for an interactive run
Max stream rows
1,000,000
Rows a streaming NDJSON or chunked CSV export may emit
Max concurrent queries
4
Semaphore permits — the true load ceiling
Max queue size
20
Waiting queries before overflow returns queue-full
Queue wait timeout
30 s
How long a query may sit in the queue before failing
Tier
What it holds
Where it lives
T1 — JVM cache
Encrypted result bytes, optionally a typed columnar store
In-process heap. Lost on restart
T2 — persistent cache
The same encrypted bytes in a compact self-describing binary format
Shared application database, or a per-connection SQLite WAL file
T3 — materialized table cache
A saved question's result, written as a real table
A customer-owned target database, readable by any BI tool
T4 — table sync
Raw source tables, replicated with optional projection and filter
An external customer connection or a Qrly-managed cache target
Note All four tiers are opt-in, per connection or per question
T1 / IN-JVM

Encrypted bytes, byte-budgeted, nearest-to-expiry eviction

The first tier lives in the process. It stores encrypted bytes rather than object graphs, and it is budgeted in bytes twice over — once per connection and once JVM-wide — with registered evictors that evict the entries nearest to expiry first. That is a deliberately unfashionable policy: it protects the entries with the most useful life left, rather than the ones that happen to have been touched recently.

Because it is in-process, it is the fastest tier and it is gone on restart. It is the right tier for a dashboard that ten people open in the same ten minutes.

T1 / COLUMNAR

An optional typed columnar store

T1 can also hold results in a columnar layout using typed primitive arrays per column — LONG, DOUBLE, STRING and BOOLEAN — rather than boxed row objects. The point is CPU-cache-friendly scans: a column of longs read sequentially out of a primitive array behaves very differently from the same data spread across a million small objects.

Hit counters and byte estimates are tracked per entry, so the columnar store is measurable rather than a matter of faith.

T2 / SHARED

The shared application database

Pointed at the shared application database, T2 gives cross-instance hits at roughly ten milliseconds: one node runs the query, every node in the cluster benefits. The registry is warmed from T2 at startup, so a rolling restart does not throw away everything the cluster had learned.

This is the sensible default for a clustered deployment where all instances serve the same tenants.

T2 / SQLITE

A per-connection SQLite WAL file

The alternative T2 backend is a SQLite file in WAL mode, one per connection, laid out on disk as {basePath}/{tenantSlug}/{orgSlug}/{connectionId}/cache.db. It is node-local rather than cluster-shared, and it buys something the shared database cannot: physical separation. One tenant's cached results are a different file, in a different directory, on a path you can point at in an audit.

Both backends carry the columnar layout in a compact self-describing binary wire format, so what T1 held in memory is what T2 writes to disk.

Refresh mode
Behaviour on expiry
Use it when
PASSIVE
Expire and miss — the next caller runs the query
Correctness matters more than the occasional slow first load
STALE
Serve the stale entry, revalidate asynchronously, guarded against the thundering herd by an in-flight key set
A busy dashboard where slightly old is better than slow
ACTIVE
Cron-driven proactive refresh — the entry is rebuilt before anyone asks
A known-heavy query on a predictable schedule, refreshed off-peak
01 / ENCRYPTION

AES-256-GCM in both tiers

Cached results are encrypted at rest in the JVM tier and in the persistent tier. The cache never stores plaintext rows — not in memory, not in the shared application database, not in the SQLite file.

This is the difference between a cache you can enable on a connection carrying customer data and one you cannot. A cache that quietly duplicates production rows into a second, less-protected store is a compliance problem wearing a performance costume.

02 / KEYS

SHA-256 over normalised SQL

The cache key is a SHA-256 hash of the organisation, the connection, the SQL with whitespace collapsed and keywords normalised, and the parameters sorted by name. Two logically identical queries that differ only in formatting or parameter ordering hit the same entry.

Because the organisation and connection are part of the key, a result can never be served across a tenant boundary — the key space is partitioned by construction, not by a filter someone has to remember to apply.

03 / CONTROL

Cache-Control, de-duplication, circuit breaker

Callers can pass NORMAL, NO_CACHE or NO_STORE to bypass or refuse to populate the cache for a specific run — useful when someone is reconciling a figure and wants to be certain they are looking at the source. Concurrent requests for the same key are de-duplicated so only one of them runs the query.

Per-connection consecutive-failure counters feed a ten-minute circuit breaker, and dispatch runs on virtual threads. A cache backend that starts failing stops being asked, rather than adding latency to every request while it fails.

04 / FAILURE

A cache failure is always a miss

This is the rule the rest of the design hangs off: cache failures degrade to a miss, never to an error. If the cache cannot answer, cannot decrypt, or cannot be reached, the query runs against the source and the user sees a result.

It means enabling caching cannot make your reporting less available than it was before you enabled it. The worst case is the performance you already had.

Write strategy
What it does
Requires
TRUNCATE_INSERT (default)
Empties the target table and reloads it
Nothing beyond write access
DROP_RECREATE
Drops the table and rebuilds it from the current result shape
Allow-drop-recreate enabled on the config
UPSERT
Merges rows by key, updating what exists and inserting what does not
Primary-key columns declared; row storage format
APPEND_SNAPSHOT
Appends each run as a dated snapshot, adding a timestamp column and a btree index
Row storage format
STORAGE

Row or columnar, in the target itself

The ROW format writes one wide table, which is what most people expect and what most external tools want. The COLUMNAR format writes one physical table per source column plus a coalescing view over them, which suits wide results where consumers select a handful of columns.

Columnar rejects UPSERT and APPEND_SNAPSHOT rather than pretending to support them — merging by key across one-table-per-column is not a thing you want happening quietly on your behalf.

TYPES

Logical categories to physical types

Results carry logical type categories, which are mapped to the target dialect's physical types on write: TEXT becomes TEXT, STRING or VARCHAR(MAX) depending on the engine; NUMBER becomes DECIMAL(p,s) when precision and scale are known; JSON is preserved natively on Postgres and MySQL and falls back to text elsewhere; ARRAY, RANGE and COMPOSITE values are flattened.

Schema drift is detected through a DDL hash, which drives auto-recreate — so a question that gains a column does not silently start writing into a table shaped for the old one.

READS

Read rewriting, with an honest header

With serve-reads-from-cache enabled, a question run is rewritten to SELECT * FROM target.table instead of re-executing against the source. When the materialized copy is missing or broken, the run falls back and says so in an X-Cache response header — MATERIALIZED-MISS or MATERIALIZED-ERROR.

You can therefore tell, per request, whether you were served from the materialized table or from the source. Caches that cannot tell you that are how people end up debugging the wrong system.

OPERATIONS

Locks, bookkeeping and a preview

Refresh follows the same passive, stale-by-TTL and active-by-cron modes as the shared cache. A per-config lock guarantees only one run is in flight, and every run is recorded — status, last run, duration, rows, error — alongside audit rows.

Batch size, statement timeout, max rows and whether drop-recreate is permitted are all configurable, and there is both a health probe and a preview-DDL endpoint so you can see exactly what Qrly intends to create before it creates it.

Sync mode
How it moves data
Honest caveat
FULL_RELOAD
Stages into <target>__staging, then swaps atomically
Reads the whole table every run
WATERMARK
Incremental on a watermark column; upserts when primary keys are defined, otherwise appends
Does not capture deletes
SNAPSHOT_DELTA
Full on the first run, watermark thereafter
Inherits the watermark mode's delete blind spot
CDC
Embedded Debezium — snapshot, then tail the change stream
Needs replication configured on the source engine
Write strategies Staging-swap Truncate-insert Upsert Append-only
01 / SWAPS

Per-dialect atomic staging swap

Full reloads never truncate the table your users are reading. Data lands in a staging table and the two are swapped: a rename on Postgres, RENAME TABLE on MySQL, sp_rename on SQL Server. Readers see the old copy right up until they see the new one.

BigQuery has no equivalent rename, so it falls back to drop, create and insert inside a single transaction — stated plainly rather than papered over, because that fallback has a visible consequence.

02 / CONSISTENCY

FK groups under one source snapshot

Replicating five related tables in five independent passes gives you five tables that were each individually correct at a different moment — which is how you end up with orphan rows in a replica nobody trusts. Qrly coordinates multi-table syncs as an FK group under a single source snapshot: repeatable read with an exported snapshot on Postgres and Redshift, InnoDB consistent snapshot on MySQL, best-effort elsewhere.

The apply order comes from a topological sort of the foreign keys, with an optional referential check afterwards. Parents land before children, by construction.

03 / THROUGHPUT

Streaming reads and native COPY

Reads are dialect-aware and streamed: a server-side cursor on Postgres, row-by-row on MySQL, tuned fetch size elsewhere, with configurable fetch and batch sizes. Writes use native COPY on Postgres and Redshift with automatic fallback to batched INSERT, and per-dialect upsert — ON CONFLICT, ON DUPLICATE KEY, SQLite's native form, or a portable MERGE.

Each batch commits, so a retry after a failure is idempotent rather than a restart from zero. Tasks run one virtual thread each behind a fair semaphore, with per-config locks and queued and in-flight counters.

04 / ROUTING

Reads follow the replica — conditionally

When a sync config exists, is enabled, serves reads and its last successful run is inside the TTL, visual question runs are transparently rewritten to the synced copy. Four conditions, all of which must hold; otherwise the query goes to the source as usual.

Native SQL is never rewritten. If you wrote the SQL yourself, Qrly runs the SQL you wrote against the connection you named — silently retargeting hand-written SQL to a replica would be an unpleasant surprise in a reconciliation.

Source engine
Capture mechanism
What Qrly provisions
PostgreSQL
Logical replication
A replication slot and publication per source
MySQL / MariaDB
Binlog
A derived unique server id
SQL Server
Database-level CDC
Per-table capture instances
Stream handling Tolerates tombstones, heartbeats, schema changes and truncates Prepared DML cached per config and operation Unowned events are dropped silently

What a run tells you afterwards

Every sync run is recorded with rows read and written, bytes, the watermark it moved from and to, status, duration and error, plus a DDL hash for drift detection. Replication that cannot be audited after the fact is replication you will eventually stop believing — the run history is what lets you answer "is the replica behind, and by how much" without opening the target database yourself.

The default posture, stated plainly

Every tier described on this page is opt-in, enabled per connection or per question. Out of the box, a Qrly query goes to the source, live, every time — through the governor, against the connection's limits, recorded in the query audit log.

  1. Caching is off. T1, T2, T3 and T4 are each enabled deliberately, on the connections and questions where they earn their place. There is no global "make it fast" switch that quietly duplicates production data.
  2. Question-level materialization is separate and narrower. It is a simpler opt-in cache for un-parameterized base reads only. Any caller-supplied parameter or interactive filter value forces a live run — a filtered question is never answered from a materialization built for the unfiltered one.
  3. Schema sync is off too. Reading your schema to power SQL autocomplete and the visual builder is opt-in, with a manual trigger and a nightly scheduler once enabled.
  4. Everything that ran is written down. The query audit log records the full SQL executed, row count, duration, status — SUCCESS, ERROR or TIMEOUT — and the error message, browsable and filterable, with retention enforced by a daily scheduler.
  5. Read-only by default at the driver. The pool hands out connections with setReadOnly(true), native SQL passes the sanitizer, and every parameter is bound. The only path that changes the shape of a customer database is Constructions, which is separately gated.
Scheduler
Cadence
Job
AlertScheduler
Hourly
Evaluates alerts, with per-cadence gating
SubscriptionScheduler
Daily 06:00
Renders and delivers subscriptions
SchemaSyncScheduler
Nightly
Refreshes connection schema metadata
MaterializationScheduler
Per minute
Question-level materialization
TableMaterializationScheduler
Per minute
T3 cron-driven refreshes
TableSyncScheduler
Per minute
T4 configs and FK groups
QueryCacheEvictionScheduler
60 s / hourly
T1 sweep, T2 cleanup, active-refresh tick
FreshnessScheduler
Every 5 min
Drives the "Data as of …" badges
AnomalyScheduler
Nightly 03:00
Anomaly detection sweep
ProactiveInsightScheduler
Hourly
Honours a per-organisation cron
ModelerEnrichmentScheduler
Nightly 02:30
Model metadata enrichment
NarrationWorker
5 s drain
Uses FOR UPDATE SKIP LOCKED
WeeklyInvestigationDigestScheduler
Mondays 07:00
Weekly investigation digest
BackupScheduler
Per-minute tick
Runs an operator-editable cron
AuditLogRetentionScheduler
Daily 03:00
Enforces query audit retention
AiCallContextPurgeScheduler
Daily 03:30
Purges AI call context, default 90 days
RiskReportScheduler
Hourly
Overdue risk-report sweep
Gate All schedulers are disabled until the platform is initialised
Does Qrly cache my data by default?

No. All four cache tiers are opt-in, enabled per connection or per question. With nothing switched on, every query runs live against the source, through the governor and the audit log.

Question-level materialization is a separate, simpler opt-in for un-parameterized base reads only — any caller-supplied parameter or interactive filter value forces a live run.

Is cached data encrypted at rest?

Yes. The in-JVM tier and the persistent tier both store AES-256-GCM encrypted bytes, so the cache never holds plaintext rows.

The persistent tier is either the shared application database, for cross-instance hits, or a per-connection SQLite WAL file laid out as {basePath}/{tenantSlug}/{orgSlug}/{connectionId}/cache.db, which gives physical separation per tenant and organisation on the node's own disk.

What stops one runaway dashboard from saturating my production database?

The query governor. Each connection gets a semaphore sized to its max-concurrent-queries setting, with a FIFO wait queue capped at max queue size. Overflow returns a queue-full error rather than piling more work onto the database, and a query that waits longer than the queue wait timeout is rejected.

Lifecycle events are persisted on a background executor, and the admin page shows live per-connection stats — active, queued, max concurrent, max queue, last wait — with a recent-events table.

What happens if the cache itself fails?

A cache failure always degrades to a miss, never to an error — the query simply runs against the source.

On top of that there is per-key de-duplication, per-connection consecutive-failure counters and a ten-minute circuit breaker, so a sick cache backend stops being consulted rather than slowing every request down.

How does CDC replication work in Qrly?

Table sync can run in CDC mode using embedded Debezium, loaded reflectively so the application still starts when it is not on the classpath. Postgres uses logical replication with a per-source slot and publication, MySQL and MariaDB use the binlog with a derived unique server id, and SQL Server uses database-level CDC with per-table capture instances.

All three are snapshot-then-tail, with credentials injected after decryption, offsets and schema history persisted, a parser tolerant of tombstones, heartbeats, schema changes and truncates, and an applier that caches prepared DML per config and operation.

Can external BI tools read Qrly's cache?

Yes, with the materialized table cache. It re-runs a saved question and writes the result as a real table in a customer-owned target database, which any external tool can point at directly.

It supports truncate-insert, drop-recreate, upsert and append-snapshot write strategies, row and columnar storage formats, and schema-drift detection through a DDL hash that drives auto-recreate.

Fast enough to trust, bounded enough to allow

A governor your DBA can read, four cache tiers you switch on deliberately, and replication that writes into a database you own.