QoreDB's universal data model: Namespace, Collection, Row

QoreDB talks to PostgreSQL, MySQL, MongoDB, SQLite, Redis, ClickHouse, SQL Server, CockroachDB and other engines. Each one exposes its own concepts: databases, schemas, keyspaces, Elasticsearch indexes, tables, views, BSON collections. To avoid having one UI per engine or branching on the driver type all over the code, the qore-core core exposes a minimal vocabulary shared by all drivers.

This vocabulary boils down to three building blocks: Namespace (where the data is), Collection (in which container), Row (a typed row of data). This article describes the exact shape of these types, what they model, and how far the abstraction deliberately goes.

Namespace: the logical container

A Namespace is a pair (database, optional schema). Its Rust definition in qore-core/src/types.rs fits in a few lines: two fields, two constructors, new(database) and with_schema(database, schema).

For two-level engines (PostgreSQL, CockroachDB, SQL Server), a Namespace carries both components. The PostgreSQL driver, for example, iterates over pg_catalog.pg_namespace and calls Namespace::with_schema(current_database, schema_name). The SQL Server driver does the same, relying on sys.schemas.

For single-level engines (MySQL, MariaDB, MongoDB, SQLite, ClickHouse, DuckDB), the schema field stays None. The MySQL driver builds its Namespace with Namespace::new(database_name), and QoreDB displays the hierarchy with a single level.

Redis lends itself to the same scheme: each logical database (db0 to db15) becomes a Namespace named db0, db1, and so on. Elasticsearch and OpenSearch, which have neither databases nor schemas in the SQL sense, expose a single Namespace carrying the cluster name.

The reasoning is straightforward: rather than inventing a model that tries to force-fit every engine, we take the greatest common denominator observed (database + optional schema) and let each driver fill it in with the semantics that match its backend. The frontend displays a homogeneous tree without having to know the subtleties of each engine.

Collection: the storage unit

A Collection carries a Namespace, a name, and a discriminant type (CollectionType). The variants are Table, View, MaterializedView, and Collection for NoSQL engines.

This discriminant matters. It lets the frontend render views differently from tables (icon, available actions), without forcing engines that don't know certain concepts to emulate them. MongoDB returns Collection objects of the Collection variant. PostgreSQL returns a mix of Table, View and MaterializedView. SQLite returns Table and View. Each driver declares what it knows, and the UI adapts.

The deliberately generic name Collection reflects the fact that it's neither a SQL table nor a BSON container as such: it's the abstract notion of a place where rows live. This choice avoids the opposite trap; calling it Table would force the vocabulary to be twisted on the NoSQL side and would leak false friends into the API.

Row and Value: the data content

A Row is an ordered vector of values. It is indexed by position, with the column metadata (ColumnInfo) carried separately in QueryResult. This is the form returned by read queries.

For mutations, QoreDB uses RowData rather than Row. RowData is a HashMap<String, Value> where the key is the column name. The distinction reflects a reality: for reading, you want a reproducible order aligned with the result's columns; for writing (INSERT/UPDATE), you want to address columns by name, without depending on an implicit order.

The Value type is the enumeration that bounds the universe of possible values: Null, Bool, Int(i64), Float(f64), Text(String), Bytes(Vec<u8>) (serialized as base64 on the wire), Json, and Array(Vec<Value>). Eight variants that cover the common trunk of what the engines support.

Engine-specific types (PostgreSQL's arbitrary-precision NUMERIC, TIMESTAMP WITH TIME ZONE, INTERVAL, ENUM, GEOMETRY) go through Text or Json depending on what best preserves the information. The driver decides, at the mapping level, what stays readable for the user. Value is not an abstraction over the engine's physical types: the TableColumn that accompanies a schema carries a data_type as a string (VARCHAR(255), NUMERIC(10,2), jsonb, ObjectId…) taken directly from the engine's catalog. Value exists only to carry cells between the driver and the frontend in a typed way.

The DataEngine trait built on these primitives

The DataEngine trait (qore-core/src/traits.rs) is the single surface that each driver implements. Its methods are expressed almost exclusively in terms of Namespace, Collection and Row: list_namespaces(session), list_collections(session, namespace, options), get_schema(session, namespace, collection), query_table(session, namespace, collection, options).

This uniformity of the contract makes a generic UI possible: the Browser component doesn't know whether you're connected to a PostgreSQL or a MongoDB, it unfolds Namespace and Collection objects. The Results component displays Row and Value without knowing where they come from. The Tauri commands (src-tauri/src/commands/) manipulate these same types and serialize them to TypeScript.

What stays at the driver level, and not the model: the SQL dialect, the transaction semantics, the precise capabilities (streaming, cancellation, explain, maintenance). This information surfaces through DriverCapabilities, which the frontend queries to enable or disable certain actions. The model remains an abstraction of structure, not an abstraction of behavior.

A minimalist model, a homogeneous UI

A minimalist model is more useful than an exhaustive one when it comes to unifying very different engines. Three building blocks, a discriminant on CollectionType, a bounded enumeration for Value. The rest lives in the driver, surfaces through DriverCapabilities and stays visible to the user.

This is what lets QoreDB add a new engine without breaking the UI and without lying to the user about what they're looking at.