Three Months of Growing RepoDB Across Database Providers

I want to take a step back and reflect on the last three months of work on RepoDB, because it has genuinely been one of the more intense stretches since I started the project.

At the start of summer, RepoDB supported the four providers most people associate with it: SQL Server, SQLite, MySQL, and PostgreSQL. Since then, I have pushed support for Oracle, IBM DB2, MariaDB, MariaDB Connector, MySQL Connector, Firebird, Vertica, SAP HANA, ClickHouse, and EnterpriseDB. That is ten new providers in roughly three months, bringing the total to fourteen.

No Two Databases Play by the Same Rules

Going in, I expected the differences between these engines to mostly be syntax — a different way to write LIMIT, a different quoting character for identifiers. That assumption did not survive contact with the actual work.

Oracle does not allow multiple SQL statements to be packed into a single round-trip the way SQL Server does, which meant QueryMultiple needed its own round-trip strategy just for that provider. Its bulk operations also needed an entirely custom array-binding path, because Oracle’s ODP.NET driver does not expose anything close to SqlBulkCopy.

DB2 brought a different kind of pain: the official driver ships as a native shared library, and getting it to load correctly in CI, across Docker health checks and container startup timing, took more iterations than the actual data-access code did.

MariaDB looked like it should just piggyback on the existing MySQL support, since both speak a very similar wire protocol. But treating them as the same thing causes real collisions the moment both providers are registered in the same process. That is what pushed me to build a dedicated RepoDb.Connector.MariaDb, with every type prefixed so a MariaDbConnection and a MySqlConnection can coexist without stepping on each other, even though both ultimately talk to the server through the same underlying driver.

Firebird had no bulk-loading protocol at all in its ADO.NET driver, so I had to hand-roll a command batcher just to get bulk operations working at a reasonable speed.

Vertica ended up templated from that same Firebird groundwork, since its driver situation was similarly sparse. Unfortunately, there is no real support to wired-protocol bulk copy, ended up to Task.Run and flagged it.

SAP HANA’s bulk copy class only exposed a synchronous API, so making it behave asynchronously meant wrapping it with Task.Run or batch via command through custom batcher — a shortcut I am not fully comfortable with, and one I have flagged to revisit.

And ClickHouse was the biggest mental shift of the batch, because it is not a transactional row-store at all. There is no identity column, so RepoDB cannot hand you back a generated key the way it does everywhere else. There is no MERGE/UPSERT, so Merge compiles down to a plain INSERT and deduplication becomes a table-engine concern instead of a write-time one. Updates and deletes are background mutations, not immediate in-place writes. None of RepoDB’s existing assumptions about “a row” held up, and the provider had to be honest about that instead of pretending ClickHouse is just another RDBMS.

Where RepoDB Stands Today

As of today, RepoDB officially supports fourteen database providers:

• ClickHouse
• EnterpriseDB
• Firebird
• IBM DB2
• MariaDB
• MariaDB Connector
• MySQL
• MySQL Connector
• Oracle
• PostgreSQL
• SAP HANA
• SQL Server
• SQLite
• Vertica

Why RepoDB’s Resolution Layer Matters More Than Ever

None of this would have been sustainable if every provider needed its own bespoke way of being called. The reason a developer can write the same Query, Insert, Merge, or BulkInsert call against SQL Server, Oracle, or ClickHouse and have it just work is because RepoDB pushes all of that variance into a small, well-defined set of resolution points — the IDbSetting, IDbHelper, IStatementBuilder, and CLR-to-database-type resolvers each provider plugs in.

Those resolutions are where the real gap-closing happens. They are what let RepoDB decide, per provider, whether an identity value comes back from the server or has to be generated by the caller, whether a merge is a real MERGE statement or a fallback INSERT, whether a bulk load goes through a native binary protocol or a hand-rolled batcher, and how a .NET type maps to whatever native type that specific engine expects. Without that layer, “move data from one provider to another” would mean rewriting data-access code every time the destination changed. With it, the same repository code can target a completely different database engine, and the only thing that changes is which resolvers get registered.

That, more than any single provider, is the part of this three-month push I am proudest of. Getting Oracle, DB2, or ClickHouse working was a lot of grinding. But proving that RepoDB’s resolution model could absorb engines this different from each other — without bending the core API out of shape — is what tells me the architecture is sound enough to keep growing.

Rethinking the Architecture in Four Layers

Going through fourteen providers back to back also forced a bigger realization: RepoDB itself needed to stop being thought of as “one ORM library” and start being thought of as a stack. That thinking is what led to the four-layer architecture I laid out in RepoDB’s New Architecture - Productivity, ORM, Low-Level Connectors and Insights — Low-Level Connectors at the foundation, ORM on top of that, Productivity on top of ORM, and Insights branching off the ORM layer to observe everything flowing through it.

That separation is not just a diagram; it is a direct consequence of everything described above. Low-Level Connectors is where all of the provider-specific pain — Oracle’s array binding, Firebird’s hand-rolled batcher, MariaDB needing its own prefixed types — actually lives, isolated from the ORM surface. ORM is what stays provider-agnostic thanks to the resolution layer, so a BulkInsert call reads the same whether it targets SQL Server or ClickHouse. And once that separation exists, Productivity-level operations like Move, Copy, Merge, and Deduplicate can be built once, on top of Bulk CRUD, instead of being reimplemented per provider. It is the same closing-the-gap idea from the resolution layer, just one level higher: the boundary between fourteen very different databases collapses into a single, consistent surface for actually moving data between them.

What This Sets Up

Fourteen providers is not the finish line. It is proof that the model scales to genuinely different kinds of databases, transactional and analytical alike, which matters a lot if RepoDB is going to keep closing the gap between how different data platforms store and move data. There is still cleanup ahead — the SAP HANA async story being the most obvious one — but the foundation for the next batch of providers is in a much better place than it was three months ago.

More to come as this keeps moving.

~ This post is refined by AI, but is reviewed and gatekept by me before posting ~