SharpCoreDB

Story behind the project

SharpCoreDB was built from the frustration with heavy database clusters for small-to-medium workloads and the limitations of plain JSON files. After 28 years of .NET development we know exactly where teams hit walls: transactions, queries, encryption and performance without external dependencies. Designed for C# 14, .NET 10 and zero-allocation principles.

Problem this solves

Full database servers (SQL Server, PostgreSQL) are too heavy for many scenarios: edge devices, offline apps, config stores, audit logs, embedded tooling. At the same time, plain JSON/CSV files are insufficient for queries, transactions, concurrency and reliable storage. Teams keep rebuilding the same wheel: file locking, parsing, indexing.

Architecture

Single-file storage engine with its own SQL parser, transactional write-ahead flow, page-based allocation and at-rest encryption. Heavy use of Span<T>, Memory<T>, ArrayPool<T> and Channel<T> for producer-consumer writes. Extensible modules for indexing, compression and replication hooks.

sequenceDiagram
    participant App
    participant DB as SharpCoreDB
    participant Storage as SingleFile
    App->>DB: ExecuteBatchSQL(statements)
    DB->>DB: Parse + Validate
    DB->>Storage: Write pages (WAL)
    Storage-->>DB: Offset + Checksum
    DB->>App: Result + Flush

Lessons learned

Designing performance, security and developer ergonomics together from day one prevents expensive rework. Use ExecuteBatchSQL + Flush instead of individual calls. Never sync-over-async. Use Lock class (C# 14), never object. After 28 years it is clear: the fastest code is code you do not have to debug in production.

Screenshots

Code snippets

// Factory + open with encryption
using SharpCoreDB;
var factory = new DatabaseFactory();
await using var db = factory.Create("app.scdb", "strong-master-password");

// Batch for performance (persists directly to engine)
var statements = new List<string>
{
    "CREATE TABLE posts (id INT PRIMARY KEY, title TEXT, created TEXT)",
    "INSERT INTO posts VALUES (1, 'Hello', '2025-01-28')"
};
db.ExecuteBatchSQL(statements);
db.Flush();
db.ForceSave();

// Zero-allocation hot path example
ReadOnlySpan<byte> data = ...;
var buffer = ArrayPool<byte>.Shared.Rent(4096);
try
{
    // write via Span
}
finally { ArrayPool<byte>.Shared.Return(buffer); }