Skip to content

jamals86/KalamDB

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1,216 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

KalamDB logo

Rust License CI Overall Tests Release Docker Pulls pg-kalam Docker Pulls

SDKs

TypeScript SDK Tests Dart SDK Tests TypeScript SDK Dart SDK

KalamDB is a SQL-first realtime state database for AI agents, chat products, and multi-tenant SaaS. It combines SQL execution, live subscriptions, pub/sub streams, and hot/cold storage in one Rust runtime.

Supported SDKs: TypeScript/JavaScript and Dart/Flutter.

PostgreSQL extension Docker image: jamals86/pg-kalam with pg_kalam preinstalled.

Frontend clients can execute SQL directly against KalamDB. This is not only a backend database layer.

Frontend-First SQL Execution

To avoid confusion: KalamDB is designed for both frontend and backend SQL execution.

  • Frontend (web/mobile/desktop): run SQL for user-scoped reads/writes and realtime subscriptions.
  • Backend/workers: run automation, cross-tenant system jobs, and service workflows.
  • Same SQL model on both sides: SELECT, INSERT, UPDATE, DELETE, SUBSCRIBE TO, CONSUME, ACK.

This is what lets chat UIs and agent apps read/write state directly and receive live updates without adding separate polling or fanout infrastructure.

Why KalamDB

  • Frontend-direct SQL and realtime subscriptions, so web and mobile apps can read, write, and stay in sync without a separate sync layer.
  • Multi-tenant isolation by default, with the same SQL returning user-scoped data safely across frontend and backend clients.
  • Built-in authentication and authorization with Basic auth, JWT, OAuth 2.0 (Google Workspace, GitHub, Azure AD), and RBAC.
  • Unified application primitives: SQL tables, live queries, pub/sub topics, consumer groups, and file attachments in one runtime.
  • Hybrid storage engine tuned for product workloads: RocksDB for fast active writes, Parquet for compressed historical storage and analytics.
  • Portable object storage support across filesystem, S3, GCS, and Azure backends.
  • Vector embeddings and vector search workflows for semantic retrieval, agent memory, and AI-powered product features.
  • Distributed clustering with Multi-Raft replication and failover for resilient production deployments.
  • First-party tooling for operators and app teams: Admin UI, kalam CLI, and official TypeScript and Dart SDKs.

60-Second Quick Start (Docker)

Single node

KALAMDB_JWT_SECRET="$(openssl rand -base64 32)" \
curl -sSL https://raw.githubusercontent.com/jamals86/KalamDB/main/docker/run/single/docker-compose.yml | docker-compose -f - up -d

3-node cluster

KALAMDB_JWT_SECRET="$(openssl rand -base64 32)" \
curl -sSL https://raw.githubusercontent.com/jamals86/KalamDB/main/docker/run/cluster/docker-compose.yml | docker-compose -f - up -d

Local run

git clone https://github.com/jamals86/KalamDB.git
cd KalamDB/backend
cargo run --bin kalamdb-server

Browser/Frontend Client Example

import { createClient, Auth } from 'kalam-link';

const client = createClient({
  url: 'http://localhost:8080',
  authProvider: async () => Auth.jwt('<user-token>'),
});

const threadSql = `
  SELECT id, role, content, created_at
  FROM chat.messages
  WHERE thread_id = 'thread_42'
`;

await client.query(`
  INSERT INTO chat.messages (thread_id, role, content)
  VALUES ('thread_42', 'user', 'hello from frontend');
`);

const unsubscribe = await client.live(
  threadSql,
  (rows) => {
    // If `chat.messages` is a USER table, each signed-in user only sees
    // their own rows even though the SQL text is identical for everyone.
    console.log('live rows', rows);
  },
  {
    subscriptionOptions: { last_rows: 20 },
  },
);

// Later
await unsubscribe();
await client.disconnect();

AI Agent Example (Topic Subscription)

Subscribe an AI agent to a KalamDB topic and process each row with an LLM — fully managed retries, backpressure, and at-least-once delivery via runAgent.

import { createClient, Auth, runAgent } from 'kalam-link';

const client = createClient({
  url: 'http://localhost:8080',
  authProvider: async () => Auth.basic('root', 'kalamdb123'),
});

const abort = new AbortController();
process.on('SIGINT', () => abort.abort());

await runAgent<{ title: string; body: string }>({
  client,
  name: 'summarizer-agent',
  topic: 'blog.posts',       // KalamDB topic to consume
  groupId: 'summarizer-v1',  // consumer group — tracks per-agent offset
  start: 'earliest',
  batchSize: 20,
  timeoutSeconds: 30,
  stopSignal: abort.signal,

  // Called for every row; return value is written back as metadata
  onRow: async ({ row }) => {
    const summary = await myLlm.summarize(row.body);
    console.log(`[${row.title}] →`, summary);
  },

  onFailed: async ({ row, error }) => {
    console.error('failed row', row, error);
  },

  retry: {
    maxAttempts: 3,
    initialBackoffMs: 250,
    maxBackoffMs: 1500,
    multiplier: 2,
  },
  ackOnFailed: true,    // commit offset even on permanent failure
});

await client.disconnect();

See examples/summarizer-agent/ for a full working example with Gemini integration.

SQL Example

CREATE NAMESPACE IF NOT EXISTS chat;

CREATE TABLE chat.messages (
  id BIGINT PRIMARY KEY DEFAULT SNOWFLAKE_ID(),
  thread_id TEXT NOT NULL,
  role TEXT NOT NULL,
  content TEXT NOT NULL,
  attachment FILE,
  created_at TIMESTAMP DEFAULT NOW()
) WITH (TYPE = 'USER');

Architecture Snapshot

  • SQL and execution: kalamdb-sql + DataFusion/Arrow.
  • Hot path: kalamdb-store (RocksDB).
  • Cold path: kalamdb-filestore (Parquet/object storage).
  • Orchestration: kalamdb-core (DDL/DML, jobs, schema registry).
  • API surface: HTTP SQL API + WebSocket realtime + Admin UI + CLI.

Best-Fit Workloads

  • AI chat history and agent memory/state.
  • Tool-call logs and human-in-the-loop workflows.
  • Live dashboards, notifications, and collaborative feeds.
  • Multi-tenant SaaS that needs strong user-level isolation.

Docs and Links

  • Docs: https://kalamdb.org/docs
  • Quick start: docs/getting-started/quick-start.md
  • TypeScript SDK: link/sdks/typescript/
  • Docker deployment: docker/run/
  • PostgreSQL extension Docker image: jamals86/pg-kalam
  • Website: https://kalamdb.org

KalamDB is under active development and evolving quickly.

About

KalamDB — a lightweight, real-time, storage-efficient SQL database. Designed for per-user data isolation, GDPR compliance, and scalable performance — ideal for the AI era.

Topics

Resources

License

Stars

Watchers

Forks

Packages

 
 
 

Contributors