v1.0.18
zoobzio January 6, 2025 Edit this page

Overview

Storage in Go often means choosing between vendor lock-in or writing abstraction layers from scratch.

Grub offers a third path: type-safe storage operations across key-value stores, blob storage, SQL databases, and vector similarity search with a consistent API.

// Same code, different backends
users := grub.NewStore[User](redis.New(client))
users := grub.NewStore[User](badger.New(db))
users := grub.NewStore[User](bolt.New(db, "users"))

// Type-safe operations
user, _ := users.Get(ctx, "user:123")
users.Set(ctx, "user:456", &User{Name: "Alice"}, time.Hour)

Swap providers without touching business logic.

Architecture

┌──────────────────────────────────────────────────────────────────────────┐
│                           Your Application                                │
├──────────────────────────────────────────────────────────────────────────┤
│  Store[T]          Bucket[T]         Database[T]         Index[T]        │
│  (Key-Value)       (Blob Storage)    (SQL)               (Vector)        │
├──────────────────────────────────────────────────────────────────────────┤
│                         Codec (JSON/Gob)                                  │
├──────────────────────────────────────────────────────────────────────────┤
│  StoreProvider     BucketProvider    edamame.Factory     VectorProvider  │
├────────┬────────┬────────┬────────┬────────┬────────┬────────┬───────────┤
│ Redis  │ Badger │ Bolt   │ S3     │ MinIO  │ GCS    │ Azure  │  Qdrant   │
└────────┴────────┴────────┴────────┴────────┴────────┴────────┴───────────┘

Each wrapper maintains type safety through generics while providers handle raw bytes. The codec layer handles serialization, defaulting to JSON.

Philosophy

Grub draws inspiration from Go's database/sql package: define interfaces, let implementations vary.

One interface per storage mode:

  • StoreProvider for key-value operations
  • BucketProvider for blob storage
  • SQL databases use edamame factories directly
  • VectorProvider for similarity search

Errors are semantic:

if errors.Is(err, grub.ErrNotFound) {
    // Handle missing record consistently across all providers
}

Four Storage Modes

Key-Value Store

For sessions, cache, configuration, and any keyed data with optional TTL.

store := grub.NewStore[Session](redis.New(client))

store.Set(ctx, "session:abc", &session, 24*time.Hour)
session, err := store.Get(ctx, "session:abc")
keys, _ := store.List(ctx, "session:", 100)

Providers: Redis, BadgerDB, BoltDB

Blob Storage

For files, media, documents, and any object with metadata.

bucket := grub.NewBucket[Document](s3.New(client, "my-bucket"))

bucket.Put(ctx, &grub.Object[Document]{
    Key:         "docs/report.json",
    ContentType: "application/json",
    Data:        Document{Title: "Q4 Report"},
})
obj, _ := bucket.Get(ctx, "docs/report.json")

Providers: AWS S3, MinIO, Google Cloud Storage, Azure Blob Storage

SQL Database

For structured records with query capabilities.

db := grub.NewDatabase[User](sqlxDB, "users", sqlite.New())

db.Set(ctx, "1", &User{ID: 1, Email: "alice@example.com"})
user, _ := db.Get(ctx, "1")
users, _ := db.Query().Where("status", "=", "active").Exec(ctx, map[string]any{"active": "enabled"})

Drivers: PostgreSQL, MariaDB, SQLite, SQL Server

For similarity search with typed metadata and filtering.

index := grub.NewIndex[Embedding](qdrant.New(client, qdrant.Config{
    Collection: "documents",
}))

index.Upsert(ctx, "doc:1", embedding, &Embedding{Category: "tech"})
results, _ := index.Search(ctx, queryVector, 10, nil)

Providers: Milvus, Pinecone, Qdrant, Weaviate

Priorities

Type Safety

Generics eliminate runtime type assertions. The compiler catches type mismatches before your code runs.

// Compiler enforces User type throughout
store := grub.NewStore[User](provider)
user, _ := store.Get(ctx, "key")  // *User, not interface{}

Consistency

Same error types across all providers. ErrNotFound means the same thing whether you're using Redis or S3.

// Works identically for any provider
if errors.Is(err, grub.ErrNotFound) {
    return createDefault()
}

Atomization

For framework internals that need field-level access (encryption, pipelines), atomic views expose the underlying structure without breaking type safety.

atomicStore := store.Atomic()
atom, _ := atomicStore.Get(ctx, "key")
// Access individual fields via atom.Spec()

When to Use Grub

Good fit:

  • Applications that may switch storage backends
  • Multi-tenant systems with different storage per tenant
  • Libraries that need storage abstraction
  • Prototyping with embedded DBs, deploying with cloud services
  • RAG applications needing portable vector storage

Consider alternatives:

  • Single-provider applications with provider-specific features
  • High-performance scenarios requiring provider optimizations
  • Applications using provider-specific query languages extensively
  • Vector search requiring provider-specific operators (see operator support matrix)
© 2026 Alexander Thorwaldson