Architecture

This document covers grub's internal design for those who need deeper understanding.

Layer Model

┌─────────────────────────────────────────────────────────────────────────┐
│                          Application Layer                               │
│         Store[T], Bucket[T], Database[T], Index[T] wrappers             │
├─────────────────────────────────────────────────────────────────────────┤
│                           Codec Layer                                    │
│                  JSONCodec, GobCodec, custom                            │
├─────────────────────────────────────────────────────────────────────────┤
│                          Provider Layer                                  │
│      StoreProvider, BucketProvider, VectorProvider interfaces           │
├─────────────────────────────────────────────────────────────────────────┤
│                        Implementation Layer                              │
│  redis, badger, bolt, s3, minio, gcs, azure, qdrant, pinecone, etc.   │
└─────────────────────────────────────────────────────────────────────────┘

Data flow for Get:

1. Application calls store.Get(ctx, "key")
2. Wrapper calls provider.Get(ctx, "key") → []byte
3. Codec decodes bytes → typed value
4. If T implements AfterLoad, the hook is called
5. Wrapper returns *T

Data flow for Set:

1. Application calls store.Set(ctx, "key", &value, ttl)
2. If T implements BeforeSave, the hook is called (error aborts)
3. Codec encodes value → []byte
4. Wrapper calls provider.Set(ctx, "key", bytes, ttl)
5. If T implements AfterSave, the hook is called

Data flow for Delete:

1. Application calls store.Delete(ctx, "key")
2. If T implements BeforeDelete, the hook is called on a zero-value T (error aborts)
3. Wrapper calls provider.Delete(ctx, "key")
4. If T implements AfterDelete, the hook is called on a zero-value T

Hooks are opt-in. Types that don't implement any hook interface skip these steps with zero overhead. Hooks fire at the wrapper layer — atomic views and raw provider calls do not trigger hooks.

Wrapper Implementation

StoreT

type Store[T any] struct {
    provider StoreProvider
    codec    Codec
    atomic   *atomic.Store[T]  // Lazily initialized
    once     sync.Once
}

func (s *Store[T]) Get(ctx context.Context, key string) (*T, error) {
    data, err := s.provider.Get(ctx, key)
    if err != nil {
        return nil, err
    }
    var v T
    if err := s.codec.Decode(data, &v); err != nil {
        return nil, err
    }
    return &v, nil
}

Lazy Atomic Initialization

Atomic views are created on first access and cached:

func (s *Store[T]) Atomic() *atomic.Store[T] {
    s.once.Do(func() {
        s.atomic = atomic.NewStore[T](s.provider, s.codec)
    })
    return s.atomic
}

Panic behavior: If T is not atomizable, Atomic() panics on first call. This is intentional—check atomizability at application startup, not runtime.

Provider Contracts

Error Semantics

TTL Handling

Batch Operations

GetBatch: Missing keys are silently omitted from the result map. No error is returned for missing keys.

results, err := store.GetBatch(ctx, []string{"exists", "missing"})
// results = {"exists": value}  // "missing" not present
// err = nil

SetBatch: Atomicity varies by provider:

• Redis: Pipelined (each operation independent)
• Badger: WriteBatch (atomic)
• Bolt: Single transaction (atomic)

Context Handling

All operations accept context.Context for cancellation and timeouts.

Provider-Specific Behavior

Transaction Support (Database)

Database[T] provides *Tx method variants for transaction support:

// Create database wrapper
db := grub.NewDatabase[User](sqlxDB, "users", sqlite.New())

// Use *Tx methods within a transaction
tx, _ := sqlxDB.BeginTxx(ctx, nil)
defer tx.Rollback()

user, _ := db.GetTx(ctx, tx, "123")
user.Name = "Updated"
_ = db.SetTx(ctx, tx, "123", user)

tx.Commit()

All operations have *Tx variants: GetTx, SetTx, DeleteTx, ExistsTx, QueryTx, SelectTx, UpdateTx, AggregateTx.

Atomic Views

Atomic views provide type-agnostic access to field structure, used by framework internals for:

• Field-level encryption
• Data pipelines
• Schema introspection

Architecture

┌─────────────────────────────────────────────────────────┐
│                    Store[T]                             │
│                        │                                │
│                        ▼                                │
│              ┌─────────────────┐                        │
│              │  atomic.Store   │ ◄── Lazily created     │
│              └────────┬────────┘                        │
│                       │                                 │
│                       ▼                                 │
│              ┌─────────────────┐                        │
│              │   atom.Spec     │ ◄── Field metadata     │
│              └─────────────────┘                        │
└─────────────────────────────────────────────────────────┘

Atomic Interfaces

// AtomicStore - type-agnostic key-value access
type AtomicStore interface {
    Spec() atom.Spec
    Get(ctx context.Context, key string) (*atom.Atom, error)
    Set(ctx context.Context, key string, a *atom.Atom, ttl time.Duration) error
    Delete(ctx context.Context, key string) error
    Exists(ctx context.Context, key string) (bool, error)
}

// AtomicBucket - type-agnostic blob access
type AtomicBucket interface {
    Spec() atom.Spec
    Get(ctx context.Context, key string) (*AtomicObject, error)
    Put(ctx context.Context, key string, obj *AtomicObject) error
    Delete(ctx context.Context, key string) error
    Exists(ctx context.Context, key string) (bool, error)
}

// AtomicDatabase - type-agnostic SQL access
type AtomicDatabase interface {
    Table() string
    Spec() atom.Spec
    Get(ctx context.Context, key string) (*atom.Atom, error)
    Set(ctx context.Context, key string, a *atom.Atom) error
    Delete(ctx context.Context, key string) error
    Exists(ctx context.Context, key string) (bool, error)
    Query(ctx context.Context, name string, params map[string]any) ([]*atom.Atom, error)
    Select(ctx context.Context, name string, params map[string]any) (*atom.Atom, error)
}

// AtomicIndex - type-agnostic vector access
type AtomicIndex interface {
    Spec() atom.Spec
    Get(ctx context.Context, id string) (*AtomicVector, error)
    Upsert(ctx context.Context, id string, vector []float32, metadata *atom.Atom) error
    Delete(ctx context.Context, id string) error
    Exists(ctx context.Context, id string) (bool, error)
    Search(ctx context.Context, vector []float32, k int, filter *atom.Atom) ([]AtomicVector, error)
    Query(ctx context.Context, vector []float32, k int, filter *vecna.Filter) ([]AtomicVector, error)
}

Atomization Requirements

Types must be atomizable (via atom package):

type User struct {
    ID    string `json:"id" atom:"id"`
    Name  string `json:"name" atom:"name"`
    Email string `json:"email" atom:"email"`
}

If T lacks atom tags or has unsupported field types, Atomic() panics.

Concurrency Guarantees

Wrappers are safe for concurrent use. Provider thread-safety depends on the underlying client (Redis client, Badger DB, etc.).

Memory Management

Object Pooling

Grub does not pool objects. Each Get allocates a new value. For high-throughput scenarios, consider:

• Application-level pooling
• Reusing structs where safe
• Using atomic views for field-level operations

Codec Allocation

• JSONCodec: Allocates per encode/decode
• GobCodec: Encoder/decoder created per operation

For performance-critical paths, consider custom codecs with pooled buffers.

Dependencies

Core Package

github.com/zoobz-io/atom      # Field atomization
github.com/zoobz-io/edamame   # SQL query building
github.com/zoobz-io/soy       # SQL execution
github.com/zoobz-io/astql     # SQL dialect rendering
github.com/jmoiron/sqlx      # SQL toolkit

Provider Packages

Each provider is a separate module with its own dependencies:

• grub/redis → github.com/redis/go-redis/v9
• grub/badger → github.com/dgraph-io/badger/v4
• grub/bolt → go.etcd.io/bbolt
• grub/s3 → github.com/aws/aws-sdk-go-v2
• grub/minio → github.com/minio/minio-go/v7
• grub/gcs → cloud.google.com/go/storage
• grub/azure → github.com/Azure/azure-sdk-for-go
• grub/qdrant → github.com/qdrant/go-client
• grub/pinecone → github.com/pinecone-io/go-pinecone/v2
• grub/milvus → github.com/milvus-io/milvus-sdk-go/v2
• grub/weaviate → github.com/weaviate/weaviate-go-client/v5

This isolation ensures consumers only pull dependencies they use.