Skip to content

README.md

Latest commit

 

History

History
79 lines (63 loc) · 3.05 KB

README.md

File metadata and controls

79 lines (63 loc) · 3.05 KB

Distributed Command Executor

A Kubernetes-native system for executing commands at scale across distributed worker nodes.

Features

  • Parallel Execution - Run commands across multiple workers simultaneously
  • Kubernetes Native - First-class CRD support with kubectl integration
  • Auto-Scaling - Dynamically scales workers based on queue depth
  • Fast - gRPC-based communication for minimal latency
  • Secure - Command allowlists/blocklists, RBAC integration

Quick Start

# Submit a job via CLI
./bin/distctl run echo "Hello, World!"

# Or via Kubernetes CRD
kubectl apply -f - <<EOF
apiVersion: distexec.io/v1
kind: DistributedJob
metadata:
  name: hello
spec:
  command: "echo"
  args: ["Hello from K8s!"]
  parallelism: 5
EOF

# Check status
kubectl get distributedjobs
./bin/distctl list

Installation

# Build
make build

# Deploy to Kubernetes
kubectl apply -f deploy/k8s/

Architecture

┌─────────────┐         ┌─────────────────────────────────┐
│   distctl   │         │            MASTER               │
│    (CLI)    │──gRPC──▶│  ┌─────────┐  ┌──────────────┐  │
└─────────────┘         │  │Scheduler│  │ Load Balancer│  │
                        │  └────┬────┘  └───────┬──────┘  │
                        │       │               │         │
                        │  ┌────▼───────────────▼──────┐  │
                        │  │      Node Registry        │  │
                        │  └───────────────────────────┘  │
                        └───────────────┬─────────────────┘
                                        │ gRPC
                 ┌──────────────────────┼──────────────────────┐
                 ▼                      ▼                      ▼
          ┌──────────┐           ┌──────────┐           ┌──────────┐
          │ Worker 1 │           │ Worker 2 │           │ Worker N │
          │(executor)│           │(executor)│           │(executor)│
          └──────────┘           └──────────┘           └──────────┘

Documentation

Topic Link
Architecture specs/00-OVERVIEW.md
Components specs/01-COMPONENTS.md
API Reference specs/02-RPC-PROTOCOL.md
Scaling specs/03-DYNAMIC-SCALING.md

License

MIT