Multi-Tenant Model-as-a-Service (MaaS) Platform

Overview

This repository contains the complete architecture, design, and proof-of-concept implementation for a multi-tenant Model-as-a-Service (MaaS) platform built on Red Hat OpenShift AI 3.0+. The platform enables multiple isolated tenants to deploy and consume LLM inference services with comprehensive RBAC, monitoring, and billing capabilities.

Architecture Components

The platform consists of three main architectural pillars:

1. Multi-Tenant RBAC

Namespace-level isolation with delegated tenant administration, external identity provider integration (IBM Verify), and self-service user management within tenant boundaries.

Key Features:

• Tenant administrators with namespace-scoped permissions
• Integration with IBM Security Verify (OIDC/OAuth2)
• No cluster-admin privileges for tenant admins
• Group-based access control with custom tenant attributes

2. Model-as-a-Service Platform

Per-tenant Gateway architecture with JWT-based authentication, OPA authorization, and multi-tier rate limiting for LLM inference workloads.

Key Features:

• Per-tenant Gateway isolation (strongest tenant separation)
• JWT authentication with group-based authorization (OPA/Rego)
• Multi-tier rate limiting (request-based and token consumption-based)
• vLLM runtime for high-performance inference
• Kuadrant for traffic management and policy enforcement

3. Monitoring & Billing

Dedicated tenant Prometheus instances with federation, automatic metric collection from KServe, and TimescaleDB-backed billing system.

Key Features:

• Per-tenant Prometheus with federation from cluster Prometheus
• Automatic PodMonitor creation by KServe
• Self-service Grafana dashboards for usage visualization
• Namespace-based metric isolation
• TimescaleDB for long-term billing data storage
• REST API for usage queries and cost calculations

---

Documentation Structure

Design Documents

1. RBAC Design Complete RBAC architecture with identity provider integration, tenant isolation, and role definitions.

2. Multi-Tenant MaaS Design Platform architecture covering Gateway API, authentication, authorization, rate limiting, and model serving.

3. Monitoring & Billing Design Monitoring architecture with Prometheus federation, metrics collection, and billing data model.

Implementation Guides

4. RBAC Implementation POC Step-by-step guide for implementing tenant RBAC with IBM Verify integration.

5. MaaS Platform POC Deployment guide for per-tenant Gateway with authentication, authorization, and rate limiting.

6. Monitoring Implementation POC Implementation guide for tenant Prometheus, Grafana, and TimescaleDB setup.

7. Sample Billing Application Complete billing application with aggregator CronJob, REST API, and cost calculation engine.

---

Current Status

Proof-of-Concept Validated:

• RBAC with IBM Verify integration (2 tenants: tenant-a, tenant-b)
• Per-tenant Gateway with AuthPolicy and RateLimitPolicy
• Model deployment with KServe and vLLM runtime
• Tenant Prometheus federation and Grafana dashboards
• TimescaleDB with billing schema and sample application
• Automatic metric collection from vLLM endpoints

Deployment Scale:

• 2 tenants (tenant-a, tenant-b)
• 1 model per tenant (granite-3-1-8b-instruct-fp8)
• Per-tenant monitoring and gateway infrastructure

---

Next Steps

1. Migrate to Red Hat SSO

Objective: Replace IBM Verify with Red Hat SSO (Keycloak) for identity management.

Rationale:

• Native OpenShift integration
• On-premise deployment option
• Better alignment with Red Hat ecosystem
• Enhanced customization and control

Implementation: See Red Hat SSO Implementation for complete deployment guide with RH SSO/Keycloak integration.

2. Expand from 1..2 to 1..n Tenants

Implementation: See Tenant automation

3. Define Custom Resource(s) for Tenant Management

Objective: Simplify tenant lifecycle with declarative Kubernetes CRDs.

4. ⚠️ Does RedHat Openshift AI dashboard suport multi-tenancy - Gotcha

---

Key Design Principles

1. Strict Isolation: Namespace-based separation with RBAC enforcement
2. Self-Service: Tenant admins manage their resources independently
3. Automation-First: Federation, PodMonitor creation, metric collection are automatic
4. Observability: Built-in monitoring and billing for usage transparency
5. Scalability: Federation and per-tenant resources enable horizontal scaling
6. Security: JWT authentication, OPA authorization, rate limiting, and network policies

---

Getting Started

For Cluster Administrators

1. Deploy RBAC Foundation

   • Follow tenant_admin_rbac_implementation.md
   • Configure IBM Verify (or RH SSO) integration
   • Create tenant namespaces and RBAC bindings

2. Deploy MaaS Platform

   • Follow maas-platform/README.md
   • Deploy shared MaaS API
   • Configure per-tenant Gateways

3. Enable Monitoring & Billing

   • Follow monitoring/README.md
   • Enable User Workload Monitoring
   • Deploy TimescaleDB and billing infrastructure

For Tenant Administrators

1. Deploy Models

   • Create LLMInferenceService in your models namespace
   • Configure HTTPRoute to your tenant Gateway
   • Verify AuthPolicy and RateLimitPolicy enforcement

2. Set Up Monitoring

   • Deploy tenant Prometheus (federation-based)
   • Deploy Grafana with usage dashboards
   • Access metrics via Grafana UI

3. Access Billing Data

   • Query billing API for usage and costs
   • Download CSV reports for billing periods
   • Monitor token consumption and request rates

---

Support and Contribution

This is a reference architecture and POC implementation. For production deployments:

• Review security policies and adjust for your environment
• Validate resource quotas and limits
• Test disaster recovery procedures
• Implement comprehensive monitoring and alerting
• Plan for multi-cluster or hybrid scenarios