[FEATURE]: Tag support with editing and validation across APIs and UI

[crivetimihai]

🧭 Epic

Title: Comprehensive Tag Support System
Goal: Enable flexible tagging capabilities across all MCP Gateway entities (tools, resources, prompts, servers, gateways) to support dynamic categorization, filtering, and organization.
Why now: The Dynamic Virtual Server API depends on tag-based filtering. Currently, no entities support tags, limiting organizational flexibility and preventing implementation of advanced features like dynamic server composition and intelligent grouping.

Enables:

• [CHORE][PLUGIN]: Architecture decisions for AI middleware framework #313
• [FEATURE][PLUGIN]: AI middleware integration / Plugin framework #319
• [FEATURE][AI]: Dynamic server catalog via rule, regexp, tags, or LLM-based selection #123

---

🧭 Type of Feature

• Enhancement to existing functionality
• New feature or capability
• Security Related (requires review)

---

🙋‍♂️ User Story 1

As a: gateway admin
I want: to assign multiple tags to any tool, resource, or prompt
So that: I can organize entities by category, team, or purpose

✅ Acceptance Criteria

Scenario: Add tags to a tool
  Given I have a tool with id "analytics-tool-1"
  When I PATCH /tools/analytics-tool-1 with {"tags": ["analytics", "finance", "internal"]}
  Then the tool has tags = ["analytics", "finance", "internal"]
  And GET /tools/analytics-tool-1 returns the tags in the response

Scenario: Update existing tags
  Given a tool with tags = ["old-tag"]
  When I PATCH the tool with {"tags": ["new-tag", "another-tag"]}
  Then the old tags are replaced with the new ones

---

🙋‍♂️ User Story 2

As a: API consumer
I want: to filter entities by one or more tags
So that: I can retrieve only relevant items

✅ Acceptance Criteria

Scenario: Filter tools by single tag
  Given tools exist with various tags
  When I GET /tools?tag=analytics
  Then I only receive tools that have "analytics" in their tags array

Scenario: Filter by multiple tags (AND logic)
  Given tools with different tag combinations
  When I GET /tools?tag=analytics&tag=internal
  Then I only receive tools that have BOTH "analytics" AND "internal" tags

Scenario: Empty result for non-existent tag
  When I GET /tools?tag=nonexistent
  Then I receive an empty array

---

🙋‍♂️ User Story 3

As a: server admin
I want: to tag servers and gateways for organizational purposes
So that: I can group related infrastructure components

✅ Acceptance Criteria

Scenario: Tag a server
  Given a server with id "prod-server-1"
  When I PATCH /servers/prod-server-1 with {"tags": ["production", "region-us-east"]}
  Then the server is tagged appropriately

Scenario: Filter servers by tag
  When I GET /servers?tag=production
  Then I only see production servers

---

🙋‍♂️ User Story 4

As a: UI developer
I want: to see all unique tags across a resource type
So that: I can build tag clouds or filter interfaces

✅ Acceptance Criteria

Scenario: Get all tool tags
  Given tools with tags: ["analytics", "finance"], ["analytics", "risk"], ["operations"]
  When I GET /tools/tags
  Then I receive ["analytics", "finance", "risk", "operations"] (unique, sorted)

Scenario: Tag frequency count
  When I GET /tools/tags?include_count=true
  Then I receive [{"tag": "analytics", "count": 2}, {"tag": "finance", "count": 1}, ...]

---

🙋‍♂️ User Story 5

As a: operations team member
I want: to bulk update tags on multiple entities
So that: I can reorganize without individual updates

✅ Acceptance Criteria

Scenario: Bulk add tag to tools
  Given a list of tool IDs
  When I POST /tools/bulk-tag with {"ids": ["tool1", "tool2"], "operation": "add", "tags": ["deprecated"]}
  Then both tools have "deprecated" added to their existing tags

Scenario: Bulk remove tag
  When I POST /tools/bulk-tag with {"ids": ["tool1", "tool2"], "operation": "remove", "tags": ["old-tag"]}
  Then "old-tag" is removed from both tools' tags

---

🙋‍♂️ User Story 6

As a: admin
I want: tag validation and normalization
So that: tags remain consistent and searchable

✅ Acceptance Criteria

Scenario: Tag normalization
  When I add tags ["Finance", "FINANCE", " finance "]
  Then they are stored as ["finance"] (lowercase, trimmed, deduplicated)

Scenario: Invalid tag rejection
  When I try to add tags ["", "a", "this-tag-is-way-too-long-and-exceeds-reasonable-limits"]
  Then I receive a validation error

Scenario: Special character handling
  When I add tags ["high-priority", "team:backend", "v2.0"]
  Then they are accepted (alphanumeric, dash, colon, dot allowed)

---

🙋‍♂️ User Story 7

As a: dynamic server user
I want: to reference tags in dynamic server rules
So that: my virtual servers automatically include tagged entities

✅ Acceptance Criteria

Scenario: Dynamic rule using tags
  Given tools tagged with "analytics"
  And a dynamic server rule: type = "tool", filter = "$.tags[?(@ == 'analytics')]"
  When I GET /dynamic/my-server/tools
  Then I see all tools tagged "analytics"

Scenario: Complex tag queries
  Given a rule: filter = "$.tags[?(@ == 'finance' || @ == 'risk')]"
  Then tools with either "finance" OR "risk" tags are included

---

📐 Design Sketch

🗄️ Database Schema Changes

-- Add tags column to each entity table
ALTER TABLE tools ADD COLUMN tags JSON DEFAULT '[]';
ALTER TABLE resources ADD COLUMN tags JSON DEFAULT '[]';
ALTER TABLE prompts ADD COLUMN tags JSON DEFAULT '[]';
ALTER TABLE servers ADD COLUMN tags JSON DEFAULT '[]';
ALTER TABLE gateways ADD COLUMN tags JSON DEFAULT '[]';

-- Create indexes for tag queries (PostgreSQL GIN index example)
CREATE INDEX idx_tools_tags ON tools USING GIN (tags);
CREATE INDEX idx_resources_tags ON resources USING GIN (tags);
-- ... repeat for other tables

🧩 API Modifications

Entity Type	Tag Endpoints	Filter Support
Tools	PATCH /tools/{id} with tags GET /tools/tags POST /tools/bulk-tag	GET /tools?tag=...
Resources	PATCH /resources/{id} with tags GET /resources/tags POST /resources/bulk-tag	GET /resources?tag=...
Prompts	PATCH /prompts/{id} with tags GET /prompts/tags POST /prompts/bulk-tag	GET /prompts?tag=...
Servers	PATCH /servers/{id} with tags GET /servers/tags	GET /servers?tag=...
Gateways	PATCH /gateways/{id} with tags GET /gateways/tags	GET /gateways?tag=...

🔧 Tag Validation Rules

class TagValidator:
    MIN_LENGTH = 2
    MAX_LENGTH = 50
    ALLOWED_PATTERN = r'^[a-z0-9][a-z0-9\-\:\.]*[a-z0-9]$'
    
    @staticmethod
    def normalize(tag: str) -> str:
        return tag.strip().lower()
    
    @staticmethod
    def validate(tag: str) -> bool:
        normalized = TagValidator.normalize(tag)
        return (
            len(normalized) >= TagValidator.MIN_LENGTH and
            len(normalized) <= TagValidator.MAX_LENGTH and
            re.match(TagValidator.ALLOWED_PATTERN, normalized)
        )

---

🎨 UI Implementation Details

📊 Tag Display in Lists (admin.html)

For each entity table (tools, resources, prompts, servers, gateways), add a new "Tags" column:

<!-- In table header -->
<th class="px-6 py-3 text-left text-xs font-medium text-gray-500 dark:text-gray-300 uppercase tracking-wider">
  Tags
</th>

<!-- In table body -->
<td class="px-6 py-4 whitespace-normal">
  <div class="flex flex-wrap gap-1">
    {% for tag in entity.tags %}
    <span class="inline-flex items-center px-2 py-0.5 rounded text-xs font-medium bg-gray-100 text-gray-800 dark:bg-gray-700 dark:text-gray-300">
      {{ tag }}
    </span>
    {% endfor %}
    {% if not entity.tags %}
    <span class="text-gray-500 dark:text-gray-400 text-xs">No tags</span>
    {% endif %}
  </div>
</td>

🏷️ Tag Input Component (admin.js)

Create a reusable tag input component with these features:

• Comma-separated input: Type tags separated by commas
• Tag pills display: Show selected tags as removable pills
• Autocomplete: Suggest existing tags as user types
• Validation: Real-time validation with error messages
• Keyboard navigation: Tab to complete, backspace to remove

class TagInput {
  constructor(elementId, options = {}) {
    this.element = document.getElementById(elementId);
    this.tags = options.initialTags || [];
    this.maxTags = options.maxTags || 20;
    this.existingTags = options.existingTags || [];
    this.onTagsChange = options.onTagsChange || (() => {});
    
    this.init();
  }
  
  init() {
    // Create wrapper with input and pills container
    this.wrapper = document.createElement('div');
    this.wrapper.className = 'tag-input-wrapper border rounded-md p-2';
    
    this.pillsContainer = document.createElement('div');
    this.pillsContainer.className = 'flex flex-wrap gap-1 mb-2';
    
    this.input = document.createElement('input');
    this.input.type = 'text';
    this.input.placeholder = 'Add tags (comma-separated)...';
    this.input.className = 'w-full outline-none';
    
    // Set up event listeners
    this.input.addEventListener('keydown', (e) => this.handleKeydown(e));
    this.input.addEventListener('input', (e) => this.handleInput(e));
    this.input.addEventListener('blur', () => this.addPendingTags());
    
    this.wrapper.appendChild(this.pillsContainer);
    this.wrapper.appendChild(this.input);
    this.element.appendChild(this.wrapper);
    
    this.renderTags();
  }
  
  addTag(tag) {
    const normalized = this.normalizeTag(tag);
    if (normalized && !this.tags.includes(normalized) && this.tags.length < this.maxTags) {
      this.tags.push(normalized);
      this.renderTags();
      this.onTagsChange(this.tags);
      return true;
    }
    return false;
  }
  
  removeTag(tag) {
    this.tags = this.tags.filter(t => t !== tag);
    this.renderTags();
    this.onTagsChange(this.tags);
  }
  
  renderTags() {
    this.pillsContainer.innerHTML = '';
    this.tags.forEach(tag => {
      const pill = document.createElement('span');
      pill.className = 'inline-flex items-center px-2 py-0.5 rounded text-xs font-medium bg-blue-100 text-blue-800';
      pill.innerHTML = `
        ${escapeHtml(tag)}
        <button type="button" class="ml-1 text-blue-600 hover:text-blue-800" data-tag="${escapeHtml(tag)}">
          <svg class="h-3 w-3" fill="currentColor" viewBox="0 0 20 20">
            <path fill-rule="evenodd" d="M4.293 4.293a1 1 0 011.414 0L10 8.586l4.293-4.293a1 1 0 111.414 1.414L11.414 10l4.293 4.293a1 1 0 01-1.414 1.414L10 11.414l-4.293 4.293a1 1 0 01-1.414-1.414L8.586 10 4.293 5.707a1 1 0 010-1.414z" clip-rule="evenodd"/>
          </svg>
        </button>
      `;
      
      pill.querySelector('button').addEventListener('click', () => this.removeTag(tag));
      this.pillsContainer.appendChild(pill);
    });
  }
}

✏️ Edit Modal Integration

For each edit modal, add a tags section:

<!-- In edit modal form -->
<div>
  <label class="block text-sm font-medium text-gray-700 dark:text-gray-300">Tags</label>
  <div id="edit-{entity}-tags-container" class="mt-1"></div>
  <p class="mt-1 text-xs text-gray-500 dark:text-gray-400">
    Add tags to categorize this {entity}. Use lowercase letters, numbers, hyphens, and underscores.
  </p>
</div>

🔍 Tag Filtering

Add a multi-select dropdown for tag filtering on each tab:

<!-- Above each entity table -->
<div class="flex items-center space-x-4 mb-4">
  <div class="relative">
    <button id="{entity}-tag-filter-btn" class="inline-flex items-center px-4 py-2 border border-gray-300 rounded-md shadow-sm text-sm font-medium text-gray-700 bg-white hover:bg-gray-50">
      <svg class="mr-2 h-4 w-4" fill="none" stroke="currentColor" viewBox="0 0 24 24">
        <path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M7 7h.01M7 3h5c.512 0 1.024.195 1.414.586l7 7a2 2 0 010 2.828l-7 7a2 2 0 01-2.828 0l-7-7A1.994 1.994 0 013 12V7a4 4 0 014-4z"/>
      </svg>
      Filter by Tags
      <span id="{entity}-tag-count" class="ml-2 px-2 py-0.5 text-xs bg-gray-200 rounded-full hidden">0</span>
    </button>
    
    <div id="{entity}-tag-filter-dropdown" class="absolute z-10 mt-2 w-64 rounded-md shadow-lg bg-white ring-1 ring-black ring-opacity-5 hidden">
      <div class="p-2">
        <input type="text" placeholder="Search tags..." class="w-full px-3 py-2 border border-gray-300 rounded-md text-sm">
      </div>
      <div class="max-h-60 overflow-y-auto">
        <div id="{entity}-tag-list" class="py-1">
          <!-- Populated dynamically -->
        </div>
      </div>
      <div class="border-t px-3 py-2">
        <button class="text-sm text-blue-600 hover:text-blue-800">Clear all</button>
      </div>
    </div>
  </div>
  
  <!-- Existing search and show inactive checkbox -->
</div>

📊 View Modal Enhancement

Update view modals to display tags prominently:

// In viewTool, viewResource, etc.
const renderTags = (tags) => {
  if (!tags || tags.length === 0) {
    return '<p><strong>Tags:</strong> <span class="text-gray-500">None</span></p>';
  }
  
  const tagBadges = tags.map(tag => 
    `<span class="inline-flex items-center px-2 py-0.5 rounded text-xs font-medium bg-gray-100 text-gray-800 dark:bg-gray-700 dark:text-gray-300 mr-1">
      ${escapeHtml(tag)}
    </span>`
  ).join('');
  
  return `
    <div>
      <strong>Tags:</strong>
      <div class="mt-1 flex flex-wrap gap-1">
        ${tagBadges}
      </div>
    </div>
  `;
};

🎯 Tag Management Tab

Add a new tab for centralized tag management:

<!-- In tab navigation -->
<a href="#tags" id="tab-tags" class="tab-link ...">
  Tag Management
</a>

<!-- Tag management panel -->
<div id="tags-panel" class="tab-panel hidden">
  <div class="bg-white shadow rounded-lg p-6 dark:bg-gray-800">
    <h2 class="text-2xl font-bold dark:text-gray-200 mb-4">Tag Overview</h2>
    
    <!-- Tag statistics grid -->
    <div class="grid grid-cols-1 md:grid-cols-2 lg:grid-cols-3 gap-4 mb-8">
      <div class="bg-gray-50 dark:bg-gray-700 p-4 rounded">
        <h3 class="font-medium text-gray-900 dark:text-gray-100">Total Tags</h3>
        <p class="text-2xl font-bold text-indigo-600" id="total-tags-count">0</p>
      </div>
      <div class="bg-gray-50 dark:bg-gray-700 p-4 rounded">
        <h3 class="font-medium text-gray-900 dark:text-gray-100">Most Used Tag</h3>
        <p class="text-lg font-medium text-gray-700 dark:text-gray-300" id="most-used-tag">-</p>
      </div>
      <div class="bg-gray-50 dark:bg-gray-700 p-4 rounded">
        <h3 class="font-medium text-gray-900 dark:text-gray-100">Tagged Entities</h3>
        <p class="text-2xl font-bold text-green-600" id="tagged-entities-count">0</p>
      </div>
    </div>
    
    <!-- Tag cloud -->
    <div class="mb-8">
      <h3 class="text-lg font-medium mb-4">Tag Cloud</h3>
      <div id="tag-cloud" class="flex flex-wrap gap-2">
        <!-- Populated dynamically with varying sizes based on usage -->
      </div>
    </div>
    
    <!-- Tag usage table -->
    <div>
      <h3 class="text-lg font-medium mb-4">Tag Usage Details</h3>
      <table class="min-w-full divide-y divide-gray-200">
        <thead class="bg-gray-50 dark:bg-gray-700">
          <tr>
            <th class="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase">Tag</th>
            <th class="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase">Tools</th>
            <th class="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase">Resources</th>
            <th class="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase">Prompts</th>
            <th class="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase">Servers</th>
            <th class="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase">Gateways</th>
            <th class="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase">Total</th>
            <th class="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase">Actions</th>
          </tr>
        </thead>
        <tbody id="tag-usage-tbody" class="bg-white divide-y divide-gray-200 dark:bg-gray-900">
          <!-- Populated dynamically -->
        </tbody>
      </table>
    </div>
  </div>
</div>

---

🔄 Alternatives Considered

• Separate tags table: More complex joins, but better normalization → decided against for performance
• Tag hierarchies: Parent/child tags → too complex for initial implementation
• Predefined tag taxonomy: Restrictive for users → free-form tags more flexible
• Tags as separate service: Over-engineering → integrated approach simpler

---

📓 Additional Context

• Tags should be case-insensitive for queries but preserve original case for display
• Consider tag autocomplete API for UI (/tags/autocomplete?prefix=...)
• Migration must handle existing data gracefully (default empty array)
• Performance consideration: Tag queries should use database indexes
• Future enhancement: Tag aliases (e.g., "ml" → "machine-learning")

---

🧭 Tasks

Area	Task	Notes	Priority
Database	[ ] Create migration to add tags columns	Use JSON column type	P1
	[ ] Add database indexes for tag queries	Platform-specific (GIN for PostgreSQL)	P1
	[ ] Add tags field to all ORM models	Default to empty list	P1
Schemas	[ ] Update Pydantic schemas (Create/Read/Update)	Add Optional[List[str]] tags field	P1
	[ ] Create TagValidator class	Normalization and validation logic	P1
	[ ] Add tag field validators with security checks	Prevent XSS, SQL injection	P1
	[ ] Add BulkTagOperation schema	For bulk updates	P2
API	[ ] Add tags to PATCH endpoints	All entity types	P1
	[ ] Add tag filter to GET list endpoints	Query parameter support	P1
	[ ] Create /tags endpoints for each entity	List unique tags	P2
	[ ] Implement bulk tag operations	Add/remove/replace	P2
Services	[ ] Update service layer for tag operations	CRUD + filtering	P1
	[ ] Implement tag normalization	On write operations	P1
	[ ] Add tag aggregation queries	For tag clouds	P2
Admin UI - Display	[ ] Add tag badges to entity list tables	Styled tag pills in admin.html	P1
	[ ] Show tags in view modals	Display tags with proper styling	P1
	[ ] Add tag count indicators	Show number of tags per entity	P2
	[ ] Implement tag tooltips	Show full tag list on hover for truncated displays	P3
Admin UI - Forms	[ ] Add tag input to Add Tool form	Multi-tag input with validation	P1
	[ ] Add tag input to Add Resource form	Multi-tag input with validation	P1
	[ ] Add tag input to Add Prompt form	Multi-tag input with validation	P1
	[ ] Add tag input to Add Server form	Multi-tag input with validation	P1
	[ ] Add tag input to Add Gateway form	Multi-tag input with validation	P1
Admin UI - Edit Modals	[ ] Add tag editor to Edit Tool modal	Editable tag list with add/remove	P1
	[ ] Add tag editor to Edit Resource modal	Editable tag list with add/remove	P1
	[ ] Add tag editor to Edit Prompt modal	Editable tag list with add/remove	P1
	[ ] Add tag editor to Edit Server modal	Editable tag list with add/remove	P1
	[ ] Add tag editor to Edit Gateway modal	Editable tag list with add/remove	P1
Admin UI - Filtering	[ ] Add tag filter dropdown to each tab	Multi-select tag filter	P2
	[ ] Implement tag search/autocomplete	Quick tag selection	P2
	[ ] Add "clear filters" button	Reset all tag filters	P2
	[ ] Persist filter state in URL	Shareable filtered views	P3
Admin UI - Tag Management	[ ] Create dedicated Tags tab	Central tag management interface	P2
	[ ] Show tag usage statistics	Count of entities per tag	P2
	[ ] Bulk tag operations UI	Select multiple entities for tagging	P3
	[ ] Tag rename/merge functionality	Admin tag maintenance	P3
Frontend - admin.js	[ ] Create tag input component	Reusable tag editor widget	P1
	[ ] Add tag validation functions	Client-side validation	P1
	[ ] Implement tag autocomplete	Suggest existing tags	P2
	[ ] Add tag filter state management	Handle multi-tag filtering	P2
	[ ] Create tag badge rendering	Safe HTML generation	P1
Frontend - Styling	[ ] Design tag pill styles	Colors, hover states, remove buttons	P1
	[ ] Create tag input styles	Match existing form design	P1
	[ ] Add tag filter dropdown styles	Consistent with other filters	P2
	[ ] Responsive tag display	Handle overflow gracefully	P1
Tests	[ ] Unit tests for tag validation	Edge cases, special characters	P1
	[ ] Integration tests for tag filtering	Single and multiple tags	P1
	[ ] Security tests for tag input	XSS, injection attempts	P1
	[ ] Test bulk operations	Success and error cases	P2
	[ ] Performance tests for tag queries	Large datasets	P2
	[ ] UI tests for tag interactions	Add, remove, filter operations	P2
Documentation	[ ] Update API documentation	New endpoints and parameters	P1
	[ ] Add tagging guide	Best practices, examples	P2
	[ ] Document migration process	For existing deployments	P1
	[ ] Create UI usage guide	How to use tags in the admin interface	P2

---

🔗 Dependencies

• Database migration framework functional
• JSON column support in target database
• Full-text search capabilities (for future tag search)

---

🎯 Success Metrics

• All entity types support tags with consistent API
• Tag queries perform within 100ms for typical datasets
• Zero data loss during migration
• Dynamic server rules can filter by tags successfully
• Admin UI provides intuitive tag management

---

---

✅ Final Quality Checklist

Code Quality & Standards:

• Include updates to schemas.py with data validation, to ensure tags follow a strict format and approved list of characters, size, etc. Included with full test coverage including security testing.
• make flake8 ruff pylint all pass
• mypy passes for new code
• make lint-web passes, no security issues
• make interrogate maintained at 100% docstring coverage, using google docstring format, and arguments typing, with doctest
• make doctest passes and maintained at > 60%
• make test passes and maintained at > 82%, test cases included with > 80% coverage for new feature code
• make smoketest passes
• make docker-run-ssl-host passes
• make docker-stop compose-stop compose-rm compose-up passes
• cd docs; make serve shows no errors and new docs render correctly

Feature Completeness:

• All entity types (tools, resources, prompts, servers, gateways) support tags
• Tag validation prevents invalid/malicious input
• Tags are properly indexed for performance
• UI displays tags consistently across all views
• Tag filtering works for all entity types
• Bulk tag operations are functional
• Tag management interface is complete
• Documentation is comprehensive

Security Review:

• Tag input sanitization prevents XSS
• SQL injection prevention for tag queries
• Tag size limits enforced
• Special characters properly escaped
• API rate limiting for tag operations

Performance Validation:

• Tag queries optimized with indexes
• Bulk operations use efficient SQL
• UI renders large tag lists smoothly
• Autocomplete performs well with many tags
• No N+1 query problems

User Experience:

• Tag input is intuitive
• Validation messages are helpful
• Tag filtering is discoverable
• Bulk operations have confirmation
• Tag management provides clear overview