Skip to content

AI/LLM detection sensor cluster (llm_api, llm_orchestrator, ai_tools, etc.) is entirely non-functional -- no rule ever produces these signals #313

Description

@squid-protocol

File: gitgalaxy/metrics/signal_processor.py (consumer), gitgalaxy/standards/language_standards.py (missing producer)
Labels: bug, appsec, threat: false-negative, priority: high

Description

8 signal keys are declared in SIGNAL_SCHEMA (analysis_lens.py) and read throughout signal_processor.py's risk calculations: llm_api, llm_orchestrator, ai_tools, llm_vector_store, ai_memory, ai_logic_loop, ml_traditional, dl_frameworks. None of them has a corresponding regex rule in language_standards.py's per-language rule dictionaries, and none is written by detector.py's extraction logic for real source code (confirmed via grep -rn '"llm_orchestrator"\|"ai_tools"\|"llm_api"' gitgalaxy/ -- only schema declarations, UI labels, and reads, zero producers).

This isn't just a cosmetic scoring gap -- it disables active security detections:

  • _calc_logic_bomb: agent_dampener adds raw_signals.get("llm_orchestrator", 0) * 3.0 -- never contributes.
  • _calc_injection_surface: the "LLM prompt-injection-to-RCE" 10x risk spike --
    if raw_signals.get("sec_high_risk_execution", 0) > 0 and (raw_signals.get("llm_orchestrator", 0) > 0 or raw_signals.get("ai_tools", 0) > 0):
    can never fire, on any file, for any repo.
  • _calc_secrets_risk: the 3x "exposed LLM API key" amplifier (raw_signals.get("llm_api", 0) > 0) can never fire.
  • The ecosystem-wide "AI Topology" summary classifies every real repository as "Non-AI / Traditional", even ones that are obviously AI/LLM-heavy.

One partial exception: llm_local_compute does get set (hit_vector[idx] = 100) in galaxyscope.py, but only for synthetic ai_model_weights binary-tensor artifacts -- never for actual source code, so it doesn't rescue any of the above.

Fix

Add regex detection rules for these 8 signal categories to language_standards.py (LLM SDK imports/calls -- openai, anthropic, langchain, llama-index, vector-store clients, agent/tool-calling frameworks, sklearn/xgboost imports, torch/tensorflow imports), matching the pattern used for every other signal category that's already wired end-to-end.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    appsecAgentic guardrails, prompt injection, and RCE sensorsbugUnintended behavior or logic failure in the enginepriority: highCore feature broken, but workarounds existthreat: false-negativeAccuracy tuning: Engine missed a bypass or vulnerability

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions