Rebuild Titan

[lowtorola]

Proposed Titan Rebuild:

• Titan should behave like Saturn, i.e. receive a notice via HTTP or pub sub from siarnaq on ANY file upload that the file object (sent in the request) should be scanned for malware (applies to code submissions and resumes)
• In the interim, siarnaq should mark the file as unverified
• Once the file has been verified/believed to be malicious, titan should send an HTTP request back to siarnaq instructing to mark the file as such (doesn't need to send the file back to siarnaq in the request)
  • Currently, Titan itself hits the database. This is a one-off occurence, and creates more to think about that we don't have to
• Upon receiving the HTTP request, siarnaq should mark the file with the given status
• Stretch goal, which may or may not be helpful: The download for ClamAV DB should be faster.
  • For example, cache it in a Google Storage bucket
• Stretch goal, Titan runs locally

Benefits of Proposed Structure:

• It will be easier to debug and log issues with titan, as previous debugging efforts in Fix Titan #786 were extremely difficult and required manual tweaking of service accounts and event triggers 🚫
• Mimics design methodology of siarnaq/saturn interaction via HTTP requests and asynchronous message-based status passing
• We don't really understand ClamAV that well, and this rebuild will effectively section it off into the Titan module where we can better reason about its behavior (and hopefully makes it easier to have ClamAV downloaded).

[n8kim1]

scratch paper: https://lucid.app/lucidspark/5a467ebd-93e1-489b-b3a1-a775169d1c87/edit?invitationId=inv_10944f2d-2e17-49ce-8a2b-c529cfb408fe&page=0_0#