EPIC: IronBank LeapfrogAI Hardening

[justinthelaw]

Description

Creating hardened IronBank "flavors" for different images that make up the LeapfrogAI stack provides the following benefits for all LeapfrogAI end-users:

1. Obtain a Certificate to Field (CtF) for deployment of LeapfrogAI to PlatformONE's PaaS
2. Gain reciprocity for other deployment platforms that accept the PlatformONE ATO
3. Enable "easier" ATO on other programs that accept IronBank images as drop-in replacements
4. Provide another open-source, hardened image option beyond Chainguard and upstream vendor containers

Acceptance Criteria

The following spikes¹ and features must be closed or completed in order to complete this EPIC:

Tasks:

Beta Give feedback

1. feat: leapfrogai api registry1 flavor #730
   dependencies documentation enhancement +3
   
2. feat: leapfrogai ui registry1 flavor #736
   dependencies documentation enhancement ui +4
   
3. spike: supabase bitnami registry1 image replacements #731
   dependencies spike +2
   
4. spike: db migrations using registry1 image replacements #735
   dependencies spike +2
   
5. spike: inference engine backends registry1 image replacements #737
   dependencies spike +2
   
6. spike: model files pvc image hardening example #979
   enhancement spike +2
   

Additional Context

Potential Blockers

The following is a running list of potential blockers, in the form of EPICs and/or issues, that could slowdown and/or disrupt the progress of this EPIC:

• refactor: istio-sidecar injection into supabase pods #460
• chore(release): correct ghcr repository directory #703
• EPIC: Implement Model Directory #623
• ADR: Bitnami Supabase deprecation continuation #968

Dependent Issues

The following is a running list of other EPICs and/or issues that are dependent on the closure of this EPIC:

• EPIC: Made for UDS #727
  • feat(flavors): ironbank versus upstream container flavors #704
  • test: airgap testing in e2e #707
• ADR: Breaking models weights out of model images #752
  • chore: Remove model weights from container images #786

Closed (Blocker Removed):

• chore: Slim down the things getting installed into the the vLLM Dockerfile #367

LeapfrogAI Images

The base image(s) include the "upstream" variant we use today, and the "possible IronBank" drop-in we could use for container re-base testing, if it exists.

1. LeapfrogAI API

   • Python 3.11 (upstream) (possible IronBank)

2. (Bitnami) Supabase Operator²

   • GoTrue (upstream)
   • JWT-CLI (upstream)
   • Kubectl (upstream)
   • OS-shell (upstream)
   • Postgrest (upstream)
   • Kong (upstream)
   • Supabase-Postgres (upstream)
   • Supabase-Postgres-Meta (upstream)
   • Supabase-Realtime (upstream)
   • Supabase-Storage (upstream)
   • Supabase-Studio (upstream)

3. Migrations (upstream) (possible IronBank)

4. LeapfrogAI UI

   • Node 18 (upstream) (possible IronBank)
   • Chainguard Node-Latest (upstream) (possible IronBank)

5. vLLM backend (upstream) (possible IronBank) (possible IronBank, but needs updating)

6. LLaMA-CPP-Python backend (upstream) (possible IronBank)

7. Text-Embeddings backend (upstream) (possible IronBank)

8. Whisper (upstream) (possible IronBank)

Resources and Links

• RepoONE
• Defense Unicorns RepoONE Sub-Project
• IronBank Harbor Registry
• IronBank Registry
• PlatformONE IL2 Slack
• Shadow's Edge Ollama IronBank repository
• Pepr P1 Developer Documentation
• Iron Bank Onboarding
• IronBank Hardening Guide
• IronBank Contributing Guide
• IronBank Fetch Resources Tool

Additional POCs

Defense Unicorns Product: @mjnagel @cmwylie19

Defense Unicorns Delivery: @gscallon @vanakema @CodyDamewood @phillip-record @americanthinker

Footnotes

1. All spikes related to this EPIC require the creation of a feat: issue after spike findings are reported ↩

2. Bitnami's Supabase Helm deployment is highly-coupled and there is a lack of the specific Bitnami-flavor images on the IronBank/Bitnami sub-project. Please see the spike required to confirm which IronBank images can be drop-in replacements for these Bitnami-flavor images. ↩

[justinthelaw]

Submitted onboarding document/questionnaire with request for the creation of all repos as outlined in the EPIC description above, with repo pattern opensource/defenseunicorns/leapfrogai/<optional sub-directory>/<container name>

[justinthelaw]

We have created a new Repo1 LeapfrogAI Project.

Below is a recommended (DRAFT) RACI for the development and maintenance of each hardened container:

IronBank Container	Responsible1	Accountable2	Consulted3	Informed4
api	@barronstone	@justinthelaw + @YrrepNoj	@YrrepNoj	@defenseunicorns/mission-ops-team @defenseunicorns/delivery @defenseunicorns/mission-ops
ui	@barronstone	@andrewrisse + @justinthelaw	@YrrepNoj @andrewrisse	@defenseunicorns/mission-ops-team @defenseunicorns/delivery @defenseunicorns/mission-ops
supabase5	@barronstone	@CollectiveUnicorn + @justinthelaw	@YrrepNoj @CollectiveUnicorn	@defenseunicorns/mission-ops-team @defenseunicorns/delivery @defenseunicorns/mission-ops
migrations6	@barronstone	@justinthelaw + @YrrepNoj	@YrrepNoj	@defenseunicorns/mission-ops-team @defenseunicorns/delivery @defenseunicorns/mission-ops
vllm	@barronstone	@justinthelaw + @YrrepNoj	@YrrepNoj	@defenseunicorns/mission-ops-team @defenseunicorns/delivery @defenseunicorns/mission-ops
llama-cpp-python	@barronstone	@justinthelaw + @YrrepNoj	@YrrepNoj	@defenseunicorns/mission-ops-team @defenseunicorns/delivery @defenseunicorns/mission-ops
whisper	@barronstone	@justinthelaw + @YrrepNoj	@YrrepNoj	@defenseunicorns/mission-ops-team @defenseunicorns/delivery @defenseunicorns/mission-ops
text-embeddings	@barronstone	@justinthelaw + @YrrepNoj	@YrrepNoj	@defenseunicorns/mission-ops-team @defenseunicorns/delivery @defenseunicorns/mission-ops

Footnotes

1. The person or team who does the work. There should only be one responsible person or team per task. E.g., Justin Law + Gregory Horvath ↩

2. The person who owns the work and is responsible for its completion. They may not be the person doing the work, but they are ultimately responsible if it's not completed. E.g., Barron Stone ↩

3. The person or people who provide input or knowledge before the work can be done and signed off on. They may use their subject knowledge to suggest changes or improvements. E.g., Jonathan Perry ↩

4. The person or people who need to be kept up to date on the progress, process execution, and quality of the work. They may also need to be informed of challenges and setbacks. E.g., Mission Operations Value Stream ↩

5. Drop-in replacement spikes and development for the following containers and/or base images (11 total): GoTrue, JWT-CLI, Kubectl, OS-shell, Postgrest, Kong, Supabase-Postgres, Supabase-Postgres-Meta, Supabase-Realtime, Supabase-Storage, Supabase-Studio ↩

6. Drop-in replacement spikes and development for migration for the following containers and/or base images (3 total): Supabase, API, and UI ↩

[gscallon]

Access update: @vanakema and I have received access to the Defense Unicorns and Leapfrog groups within Iron Bank and on the VAT

[andrewrisse]

blocked until presentation @justinthelaw

[justinthelaw]

Quick Start slides for IronBank Hardening: https://docs.google.com/presentation/d/1RQnkm4wnq2kvITh36rdYDeaHtEHq1YXP-eHKwOgZ6VE/edit#slide=id.g28fa4b3dbec_0_41

[justinthelaw]

Updates to this EPIC:

1. Due to Bitnami's end-of-life for Supabase Bitnami charts and containers, the Bitnami/Supabase task may change in scope depending on the results of this issue
2. The registry1 flavored API is now merged into the main branch of LeapfrogAI, which includes the creation of upstream flavors by default
3. The RACI has been updated with current POCs for each component