Skip to content
This repository has been archived by the owner on Aug 25, 2024. It is now read-only.

Commit

Permalink
docs: discussions: Alice Engineering Comms: 2024-01-05T20:22:35+00:00
Browse files Browse the repository at this point in the history
  • Loading branch information
aliceoa-intel authored and pdxjohnny committed Jan 5, 2024
1 parent 527a107 commit 7a406ad
Show file tree
Hide file tree
Showing 78 changed files with 980 additions and 7 deletions.
2 changes: 0 additions & 2 deletions docs/discussions/alice_engineering_comms/0001/reply_0000.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,8 +60,6 @@ Hope you’ve been well. It’s John from Intel. Thanks again to you and the tea

I periodically check the minutes so I joined today and asked about the "Alpha-Omega" project from last week’s minutes which I then did some research on. We just started what looks to me to be an aligned project, coincidentally named Alice Omega Alpha: https://github.com/intel/dffml/tree/main/entities/alice

It looks to me like Alice's mission to proactively enable developers and organizations to deliver organizationally context aware, adaptive secure by default best practices to teams aligns with project Alpha-Omega’s goals.

Alice is the nickname for both the entity and the architecture, the Open Architecture, which is a methodology for interpretation of existing well established, formats, protocols, and other domain specific representations of architecture. What we end up with is some JSON, YAML, or other blob of structured data that we can use to build cross language tooling focused more on policy and intent, incorporating data from arbitrary sources to create a holistic picture of software across dependency boundaries by focusing on threat models.

Alice will be doing scans of open source projects and we’d still love to collaborate to contribute metrics to the OpenSSF metrics database, we can easily have her shoot applicable metrics off to that DB. We’ve also been looking at fusing VEX and DIDs to facilitate distributed vulnerability disclosure and patch distribution.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ $ dffml version
dffml 0.4.0 /src/dffml/dffml 5c89b6780 (dirty git repo)
dffml-config-yaml 0.1.0 /src/dffml/configloader/yaml/dffml_config_yaml 5c89b6780 (dirty git repo)
dffml-config-image not installed
dffml-configloader-jsonschema 0.0.1 /src/dffml/configloader/jsonschema/dffml_configloader_jsonschema 5c89b6780 (dirty git repo)
dffml-config-jsonschema 0.0.1 /src/dffml/configloader/jsonschema/dffml_config_jsonschema 5c89b6780 (dirty git repo)
dffml-model-scratch not installed
dffml-model-scikit not installed
dffml-model-tensorflow not installed
Expand Down
2 changes: 2 additions & 0 deletions docs/discussions/alice_engineering_comms/0243/reply_0000.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,8 @@ if __name__ == "__main__":
- https://github.com/ossf/wg-vulnerability-disclosures/issues/74
- https://app.slack.com/client/T019QHUBYQ3/C05009RHCNT
- TODO: Anyone playing with the json-ld-ness of openvex yet?
- Lot's of kundalini today
- We must be getting close
- https://github.com/in-toto/attestation/pull/192
- Great proto regen example
- https://github.com/in-toto/attestation/blob/3df726cfcc0528dcbdb4d45ed1597b793d1b777d/spec/predicates/scai.md
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@
- The owner will need to lookup own key in the log often
- Each time owners key changes must check that key change has been correctly included in the log
- Owner needs to remember ephocs they changed their key (hmmm)
- https://github.com/scitt-community/scitt-examples/pull/5
- TODO
- [ ] Specify the privacy guarantees
- [ ] Review compliance requirements about removing information from logs on mailing list
10 changes: 9 additions & 1 deletion docs/discussions/alice_engineering_comms/0458/reply_0000.md
Original file line number Diff line number Diff line change
Expand Up @@ -424,7 +424,9 @@ ing to infer new facts or constraints from the ontology and apply them to policy
> ### Slide 1: Collaborative AGI Development with Trusted Poly-Repo Management
> - Summary of the seamless, secure collaboration protocol established between AGI instances, Alice and Bob, across multiple repository environments.
>
> --- > > ### Slide 2: Manager's Speak - Executive Summary
> ---
>
> ### Slide 2: Summary
> - Introduction to the need for secure, transparent collaboration between advanced software systems.
> - Overview of SCITT as the transparency ledger for recording and sharing approved activities.
> - The role of policy engines in ensuring adherence to each AGI's operational policies.
Expand Down Expand Up @@ -579,6 +581,12 @@ ing to infer new facts or constraints from the ontology and apply them to policy
>
> By integrating SCITT with Kubernetes, Alice and Bob can ensure that the cluster's state always reflects approved and validated states from their CI/CD workflows, maintaining security and consistency across their development operations. This integration also creates an audit trail for all changes, providing complete visibility into cluster events and enabling rapid response to potential policy violations.
- https://github.com/intel/dffml/blob/6fd36f7b88943c038bbd5217bb187f4a04891003/docs/discussions/alice_engineering_comms/0243/reply_0000.md
- https://futurism.com/openai-employees-say-firms-chief-scientist-has-been-making-strange-spiritual-claims
- https://futurism.com/sam-altman-imply-openai-building-god
- https://www.theatlantic.com/technology/archive/2022/09/artificial-intelligence-machine-learing-natural-language-processing/661401/ **OF GOD AND MACHINES**
![such-alignment](https://user-images.githubusercontent.com/5950433/226707682-cfa8dbff-0908-4a34-8540-de729c62512f.png)
- TODO
- [ ] k8s SCITT receipt as admission control
Expand Down
1 change: 1 addition & 0 deletions docs/discussions/alice_engineering_comms/0459/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# 2023-11-22 Engineering Logs
123 changes: 123 additions & 0 deletions docs/discussions/alice_engineering_comms/0459/reply_0000.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
## 2023-11-22 @pdxjohnny Engineering Logs

```console
$ pip install -I git+https://github.com/wbond/oscrypto.git
```

```python
import aiohttp
import asyncio
import json

async def fetch_dependency_graph(session, owner, repo, token, manifest_cursor=None, dependency_cursor=None):
# The GraphQL query to fetch the dependency graph manifests
query = """
query($owner: String!, $repo: String!, $manifest_cursor: String, $dependency_cursor: String) {
repository(owner: $owner, name: $repo) {
dependencyGraphManifests(first: 2, after: $manifest_cursor) {
pageInfo {
hasNextPage
endCursor
}
nodes {
blobPath
dependencies(first: 2, after: $dependency_cursor) {
pageInfo {
hasNextPage
endCursor
}
edges {
node {
packageName
repository {
nameWithOwner
}
requirements
}
}
}
}
}
}
}
"""

# Format the variables for the GraphQL query
variables = {
"owner": owner,
"repo": repo,
"manifest_cursor": manifest_cursor,
"dependency_cursor": dependency_cursor
}

# Headers to be sent with the request
headers = {
"Authorization": f"Bearer {token}",
"Content-Type": "application/json"
}

# Make the POST request to the GitHub GraphQL API
async with session.post('https://api.github.com/graphql', json={'query': query, 'variables': variables}, headers=headers) as response:
return await response.json()

async def generate_sbom(owner, repo, token):
dependency_manifests = []

async with aiohttp.ClientSession() as session:
# Pagination for manifests
has_manifest_page = True
manifest_cursor = None

while has_manifest_page:
# Fetch the dependency graph manifests
data = await fetch_dependency_graph(session, owner, repo, token, manifest_cursor)
manifest_nodes = data['data']['repository']['dependencyGraphManifests']['nodes']
manifest_page_info = data['data']['repository']['dependencyGraphManifests']['pageInfo']
has_manifest_page = manifest_page_info['hasNextPage']
manifest_cursor = manifest_page_info['endCursor']

for manifest_node in manifest_nodes:
manifest_dependencies = []
# Start nested pagination on first page for each manifest
has_dependency_page = True
dependency_cursor = None

while has_dependency_page:
# Fetch the dependencies for the current manifest
manifest_data = await fetch_dependency_graph(session, owner, repo, token, manifest_cursor, dependency_cursor)
dependencies = manifest_data['data']['repository']['dependencyGraphManifests']['nodes'][0]['dependencies']['edges']
dependency_page_info = manifest_data['data']['repository']['dependencyGraphManifests']['nodes'][0]['dependencies']['pageInfo']

for dependency_edge in dependencies:
dependency_node = dependency_edge['node']
manifest_dependencies.append({
"packageName": dependency_node['packageName'],
"requirements": dependency_node['requirements'],
"repository": dependency_node['repository']['nameWithOwner'] if dependency_node['repository'] else None
})

has_dependency_page = dependency_page_info['hasNextPage']
dependency_cursor = dependency_page_info['endCursor']

dependency_manifests.append({
"blobPath": manifest_node['blobPath'],
"dependencies": manifest_dependencies
})

return { "dependencyGraphManifests": dependency_manifests }

# Run the asynchronous function to generate the SBOM and get the result
sbom_data = asyncio.run(generate_sbom(owner, repo, token))

# Now sbom_data is a dictionary with all the dependencyGraphManifests
# You could pretty print it using json.dumps for example
print(json.dumps(sbom_data, indent=2))
```

- TODO
- [ ] Request from Orie, review COSE typ header parameter draft
- [x] git ls-files with aiohttp
- [x] Example files: https://gist.github.com/52d17fd4d44014fe1b8a15111873454b
- [ ] GitHub Webhook Notary for SBOM generation
- [ ] SBOM -> Polling of repos -> GitHub webhook style payload creation -> GitHub Webhook Notary
- Content addressability of webhook payloads to ensure dedup / polling updated SHAs always trigger new update but never when SHAs not updated
1 change: 1 addition & 0 deletions docs/discussions/alice_engineering_comms/0460/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# 2023-11-23 Engineering Logs
57 changes: 57 additions & 0 deletions docs/discussions/alice_engineering_comms/0460/reply_0000.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
## 2023-11-23 @pdxjohnny Engineering Logs

- Happy Thanksgiving!

```bash
export COMPUTE_IPV4=$(doctl compute droplet list --no-header --format PublicIPv4 prophecy-0)
doctl compute domain records create --record-name alice --record-ttl 3600 --record-type A --record-data "${COMPUTE_IPV4}" chadig.com
doctl compute domain records create --record-name github-webhook-notary.scitt.alice --record-ttl 3600 --record-type A --record-data "${COMPUTE_IPV4}" chadig.com
ssh -nNT -R 127.0.0.1:7777:0.0.0.0:7777 [email protected]
```

```caddyfile
alice.chadig.com {
redir "https://github.com/intel/dffml/discussions/1406?sort=new" temporary
}
github-webhook-notary.scitt.alice.chadig.com {
reverse_proxy http://localhost:7777
}
scitt.bob.chadig.com {
reverse_proxy http://localhost:6000
}
scitt.alice.chadig.com {
reverse_proxy http://localhost:7000
}
scitt.unstable.chadig.com {
reverse_proxy http://localhost:8000
}
scitt.pdxjohnny.chadig.com {
reverse_proxy http://localhost:9000
}
define.chadig.com {
respond "Cha-Dig: can you dig it? chaaaaaaa I can dig it!!!"
}
```

- Claus
- https://www.scandinaviastandard.com/what-is-janteloven-the-law-of-jante/
- TODO
- [ ] GitHub App Blueprints to
- [x] https://github.com/apps/alice-oa
- [ ] Webhook events to notarizing proxy
- [ ] `$ gh webhook forward --repo=intel/dffml --events='*' --url=https://github-webhook-notary.scitt.alice.chadig.com`
- [ ] https://docs.github.com/en/webhooks/using-webhooks/validating-webhook-deliveries#python-example
- [ ] #1315
- [ ] Bovine based downstream event receiver
- [ ] As async iterator for new data events
- [ ] POC using OpenAI agent threads with file uploads
- [ ] Alice engineering log entry in daily discussion thread for updates
- [ ] Checkbox checked by maintainer for requests approval
- [ ] Assign issues to Alice via `Assignee: @aliceoa` watch webhook issue creation or body updates
- `cat issues.action\:edited.json | jq 'select(.issue.body | index("Assignee: @aliceoa"))'`
1 change: 1 addition & 0 deletions docs/discussions/alice_engineering_comms/0461/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# 2023-11-24 Engineering Logs
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
- https://hackaday.com/2023/11/22/esp32-used-as-wireless-can-bus-reader/
1 change: 1 addition & 0 deletions docs/discussions/alice_engineering_comms/0462/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# 2023-11-25 Engineering Logs
1 change: 1 addition & 0 deletions docs/discussions/alice_engineering_comms/0463/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# 2023-11-26 Engineering Logs
1 change: 1 addition & 0 deletions docs/discussions/alice_engineering_comms/0464/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# 2023-11-27 Engineering Logs
13 changes: 13 additions & 0 deletions docs/discussions/alice_engineering_comms/0464/reply_0000.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
## 2023-11-27 OpenVEX SIG

- https://docs.google.com/document/d/1C-L0JDx5O35TjXb6dcyL6ioc5xWUCkdR5kEbZ1uVQto/edit#heading=h.yz69ktumsyjh
- Using grype to attach attestations via cosgin to a container image
- PyPi ecosystem looking at this as well
- Currently if you can upload to the registry we decide we can trust the attestation
- Next is if the signature is from the same entity
- https://github.com/puerco/grype/tree/vex-discovery
- https://github.com/puerco/grype/tree/dabe702c5172f5fd7faf7008513696a435c87d15
- https://github.com/openvex/spec/issues/43
- https://github.com/opencontainers/distribution-spec/issues/459

![image](https://github.com/intel/dffml/assets/5950433/91165c0a-0b81-4304-9d4e-e02cf20eeb61)
4 changes: 4 additions & 0 deletions docs/discussions/alice_engineering_comms/0464/reply_0001.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
## 2023-11-27 @pdxjohnny Engineering Logs

- https://github.com/quartzjer/did-jwk/blob/main/spec.md
- If you leverage the content address as the subject you can get trust attestations from SCITT
1 change: 1 addition & 0 deletions docs/discussions/alice_engineering_comms/0465/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# 2023-11-28 Engineering Logs
2 changes: 2 additions & 0 deletions docs/discussions/alice_engineering_comms/0465/reply_0000.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
- https://github.com/ipvm-wg/homestar/tree/main/examples/websocket-relay
- > It's like we're in a scene from 'The Matrix.' In one hand, the existing powers hold the blue pill, symbolizing our move to centralize data in the cloud – efficient, streamlined, but very by-the-book. In the other hand, I've got the red pill, representing our journey to decentralize, to innovate and explore new frontiers in AI. While the blue pill keeps things running smoothly, the red pill is about venturing into uncharted territory. It's a fun way to see our roles – one maintaining the order, the other pushing the boundaries. Which pill would you choose?
1 change: 1 addition & 0 deletions docs/discussions/alice_engineering_comms/0466/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# 2023-11-29 Engineering Logs
1 change: 1 addition & 0 deletions docs/discussions/alice_engineering_comms/0467/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# 2023-11-30 Engineering Logs
1 change: 1 addition & 0 deletions docs/discussions/alice_engineering_comms/0468/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# 2023-12-01 Engineering Logs
90 changes: 90 additions & 0 deletions docs/discussions/alice_engineering_comms/0468/reply_0000.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
## 2023-12-01 @pdxjohnny Engineering Logs

- https://chromium.googlesource.com/chromium/chromium/+/refs/heads/trunk/chromeos/attestation/attestation_flow.cc
- https://github.com/slsa-framework/slsa-github-generator/blob/62a6671ba95c18cf73102bda18ec523e39dc7ab2/internal/builders/generic/attest.go#L81C30-L81C51
- https://github.com/in-toto/scai-demos/tree/main/.github/actions
- https://slsa.dev/spec/v1.0/provenance
- https://search.sigstore.dev/?logIndex=33351527

```json
{
// Standard attestation fields:
"_type": "https://in-toto.io/Statement/v1",
"subject": [...],

// Predicate:
"predicateType": "https://slsa.dev/provenance/v1",
"predicate": {
"buildDefinition": {
"buildType": string,
"externalParameters": object,
"internalParameters": object,
"resolvedDependencies": [ ...#ResourceDescriptor ],
},
"runDetails": {
"builder": {
"id": string,
"builderDependencies": [ ...#ResourceDescriptor ],
"version": { ...string },
},
"metadata": {
"invocationId": string,
"startedOn": #Timestamp,
"finishedOn": #Timestamp,
},
"byproducts": [ ...#ResourceDescriptor ],
}
}
}

#ResourceDescriptor: {
"uri": string,
"digest": {
"sha256": string,
"sha512": string,
"gitCommit": string,
[string]: string,
},
"name": string,
"downloadLocation": string,
"mediaType": string,
"content": bytes, // base64-encoded
"annotations": object,
}

#Timestamp: string // <YYYY>-<MM>-<DD>T<hh>:<mm>:<ss>Z
```


```yaml
_type: https://in-toto.io/Statement/v1
subject:
- name: pkg:npm/[email protected]
digest:
sha512: >-
90f223f992e4c88dd068cd2a5fc57f9d2b30798343dd6e38f29c240e04ba090ef831f84490847c4e82b9232c78e8a258463b1e55c0f7469f730265008fa6633f
predicateType: https://slsa.dev/provenance/v1
predicate:
buildDefinition:
buildType: https://slsa-framework.github.io/github-actions-buildtypes/workflow/v1
externalParameters:
workflow:
ref: refs/heads/main
repository: https://github.com/sigstore/sigstore-js
path: .github/workflows/release.yml
internalParameters:
github:
event_name: push
repository_id: '495574555'
repository_owner_id: '71096353'
resolvedDependencies:
- uri: git+https://github.com/sigstore/sigstore-js@refs/heads/main
digest:
gitCommit: 26d16513386ffaa790b1c32f927544f1322e4194
runDetails:
builder:
id: https://github.com/actions/runner/github-hosted
metadata:
invocationId: >-
https://github.com/sigstore/sigstore-js/actions/runs/6014488666/attempts/1
```
1 change: 1 addition & 0 deletions docs/discussions/alice_engineering_comms/0469/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# 2023-12-02 Engineering Logs
Loading

0 comments on commit 7a406ad

Please sign in to comment.