Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Access Edges #2195

Merged
merged 55 commits into from
Oct 8, 2024
Merged
Show file tree
Hide file tree
Changes from 37 commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
220a078
access edge creator first draft
meln1k Sep 16, 2024
816297f
refactor the code a bit
meln1k Sep 18, 2024
afc8d69
handle service linked roles when checking SCPs policies
meln1k Sep 18, 2024
dd43128
fix the source in explicit deny check
meln1k Sep 18, 2024
066a650
implement basic scp control group checks
meln1k Sep 18, 2024
4fd45ec
evaluate SCPs correctly
meln1k Sep 18, 2024
f93e96d
adjust resource based policies
meln1k Sep 18, 2024
93df70a
fix how identity-based policies work
meln1k Sep 18, 2024
9bbd05f
cleanup
meln1k Sep 18, 2024
802bc42
add explicit some more tests
meln1k Sep 20, 2024
6a273be
linter fix
meln1k Sep 20, 2024
95aede1
more fixes
meln1k Sep 20, 2024
066eb65
add even more tests
meln1k Sep 22, 2024
13ac740
linter fixes
meln1k Sep 23, 2024
9177edd
Merge branch 'main' into nm/access-edge-creator-process
meln1k Sep 23, 2024
f636580
PR feedback
meln1k Sep 23, 2024
a7b1af5
another linter fix
meln1k Sep 23, 2024
6571a60
Merge branch 'main' into nm/access-edge-creator-process
meln1k Sep 23, 2024
5bca38a
performance improvements
meln1k Sep 23, 2024
ad3aa3c
arangodb devcontainer update
meln1k Sep 23, 2024
b8d31e1
linter fix
meln1k Sep 24, 2024
dcccb29
Merge branch 'main' into nm/access-edge-creator-process
meln1k Sep 24, 2024
d625b71
feature flag
meln1k Sep 25, 2024
4406f65
make feature flag false by default
meln1k Sep 25, 2024
a90f579
style fix
meln1k Sep 25, 2024
4b88253
Merge branch 'main' into nm/access-edge-creator-process
meln1k Sep 25, 2024
0c598d1
more PR feedback
meln1k Sep 25, 2024
b92b726
more pr feedback fixes
meln1k Sep 25, 2024
57d7970
linter fix
meln1k Sep 26, 2024
45ce83c
logging improvement
meln1k Sep 27, 2024
3fff4bb
linter
meln1k Sep 27, 2024
281d9c0
fix settings.json
meln1k Sep 27, 2024
8be14bb
add edge bugfix
meln1k Sep 27, 2024
5dd7c47
remove debug logging
meln1k Sep 27, 2024
f798bce
do not create self loop edges.
meln1k Sep 27, 2024
7304da8
rename access -> iam edges
meln1k Sep 27, 2024
0ec88a7
export iam reported section
meln1k Sep 27, 2024
10e0f66
PR feedback
meln1k Oct 1, 2024
b5120de
Merge branch 'main' into nm/access-edge-creator-process
meln1k Oct 1, 2024
700d118
linters and tests
meln1k Oct 1, 2024
39de959
collect more principals
meln1k Oct 2, 2024
fabebcf
debug logging
meln1k Oct 2, 2024
ef34152
do not check every action on AWS readonly policy
meln1k Oct 4, 2024
eb6325c
Merge branch 'main' into nm/access-edge-creator-process
meln1k Oct 4, 2024
1432a29
remove progress bar
meln1k Oct 4, 2024
638ee68
fetch permission boundaries for users and roles
meln1k Oct 7, 2024
eedcfc0
linter fix
meln1k Oct 7, 2024
45a1bee
collect more resource-based policies
meln1k Oct 8, 2024
f68420c
pr feedback
meln1k Oct 8, 2024
f954cb4
Merge branch 'main' into nm/access-edge-creator-process
meln1k Oct 8, 2024
0777803
set access edges flag to disabled
meln1k Oct 8, 2024
06b03e7
lintex fix
meln1k Oct 8, 2024
acad041
Merge branch 'main' into nm/access-edge-creator-process
meln1k Oct 8, 2024
c32fdd1
a bit of more fixes
meln1k Oct 8, 2024
dd181a0
remove default arn
meln1k Oct 8, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .devcontainer/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \
&& apt-get -y install --no-install-recommends vim wget


RUN wget -qO- https://download.arangodb.com/arangodb310/DEBIAN/Release.key | apt-key add -
RUN echo 'deb https://download.arangodb.com/arangodb310/DEBIAN/ /' | tee /etc/apt/sources.list.d/arangodb.list
RUN wget -qO- https://download.arangodb.com/arangodb311/DEBIAN/Release.key | apt-key add -
RUN echo 'deb https://download.arangodb.com/arangodb311/DEBIAN/ /' | tee /etc/apt/sources.list.d/arangodb.list
RUN apt-get install -y apt-transport-https
RUN apt-get update
RUN apt-get install -y arangodb3-client
Expand Down
2 changes: 1 addition & 1 deletion .devcontainer/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ services:
# (Adding the "ports" property to this file will not forward from the container.)

arangodb:
image: arangodb:3.10.3
image: arangodb:3.11.11
restart: unless-stopped
# Uncomment the lines below in case you want to keep arangodb data in a separate volume
# volumes:
Expand Down
13 changes: 11 additions & 2 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,16 @@
],
"python.testing.unittestEnabled": false,
"python.testing.pytestEnabled": false,
"python.formatting.provider": "black",
"[python]": {
"editor.defaultFormatter": "ms-python.black-formatter",
"editor.formatOnSave": true
},
"black-formatter.args": [
"--line-length",
"120",
"--target-version",
"py39"
],
"search.exclude": {
"**/*.code-search": true,
"**/bower_components": true,
Expand All @@ -22,4 +31,4 @@
"python.analysis.extraPaths": [
"fixlib"
]
}
}
71 changes: 68 additions & 3 deletions fixlib/fixlib/baseresources.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,16 @@
from abc import ABC
from copy import deepcopy
from datetime import datetime, timezone, timedelta
from enum import Enum, unique
from enum import Enum, StrEnum, unique
from functools import wraps, cached_property
from typing import Dict, Iterator, List, ClassVar, Optional, TypedDict, Any, TypeVar, Type, Callable, Set, Tuple
from collections import defaultdict

from attr import resolve_types
from attrs import define, field, Factory
from attrs import define, field, Factory, frozen, evolve
from prometheus_client import Counter, Summary

from fixlib.json import from_json as _from_json, to_json as _to_json
from fixlib.json import from_json as _from_json, to_json as _to_json, to_json_str
from fixlib.logger import log
from fixlib.types import Json
from fixlib.utils import make_valid_timestamp, utc_str
Expand Down Expand Up @@ -68,6 +68,7 @@ class ModelReference(TypedDict, total=False):
class EdgeType(Enum):
default = "default"
delete = "delete"
iam = "iam"

@staticmethod
def from_value(value: Optional[str] = None) -> EdgeType:
Expand Down Expand Up @@ -1610,4 +1611,68 @@ def delete(self, graph: Any) -> bool:
return False


class PolicySourceKind(StrEnum):
Principal = "principal" # e.g. IAM user, attached policy
Group = "group" # policy comes from an IAM group
Resource = "resource" # e.g. s3 bucket policy
meln1k marked this conversation as resolved.
Show resolved Hide resolved


ResourceConstraint = str

ConditionString = str


@frozen
class PolicySource:
kind: PolicySourceKind
uri: str


class HasResourcePolicy(ABC):
# returns a list of all policies that affects the resource (inline, attached, etc.)
def resource_policy(self, builder: Any) -> List[Tuple[PolicySource, Json]]:
raise NotImplementedError


@frozen
class PermissionCondition:
# if nonempty and any evals to true, access is granted, otherwise implicitly denied
allow: Optional[Tuple[ConditionString, ...]] = None
# if nonempty and any is evals to false, access is implicitly denied
boundary: Optional[Tuple[ConditionString, ...]] = None
# if nonempty and any evals to true, access is explicitly denied
deny: Optional[Tuple[ConditionString, ...]] = None


@frozen
class PermissionScope:
source: PolicySource
constraints: Tuple[ResourceConstraint, ...] # aka resource constraints
conditions: Optional[PermissionCondition] = None

def with_deny_conditions(self, deny_conditions: List[Json]) -> "PermissionScope":
c = self.conditions or PermissionCondition()
return evolve(self, conditions=evolve(c, deny=tuple([to_json_str(c) for c in deny_conditions])))

def with_boundary_conditions(self, boundary_conditions: List[Json]) -> "PermissionScope":
c = self.conditions or PermissionCondition()
return evolve(self, conditions=evolve(c, boundary=tuple([to_json_str(c) for c in boundary_conditions])))

def has_no_condititons(self) -> bool:
if self.conditions is None:
return True

if self.conditions.allow is None and self.conditions.boundary is None and self.conditions.deny is None:
return True

return False


@frozen
class AccessPermission:
action: str
level: str
meln1k marked this conversation as resolved.
Show resolved Hide resolved
scopes: Tuple[PermissionScope, ...]


resolve_types(BaseResource) # noqa
12 changes: 8 additions & 4 deletions fixlib/fixlib/graph/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ def merge(self, graph: Graph, skip_deferred_edges: bool = False) -> None:

try:
self._log_edge_creation = False
self.update(edges=graph.edges, nodes=graph.nodes)
self.update(edges=graph.edges(keys=True, data=True), nodes=graph.nodes)
self.deferred_edges.extend(graph.deferred_edges)
finally:
self._log_edge_creation = True
Expand Down Expand Up @@ -640,17 +640,21 @@ def export_graph(self) -> None:
if not self.found_replace_node:
log.warning(f"No nodes of kind {self.graph_merge_kind.kind} found in graph")
start_time = time()
for edge in self.graph.edges:
for edge in self.graph.edges(keys=True, data=True):
meln1k marked this conversation as resolved.
Show resolved Hide resolved
from_node = edge[0]
to_node = edge[1]
if not isinstance(from_node, BaseResource) or not isinstance(to_node, BaseResource):
log.error(f"One of {from_node} and {to_node} is no base resource")
continue
edge_dict = {"from": from_node.chksum, "to": to_node.chksum}
if len(edge) == 3:
key = edge[2]
if key := edge[2]:
if isinstance(key, EdgeKey) and key.edge_type != EdgeType.default:
edge_dict["edge_type"] = key.edge_type.value

if key.edge_type == EdgeType.iam:
meln1k marked this conversation as resolved.
Show resolved Hide resolved
data = edge[3]
if reported := data.get("reported"):
edge_dict["reported"] = reported
edge_json = json.dumps(edge_dict) + "\n"
self.tempfile.write(edge_json.encode())
self.total_lines += 1
Expand Down
4 changes: 2 additions & 2 deletions fixlib/tox.ini
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ commands = flake8 --verbose
commands= pytest

[testenv:black]
commands = black --line-length 120 --check --diff --target-version py39 .
commands = black --line-length 120 --check --diff --target-version py311 .

[testenv:mypy]
commands= mypy --install-types --non-interactive --python-version 3.9 --strict fixlib
commands= mypy --install-types --non-interactive --python-version 3.11 --strict fixlib
Loading
Loading