Add Crown Jewel resource #40914

jakule · 2024-04-25T18:22:09Z

Add Crown Jewel resource described here https://github.com/gravitational/access-graph/blob/main/rfd/0005-crown-jewel.md

Depends on #41168

changelog: Add support for crown jewel resource type

api/types/crownjewel/crownjewel.go

github-actions · 2024-05-01T03:56:42Z

The PR changelog entry failed validation: Changelog entry not found in the PR body. Please add a "no-changelog" label to the PR, or changelog lines starting with changelog: followed by the changelog entries for the PR.

lib/auth/crownjewel/crownjewelv1/service.go

rosstimothy · 2024-05-01T15:13:36Z

lib/auth/crownjewel/crownjewelv1/service.go

+		return nil, trace.Wrap(err)
+	}
+
+	rsp, nextToken, err := s.backend.ListCrownJewels(ctx, req.PageSize, req.PageToken)


Will this read from the cache of the backend?

Backend per my understanding, as most services from what I see. Should we always read from cache?

There is a slight nuance to how reads are directed to the backend vs cache. As described here anything using the auth.Server can direct you to either the cache or backend depending on which of the two embedded items implement the function call, with priority going to the cache. In most cases, if an item is cached it should probably be read from the cache and not the backend.

Changed, I also updated the Get method to read from cache

lib/auth/crownjewel/crownjewelv1/service_test.go

lib/cache/collections.go

lib/services/local/crown_jewels.go

lib/services/local/crown_jewels_test.go

tool/tctl/common/resource_command.go

rosstimothy · 2024-05-01T16:39:00Z

lib/services/local/crown_jewels.go

+}
+
+func (s *CrownJewelsService) UpdateCrownJewel(ctx context.Context, crownJewel *crownjewelv1.CrownJewel) (*crownjewelv1.CrownJewel, error) {
+	r, err := s.service.UpdateResource(ctx, crownJewel)


Should this be calling ConditionalUpdateResource? Do we want crown jewels to be protected from concurrent updates via optimistic locking?

Sure, but I don't see this wrapper to implement this.

😞 I can push a PR to include it if in the service wrapper.

I'll update it after your PR is merged.

Updated 8ac3bee

public-teleport-github-review-bot · 2024-05-02T18:05:46Z

@jakule - this PR will require admin approval to merge due to its size. Consider breaking it up into a series smaller changes.

rosstimothy · 2024-05-02T17:28:45Z

lib/auth/crownjewel/crownjewelv1/service.go

+	// Logger is the logger to use.
+	Logger logrus.FieldLogger


Looks like this was missed during cleanup.

Suggested change

// Logger is the logger to use.

Logger logrus.FieldLogger

Removed 🤦

rosstimothy · 2024-05-02T18:29:19Z

lib/auth/crownjewel/crownjewelv1/service.go

+		return nil, trace.Wrap(err)
+	}
+
+	rsp, nextToken, err := s.backend.ListCrownJewels(ctx, req.PageSize, req.PageToken)


There is a slight nuance to how reads are directed to the backend vs cache. As described here anything using the auth.Server can direct you to either the cache or backend depending on which of the two embedded items implement the function call, with priority going to the cache. In most cases, if an item is cached it should probably be read from the cache and not the backend.

lib/services/crown_jewel.go

rosstimothy · 2024-05-02T18:39:36Z

lib/services/local/crown_jewels.go

+}
+
+func (s *CrownJewelsService) UpdateCrownJewel(ctx context.Context, crownJewel *crownjewelv1.CrownJewel) (*crownjewelv1.CrownJewel, error) {
+	r, err := s.service.UpdateResource(ctx, crownJewel)


😞 I can push a PR to include it if in the service wrapper.

tool/tctl/common/collection.go

lib/auth/clt.go

lib/auth/crownjewel/object.go

lib/services/crown_jewel.go

espadolini · 2024-05-03T14:39:22Z

api/client/crownjewel/crownjewel.go

+	if err != nil {
+		return nil, trace.Wrap(err)
+	}
+	return rsp, trace.Wrap(err)


Suggested change

return rsp, trace.Wrap(err)

return rsp, nil

espadolini · 2024-05-03T14:39:39Z

api/client/crownjewel/crownjewel.go

+	if err != nil {
+		return nil, trace.Wrap(err)
+	}
+	return rsp, trace.Wrap(err)


Suggested change

return rsp, trace.Wrap(err)

return rsp, nil

espadolini · 2024-05-03T14:39:48Z

api/client/crownjewel/crownjewel.go

+	if err != nil {
+		return nil, trace.Wrap(err)
+	}
+	return rsp, trace.Wrap(err)


Suggested change

return rsp, trace.Wrap(err)

return rsp, nil

espadolini · 2024-05-03T14:40:02Z

api/client/crownjewel/crownjewel.go

+	if err != nil {
+		return nil, trace.Wrap(err)
+	}
+	return rsp, trace.Wrap(err)


Suggested change

return rsp, trace.Wrap(err)

return rsp, nil

espadolini · 2024-05-03T14:43:45Z

api/proto/teleport/crownjewel/v1/crownjewel_service.proto

You could get rid of message DeleteAllCrownJewelsRequest too, while you're touching this.

espadolini · 2024-05-03T15:34:35Z

lib/auth/crownjewel/object.go

+//   - Matcher.Name or Matcher.Labels
+func ValidateCrownJewel(jewel *crownjewelv1.CrownJewel) error {
+	switch {
+	case jewel == nil:


for this and the later parts: a nil pointer protobuf message is a valid read-only message that has all fields zeroed; should we start using getters whenever possible? we've kind of avoided that for gogoproto, which doesn't always generate getters and can store submessages as structs, but that's not how the convention goes for protobuf

lib/auth/crownjewel/object.go

espadolini · 2024-05-03T15:41:10Z

lib/cache/cache.go

+		return nil, trace.Wrap(err)
+	}
+	defer rg.Release()
+	return rg.reader.GetCrownJewel(ctx, name)


Would it make sense to read through the cache on a NotFound? We do that for some resources like roles, for example.

It would be good to have one logic for all objects, so we don't need to reinvent the wheel for every new object. That being said I don't think we need that and from what I see most of our objects don't have this fallback implemented.

espadolini · 2024-05-03T15:48:28Z

lib/services/crown_jewel.go

+		object = proto.Clone(object).(*crownjewelv1.CrownJewel)
+		object.Metadata.Revision = ""
+	}
+	data, err := utils.FastMarshal(object)


no protojson? 🥺

r0mant

Bot.

This commit modifies the labels from a map to repeated TeleportLabel in CrownJewel proto, and accordingly updates the getter function in proto generated code. The newly introduced TeleportLabel type contains a Key and repeated values for improved flexibility. Clean up Update and restructure CrownJewel service proto The code modifications include renaming and restructuring several types and altering various functions in the CrownJewel service proto. Major changes involve renaming DeleteCrownJewel Rename 'kind' to 'kinds' in crownjewel API This commit changes the 'kind' field to 'kinds' in the crownjewel API to better reflect its purpose of holding multiple resource types. Changes are made across several files - 'crownjewel.go', 'crownjewel.pb.go' and 'crownjewel.proto'. After rebase fixes Remove CrownJewel type from several files This commit represents the removal of the CrownJewel type from several files. CrownJewel type references are deleted in certain Go files, and imports of CrownJewel in multiple files are being adjusted. The specifics of why this type is removed is not evident from the diffs, but the deletions will affect the related functionalities. Add CrownJewel update functionality This update enables the 'UpdateCrownJewel' functionality. If a 'CrownJewel' resource already exists, it can now be updated instead of returning an error. This enhances flexibility when dealing with resource management. It also introduces the ARN field into the 'AWSMatcher' service. Update CrownJewel service with pagination and improve matchers This update adds pagination to the CrownJewel service for more efficient data handling. It also improves the matching capabilities with the inclusion of Teleport and AWS matchers and allows for selective querying with these matchers. Additionally, it includes various changes to structure and naming for better clarity and ease of use. Refactor CrownJewel protos to use ResourceHeader This refactoring replaces the previous structure of CrownJewel proto's fields Kind, Sub_kind, Version, and Metadata with a single object called Header (ResourceHeader). It simplifies the code structure and harmonizes it with other protos. Now, Header contains the essential attributes that were previously distributed across multiple separate fields. Add GetCrownJewels method to cache and accessgraph The function GetCrownJewels has been added to the cache and the accessgraph. It retrieves all CrownJewel instances from the cache. Additionally, CrownJewel retrieval functionality has been removed from auth server and existing usages have been refactored. Implement Crown Jewel creation feature Updated the method `createCrownJewel` in the resources command to handle Crown Jewel creation. Introduced a new gRPC service `crownjewelv1` to handle client-server interactions for Crown Jewel objects. Crown Jewel expiry configuration has now also been enabled. Add missing files I have no words... Update authentication and event handling Refactored the argument from 'auth.ClientI' to '*auth.Client' in 'createCrownJewel' function in 'resource_command.go'. Removed unused import 'durationpb' in 'auth_with_roles.go'. Added a new event code 'AccessGraphAccessPathChangedCode' in 'codes.go' and 'AccessGraphAccessPathChanged' in 'api.go' for better monitoring and logging of access path changes. Add access path changed event and relevant changes. update api add grpc update api add cache support add protos

Updated API methods related to the Crown Jewel resource to improve clarity and consistency. Renamed `GetCrownJewels` to `ListCrownJewels`, added pagination parameters, and ensured uniform handling of the Crown Jewel resource across different parts of the codebase. Updated relevant comments for better code understanding.

This commit removes the CrownJewel converter and updates references from the old CrownJewel type to the new protobuf-defined CrownJewel type, streamlining the codebase. The commit also disables CrownJewel deletion since it is not supported yet. Some minor errors were fixed regarding CrownJewel resource names.

The crown jewel deletion feature has been enabled by uncommenting and simplifying the code block. Now, the delete action is directly performed on the specific 'crown jewel' mentioned based on its name. The previously unsupported action returns an informative success message after the deletion.

In both the cache test and access control files, CrownJewel resource has been included. This provides the system's ability to manage and test CrownJewel resources. Additionally, it allows the handling of CrownJewel resources in client events.

Updated the copyright dates in several files to reflect the new year. Additionally, license headers have been added to some files where they were previously missing, ensuring the proper copyrights and GPL notice are visible.

Fixed various typographical errors in CrownJewel's related functions and removed a TODO comment. Improved function and function calls' names that were incorrectly referring to different modules, making the code easier to read and understand across different CrownJewel services.

The validation has been added for Teleport and AWS matchers in the service layer to ensure that essential details are not missed when setting matchers. This includes checks to ensure that kinds and labels for Teleport, and types and tags for AWS matchers are properly set.

This commit adds the ability to update a CrownJewel resource within the ResourceCommand. It also includes refactoring of related services, condensed code logic, and improved utilization of existing methods for marshaling/unmarshaling CrownJewel. Specific changes in service.go and crown_jewels.go files have reduced code redundancy and enhanced the code structure.

This commit moves the crown_jewels_test from the local to the services library and adds test cases for unmarshalling Crown Jewels data. Also, import statements for labelv1 and utils were introduced to support these changes.

Revised the crown jewels test file by including AWS matchers section in the CrownJewel object. The change ensures to specifically match AWS infrastructure following certain parameters, thus enhancing the specificity and flexibility of our matchers. Apart from this, minor reordering of objects has also been done for better code organization.

The commit adds the UpsertCrownJewel function in the service.go file, which enables upsert operations on crown jewel resources. Alongside, the UpsertCrownJewelRequest proto message is added to the crownjewel_service.pb.go file to handle such requests.

This change refactors the CrownJewelsClient to use the upsert method when the force flag is on. The code has been streamlined by eliminating the need for 'already exist' error checking and separate create and update methods.

A GetCrownJewel function has been added to the Service, which retrieves a specific CrownJewel resource. Also, the CrownJewel validation process has been refactored, moving the validateCrownJewel function from service.go into the crownjewel library and renaming it to ValidateCrownJewel. These changes improve modularity and code organization.

The checkAccessToKind function now also checks for the VerbCreate permission. Additionally, a new test file, object_test.go, was added to validate the functionality of the CrownJewel object. This includes various validation checks such as "NilCrownJewel", "ValidCrownJewel", "MissingMatchers", among others.

The codebase was updated to improve CrownJewel resources handling. This includes modifications to list, get, create, update, upsert, and delete functions in several files. Additionally, a validation function has been implemented for CrownJewel objects, ensuring required fields are present. Depreciated logger service has also been removed.

The CrownJewel service has been enhanced by integrating it with cache. The main changes include adding "Cache" as part of ServiceConfig and using cache to fetch crown jewels instead of directly using the backend services. The cache interface with the methods "ListCrownJewels" and "GetCrownJewel" has been added to handle these operations.

In this commit, we added the Cache field to the service configuration used in the tests for the crownjewelv1 service. This ensures that the service configuration used in the tests matches the expected real-world configuration.

This commit changes the CrownJewelsService's `UpdateCrownJewel` method to use the `ConditionalUpdateResource` method instead of the `UpdateResource` method. This change will provide a more appropriate update strategy for the specific needs of the CrownJewelsService.

…lCrownJewelsRequest The 'Cache' in crownjewel service has been replaced with 'Reader' to improve code readability and to match with its functionality. Also, the DeleteAllCrownJewelsRequest functionality was removed as it posed a high risk potential for data loss and it wasn

This commit updates the test suite of the Crown Jewel service. An object is fetched from the backend before updating its expiry time, ensuring the revision is populated. In addition, a new test is introduced to handle the scenario where the revision is missing when updating a Crown Jewel.

This commit replaces the usage of the `protojson` method for marshaling and unmarshaling protobuf objects in `crown_jewel.go` with the `FastMarshal` and `FastUnmarshal` methods from `utils`. protojson is failing to unmarshal resources.

Updated the test condition in crown_jewels_test.go to use Proto.Equal for object comparisons. Also, swapped out the FastMarshal and FastUnmarshal methods with protojson's Marshal and Unmarshal in the CrownJewels service.

* Update labels datatype in Teleport crownjewel proto This commit modifies the labels from a map to repeated TeleportLabel in CrownJewel proto, and accordingly updates the getter function in proto generated code. The newly introduced TeleportLabel type contains a Key and repeated values for improved flexibility. Clean up Update and restructure CrownJewel service proto The code modifications include renaming and restructuring several types and altering various functions in the CrownJewel service proto. Major changes involve renaming DeleteCrownJewel Rename 'kind' to 'kinds' in crownjewel API This commit changes the 'kind' field to 'kinds' in the crownjewel API to better reflect its purpose of holding multiple resource types. Changes are made across several files - 'crownjewel.go', 'crownjewel.pb.go' and 'crownjewel.proto'. After rebase fixes Remove CrownJewel type from several files This commit represents the removal of the CrownJewel type from several files. CrownJewel type references are deleted in certain Go files, and imports of CrownJewel in multiple files are being adjusted. The specifics of why this type is removed is not evident from the diffs, but the deletions will affect the related functionalities. Add CrownJewel update functionality This update enables the 'UpdateCrownJewel' functionality. If a 'CrownJewel' resource already exists, it can now be updated instead of returning an error. This enhances flexibility when dealing with resource management. It also introduces the ARN field into the 'AWSMatcher' service. Update CrownJewel service with pagination and improve matchers This update adds pagination to the CrownJewel service for more efficient data handling. It also improves the matching capabilities with the inclusion of Teleport and AWS matchers and allows for selective querying with these matchers. Additionally, it includes various changes to structure and naming for better clarity and ease of use. Refactor CrownJewel protos to use ResourceHeader This refactoring replaces the previous structure of CrownJewel proto's fields Kind, Sub_kind, Version, and Metadata with a single object called Header (ResourceHeader). It simplifies the code structure and harmonizes it with other protos. Now, Header contains the essential attributes that were previously distributed across multiple separate fields. Add GetCrownJewels method to cache and accessgraph The function GetCrownJewels has been added to the cache and the accessgraph. It retrieves all CrownJewel instances from the cache. Additionally, CrownJewel retrieval functionality has been removed from auth server and existing usages have been refactored. Implement Crown Jewel creation feature Updated the method `createCrownJewel` in the resources command to handle Crown Jewel creation. Introduced a new gRPC service `crownjewelv1` to handle client-server interactions for Crown Jewel objects. Crown Jewel expiry configuration has now also been enabled. Add missing files I have no words... Update authentication and event handling Refactored the argument from 'auth.ClientI' to '*auth.Client' in 'createCrownJewel' function in 'resource_command.go'. Removed unused import 'durationpb' in 'auth_with_roles.go'. Added a new event code 'AccessGraphAccessPathChangedCode' in 'codes.go' and 'AccessGraphAccessPathChanged' in 'api.go' for better monitoring and logging of access path changes. Add access path changed event and relevant changes. update api add grpc update api add cache support add protos * Fixes after rebase * Refactor Crown Jewel resource methods Updated API methods related to the Crown Jewel resource to improve clarity and consistency. Renamed `GetCrownJewels` to `ListCrownJewels`, added pagination parameters, and ensured uniform handling of the Crown Jewel resource across different parts of the codebase. Updated relevant comments for better code understanding. * Removed CrownJewel converter, updated CrownJewel references This commit removes the CrownJewel converter and updates references from the old CrownJewel type to the new protobuf-defined CrownJewel type, streamlining the codebase. The commit also disables CrownJewel deletion since it is not supported yet. Some minor errors were fixed regarding CrownJewel resource names. * Implement Crown Jewel deletion functionality The crown jewel deletion feature has been enabled by uncommenting and simplifying the code block. Now, the delete action is directly performed on the specific 'crown jewel' mentioned based on its name. The previously unsupported action returns an informative success message after the deletion. * Refactored function 'allCombinations' and added authorization checks Moved 'allCombinations' function into 'utils' package, renamed it to 'Combinations', and updated its references in 'dbobjectimportrulev1' and 'dbobjectv1' tests. Additionally, introduced admin authorization checks to 'CreateCrownJewels', 'UpdateCrownJewels', and 'DeleteCrownJewels'. Created corresponding tests for these changes. * Add CrownJewel service to cache tests The CrownJewel service has been added to the cache test file. The service creation is implemented and assigned to new fields in multiple methods within the cache tests. A new test function `TestCrownJewel` has also been added to validate CRUD operations for the service. * Add access path changed event and update Crown Jewel tests Added a new event code for when an access path changes in the access graph, and introduced new API calls to handle these situations. Also, updated crown jewel related tests and fixed copyright year in several files. * Update protobuf import path The protobuf import path was updated from "github.com/golang/protobuf/proto" to "google.golang.org/protobuf/proto" to ensure correct use and to prevent potential issues, while maintaining functionality in crown_jewels.go. * Refactor CrownJewel and AccessGraph services This commit optimizes the ListCrownJewelsResponse in the CrownJewel service by directly assigning the response instead of using an extra loop. Additionally, the EventsStream in the AccessGraph service and related tests have been updated to EventsStreamV2, reflecting the implementation of new versions of these methods. * Add CrownJewel resource in access controls and tests In both the cache test and access control files, CrownJewel resource has been included. This provides the system's ability to manage and test CrownJewel resources. Additionally, it allows the handling of CrownJewel resources in client events. * Simplify implementation * Update copyright dates and add license headers Updated the copyright dates in several files to reflect the new year. Additionally, license headers have been added to some files where they were previously missing, ensuring the proper copyrights and GPL notice are visible. * Refactor CrownJewel codebase for better readability Fixed various typographical errors in CrownJewel's related functions and removed a TODO comment. Improved function and function calls' names that were incorrectly referring to different modules, making the code easier to read and understand across different CrownJewel services. * Add validation for Teleport and AWS matchers in service The validation has been added for Teleport and AWS matchers in the service layer to ensure that essential details are not missed when setting matchers. This includes checks to ensure that kinds and labels for Teleport, and types and tags for AWS matchers are properly set. * gci * Add CrownJewel update functionality and refactor related services This commit adds the ability to update a CrownJewel resource within the ResourceCommand. It also includes refactoring of related services, condensed code logic, and improved utilization of existing methods for marshaling/unmarshaling CrownJewel. Specific changes in service.go and crown_jewels.go files have reduced code redundancy and enhanced the code structure. * Relocate and update crown_jewels_test file This commit moves the crown_jewels_test from the local to the services library and adds test cases for unmarshalling Crown Jewels data. Also, import statements for labelv1 and utils were introduced to support these changes. * Update crown jewels test with AWS matchers Revised the crown jewels test file by including AWS matchers section in the CrownJewel object. The change ensures to specifically match AWS infrastructure following certain parameters, thus enhancing the specificity and flexibility of our matchers. Apart from this, minor reordering of objects has also been done for better code organization. * Added upsertCrownJewel function and related proto message The commit adds the UpsertCrownJewel function in the service.go file, which enables upsert operations on crown jewel resources. Alongside, the UpsertCrownJewelRequest proto message is added to the crownjewel_service.pb.go file to handle such requests. * Refactor CrownJewelsClient to use upsert This change refactors the CrownJewelsClient to use the upsert method when the force flag is on. The code has been streamlined by eliminating the need for 'already exist' error checking and separate create and update methods. * Add GetCrownJewel functionality and refactor validation A GetCrownJewel function has been added to the Service, which retrieves a specific CrownJewel resource. Also, the CrownJewel validation process has been refactored, moving the validateCrownJewel function from service.go into the crownjewel library and renaming it to ValidateCrownJewel. These changes improve modularity and code organization. * Add VerbCreate check and test for CrownJewel validation The checkAccessToKind function now also checks for the VerbCreate permission. Additionally, a new test file, object_test.go, was added to validate the functionality of the CrownJewel object. This includes various validation checks such as "NilCrownJewel", "ValidCrownJewel", "MissingMatchers", among others. * Refactor CrownJewel resource handling and validation The codebase was updated to improve CrownJewel resources handling. This includes modifications to list, get, create, update, upsert, and delete functions in several files. Additionally, a validation function has been implemented for CrownJewel objects, ensuring required fields are present. Depreciated logger service has also been removed. * Add cache implementation to CrownJewel service The CrownJewel service has been enhanced by integrating it with cache. The main changes include adding "Cache" as part of ServiceConfig and using cache to fetch crown jewels instead of directly using the backend services. The cache interface with the methods "ListCrownJewels" and "GetCrownJewel" has been added to handle these operations. * Add Cache to service configuration in crownjewelv1 tests In this commit, we added the Cache field to the service configuration used in the tests for the crownjewelv1 service. This ensures that the service configuration used in the tests matches the expected real-world configuration. * Update CrownJewelsService to use ConditionalUpdateResource This commit changes the CrownJewelsService's `UpdateCrownJewel` method to use the `ConditionalUpdateResource` method instead of the `UpdateResource` method. This change will provide a more appropriate update strategy for the specific needs of the CrownJewelsService. * Replace 'Cache' with 'Reader' in crownjewel service & remove DeleteAllCrownJewelsRequest The 'Cache' in crownjewel service has been replaced with 'Reader' to improve code readability and to match with its functionality. Also, the DeleteAllCrownJewelsRequest functionality was removed as it posed a high risk potential for data loss and it wasn * Update tests for Crown Jewel service This commit updates the test suite of the Crown Jewel service. An object is fetched from the backend before updating its expiry time, ensuring the revision is populated. In addition, a new test is introduced to handle the scenario where the revision is missing when updating a Crown Jewel. * Refactor protobuf marshaling with utils This commit replaces the usage of the `protojson` method for marshaling and unmarshaling protobuf objects in `crown_jewel.go` with the `FastMarshal` and `FastUnmarshal` methods from `utils`. protojson is failing to unmarshal resources. * Update CrownJewel test and serialization methods Updated the test condition in crown_jewels_test.go to use Proto.Equal for object comparisons. Also, swapped out the FastMarshal and FastUnmarshal methods with protojson's Marshal and Unmarshal in the CrownJewels service. * Add missing license --------- Co-authored-by: Tiago Silva <[email protected]>

* Add CrownJewel GRPC service and related messages (#40656) * Add CrownJewel GRPC service and related messages This commit introduces the CrownJewel service, along with its corresponding request and response messages. The changes include new proto definitions for creating, getting, updating, and deleting CrownJewels. Functions for pagination are also introduced for the GetCrownJewels method. This new service will provide methods to manage critical resources considered 'CrownJewels' for the organization. * Update TeleportMatcher labels to use TeleportLabel This update changes the previous map data type of 'labels' in TeleportMatcher to use TeleportLabel, allowing for more flexibility. Also, the TeleportLabel message is created with 'key' and 'values' attributes in the api/proto/teleport directory. This new structure complements the changes made in crownjewel.pb.go. * Update label and tag value types in crownjewel.proto and crownjewel.pb.go This commit changes the value types of labels and tags in crownjewel.proto and crownjewel.pb.go. Instead of using google.protobuf.StringValue, the label now uses simple strings for values, and a list of google.protobuf.StringValue for the tag values. * RFD 153 * Rename CrownJewels proto messages Updated protobuf message naming for consistency and clarity. The GetCrownJewelsRequest and GetCrownJewelsResponse were renamed to ListCrownJewelsRequest and ListCrownJewelsResponse respectively. This change better reflects their functionality. * Rename CrownJewels proto messages Updated protobuf message naming for consistency and clarity. The GetCrownJewelsRequest and GetCrownJewelsResponse were renamed to ListCrownJewelsRequest and ListCrownJewelsResponse respectively. This change better reflects their functionality. * Update import and label reference in teleport's crownjewel.proto This commit replaces the TeleportLabel message defined within the crownjewel.proto with an import from label.proto. It refers to labels under the namespace teleport.label.v1, improving modularity and reducing code duplication. * Introduce Access Graph EventsStreamV2 (#40807) * Introduce Access Graph EventsStreamV2 A new GRPC method EventsStreamV2 is introduced. The new method allows returning responses from the Access Graph. The new API is required by crown jewel implementation. The new API will be used to emit audit events from the access graph. * Add new fields to Event protobuf for better tracking Added two new fields, AffectedResourceName and AffectedResourceSource, to the Event protobuf. These additions allow for a more detailed and efficient tracking of changes and their sources, improving overall monitoring and event handling. * Add AccessPathChanged event and update service file A new proto file, events.proto, has been added to define the AccessPathChanged event, detailing changes in access paths. The access_graph_service.proto has been updated to use this new event instead of the legacy one. The relevant changes are also reflected in the generated Go file, access_graph_service.pb.go. * Missing files * Add Crown Jewel resource (#40914) * Update labels datatype in Teleport crownjewel proto This commit modifies the labels from a map to repeated TeleportLabel in CrownJewel proto, and accordingly updates the getter function in proto generated code. The newly introduced TeleportLabel type contains a Key and repeated values for improved flexibility. Clean up Update and restructure CrownJewel service proto The code modifications include renaming and restructuring several types and altering various functions in the CrownJewel service proto. Major changes involve renaming DeleteCrownJewel Rename 'kind' to 'kinds' in crownjewel API This commit changes the 'kind' field to 'kinds' in the crownjewel API to better reflect its purpose of holding multiple resource types. Changes are made across several files - 'crownjewel.go', 'crownjewel.pb.go' and 'crownjewel.proto'. After rebase fixes Remove CrownJewel type from several files This commit represents the removal of the CrownJewel type from several files. CrownJewel type references are deleted in certain Go files, and imports of CrownJewel in multiple files are being adjusted. The specifics of why this type is removed is not evident from the diffs, but the deletions will affect the related functionalities. Add CrownJewel update functionality This update enables the 'UpdateCrownJewel' functionality. If a 'CrownJewel' resource already exists, it can now be updated instead of returning an error. This enhances flexibility when dealing with resource management. It also introduces the ARN field into the 'AWSMatcher' service. Update CrownJewel service with pagination and improve matchers This update adds pagination to the CrownJewel service for more efficient data handling. It also improves the matching capabilities with the inclusion of Teleport and AWS matchers and allows for selective querying with these matchers. Additionally, it includes various changes to structure and naming for better clarity and ease of use. Refactor CrownJewel protos to use ResourceHeader This refactoring replaces the previous structure of CrownJewel proto's fields Kind, Sub_kind, Version, and Metadata with a single object called Header (ResourceHeader). It simplifies the code structure and harmonizes it with other protos. Now, Header contains the essential attributes that were previously distributed across multiple separate fields. Add GetCrownJewels method to cache and accessgraph The function GetCrownJewels has been added to the cache and the accessgraph. It retrieves all CrownJewel instances from the cache. Additionally, CrownJewel retrieval functionality has been removed from auth server and existing usages have been refactored. Implement Crown Jewel creation feature Updated the method `createCrownJewel` in the resources command to handle Crown Jewel creation. Introduced a new gRPC service `crownjewelv1` to handle client-server interactions for Crown Jewel objects. Crown Jewel expiry configuration has now also been enabled. Add missing files I have no words... Update authentication and event handling Refactored the argument from 'auth.ClientI' to '*auth.Client' in 'createCrownJewel' function in 'resource_command.go'. Removed unused import 'durationpb' in 'auth_with_roles.go'. Added a new event code 'AccessGraphAccessPathChangedCode' in 'codes.go' and 'AccessGraphAccessPathChanged' in 'api.go' for better monitoring and logging of access path changes. Add access path changed event and relevant changes. update api add grpc update api add cache support add protos * Fixes after rebase * Refactor Crown Jewel resource methods Updated API methods related to the Crown Jewel resource to improve clarity and consistency. Renamed `GetCrownJewels` to `ListCrownJewels`, added pagination parameters, and ensured uniform handling of the Crown Jewel resource across different parts of the codebase. Updated relevant comments for better code understanding. * Removed CrownJewel converter, updated CrownJewel references This commit removes the CrownJewel converter and updates references from the old CrownJewel type to the new protobuf-defined CrownJewel type, streamlining the codebase. The commit also disables CrownJewel deletion since it is not supported yet. Some minor errors were fixed regarding CrownJewel resource names. * Implement Crown Jewel deletion functionality The crown jewel deletion feature has been enabled by uncommenting and simplifying the code block. Now, the delete action is directly performed on the specific 'crown jewel' mentioned based on its name. The previously unsupported action returns an informative success message after the deletion. * Refactored function 'allCombinations' and added authorization checks Moved 'allCombinations' function into 'utils' package, renamed it to 'Combinations', and updated its references in 'dbobjectimportrulev1' and 'dbobjectv1' tests. Additionally, introduced admin authorization checks to 'CreateCrownJewels', 'UpdateCrownJewels', and 'DeleteCrownJewels'. Created corresponding tests for these changes. * Add CrownJewel service to cache tests The CrownJewel service has been added to the cache test file. The service creation is implemented and assigned to new fields in multiple methods within the cache tests. A new test function `TestCrownJewel` has also been added to validate CRUD operations for the service. * Add access path changed event and update Crown Jewel tests Added a new event code for when an access path changes in the access graph, and introduced new API calls to handle these situations. Also, updated crown jewel related tests and fixed copyright year in several files. * Update protobuf import path The protobuf import path was updated from "github.com/golang/protobuf/proto" to "google.golang.org/protobuf/proto" to ensure correct use and to prevent potential issues, while maintaining functionality in crown_jewels.go. * Refactor CrownJewel and AccessGraph services This commit optimizes the ListCrownJewelsResponse in the CrownJewel service by directly assigning the response instead of using an extra loop. Additionally, the EventsStream in the AccessGraph service and related tests have been updated to EventsStreamV2, reflecting the implementation of new versions of these methods. * Add CrownJewel resource in access controls and tests In both the cache test and access control files, CrownJewel resource has been included. This provides the system's ability to manage and test CrownJewel resources. Additionally, it allows the handling of CrownJewel resources in client events. * Simplify implementation * Update copyright dates and add license headers Updated the copyright dates in several files to reflect the new year. Additionally, license headers have been added to some files where they were previously missing, ensuring the proper copyrights and GPL notice are visible. * Refactor CrownJewel codebase for better readability Fixed various typographical errors in CrownJewel's related functions and removed a TODO comment. Improved function and function calls' names that were incorrectly referring to different modules, making the code easier to read and understand across different CrownJewel services. * Add validation for Teleport and AWS matchers in service The validation has been added for Teleport and AWS matchers in the service layer to ensure that essential details are not missed when setting matchers. This includes checks to ensure that kinds and labels for Teleport, and types and tags for AWS matchers are properly set. * gci * Add CrownJewel update functionality and refactor related services This commit adds the ability to update a CrownJewel resource within the ResourceCommand. It also includes refactoring of related services, condensed code logic, and improved utilization of existing methods for marshaling/unmarshaling CrownJewel. Specific changes in service.go and crown_jewels.go files have reduced code redundancy and enhanced the code structure. * Relocate and update crown_jewels_test file This commit moves the crown_jewels_test from the local to the services library and adds test cases for unmarshalling Crown Jewels data. Also, import statements for labelv1 and utils were introduced to support these changes. * Update crown jewels test with AWS matchers Revised the crown jewels test file by including AWS matchers section in the CrownJewel object. The change ensures to specifically match AWS infrastructure following certain parameters, thus enhancing the specificity and flexibility of our matchers. Apart from this, minor reordering of objects has also been done for better code organization. * Added upsertCrownJewel function and related proto message The commit adds the UpsertCrownJewel function in the service.go file, which enables upsert operations on crown jewel resources. Alongside, the UpsertCrownJewelRequest proto message is added to the crownjewel_service.pb.go file to handle such requests. * Refactor CrownJewelsClient to use upsert This change refactors the CrownJewelsClient to use the upsert method when the force flag is on. The code has been streamlined by eliminating the need for 'already exist' error checking and separate create and update methods. * Add GetCrownJewel functionality and refactor validation A GetCrownJewel function has been added to the Service, which retrieves a specific CrownJewel resource. Also, the CrownJewel validation process has been refactored, moving the validateCrownJewel function from service.go into the crownjewel library and renaming it to ValidateCrownJewel. These changes improve modularity and code organization. * Add VerbCreate check and test for CrownJewel validation The checkAccessToKind function now also checks for the VerbCreate permission. Additionally, a new test file, object_test.go, was added to validate the functionality of the CrownJewel object. This includes various validation checks such as "NilCrownJewel", "ValidCrownJewel", "MissingMatchers", among others. * Refactor CrownJewel resource handling and validation The codebase was updated to improve CrownJewel resources handling. This includes modifications to list, get, create, update, upsert, and delete functions in several files. Additionally, a validation function has been implemented for CrownJewel objects, ensuring required fields are present. Depreciated logger service has also been removed. * Add cache implementation to CrownJewel service The CrownJewel service has been enhanced by integrating it with cache. The main changes include adding "Cache" as part of ServiceConfig and using cache to fetch crown jewels instead of directly using the backend services. The cache interface with the methods "ListCrownJewels" and "GetCrownJewel" has been added to handle these operations. * Add Cache to service configuration in crownjewelv1 tests In this commit, we added the Cache field to the service configuration used in the tests for the crownjewelv1 service. This ensures that the service configuration used in the tests matches the expected real-world configuration. * Update CrownJewelsService to use ConditionalUpdateResource This commit changes the CrownJewelsService's `UpdateCrownJewel` method to use the `ConditionalUpdateResource` method instead of the `UpdateResource` method. This change will provide a more appropriate update strategy for the specific needs of the CrownJewelsService. * Replace 'Cache' with 'Reader' in crownjewel service & remove DeleteAllCrownJewelsRequest The 'Cache' in crownjewel service has been replaced with 'Reader' to improve code readability and to match with its functionality. Also, the DeleteAllCrownJewelsRequest functionality was removed as it posed a high risk potential for data loss and it wasn * Update tests for Crown Jewel service This commit updates the test suite of the Crown Jewel service. An object is fetched from the backend before updating its expiry time, ensuring the revision is populated. In addition, a new test is introduced to handle the scenario where the revision is missing when updating a Crown Jewel. * Refactor protobuf marshaling with utils This commit replaces the usage of the `protojson` method for marshaling and unmarshaling protobuf objects in `crown_jewel.go` with the `FastMarshal` and `FastUnmarshal` methods from `utils`. protojson is failing to unmarshal resources. * Update CrownJewel test and serialization methods Updated the test condition in crown_jewels_test.go to use Proto.Equal for object comparisons. Also, swapped out the FastMarshal and FastUnmarshal methods with protojson's Marshal and Unmarshal in the CrownJewels service. * Add missing license --------- Co-authored-by: Tiago Silva <[email protected]> * Remove double import of crownjewel package. (#41370) * Add "Access Graph Path Changed" event tracking (#42029) * Add "Access Graph Path Changed" event tracking The update introduces support for access path changes. The necessary objects, events, and methods are adjusted accordingly. Icon representation for the new event in the Audit Event List is also added. * Add affected_resource_type field * Update audit event messages and add resource kind Updated the description and formatting of audit events when an access graph path is changed. Additionally, the "affected_resource_kind" field was added to provide more information about the affected resource. * fmt * Update access path change message in audit events The message text for changes to access paths in audit/audit events has been updated. This modification improves the clarity of the description for such events. * Rename AccessGraphAccessPathChanged (#42372) #42029 in Teleport renamed AccessGraphAccessPathChanged to AccessGraphAccessPathChangedEvent. This change fixed the build error caused by that. * Crown Jewel fixes (#42649) * Crown Jewel fixes * Allow passing multiple resource names as crown jewel * Rename requests fields * Add kind validation * GCI * backport testResources153 --------- Co-authored-by: Tiago Silva <[email protected]>

rosstimothy reviewed Apr 25, 2024

View reviewed changes

api/types/crownjewel/crownjewel.go Outdated Show resolved Hide resolved

jakule force-pushed the jakule/crown-jewel-1 branch 2 times, most recently from 777692c to b31c11b Compare April 30, 2024 14:26

jakule marked this pull request as ready for review May 1, 2024 03:56

github-actions bot added audit-log Issues related to Teleports Audit Log size/lg tctl tctl - Teleport admin tool labels May 1, 2024

github-actions bot requested review from gzdunek and rosstimothy May 1, 2024 03:56

rosstimothy reviewed May 1, 2024

View reviewed changes

jakule force-pushed the jakule/crown-jewel-1 branch 2 times, most recently from f8198f8 to b4202ca Compare May 2, 2024 17:06

rosstimothy self-requested a review May 2, 2024 17:26

rosstimothy reviewed May 2, 2024

View reviewed changes

jakule requested a review from rosstimothy May 2, 2024 21:11

rosstimothy approved these changes May 2, 2024

View reviewed changes

jakule force-pushed the jakule/crown-jewel-1 branch from f4751d4 to d424655 Compare May 3, 2024 14:17

tigrato approved these changes May 3, 2024

View reviewed changes

espadolini approved these changes May 3, 2024

View reviewed changes

jakule force-pushed the jakule/crown-jewel-1 branch from 8ac3bee to 80bf335 Compare May 6, 2024 20:02

r0mant approved these changes May 6, 2024

View reviewed changes

public-teleport-github-review-bot bot removed the request for review from gzdunek May 6, 2024 20:10

tigrato and others added 5 commits May 7, 2024 11:22

Fixes after rebase

8594cf6

jakule added 21 commits May 7, 2024 11:22

Simplify implementation

5be31f7

Update copyright dates and add license headers

634fe16

Updated the copyright dates in several files to reflect the new year. Additionally, license headers have been added to some files where they were previously missing, ensuring the proper copyrights and GPL notice are visible.

gci

bbe1ab8

Relocate and update crown_jewels_test file

9eafcfd

This commit moves the crown_jewels_test from the local to the services library and adds test cases for unmarshalling Crown Jewels data. Also, import statements for labelv1 and utils were introduced to support these changes.

Refactor CrownJewelsClient to use upsert

651c9ee

This change refactors the CrownJewelsClient to use the upsert method when the force flag is on. The code has been streamlined by eliminating the need for 'already exist' error checking and separate create and update methods.

Add Cache to service configuration in crownjewelv1 tests

d78c7cb

In this commit, we added the Cache field to the service configuration used in the tests for the crownjewelv1 service. This ensures that the service configuration used in the tests matches the expected real-world configuration.

Refactor protobuf marshaling with utils

0cc7e5e

This commit replaces the usage of the `protojson` method for marshaling and unmarshaling protobuf objects in `crown_jewel.go` with the `FastMarshal` and `FastUnmarshal` methods from `utils`. protojson is failing to unmarshal resources.

Update CrownJewel test and serialization methods

4c47ce3

Updated the test condition in crown_jewels_test.go to use Proto.Equal for object comparisons. Also, swapped out the FastMarshal and FastUnmarshal methods with protojson's Marshal and Unmarshal in the CrownJewels service.

jakule force-pushed the jakule/crown-jewel-1 branch from c5eae56 to 4c47ce3 Compare May 7, 2024 15:22

Add missing license

f292e04

jakule added this pull request to the merge queue May 7, 2024

Merged via the queue into master with commit bc53a82 May 7, 2024
42 checks passed

jakule deleted the jakule/crown-jewel-1 branch May 7, 2024 16:05

jakule mentioned this pull request Jun 13, 2024

[v15] Crown jewel #42866

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Crown Jewel resource #40914

Add Crown Jewel resource #40914

jakule commented Apr 25, 2024 •

edited

Loading

github-actions bot commented May 1, 2024

rosstimothy May 1, 2024

jakule May 1, 2024

rosstimothy May 2, 2024

jakule May 2, 2024

rosstimothy May 1, 2024

jakule May 2, 2024

rosstimothy May 2, 2024

rosstimothy May 2, 2024

jakule May 2, 2024

jakule May 3, 2024 •

edited

Loading

public-teleport-github-review-bot bot commented May 2, 2024

rosstimothy May 2, 2024

jakule May 2, 2024

rosstimothy May 2, 2024

rosstimothy May 2, 2024

espadolini May 3, 2024

espadolini May 3, 2024

espadolini May 3, 2024

espadolini May 3, 2024

espadolini May 3, 2024

espadolini May 3, 2024

espadolini May 3, 2024

jakule May 6, 2024

espadolini May 3, 2024

jakule May 6, 2024

r0mant left a comment

Add Crown Jewel resource #40914

Add Crown Jewel resource #40914

Conversation

jakule commented Apr 25, 2024 • edited Loading

github-actions bot commented May 1, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jakule May 3, 2024 • edited Loading

Choose a reason for hiding this comment

public-teleport-github-review-bot bot commented May 2, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

r0mant left a comment

Choose a reason for hiding this comment

jakule commented Apr 25, 2024 •

edited

Loading

jakule May 3, 2024 •

edited

Loading