add a create/update/changeId/delete posthook (#193) #242

davidcoutadeur · 2023-10-23T14:40:41Z

No description provided.

davidcoutadeur · 2023-10-23T14:46:13Z

@soisik or @rouazana if you have some time, could you review this PR?

soisik · 2023-10-25T09:55:14Z

@davidcoutadeur please rebase your PR since we have merged some important refactoring commits since you have sent this.

davidcoutadeur · 2023-10-25T09:56:27Z

@davidcoutadeur please rebase your PR since we have merged some important refactoring commits since you have sent this.

Yes, I am working on this! Coming soon.

davidcoutadeur · 2023-10-25T10:11:31Z

Done

davidcoutadeur · 2023-10-25T10:13:09Z

The test is missing, I am working on it.

davidcoutadeur · 2023-10-25T16:01:36Z

I have just added the test. Everything seems fine.

rouazana · 2023-10-25T19:29:30Z

I have some little remarks about the code itself, but nothing very important.

Before that I would like to understand the purpose. I understand that you want to be able to launch a process after each modification/creation/deletion.
My questions are:

Why are you using json as an argument to pass the modifications? Given how LSC works, and for consistency, I would more have expected to use LDIF, no?
Also it seems strange to have a command argument which can be pretty big (think of changing the members of a group for example), so why not using standard input for that instead?

davidcoutadeur · 2023-10-26T09:15:20Z

Hello

I have some little remarks about the code itself, but nothing very important.

Before that I would like to understand the purpose. I understand that you want to be able to launch a process after each modification/creation/deletion. My questions are:
* Why are you using json as an argument to pass the modifications? Given how LSC works, and for consistency, I would more have expected to use LDIF, no?

It is a topic open to discussion indeed.

The main reason is that I'd like to keep the generic side of the LSC object LscModifications. If later we have a connector to another protocol, this would permit to have a generic modification object sent in the script. In this context, JSON seemed a simple way to sent this modification object.

* Also it seems strange to have a command argument which can be pretty big (think of changing the members of a group for example), so why not using standard input for that instead?

Good point indeed. I'll do the modification to use the standard input instead.

rouazana · 2023-10-26T10:23:09Z

The main reason is that I'd like to keep the generic side of the LSC object LscModifications. If later we have a connector to another protocol, this would permit to have a generic modification object sent in the script. In this context, JSON seemed a simple way to sent this modification object.

LscModifications will always be the object to pass the modifications, whatever the protocol is used in the services.
I think the LDIF format is nearer from LscModifications than JSON, hence my suggestion to use LDIF.

soisik · 2023-10-26T12:46:59Z

LscModifications will always be the object to pass the modifications, whatever the protocol is used in the services. I think the LDIF format is nearer from LscModifications than json, hence my suggestion to use ldif.

Sorry to step in, but we could have both, adding an option in LSC configuration to choose the input format for hooks ?
It would also be nice a feature to add to executable plugin - which use ldif as input format - to choose between ldif or json as these scripts are often calling web services, and handling data in json is easier than handling ldif for many developers nowadays.

rouazana · 2023-10-26T13:18:39Z

Yes it could be an interesting option!

Then did you check how binary attributes are encoded in JSON?

…guments (#193)

davidcoutadeur · 2023-10-26T16:51:24Z

I have pushed a new commit with:

new option "outputFormat" for the LSC modifications ("ldif" or "json")
LSC is now pushing LSC modifications as input and not argument anymore

davidcoutadeur · 2023-10-26T16:56:59Z

Then did you check how binary attributes are encoded in JSON?

I have made a check with jpegPhoto, which seems to be passed as base64 in LSC modification (and so in JSON)

davidcoutadeur · 2023-10-26T17:15:59Z

Also tested with userCertificate;binary attribute

The data seems encoded in base64 as well:

[ {
  "attributeName" : "userCertificate;binary",
  "values" : [ "MIIDlzCCAn+..." ],
  "operation" : "REPLACE_VALUES"
} ]

rouazana

I left several style comments, but this PR is pretty good!

I like the functionality and the implementation is pretty nice and straightforward, good job!

rouazana · 2023-10-26T22:04:13Z

src/main/java/org/lsc/Hooks.java

@@ -63,31 +64,56 @@ public class Hooks {
 	 *
 	 * return nothing
 	 */
-	public final static void postSyncHook(final String hook, final LscModifications lm) {
+	public final static void postSyncHook(final String hook, final String outputFormat, final LscModifications lm) {


style: in general in Java we don't use static methods. You can construct the object in the constructor of the caller.

rouazana · 2023-10-26T22:05:02Z

src/main/java/org/lsc/Hooks.java

-			String jsonModifications = null;
+
+			String format = "";
+			if( outputFormat.equals("json") ) {


the parsing of the outputFormat should be done previously, here you should be able to manipulate only an Enum

rouazana · 2023-10-26T22:06:36Z

src/main/java/org/lsc/Hooks.java

+					if( format.equals("json") ) {
+						modifications = getJsonModifications(lm);
+					}
+					else {
+						modifications = LdifLayout.format(lm);
+					}
+					callHook("create", hook, lm.getMainIdentifier(), format, modifications);


these lines can be easily factorized, and in this case easily integrated in the callHook method

rouazana · 2023-10-26T22:07:12Z

src/main/java/org/lsc/Hooks.java

+	* Method calling the hook
+	*
+	* return nothing
+	*/


I'm not sure such comments are really useful

rouazana · 2023-10-26T22:08:14Z

src/main/java/org/lsc/beans/syncoptions/PropertiesBasedSyncOptions.java

@@ -197,6 +197,13 @@ public String getCondition(LscModificationType operation) {
 		return result;
 	}

+	public String getPostHookOutputFormat() {
+		if (conf.getHooks() == null || conf.getHooks().getOutputFormat() == null) {
+			return "";


As said earlier, the result of this method should be an enum.
Also, I would make clearer here which is the default value.

rouazana · 2023-10-26T22:14:50Z

src/main/java/org/lsc/beans/syncoptions/PropertiesBasedSyncOptions.java

+
+	public String getCreatePostHook() {
+		if (conf.getHooks() == null || conf.getHooks().getCreatePostHook() == null) {
+			return "";


style: I usually prefer returning an Optional to indicate there is a value or not.

But here I did not check what are the consequences, it can be a little more complicated to handle if you are not used to.

rouazana · 2023-10-26T22:16:03Z

src/test/java/org/lsc/Ldap2LdapHookSyncTest.java

+		reloadJndiConnections();
+
+		ObjectMapper mapperCreatedEntry = new ObjectMapper();
+		JsonNode expectedCreatedEntry = mapperCreatedEntry.readTree("[ { \"attributeName\" : \"objectClass\", \"values\" : [ \"inetOrgPerson\", \"person\", \"top\" ], \"operation\" : \"ADD_VALUES\" }, { \"attributeName\" : \"cn\", \"values\" : [ \"CN0001-hook\" ], \"operation\" : \"ADD_VALUES\" }, { \"attributeName\" : \"sn\", \"values\" : [ \"CN0001-hook\" ], \"operation\" : \"ADD_VALUES\" } ]");


could you add a binary attribute to your test? It will allow to check for regression regarding the handling of binary attributes in hooks.

rouazana · 2023-10-26T22:16:54Z

src/test/java/org/lsc/Ldap2LdapHookSyncTest.java

+			hookReader.close();
+			hookFile.delete(); // remove hook log
+		} catch (Exception e) {
+			fail("Error while reading hook-ldif-" + operation + ".log");


you don't really need to try/catch in test, the test will fail anyway in case of unexpected exception.

rouazana · 2023-10-26T22:17:09Z

src/test/java/org/lsc/Ldap2LdapHookSyncTest.java

+			hookReader.close();
+			hookFile.delete(); // remove hook log
+		} catch (Exception e) {
+			fail("Error while reading hook-json-" + operation + ".log");


rouazana · 2023-10-26T22:17:19Z

src/test/java/org/lsc/Ldap2LdapHookSyncTest.java

+				JsonNode hookOperation = mapper.readTree(jp);
+				assertEquals(hookOperation, expectedEntry);
+			}
+			catch (Exception e) {


davidcoutadeur · 2023-10-27T15:18:37Z

Hello,

I have taken all remarks in consideration, and made a new commit 68cd544

3 points remaining:

about returning an Option attribute for getCreatePostHook: I have made a fix, but I am not sure it's what you expected @rouazana. Could you give a look?
~~about the binary attribute, it's a good idea. For now I am stuck because I can't figure out how to add a custom binary attribute in a LSC task. (see below)~~
about try / catch in the test file, I knew it wasn't necessary, but I preferred this custom output. (it is a more simple and clearer message for the developper)

The naive way to add a custom binary attribute: (does not work because atob unavailability)

        <dataset>
          <name>userCertificate;binary</name>
          <policy>FORCE</policy>
          <forceValues>
            <string>js:atob("MIIDkTCCAnmgAwIBAgIUDhx/9qofTrT+yNFFvihdDn7rjOQwDQYJKoZIhvcNAQELBQAwWDELMAkGA1UEBhMCRlIxDTALBgNVBAgMBHRlc3QxDTALBgNVBAcMBHRlc3QxDTALBgNVBAoMBHRlc3QxDTALBgNVBAsMBHRlc3QxDTALBgNVBAMMBHRlc3QwHhcNMjMxMDI3MTQzMDQxWhcNMzMxMDI0MTQzMDQxWjBYMQswCQYDVQQGEwJGUjENMAsGA1UECAwEdGVzdDENMAsGA1UEBwwEdGVzdDENMAsGA1UECgwEdGVzdDENMAsGA1UECwwEdGVzdDENMAsGA1UEAwwEdGVzdDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKdWt0QgFnEi7a1hIJQv4ZdOM5y0GGLHQYNrUNSReArvpkYUY5zasFNVzVHCApRuj0t1NMDrn1gNzKkxTIbYGaWRSn+21J0ow+Nxh2TAQW8dkJnWTksCfyGGGItI5q3ST3EUKnepaAzUYYENcSHRyx7UY/3XuzcW0aGhy4PrVTIHBpyLq0Uzv8nH5nbWM+LYt6YbQMmlAz/psTXIC2dfEZhUb4plLGSo7rZxM5geC6Z+os+I8+uw+mGjps1VP7eGq0jCGHNs2rUHMqBNgLvwMH2WlMXo/iNarAb8fUEPdp59FwiTygBlWAn6GoKHJ1HWPpqMxdtjL2Y5+ZMcp70eJqcCAwEAAaNTMFEwHQYDVR0OBBYEFLwffjUBL/Rp4a6MgeCJiFnCZFu8MB8GA1UdIwQYMBaAFLwffjUBL/Rp4a6MgeCJiFnCZFu8MA8GA1UdEwEB/wQFMAMBAf8wDQYJKoZIhvcNAQELBQADggEBACjwsg1Z9PyauoKAhkIfyPTEnlEOCo1nN37c2vnH4fyY6fuBdV6GWtk/u9FCuDmYT/4KDRxe33ZUChwSUX0INgamOarWRES3UoPC1GeOvuMf7uustEMLcHAYZVKXSZUrsOjw+VIZ5XrD6GDE64QtvW5Ve3jf43aGgLf27NF0vhF9+gHOZjjBT33S977HUutMUKfRu9PdHAn8Yb1FmSbAvqqK+SAjn6cJC8l5yS5t0BSNQGbKSA8bPzvWI9HXYVvb+ym6GDrsr+Zad3NrqUSZGzS2JFEDVD9aAikldXu6g02fA5A7nufVePmaG7iTyylO/ZU2lTiJ0SHc2DnO0pg2i+0=")</string>
          </forceValues>
        </dataset>

davidcoutadeur · 2023-10-27T18:34:20Z

test with the binary attribute is now done in d9271d4 thanks to @soisik advice

rouazana

Sorry again a bunch of comments, but it starts to look really nice!

rouazana · 2023-10-27T21:26:15Z

src/main/java/org/lsc/Hooks.java


-		if( hook != null && ! hook.equals("") )
+		if( hook != null && hook.isPresent() )


It looks better.
The principle of an optional is to not be null, so you can remove the null check.

Also if instead of a String operationType you continue using LscModificationType, the code can be very simplified by:

hook.ifPresent(h -> callHook(lm.getOperation(), h, lm.getMainIdentifier(), outputFormat, lm)

rouazana · 2023-10-27T21:28:37Z

src/main/java/org/lsc/Hooks.java

+			LOGGER.error("Error while encoding LSC modifications to json");
+			LOGGER.error("encoding error: ", e);


Suggested change

LOGGER.error("Error while encoding LSC modifications to json");

LOGGER.error("encoding error: ", e);

LOGGER.error("Error while encoding LSC modifications to json", e);

rouazana · 2023-10-27T21:31:02Z

src/main/java/org/lsc/Hooks.java

+			LOGGER.error("Error while calling {} posthook {} with format {} for {}",
+					operationType,
+					hook,
+					outputFormat.toString(),
+					identifier);
+			LOGGER.error("posthook error: ", e);


Suggested change

LOGGER.error("Error while calling {} posthook {} with format {} for {}",

operationType,

hook,

outputFormat.toString(),

identifier);

LOGGER.error("posthook error: ", e);

LOGGER.error("Error while calling {} posthook {} with format {} for {}",

operationType,

hook,

outputFormat.toString(),

identifier,

e);

should work also

rouazana · 2023-10-27T21:31:33Z

src/main/java/org/lsc/beans/syncoptions/ForceSyncOptions.java

-	public String getCreatePostHook() {
-		return "";
+	public Optional<String> getCreatePostHook() {
+		return Optional.ofNullable(null);


Suggested change

return Optional.ofNullable(null);

return Optional.empty();

rouazana · 2023-10-27T21:31:45Z

src/main/java/org/lsc/beans/syncoptions/ForceSyncOptions.java

-	public String getDeletePostHook() {
-		return "";
+	public Optional<String> getDeletePostHook() {
+		return Optional.ofNullable(null);


Suggested change

return Optional.ofNullable(null);

return Optional.empty();

rouazana · 2023-10-27T21:32:20Z

src/main/java/org/lsc/beans/syncoptions/ForceSyncOptions.java

-	public String getPostHook(LscModificationType operation) {
-		return "";
+	public Optional<String> getPostHook(LscModificationType operation) {
+		return Optional.ofNullable(null);


Suggested change

return Optional.ofNullable(null);

return Optional.empty();

rouazana · 2023-10-27T21:34:48Z

src/main/java/org/lsc/beans/syncoptions/PropertiesBasedSyncOptions.java

-	public String getPostHook(LscModificationType operation) {
-		String result = "";
+	public Optional<String> getPostHook(LscModificationType operation) {
+		Optional<String> result = Optional.ofNullable(null);


you can simplify this method by removing the result method and returning directly into each case

rouazana · 2023-10-27T21:36:48Z

src/main/java/org/lsc/runnable/CleanEntryRunner.java

-					Hooks.postSyncHook(hook, outputFormat, lm);
+					Optional<String> hook = syncOptions.getDeletePostHook();
+					OutputFormat outputFormat = syncOptions.getPostHookOutputFormat();
+					Hooks hookObject = new Hooks();


hookObject (you can name it hooks) should be initialized in the constructor.

rouazana · 2023-10-27T21:37:10Z

src/main/java/org/lsc/runnable/SynchronizeEntryRunner.java

-				Hooks.postSyncHook(hook, outputFormat, lm);
+				Optional<String> hook = task.getSyncOptions().getPostHook(modificationType);
+				OutputFormat outputFormat = task.getSyncOptions().getPostHookOutputFormat();
+				Hooks hookObject = new Hooks();


rouazana · 2023-10-27T21:37:38Z

src/main/java/org/lsc/runnable/SynchronizeEntryRunner.java

+				Optional<String> hook = task.getSyncOptions().getPostHook(modificationType);
+				OutputFormat outputFormat = task.getSyncOptions().getPostHookOutputFormat();
+				Hooks hookObject = new Hooks();
+				hookObject.postSyncHook(hook, outputFormat, lm);


also I think you can inline the parameters

davidcoutadeur · 2023-10-30T10:33:22Z

commit 45d8a32 is fixing the last bunch of comments.

…#193) (#242)

davidcoutadeur linked an issue Oct 23, 2023 that may be closed by this pull request

Hook / Trigger system for add/delete/modify/modrdn #193

Closed

coudot requested review from rouazana and soisik October 23, 2023 15:32

coudot added this to the 2.2 milestone Oct 23, 2023

davidcoutadeur force-pushed the 193-hook branch from f01fefd to 59c91f6 Compare October 25, 2023 10:11

add a create/update/changeId/delete posthook (#193)

6d51dfe

davidcoutadeur force-pushed the 193-hook branch from 59c91f6 to 6d51dfe Compare October 25, 2023 16:00

add option outputFormat + send lsc modifications as inputs and not ar…

2f2ecb6

…guments (#193)

rouazana reviewed Oct 26, 2023

View reviewed changes

fix remarks in #242 review about hooks (#193)

68cd544

add a hook test with a binary attribute (#193)

d9271d4

rouazana reviewed Oct 27, 2023

View reviewed changes

fix the last bunch of remarks about hooks (#193 #242)

45d8a32

rouazana approved these changes Nov 1, 2023

View reviewed changes

davidcoutadeur merged commit 505580e into master Dec 11, 2023
1 check failed

davidcoutadeur pushed a commit that referenced this pull request Dec 11, 2023

fix test regression brought by create/update/changeId/delete posthook (…

62eda9f

…#193) (#242)

davidcoutadeur deleted the 193-hook branch December 11, 2023 16:43


		if( hook != null && ! hook.equals("") )
		if( hook != null && hook.isPresent() )

		LOGGER.error("Error while encoding LSC modifications to json");
		LOGGER.error("encoding error: ", e);

add a create/update/changeId/delete posthook (#193) #242

add a create/update/changeId/delete posthook (#193) #242

Conversation

davidcoutadeur commented Oct 23, 2023

davidcoutadeur commented Oct 23, 2023

soisik commented Oct 25, 2023

davidcoutadeur commented Oct 25, 2023

davidcoutadeur commented Oct 25, 2023

davidcoutadeur commented Oct 25, 2023

davidcoutadeur commented Oct 25, 2023

rouazana commented Oct 25, 2023

davidcoutadeur commented Oct 26, 2023

rouazana commented Oct 26, 2023

soisik commented Oct 26, 2023 • edited Loading

rouazana commented Oct 26, 2023

davidcoutadeur commented Oct 26, 2023

davidcoutadeur commented Oct 26, 2023

davidcoutadeur commented Oct 26, 2023

rouazana left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

davidcoutadeur commented Oct 27, 2023 • edited Loading

davidcoutadeur commented Oct 27, 2023

rouazana left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

davidcoutadeur commented Oct 30, 2023

soisik commented Oct 26, 2023 •

edited

Loading

davidcoutadeur commented Oct 27, 2023 •

edited

Loading