Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(v2): Split powertools-idempotency module to sub-modules and add redis implementation #1513

Closed
wants to merge 33 commits into from
Closed
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
0ec7f4a
Introducing sub-modules in the idempotency module
eldimi Nov 8, 2023
575e7a9
Updating the idempotency documentation
eldimi Nov 8, 2023
6382089
Move test resource to the dynamodb idempotency sub-module
eldimi Nov 9, 2023
914359d
Renaming common module to core and refactoring package name for repo …
eldimi Nov 9, 2023
336cab3
fix spotbugs for multiple modules and new pr_artifacts
eldimi Nov 9, 2023
dca1f0e
fix spotbugs for multiple modules
eldimi Nov 9, 2023
d71f9ac
fix spotbugs for multiple modules - with pom property
eldimi Nov 9, 2023
a9ef2c4
fix spotbugs for multiple modules - multiple spotbugs files
eldimi Nov 9, 2023
c017948
Try fixing java8 build
eldimi Nov 10, 2023
bcbd956
fix aspectj weaving
jeromevdl Nov 13, 2023
9f44ebf
Clean-up pom dependencies and remove awssdk#StringUtils usage
eldimi Nov 14, 2023
6da6aaf
Fix sonar issues
eldimi Nov 14, 2023
e144be5
Adding redis implementation for idempotency
eldimi Nov 24, 2023
30722ba
Fix string replacement and add unit test
eldimi Nov 24, 2023
7ff5708
Address sonar findings
eldimi Nov 24, 2023
d2e4efa
E2E test for idempotency redis implementation
eldimi Nov 28, 2023
d00b5e2
Adding instructions to bootstrap cdk in e2e README file
eldimi Nov 28, 2023
b3e3d7d
Add documentation for redis idempotency
eldimi Nov 28, 2023
0501b8f
Add support for Redis Cluster, improve documentation and e2e tests
eldimi Dec 8, 2023
0582f24
docs improvements - Apply suggestions from code review
eldimi Dec 13, 2023
fce92df
Apply checkstyle to fix import order
eldimi Dec 13, 2023
0396be8
Merge branch 'chore/v2-split-idempotency-mod' of github.com:aws-power…
eldimi Dec 13, 2023
1a47aef
Merge branch 'v2' into chore/v2-split-idempotency-mod
eldimi Dec 19, 2023
df7452f
Fix build
eldimi Dec 19, 2023
4975dc9
Fix spotbugs target and build
eldimi Dec 19, 2023
65318c9
Fix e2e build for Java8
eldimi Dec 19, 2023
6cce077
Fix e2e tests build
eldimi Dec 19, 2023
0c1cc38
Exposes JedisClientConfig to allow custom config (ssl, db, etc..), im…
eldimi Dec 27, 2023
796029d
Add database config hint in the documentation
eldimi Dec 27, 2023
4d2c716
Address spotbugs
eldimi Dec 27, 2023
81417e1
Include memoryDB support in the documentation
eldimi Dec 27, 2023
a200e64
Apply suggestions from code review
eldimi Jan 4, 2024
a82e4cd
Minor enhancements
eldimi Jan 22, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .github/workflows/pr_artifacts_size.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,8 @@ on:
- 'powertools-core/**' # not in v2
- 'powertools-common/**' # v2 only
- 'powertools-e2e-tests/**'
- 'powertools-idempotency/**'
- 'powertools-idempotency-core/**'
- 'powertools-idempotency-dynamodb/**'
- 'powertools-large-messages/**'
- 'powertools-logging/**'
- 'powertools-metrics/**'
Expand Down
12 changes: 6 additions & 6 deletions docs/utilities/idempotency.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ Depending on your version of Java (either Java 1.8 or 11+), the configuration sl
...
<dependency>
<groupId>software.amazon.lambda</groupId>
<artifactId>powertools-idempotency</artifactId>
<artifactId>powertools-idempotency-dynamodb</artifactId>
<version>{{ powertools.version }}</version>
</dependency>
...
Expand All @@ -56,7 +56,7 @@ Depending on your version of Java (either Java 1.8 or 11+), the configuration sl
<aspectLibraries>
<aspectLibrary>
<groupId>software.amazon.lambda</groupId>
<artifactId>powertools-idempotency</artifactId>
<artifactId>powertools-idempotency-dynamodb</artifactId>
</aspectLibrary>
</aspectLibraries>
</configuration>
Expand All @@ -80,7 +80,7 @@ Depending on your version of Java (either Java 1.8 or 11+), the configuration sl
...
<dependency>
<groupId>software.amazon.lambda</groupId>
<artifactId>powertools-idempotency</artifactId>
<artifactId>powertools-idempotency-dynamodb</artifactId>
<version>{{ powertools.version }}</version>
</dependency>
...
Expand All @@ -101,7 +101,7 @@ Depending on your version of Java (either Java 1.8 or 11+), the configuration sl
<aspectLibraries>
<aspectLibrary>
<groupId>software.amazon.lambda</groupId>
<artifactId>powertools-idempotency</artifactId>
<artifactId>powertools-idempotency-dynamodb</artifactId>
</aspectLibrary>
</aspectLibraries>
</configuration>
Expand Down Expand Up @@ -131,7 +131,7 @@ Depending on your version of Java (either Java 1.8 or 11+), the configuration sl
}

dependencies {
aspect 'software.amazon.lambda:powertools-idempotency:{{ powertools.version }}'
aspect 'software.amazon.lambda:powertools-idempotency-dynamodb:{{ powertools.version }}'
}

sourceCompatibility = 11 // or higher
Expand All @@ -151,7 +151,7 @@ Depending on your version of Java (either Java 1.8 or 11+), the configuration sl
}

dependencies {
aspect 'software.amazon.lambda:powertools-idempotency:{{ powertools.version }}'
aspect 'software.amazon.lambda:powertools-idempotency-dynamodb:{{ powertools.version }}'
}

sourceCompatibility = 1.8
Expand Down
4 changes: 2 additions & 2 deletions examples/powertools-examples-idempotency/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@
</dependency>
<dependency>
<groupId>software.amazon.lambda</groupId>
<artifactId>powertools-idempotency</artifactId>
<artifactId>powertools-idempotency-dynamodb</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
Expand Down Expand Up @@ -117,7 +117,7 @@
</aspectLibrary>
<aspectLibrary>
<groupId>software.amazon.lambda</groupId>
<artifactId>powertools-idempotency</artifactId>
<artifactId>powertools-idempotency-dynamodb</artifactId>
</aspectLibrary>
</aspectLibraries>
</configuration>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
import software.amazon.lambda.powertools.idempotency.Idempotency;
import software.amazon.lambda.powertools.idempotency.IdempotencyConfig;
import software.amazon.lambda.powertools.idempotency.Idempotent;
import software.amazon.lambda.powertools.idempotency.persistence.DynamoDBPersistenceStore;
import software.amazon.lambda.powertools.idempotency.persistence.dynamodb.DynamoDBPersistenceStore;
import software.amazon.lambda.powertools.logging.Logging;
import software.amazon.lambda.powertools.utilities.JsonConfig;

Expand Down
1 change: 0 additions & 1 deletion pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -381,7 +381,6 @@
</configuration>
<executions>
<execution>
<phase>process-sources</phase>
<goals>
<goal>compile</goal>
<goal>test-compile</goal>
Expand Down
1 change: 1 addition & 0 deletions powertools-batch/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
</build>

<artifactId>powertools-batch</artifactId>

<dependencies>
<dependency>
<groupId>com.amazonaws</groupId>
Expand Down
91 changes: 6 additions & 85 deletions powertools-idempotency/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -25,68 +25,24 @@
</parent>

<artifactId>powertools-idempotency</artifactId>
<packaging>jar</packaging>
<packaging>pom</packaging>

<name>Powertools for AWS Lambda (Java) library Idempotency</name>
<description>

</description>
<url>https://aws.amazon.com/lambda/</url>
<issueManagement>
<system>GitHub Issues</system>
<url>https://github.com/aws-powertools/powertools-lambda-java/issues</url>
</issueManagement>
<scm>
<url>https://github.com/aws-powertools/powertools-lambda-java.git</url>
</scm>
<developers>
<developer>
<name>Powertools for AWS Lambda team</name>
<organization>Amazon Web Services</organization>
<organizationUrl>https://aws.amazon.com/</organizationUrl>
</developer>
</developers>

<distributionManagement>
<snapshotRepository>
<id>ossrh</id>
<url>https://aws.oss.sonatype.org/content/repositories/snapshots</url>
</snapshotRepository>
</distributionManagement>

<modules>
<module>powertools-idempotency-core</module>
<module>powertools-idempotency-dynamodb</module>
</modules>

<dependencies>
<dependency>
<groupId>software.amazon.lambda</groupId>
<artifactId>powertools-common</artifactId>
</dependency>
<dependency>
<groupId>software.amazon.lambda</groupId>
<artifactId>powertools-serialization</artifactId>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-lambda-java-core</artifactId>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>dynamodb</artifactId>
<exclusions>
<exclusion>
<groupId>software.amazon.awssdk</groupId>
<artifactId>netty-nio-client</artifactId>
</exclusion>
<exclusion>
<groupId>software.amazon.awssdk</groupId>
<artifactId>apache-client</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>url-connection-client</artifactId>
<version>${aws.sdk.version}</version>
</dependency>

<!-- Test dependencies -->
<dependency>
<groupId>org.junit.jupiter</groupId>
Expand All @@ -103,11 +59,6 @@
<artifactId>junit-pioneer</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.aspectj</groupId>
<artifactId>aspectjweaver</artifactId>
Expand All @@ -128,20 +79,6 @@
<artifactId>aws-lambda-java-tests</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>DynamoDBLocal</artifactId>
<version>[1.12,2.0)</version>
<scope>test</scope>
</dependency>
<!--Needed when building locally on M1 Mac-->
<dependency>
<groupId>io.github.ganadist.sqlite4java</groupId>
<artifactId>libsqlite4java-osx-aarch64</artifactId>
<version>1.0.392</version>
<scope>test</scope>
<type>dylib</type>
</dependency>
</dependencies>
<build>
<plugins>
Expand Down Expand Up @@ -172,22 +109,6 @@
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>3.3.0</version>
<configuration>
<archive>
<manifestEntries>
<Automatic-Module-Name>software.amazon.awssdk.enhanced.dynamodb</Automatic-Module-Name>
</manifestEntries>
</archive>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-checkstyle-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
42 changes: 42 additions & 0 deletions powertools-idempotency/powertools-idempotency-core/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
~ Copyright 2023 Amazon.com, Inc. or its affiliates.
~ Licensed under the Apache License, Version 2.0 (the
~ "License"); you may not use this file except in compliance
~ with the License. You may obtain a copy of the License at
~ http://www.apache.org/licenses/LICENSE-2.0
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
~
-->

<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://maven.apache.org/POM/4.0.0"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<parent>
<groupId>software.amazon.lambda</groupId>
<artifactId>powertools-idempotency</artifactId>
<version>2.0.0-SNAPSHOT</version>
</parent>

<artifactId>powertools-idempotency-core</artifactId>
<packaging>jar</packaging>

<name>Powertools for AWS Lambda (Java) library Idempotency - Core</name>
<description>
Idempotency module common implementation
</description>

<dependencies>
<dependency>
<groupId>software.amazon.lambda</groupId>
<artifactId>powertools-serialization</artifactId>
</dependency>
<!-- Test dependencies -->
</dependencies>
</project>
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,10 @@
package software.amazon.lambda.powertools.idempotency;

import com.amazonaws.services.lambda.runtime.Context;
import java.time.Duration;
import software.amazon.lambda.powertools.idempotency.internal.cache.LRUCache;

import java.time.Duration;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you don't have checkstyle applied.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should be fixed now. I think it is because we only format uncommitted and if I forget to format before commit, I am missing some files..


/**
* Configuration of the idempotency feature. Use the {@link Builder} to create an instance.
*/
Expand Down Expand Up @@ -92,7 +93,7 @@ public static class Builder {

private int localCacheMaxItems = 256;
private boolean useLocalCache = false;
private long expirationInSeconds = 60 * 60; // 1 hour
private long expirationInSeconds = 60 * 60L; // 1 hour
private String eventKeyJMESPath;
private String payloadValidationJMESPath;
private boolean throwOnNoIdempotencyKey = false;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
package software.amazon.lambda.powertools.idempotency;

import com.amazonaws.services.lambda.runtime.Context;

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,8 @@

package software.amazon.lambda.powertools.idempotency.internal;

import static software.amazon.lambda.powertools.idempotency.persistence.DataRecord.Status.EXPIRED;
import static software.amazon.lambda.powertools.idempotency.persistence.DataRecord.Status.INPROGRESS;

import com.amazonaws.services.lambda.runtime.Context;
import com.fasterxml.jackson.databind.JsonNode;
import java.time.Instant;
import java.util.OptionalInt;
import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.reflect.MethodSignature;
import org.slf4j.Logger;
Expand All @@ -37,6 +32,12 @@
import software.amazon.lambda.powertools.idempotency.persistence.DataRecord;
import software.amazon.lambda.powertools.utilities.JsonConfig;

import java.time.Instant;
eldimi marked this conversation as resolved.
Show resolved Hide resolved
import java.util.OptionalInt;

import static software.amazon.lambda.powertools.idempotency.persistence.DataRecord.Status.EXPIRED;
import static software.amazon.lambda.powertools.idempotency.persistence.DataRecord.Status.INPROGRESS;

/**
* Internal class that will handle the Idempotency, and use the {@link software.amazon.lambda.powertools.idempotency.persistence.PersistenceStore}
* to store the result of previous calls.
Expand Down
eldimi marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,8 @@

package software.amazon.lambda.powertools.idempotency.internal;

import static software.amazon.lambda.powertools.common.internal.LambdaHandlerProcessor.placedOnRequestHandler;

import com.amazonaws.services.lambda.runtime.Context;
import com.fasterxml.jackson.databind.JsonNode;
import java.lang.annotation.Annotation;
import java.lang.reflect.Method;
import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
Expand All @@ -33,6 +29,11 @@
import software.amazon.lambda.powertools.idempotency.exceptions.IdempotencyConfigurationException;
import software.amazon.lambda.powertools.utilities.JsonConfig;

import java.lang.annotation.Annotation;
import java.lang.reflect.Method;

import static software.amazon.lambda.powertools.common.internal.LambdaHandlerProcessor.placedOnRequestHandler;

/**
* Aspect that handles the {@link Idempotent} annotation.
* It uses the {@link IdempotencyHandler} to actually do the job.
Expand Down
Loading