-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Pubsub new tests e2e #35
base: develop
Are you sure you want to change the base?
Conversation
# the License. | ||
|
||
@PubSub_Source | ||
Feature: PubSub Source - Verification of PubSub to PubSub successful data transfer in different formats. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Verification of successful data transfer from DataStream PubSub source to PubSub sink using different file formats.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
changed to above
Then Connect plugins: "Pub/Sub" and "Pub/Sub2" to establish connection | ||
Then Navigate to the properties page of plugin: "Pub/Sub" | ||
Then Override Service account details if set in environment variables | ||
Then Enter input plugin property: "referenceName" with value: "BQReferenceName" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is this BQReferenceName? Use relevant value as per plugin which is used.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changed
Then Enter PubSub source property topic name | ||
Then Validate "Pub/Sub" plugin properties | ||
And Close the Plugin Properties page | ||
Then Open the PubSub sink properties |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use the latest steps from framework, update the steps with latest framework steps wherever required. Like here use step---- Navigate to the properties page of plugin: "Pub/Sub". Change in all the scenarios.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changed wherever needed
@@ -33,6 +42,32 @@ public void sourceIsPubSub() { | |||
|
|||
@Then("Open the PubSub source properties") | |||
public void openThePubSubSourceProperties() { | |||
openSourcePluginProperties("GooglePublisher"); | |||
openSourcePluginProperties("pubsub"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This change is not required if latest framework steps are used. Please check and revert.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reverted the change
try { | ||
publisher = Publisher.newBuilder(topicName).build(); | ||
|
||
// List<String> messages = Arrays.asList("first message", "second message"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove commented code, if not required.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
REmoved
@@ -1141,7 +1179,7 @@ public static void createSinkBQDedupeTable() throws IOException, InterruptedExce | |||
|
|||
@Before(value = "@BQ_INSERT_INT_SOURCE_TEST") | |||
public static void createSourceBQTable() throws IOException, InterruptedException { | |||
bqSourceTable = "E2E_TARGET_" + UUID.randomUUID().toString().replaceAll("-", "_"); | |||
bqSourceTable = "E2E_SOURCE_" + UUID.randomUUID().toString().replaceAll("-", "_"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why are we changing BQ related in Pubsub
@@ -1141,7 +1179,7 @@ public static void createSinkBQDedupeTable() throws IOException, InterruptedExce | |||
|
|||
@Before(value = "@BQ_INSERT_INT_SOURCE_TEST") | |||
public static void createSourceBQTable() throws IOException, InterruptedException { | |||
bqSourceTable = "E2E_TARGET_" + UUID.randomUUID().toString().replaceAll("-", "_"); | |||
bqSourceTable = "E2E_SOURCE_" + UUID.randomUUID().toString().replaceAll("-", "_"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is this change regarding ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
reverted
@Then("Subscribe to the messages") | ||
public void subscribeToTheMessages() throws InterruptedException { | ||
TimeUnit time = TimeUnit.SECONDS; | ||
time.sleep(60); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it will hault the code execution , can we use implicit wait ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We have to use it, otherwise steps run very fast and they fail
// Allow the subscriber to run for 30s unless an unrecoverable error occurs. | ||
subscriber.awaitTerminated(200, TimeUnit.SECONDS); | ||
} catch (TimeoutException timeoutException) { | ||
// Shut down the subscriber after 30s. Stop receiving messages. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
are these commented lines needed in this file ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes , its some description about the code
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also I do not see any runner file for these tests , how are we executing them ? is runner file not needed ?
@Then("Publish the messages") | ||
public void publishTheMessage() throws IOException, InterruptedException { | ||
TimeUnit time = TimeUnit.SECONDS; | ||
time.sleep(120); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
time.sleep will hault the program ,
|
||
@Then("Enter PubSub source property topic name") | ||
public void enterPubSubSourcePropertyTopicName() { | ||
PubSubActions.enterPubSubTopic(TestSetupHooks.pubSubSourceTopic); } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add one line space.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
added
Then Wait for pipeline to be in status: "Running" with a timeout of 240 seconds | ||
Then Publish the messages | ||
Then Subscribe to the messages | ||
Then Validate OUT record count is equal to IN record count |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can add validation step to validate the message content. Common step to be added for all the scenarios.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No we cannot add and validation step to check the message content, this is asynchronous transfer and it will not always send messages properly as already discussed with rahul
pubSubSourceSubscription = "cdf-e2e-test-" + UUID.randomUUID(); | ||
PubSubClient.createSubscription(pubSubSourceSubscription , pubSubSourceTopic); | ||
BeforeActions.scenario.write("Source PubSub subscription " + pubSubSourceSubscription); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add a one line space.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
added
PubSubClient.createSubscription(pubSubSourceSubscription , pubSubSourceTopic); | ||
BeforeActions.scenario.write("Source PubSub subscription " + pubSubSourceSubscription); | ||
} | ||
@After(order = 1, value = "@PUBSUB_SOURCE_TESTt") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cross check the Hook name used on scenarios, remove extra 't' letter in hooks if not present.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
removed , used only to run scenarios
pubSubSourceTopic, dataMessagesList); | ||
} | ||
|
||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
remove extra line.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
removed
subscriptionName = ProjectSubscriptionName.of | ||
(PluginPropertyUtils.pluginProp(ConstantsUtil.PROJECT_ID), subscriptionId); | ||
subscriptionAdminClient.createSubscription(subscriptionName, topicName, PushConfig.getDefaultInstance(), 60); | ||
System.out.println("Subscription created: " + subscriptionName.toString()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove system.out.println line. Use logger
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
used logger
System.out.println("Subscription created: " + subscriptionName.toString()); | ||
} catch (AlreadyExistsException e) { | ||
assert subscriptionName != null; | ||
System.out.println("Subscription already exists: " + subscriptionName.toString()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove system.out.println line. Use logger
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
used logger
if (throwable instanceof ApiException) { | ||
ApiException apiException = ((ApiException) throwable); | ||
// details on the API exception | ||
System.out.println(apiException.getStatusCode().getCode()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove and use logger. Cross check and remove wherever present.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
removed from everywhere
BeforeActions.scenario.write("Source PubSub topic " + pubSubSourceTopic); | ||
} | ||
|
||
@Before(order = 1, value = "@PUBSUB_SUBSCRIPTION_TEST") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add the After hook for the Subscription as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
added hook
@PubSub_Source | ||
Feature: PubSub Source - Verification of PubSub to PubSub successful data transfer in different formats. | ||
|
||
@PUBSUB_SOURCE_TEST @PUBSUB_SINK_TEST @PUBSUB_SUBSCRIPTION_TEST @PUBSUB_MESSAGE_TEST |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Where is @PUBSUB_MESSAGE_TEST hook details in TestSetupHooks file?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removed the hook,
# License for the specific language governing permissions and limitations under | ||
# the License. | ||
|
||
@PubSub_Source |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Where is the TestRunner file changes for PubSub Source feature? Please add the TestRunner File for PubSub Source tests having PubSub_Source tag.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
which runner file? pubsub is already added in the runner file
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In existing, It has only the runner file for sink, add runner file for source as well to execute the tests, as because you are using @PubSub_Source tag. Without this how the tests are running? Please add the runner file for source.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changed the tag name , added in runner file aswell
# License for the specific language governing permissions and limitations under | ||
# the License. | ||
|
||
@PubSub_Source |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, Add the Relevant Folder name like 'PubSubDataStream'
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changed to above
MessageReceiver receiver = | ||
(PubsubMessage message, AckReplyConsumer consumer) -> { | ||
// Handle incoming message, then ack the received message. | ||
System.out.println("Id: " + message.getMessageId()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
use logger for println and printf
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changed to logger
subscriber = Subscriber.newBuilder(subscriptionName, receiver).build(); | ||
// Start the subscriber. | ||
subscriber.startAsync().awaitRunning(); | ||
System.out.printf("Listening for messages on %s:\n", subscriptionName.toString()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't use Sysout printf in code. Use logger
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
changed
Then Save the pipeline | ||
Then Deploy the pipeline | ||
Then Run the Pipeline in Runtime | ||
Then Wait for pipeline to be in status: "Running" with a timeout of 240 seconds |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we really need the timeout of 240 seconds for runtime pipelines?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes we do after the pipeline runs for this much time only then we are sending the messages
Then Navigate to the properties page of plugin: "Pub/Sub" | ||
Then Override Service account details if set in environment variables | ||
Then Enter input plugin property: "referenceName" with value: "BQReferenceName" | ||
Then Click on the Macro button of Property: "topic" and set the value to: "PubSubSourceTopic" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can we rename it to pubSubSourceTopic, pubSubSourceSubscription?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changed the name
Then Verify the pipeline status is "Stopped" | ||
|
||
@PUBSUB_SOURCE_TEST @PUBSUB_SINK_TEST @PUBSUB_SUBSCRIPTION_TEST @PUBSUB_MESSAGE_TEST | ||
Scenario: Validate successful transfer of records from PubSub(source) to PubSub(sink) having formats TEXT at both source and sink. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Validate the successful transfer of records from a pubSub source to a pubSub sink with format Text at both source and sink.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changed to above
Then Verify the pipeline status is "Stopped" | ||
|
||
@PUBSUB_SOURCE_TEST @PUBSUB_SINK_TEST @PUBSUB_SUBSCRIPTION_TEST @PUBSUB_MESSAGE_TEST | ||
Scenario: Validate successful transfer of records from PubSub(source) to PubSub(sink) having formats TEXT at source and JSON at sink. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Validate the successful transfer of records from a pubSub source to a pubSub sink with format Text at source and Json at sink.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changed to above
pubSubSourceSubscription=dummy | ||
pubSubSourceTopic=dummy | ||
firstMessage=hellomessage | ||
secondMessage=himessage |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
hiMessage
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
changed
@@ -243,6 +243,10 @@ pubSubErrorThreshold=0 | |||
pubSubStringValue=one | |||
pubSubNegativeValue=-100 | |||
pubsubDelimiter=@ | |||
pubSubSourceSubscription=dummy | |||
pubSubSourceTopic=dummy | |||
firstMessage=hellomessage |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
helloMessage
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
changed
@@ -36,6 +56,22 @@ public static Topic createTopic(String topicId) throws IOException { | |||
} | |||
} | |||
|
|||
// Create the subscription |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add the relevant comment under this
/**
Describe the function.
*/
TopicName topicName = TopicName.of(PluginPropertyUtils.pluginProp(ConstantsUtil.PROJECT_ID), topicId); | ||
subscriptionName = ProjectSubscriptionName.of | ||
(PluginPropertyUtils.pluginProp(ConstantsUtil.PROJECT_ID), subscriptionId); | ||
subscriptionAdminClient.createSubscription(subscriptionName, topicName, PushConfig.getDefaultInstance(), 60); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we need 60 int value?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
its for deadline acknowlodgement
(PluginPropertyUtils.pluginProp(ConstantsUtil.PROJECT_ID), subscriptionId); | ||
subscriptionAdminClient.createSubscription(subscriptionName, topicName, PushConfig.getDefaultInstance(), 60); | ||
System.out.println("Subscription created: " + subscriptionName.toString()); | ||
} catch (AlreadyExistsException e) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you try with StatusRuntimeException
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it works with this only for messaging
subscriptionAdminClient.createSubscription(subscriptionName, topicName, PushConfig.getDefaultInstance(), 60); | ||
System.out.println("Subscription created: " + subscriptionName.toString()); | ||
} catch (AlreadyExistsException e) { | ||
assert subscriptionName != null; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add
if ("ALREADY_EXISTS".equals(e.getStatus().getCode().name())) {
use logger and print this("Subscription already exists: {}", subscriptionName.toString());
} else {
use logger and print ("Error creating subscription", e)
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
added the above code
pubSubSourceTopic = StringUtils.EMPTY; | ||
} catch (Exception e) { | ||
if (e.getMessage().contains("Invalid resource name given") || e.getMessage().contains("Resource not found")) { | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Complete the catch block
catch (Exception e) {
if (e.getMessage().contains("Invalid resource name given") || e.getMessage().contains("Resource not found")) {
BeforeActions.scenario.write("Source PubSub topic " + pubSubSourceTopic + " does not exist.");
} else {
Assert.fail(e.getMessage());
}
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
completed the catch block
|
||
public static void publishWithErrorHandlerExample(String projectId, String topicId, List<String> dataMessages) | ||
throws IOException, InterruptedException { | ||
TopicName topicName = TopicName.of(PluginPropertyUtils.pluginProp(ConstantsUtil.PROJECT_ID), topicId); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we use the projectId Variable from the parameter?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this code was already used before and its taking project id from plugin properties inly
@@ -422,6 +422,58 @@ public void testJsonStringWithEmptyArray() throws IOException { | |||
} | |||
} | |||
|
|||
@Test | |||
public void testJsonStringWithStringArray() throws IOException { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why we have added unit tests?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
removed
ApiException apiException = ((ApiException) throwable); | ||
/** | ||
* details on the API exception. | ||
*/ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fix indentation and line spacing
|
||
public static void subscribeAsyncExample(String projectId, String subscriptionId) { | ||
ProjectSubscriptionName subscriptionName = ProjectSubscriptionName.of(PluginPropertyUtils.pluginProp | ||
(ConstantsUtil.PROJECT_ID), subscriptionId); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
use projectId from the function parameter
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
changed
subscriber.awaitTerminated(200, TimeUnit.SECONDS); | ||
} catch (TimeoutException timeoutException) { | ||
/** | ||
* Shut down the subscriber after 30s. Stop receiving messages. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
add logger to print the timeout exception in the catch block
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
added the logger
writer.beginObject(); | ||
for (Schema.Field recordField : Objects.requireNonNull(record.getSchema().getFields())) { | ||
if (recordSchema.getField(recordField.getName()) != null) { | ||
BigQueryRecordToJson.write(writer, recordField.getName(), record.get(recordField.getName()), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Revert this file.
|
||
/** | ||
* Represents PubSub client. | ||
*/ | ||
public class PubSubClient { | ||
|
||
private static final Logger logger = LoggerFactory.getLogger(SeleniumDriver.class); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why there is SeleniumDriver.class in the Logger var?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changed to pubsub client class
@@ -56,4 +101,92 @@ public static String getTopicCmekKey(String topicId) throws IOException { | |||
return getTopic(topicId).getKmsKeyName(); | |||
} | |||
|
|||
|
|||
public static void publishWithErrorHandlerExample(String projectId, String topicId, List<String> dataMessages) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
change the function name to something more relatable, try removing example from the method name
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
publishMessagesWithPubSub
public static void publishWithErrorHandlerExample(String projectId, String topicId, List<String> dataMessages) | ||
throws IOException, InterruptedException { | ||
TopicName topicName = TopicName.of(projectId, topicId); | ||
Publisher publisher = null; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Instead of creating a new Publisher instance for each message, can you reuse the same Publisher instance for all messages. Creating a Publisher is an expensive operation, and reusing it can significantly improve performance.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Or do we have the After hook for the publisher where it delete it every time?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As Discussed we already have the after hooks for deletion of topics and subscription
Added BQ Retry
Added BQ Retry
3b4478e
to
f5807a5
Compare
fc2efdf
to
5f4f137
Compare
No description provided.