Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Addition of Upsert operation in Salesforce sink Plugin #17

Open
wants to merge 2 commits into
base: develop
Choose a base branch
from

Conversation

anilm67
Copy link
Collaborator

@anilm67 anilm67 commented Jul 25, 2024

No description provided.

@@ -84,6 +84,12 @@ public static void createTempSourceBQTable() throws IOException, InterruptedExce
PluginPropertyUtils.pluginProp("InsertBQDataQueryFile"));
}

@Before(order = 1, value = "@BQ_FAILURE_TEST")
public static void createTempSourceInvalidBQTable() throws IOException, InterruptedException {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where are we using this hook?

@@ -178,14 +178,14 @@ public static List<JsonObject> queryObject(String id, String objectName) {
public static void deletePushTopic(String pushTopicName) {
try {
PartnerConnection partnerConnection = new PartnerConnection(
Authenticator.createConnectorConfig(AuthenticatorCredentials.fromParameters(USERNAME, PASSWORD + SECURITYTOKEN,

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the unnecessary/non-targeted changes


QueryResult queryResult = SalesforceStreamingSourceConfig.runQuery(
partnerConnection,
String.format("SELECT Id FROM PushTopic WHERE Name = '%s'", pushTopicName)
partnerConnection,

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the unnecessary/non-targeted changes

@@ -204,7 +204,7 @@ public static void deletePushTopic(String pushTopicName) {
} catch (ConnectionException e) {
String message = SalesforceConnectionUtil.getSalesforceErrorMessageFromException(e);
throw new InvalidStageException(
String.format("Cannot connect to Salesforce API with credentials specified due to error: %s", message), e);
String.format("Cannot connect to Salesforce API with credentials specified due to error: %s", message), e);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the unnecessary/non-targeted changes

@@ -269,7 +269,7 @@ public static String queryObjectId(String objectName) {
return uniqueRecordId;
}

public static void updateObject(String id, String objectName) {
public static void updateObject(String id, String objectName) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the unnecessary/non-targeted changes

create table `DATASET.TABLE_NAME` (Name STRING, Col_Timestamp__c TIMESTAMP, Col_Date__c DATE, Col_Currency__c FLOAT64,
Col_Email__c STRING, Col_Number__c FLOAT64, Col_GeoLocation__Latitude__s FLOAT64,
Col_GeoLocation__Longitude__s FLOAT64, Col__c STRING, Col_Url__c STRING, Col_Time__c TIME, Col_Text__c STRING)
create table `DATASET.TABLE_NAME` (Id__c FLOAT64, Name STRING, Col_Timestamp__c TIMESTAMP, Col_Date__c DATE, Col_Currency__c FLOAT64,

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this change done to fix the broken environment dependent tests?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we have done this change to fix the broken environment dependent tests.

insert into `DATASET.TABLE_NAME` (Name, Col_Timestamp__c, Col_Date__c, Col_Currency__c, Col_Email__c, Col_Number__c,
Col_GeoLocation__Latitude__s, Col_GeoLocation__Longitude__s, Col__c, Col_Url__c, Col_Time__c, Col_Text__c) values
('adam','2019-03-10 04:50:01 UTC','2021-01-28',61.823765812,'[email protected]',898365444,37.794116,-122.3432,
insert into `DATASET.TABLE_NAME` (Id__c, Name, Col_Timestamp__c, Col_Date__c, Col_Currency__c, Col_Email__c, Col_Number__c,

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this change done to fix the broken environment dependent tests?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we have done this change to fix the broken environment dependent tests.

@@ -18,7 +18,7 @@ invalid.admin.consumer.secret=lmnop891011
#SOQL Query
simple.query=SELECT Id, Name, Phone FROM Account
test.query=SELECT Id,Name,Col_Timestamp__c,Col_Date__c,Col_Currency__c,Col_Email__c,Col_Number__c,\
Col_GeoLocation__Latitude__s,Col_GeoLocation__Longitude__s,Col__c,Col_Url__c,Col_Time__c,Col_Text__c FROM Automation_custom__c
Col__c,Col_Url__c,Col_Time__c,Col_Text__c FROM Automation_custom__c

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this change done to fix the broken environment dependent tests?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we have done this change to fix the broken environment dependent tests.

"Col_Currency__c":123.456,"Col_Email__c":"[email protected]","Col_Number__c":1008.0,\
"Col__c":"984746334","Col_Url__c":"abc/123","Col_Time__c":"05:00:00.000Z","Col_Text__c":"shsss"}

url = http://10.132.0.45:3128

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this property used for?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Property removed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants