Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IllegalArgumentException: Project ID required for service #153

Open
HitenUnify opened this issue Sep 16, 2024 · 1 comment
Open

IllegalArgumentException: Project ID required for service #153

HitenUnify opened this issue Sep 16, 2024 · 1 comment

Comments

@HitenUnify
Copy link

Caused by: java.lang.IllegalArgumentException: A project ID is required for this service but could not be determined from the builder or the environment.  Please set a project ID using the builder.
	at com.google.common.base.Preconditions.checkArgument(Preconditions.java:143)
	at com.google.cloud.ServiceOptions.<init>(ServiceOptions.java:340)
	at com.google.cloud.bigquery.BigQueryOptions.<init>(BigQueryOptions.java:94)
	at com.google.cloud.bigquery.BigQueryOptions.<init>(BigQueryOptions.java:31)
	at com.google.cloud.bigquery.BigQueryOptions$Builder.build(BigQueryOptions.java:89)
	at com.google.cloud.flink.bigquery.services.BigQueryServicesImpl$QueryDataClientImpl.<init>(BigQueryServicesImpl.java:248)
	at com.google.cloud.flink.bigquery.services.BigQueryServicesImpl.createQueryDataClient(BigQueryServicesImpl.java:89)
	at com.google.cloud.flink.bigquery.services.BigQueryServicesFactory.queryClient(BigQueryServicesFactory.java:94)
	at com.google.cloud.flink.bigquery.sink.serializer.BigQuerySchemaProviderImpl.getTableSchemaFromOptions(BigQuerySchemaProviderImpl.java:153)
	at com.google.cloud.flink.bigquery.sink.serializer.BigQuerySchemaProviderImpl.<init>(BigQuerySchemaProviderImpl.java:60)

happens even though it is clearly defined here

CredentialsOptions credentials = getCredentialsOptions(destination);
BigQueryConnectOptions sinkConnectOptions =
				BigQueryConnectOptions.builder().setProjectId(projectId)
					.setDataset(dataset).setTable(destinationTableId).setCredentialsOptions(credentials).build();
BigQuerySchemaProvider schemaProvider = new BigQuerySchemaProviderImpl(sinkConnectOptions);

Core Issue seems to be at

com.google.cloud.flink.bigquery.services.BigQueryServicesImpl.QueryDataClientImpl#QueryDataClientImpl

public QueryDataClientImpl(CredentialsOptions options) {
            bigQuery =
                    BigQueryOptions.newBuilder()
                            .setCredentials(options.getCredentials())
                            .build()
                            .getService();
            bigquery = BigQueryUtils.newBigqueryBuilder(options).build();
        }

The builder

BigQueryOptions.newBuilder()
                            .setCredentials(options.getCredentials())
                            .build()
                            .getService();

doesn't have projectID context, unless passed through ENV, which I can't do in my flink project.

Thanks!

@jayehwhyehentee
Copy link
Collaborator

@HitenUnify Can you share the entire application code that you are trying to run here?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants