Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEAT] snapshot upload to azure china blob #26

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

[FEAT] snapshot upload to azure china blob #26

wants to merge 1 commit into from

Conversation

vikramhansawat
Copy link

@vikramhansawat vikramhansawat commented Nov 22, 2022 •
edited
Loading

This feature request will allow vault_raft_snapshot_agent to upload the snapshots to AzureChinaCloud. Currently only AzurePublicCloud is supported.
FYI, you can see that by default blob.core.windows.net is used as a blob storage url:

Nov 18 10:24:27 server000000 vault_raft_snapshot_agent[20019]: 2022/11/18 10:24:27 Reading configuration...
Nov 18 10:24:27 000000 vault_raft_snapshot_agent[20019]: 2022/11/18 10:24:27 Successfully created local snapshot to /data/vault/raft/snapshots/raft_snapshot-1668767067323825425.snap
Nov 18 10:24:27 000000 /usr/bin/vault_raft_snapshot_agent[20019]: 2022/11/18 10:24:27 ==> REQUEST/RESPONSE (Try=1/55.398133ms, OpTime=55.540135ms) -- REQUEST ERROR
                                                                                                        PUT https://XXX.blob.core.windows.net/serverbackup/raft_snapshot-1668767067323825425.snap?timeout=61
                                                                                                        Authorization: REDACTED
                                                                                                        Content-Length: [30667]
                                                                                                        User-Agent: [Azure-Storage/0.7 (go1.17.12; linux)]
                                                                                                        X-Ms-Blob-Cache-Control: []
                                                                                                        X-Ms-Blob-Content-Disposition: []
                                                                                                        X-Ms-Blob-Content-Encoding: []
                                                                                                        X-Ms-Blob-Content-Language: []
                                                                                                        X-Ms-Blob-Content-Type: []
                                                                                                        X-Ms-Blob-Type: [BlockBlob]
                                                                                                        X-Ms-Client-Request-Id: [66a7f0ae-243f-4d23-6256-64598cd1338c]
                                                                                                        X-Ms-Version: [2018-11-09]
                                                                                                        x-ms-date: [Fri, 18 Nov 2022 10:24:27 GMT]
                                                                                                        --------------------------------------------------------------------------------
                                                                                                        ERROR:
                                                                                                     -> github.com/Azure/azure-pipeline-go/pipeline.NewError, /root/go/pkg/mod/github.com/!azure/[email protected]/pipeline/error.go:154
                                                                                                     HTTP request failed

                                                                                                     Put "https://XXXX.blob.core.windows.net/vaultserverbackup/raft_snapshot-1668767067323825425.snap?timeout=61": dial tcp: lookup XXXX.blob.core.windo

                                                                                                     goroutine 1 [running]:
                                                                                                     github.com/Azure/azure-storage-blob-go/azblob.stack()
                                                                                                             /root/go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_policy_request_log.go:146 +0x65
                                                                                                     github.com/Azure/azure-storage-blob-go/azblob.NewRequestLogPolicyFactory.func1.1({0xdfe988, 0xc000034cc0}, {0xc00093e368})
                                                                                                             /root/go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_policy_request_log.go:96 +0x733
                                                                                                     github.com/Azure/azure-pipeline-go/pipeline.PolicyFunc.Do(0xc506c0, {0xdfe988, 0xc000034cc0}, {0xd})
                                                                                                             /root/go/pkg/mod/github.com/!azure/[email protected]/pipeline/core.go:43 +0x2f
                                                                                                     github.com/Azure/azure-storage-blob-go/azblob.(*SharedKeyCredential).New.func1({0xdfe988, 0xc000034cc0}, {0xdf8475800})
                                                                                                             /root/go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_credential_shared_key.go:66 +0x3cd
                                                                                                     github.com/Azure/azure-pipeline-go/pipeline.PolicyFunc.Do(0xdfe918, {0xdfe988, 0xc000034cc0}, {0x7})
                                                                                                             /root/go/pkg/mod/github.com/!azure/[email protected]/pipeline/core.go:43 +0x2f
                                                                                                     github.com/Azure/azure-storage-blob-go/azblob.NewRetryPolicyFactory.func1.1({0xdfe918, 0xc0000a6180}, {0xc00093e668})
                                                                                                             /root/go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_policy_retry.go:204 +0x96e
                                                                                                     github.com/Azure/azure-pipeline-go/pipeline.PolicyFunc.Do(0xc09500, {0xdfe918, 0xc0000a6180}, {0x16})
                                                                                                             /root/go/pkg/mod/github.com/!azure/[email protected]/pipeline/core.go:43 +0x2f
                                                                                                     github.com/Azure/azure-storage-blob-go/azblob.NewUniqueRequestIDPolicyFactory.func1.1({0xdfe918, 0xc0000a6180}, {0xc00093e708})
                                                                                                             /root/go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_policy_unique_request_id.go:19 +0x163
                                                                                                     github.com/Azure/azure-pipeline-go/pipeline.PolicyFunc.Do(0xc09500, {0xdfe918, 0xc0000a6180}, {0xa})
                                                                                                             /root/go/pkg/mod/github.com/!azure/[email protected]/pipeline/core.go:43 +0x2f
                                                                                                     github.com/Azure/azure-storage-blob-go/azblob.NewTelemetryPolicyFactory.func1.1({0xdfe918, 0xc0000a6180}, {0xc000010430})
                                                                                                             /root/go/pkg/mod/github.com/!azure/[email protected]/azblob/zc_policy_telemetry.go:34 +0x11c
                                                                                                     github.com/Azure/azure-pipeline-go/pipeline.PolicyFunc.Do(0xc0000735c0, {0xdfe918, 0xc0000a6180}, {0x41a025})
                                                                                                             /root/go/pkg/mod/github.com/!azure/[email protected]/pipeline/core.go:43 +0x2f
                                                                                                     github.com/Azure/azure-pipeline-go/pipeline.(*pipeline).Do(0xc5c0c0, {0xdfe918, 0xc0000a6180}, {0xdece40, 0xc00023f6b0}, {0xc000038d88})
                                                                                                             /root/go/pkg/mod/github.com/!azure/[email protected]/pipeline/core.go:129 +0x54
                                                                                                     github.com/Azure/azure-storage-blob-go/azblob.blockBlobClient.Upload({{{{0xc000038d80, 0x5}, {0x0, 0x0}, 0x0, {0xc000038d88, 0x25}, {0xc0000cc040, 0x39}, {0x0, ...}, ...
                                                                                                             /root/go/pkg/mod/github.com/!azure/[email protected]/azblob/zz_generated_block_blob.go:436 +0x4b8
                                                                                                     github.com/Azure/azure-storage-blob-go/azblob.BlockBlobURL.Upload({{{{{...}, {...}}}}, {{{{...}, {...}, 0x0, {...}, {...}, {...}, _, {...}, ...}, ...}}}, ...)
                                                                                                             /root/go/pkg/mod/github.com/!azure/[email protected]/azblob/url_block_blob.go:64 +0x1fc
                                                                                                     github.com/Azure/azure-storage-blob-go/azblob.(*uploadStreamToBlockBlobOptions).end(0xc0000e2000, {0xdfe918, 0xc0000a6180})
                                      ```

@vikramhansawat vikramhansawat changed the title [FEAT] azure china blob upload [FEAT] snapshot upload on azure china blob Nov 22, 2022
@vikramhansawat vikramhansawat changed the title [FEAT] snapshot upload on azure china blob [FEAT] snapshot upload to azure china blob Nov 22, 2022
@vikramhansawat
Copy link
Author

@Lucretius can you please review the pull request?

Argelbargel added a commit to Argelbargel/vault-raft-snapshot-agent that referenced this pull request Sep 14, 2023
@Argelbargel
Azure cloud domain (#8)
50ee232 
* allow configuration of azure-cloud-domain (based on Lucretius#26)

---------

Co-authored-by: Karsten Kraus <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@vikramhansawat