Add minio scheme for S3-style storage if you're not using Amazon S3

[PeterGrace]

This PR adds a new scheme for the s3-bucket argument, minio://, which allows you to specify a custom endpoint for your s3 connection rather than using the AWS defaults. This allows you to use S3-compatible storage systems such as MinIO.

I am an extreme novice at Go programming, so if I've done anything weird here that might break other things, I apologize in advance. I wanted to provide this as a proof of concept to indicate how easy it is to support third-party S3-compatible endpoints.
If this is close to mergable, please feel free to suggest any amendments.

[derailed]

I think this could be simplified the config is the thing that varies based on host info. So something like this could be better...

cfg := aws.Config{LogLevel: aws.LogLevel(aws.LogDebugWithRequestErrors)})}
if host != "" {
  cfg.Endpoint = aws.String(host)
}
s, err := session.NewSession(&cfg)
if err != nil {
  log.Fatal().Err(err).Msg("Create S3 Session")
}

[derailed]

A test should be provided here to make sure we're doing the right thing with minio urls

[OllowainT]

Would be really nice, if this feature will be finalized and integrated
Edit: I just found out that minio is already working. I just had to use the parameters as follows (k8s):

• --s3-bucket=""
• --s3-endpoint={serveradress}/{bucketname}
• --s3-region={regionname}

[rbjorklin]

@OllowainT did you manage to upload anything to a non s3 bucket?

[OllowainT]

@OllowainT did you manage to upload anything to a non s3 bucket?

@rbjorklin: Yes, as I pointed out, I had to add the three args:

• --s3-bucket=""
• --s3-endpoint={serveradress}/{bucketname}
• --s3-region={regionname}

Its just a workaround, but for me its ok for now. You need to add --s3-bucket option with quotes. Unfortunately it also creates the directory in the bucket with quotes...

Here the Spec part of my CronJob

        spec:
          serviceAccountName: {{ .Values.popeye.accountname }}
          restartPolicy: Never
          containers:
          - name: popeye
            image: "{{ .Values.popeye.image.name }}:{{ .Values.popeye.image.tag }}"
            ports:
            - containerPort: 9216
              protocol: TCP
            imagePullPolicy: IfNotPresent
            command: ["/bin/popeye"]
            args:
              - --cluster-name
              - {{.Values.global.clusterName}}
              - -f
              - /etc/config/popeye/spinach.yml
              - --force-exit-zero=true
              - --all-namespaces
              - -o
              - html
              - --pushgateway-address
              - prometheus-pushgateway.prometheus.svc.cluster.local:9091
              # because of a popeye bug, we cannot use --s3-bucket without quotations - in the bucket the directory has also quotations
              - --s3-bucket="{{.Values.global.squadKey}}"
              - --s3-endpoint={{ template "s3.endpoint" . }}
              - --s3-region={{.Values.popeye.cronjob.s3.region}}
            env:
              - name: AWS_SHARED_CREDENTIALS_FILE
                value: "/creds/credentials"
            resources:
              limits:
                memory: 96Mi
              requests:
                cpu: 100m
                memory: 32Mi
            volumeMounts:
              - name: spinach
                mountPath: /etc/config/popeye

Hope this helps