Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Three legged model translation requirements #14

Open
wants to merge 125 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
125 commits
Select commit Hold shift + click to select a range
8fceaf6
added client as param that can be used with urlfetch
Jul 14, 2018
41ddc44
updated dm services with http.Client param
Jul 16, 2018
921034e
added client to method signature since context is required by urlfetch
Jul 16, 2018
66e7798
removed client from struct
Jul 16, 2018
3275a9c
have to pass client to auth
Jul 16, 2018
9281574
add client as param to auth interface
Jul 16, 2018
09fd21d
changed auth namespace to outer-labs
Jul 16, 2018
fe848ee
added client as param to auth
Jul 16, 2018
d70ed87
debugging invalid url
Jul 17, 2018
4c2320d
fixed tests
Jul 17, 2018
e0c8b41
try without https
Jul 17, 2018
783f908
hardcode path
Jul 17, 2018
a9d5963
updated to use reader for upload
Jul 17, 2018
404d082
fixed missing dependencies
Jul 17, 2018
c0626e9
modified reader
Jul 17, 2018
2b80982
updated md package with client
Jul 17, 2018
da2df95
updated authention call to use supplied client
Jul 17, 2018
56ab619
updated namespace to outer-labs
Jul 17, 2018
8d42576
added manifest api call
Jul 18, 2018
68a2739
replace object id param with urn
Jul 18, 2018
1742b57
POST > GET
Jul 18, 2018
693e179
updates to serializers
Jul 18, 2018
db7072b
Reverted client param changes
Jul 26, 2018
43482a2
added thumbnail request
Jul 30, 2018
faae880
missed io import
Jul 30, 2018
4947283
return open thumb stream
Jul 30, 2018
b5bff40
removed pointer
Jul 30, 2018
04df793
removed 2d translation
Aug 12, 2018
c54a9e3
added 2d view type back in, and including thumbnail in spec
Aug 12, 2018
0ead07a
fixed stupid newline
Aug 12, 2018
0a0e0d3
added broad format spec
Aug 12, 2018
829bb83
made format array generic
Aug 12, 2018
8fb5d97
made format array generic
Aug 12, 2018
e8b8eaa
reverted format spec changes
Aug 15, 2018
0d65fcb
added method to get properties
Aug 22, 2018
9593759
fixed url for properties
Aug 22, 2018
e3ecb26
changed object id from string to long
Aug 22, 2018
89418df
added metadata request
Aug 22, 2018
a5f12d0
added public metadata method
Aug 22, 2018
642dbb4
split properties into stream and object requests
Aug 22, 2018
803ba50
fixed logical error
Aug 22, 2018
d306e11
added forceget param to properties request
Sep 20, 2018
894dadb
Implemented strongly typed errors for better handling
Nov 7, 2018
e7b9cc6
fetch object tree
Nov 20, 2018
995a03f
objectid as int64
Nov 21, 2018
7c444c1
Ignores mac .DS_Store files
monicatie Dec 5, 2019
886bbcd
uncomments hub sketch
Dec 5, 2019
05809dc
progress towards listing hubs
Dec 6, 2019
bd172f1
duped work
Dec 6, 2019
c8c4bef
hubs progress
Dec 7, 2019
d48edd5
specify Hubs struct as HubDetails
Dec 7, 2019
4bb7b55
sketches out body structure of projects
Dec 7, 2019
56aecea
building out hubs, projects, and a common file for the dm package
Dec 9, 2019
e621585
update hubs and tests for required functionality (gets ride or superf…
Dec 10, 2019
d79baad
all hub tests passing
Dec 10, 2019
a84e03b
failing object test and failing bucket test
Dec 10, 2019
929754d
abstracting out cross method structs to common.go
Dec 10, 2019
0c51831
ListProjects and GetProjectDetails with tests
Dec 10, 2019
d802387
folders complete with passing tests
Dec 11, 2019
1b113d0
initial item functionality and tests complete
Dec 11, 2019
81ad1a1
addresses poor naming, updates to refect function purpose
Dec 11, 2019
dbf63d0
Adds GetTopFolders functionality to projects
Dec 11, 2019
f435412
adds Tip and Versions functionality to items
Dec 11, 2019
9bdfe5d
add array for content
Dec 12, 2019
1e08d82
Try unwrapping datadetails from projectdetails
Dec 12, 2019
d6693a5
Unwrap datadetails from folderdetails
Dec 12, 2019
f3571b9
be picky about json structure for unmarshalling
Dec 12, 2019
1ea0d0e
Merge pull request #1 from outer-labs/extend
camillaol Dec 13, 2019
90a1672
import paths
camillaol Dec 19, 2019
8077df2
Fixes folder contents return to add missing 'included' array
monicatie Dec 19, 2019
9106620
Adds Storage to Relationships struct
monicatie Dec 19, 2019
383a539
Adds DownloadObject to BucketAPI and adds tests
monicatie Dec 19, 2019
c105623
Merge branch 'master' of github.com:outer-labs/forge-api-go-client
camillaol Jan 6, 2020
bbf41d3
reduce omitempty
camillaol Jan 9, 2020
d912565
Merge branch 'master' of github.com:outer-labs/forge-api-go-client
camillaol Jan 15, 2020
cc4ce2d
refactored interfaces to have parity with Forge responses and cleaner…
camillaol Jan 16, 2020
c9eb5a0
get rid of local paths for github import
camillaol Jan 16, 2020
e5d1051
formatting for the commons and struct replacement for the request scr…
camillaol Jan 16, 2020
ed66d23
filling in a few gaps i missed for forge item responses in the commons
camillaol Jan 16, 2020
89c1c84
data type fixes
camillaol Jan 16, 2020
4a964c7
change ForegeResponse.data from object to array
camillaol Jan 16, 2020
c1239a5
distinguish between array and object responses. fixes function call f…
camillaol Jan 16, 2020
3064d06
make scopes array
camillaol Jan 16, 2020
ee93e27
change objectCount to int
camillaol Jan 16, 2020
270b3e0
storageSize to int
camillaol Jan 16, 2020
2a51971
Formatting for readability
camillaol Jan 21, 2020
2e53e67
Merge pull request #5 from outer-labs/improve-the-commons
camillaol Jan 31, 2020
eacd273
add getHubs to list hubs for a given user
camillaol Mar 6, 2020
caea77c
trying out some new three legged api call methods
camillaol Mar 6, 2020
8aad503
wip - testing out func instead of method, bearer as func arg
camillaol Mar 6, 2020
71f12c8
updates bucket, folders, hubs, and projects with three-legged auth me…
camillaol Mar 10, 2020
0f4765d
updates hubs to use proper method for retrieving refreshToken
camillaol Mar 10, 2020
bcda478
whitespace ._.
camillaol Mar 10, 2020
b398f66
wrap credentials + desired endpoint in an API struct: first try with …
camillaol Mar 10, 2020
eaa9ae7
updates folders, hubs, items, to create new wrapped credential struct…
camillaol Mar 10, 2020
1400220
couple error fixes
camillaol Mar 11, 2020
0c7b543
basic token refreshing logic
yourheropaul Mar 11, 2020
58710e6
typo fix; update pointer properly
yourheropaul Mar 11, 2020
d69a3be
Merge pull request #9 from outer-labs/feature/3l-token-refresh
camillaol Mar 11, 2020
4dc2e38
updates folder apis to match hub extension for three legged auth and …
camillaol Mar 11, 2020
04f7fa7
updates path retrieval, removes duplicate refresh token call in getfo…
camillaol Mar 11, 2020
9fb5efe
remove redundant and failing refreshtoken calls in folders_three_legg…
camillaol Mar 11, 2020
a46a242
calculate expiration time for access token in three legged auth
camillaol Mar 12, 2020
c795d4c
move ExpireTime from Bearer to folderAPI struct
camillaol Mar 12, 2020
52b11dc
add expiration time handling to hubAPI
camillaol Mar 12, 2020
c0327ce
adds time.Now() as default to AuthData literal for 2L and 3L structs
camillaol Mar 12, 2020
93d0def
change time type from string to time.Time
camillaol Mar 12, 2020
64475fa
import time to hubs
camillaol Mar 12, 2020
6ee613c
introduced async token refresher
yourheropaul Mar 12, 2020
1f4ad7b
Merge pull request #10 from outer-labs/feature/3l-auth-api-calls-async
camillaol Mar 12, 2020
7f88529
require timestamp in three-legged auth
yourheropaul Mar 17, 2020
4942a5c
Merge pull request #11 from outer-labs/feature/issue-71_store_token_i…
camillaol Mar 17, 2020
79e0e87
minor refactor and gofmt for JSON structs
yourheropaul Mar 19, 2020
d18bc14
Merge pull request #12 from outer-labs/feature/issue-73_bim-picker-th…
camillaol Mar 20, 2020
2b04697
Merge pull request #8 from outer-labs/3l-auth-api-calls
camillaol Mar 24, 2020
456d3bf
sketch 3l access to thumbnail fetching
camillaol Jun 30, 2020
d511075
add token_refresher method to md pacakge
camillaol Jun 30, 2020
4c32cdc
update md thumbnail path and return tsruct
camillaol Jun 30, 2020
b51402f
change token type in 3l md creds
camillaol Jul 1, 2020
729e7ef
create 3l method equivalents for all logic that is required by model …
camillaol Jul 2, 2020
414c106
3l api for 3l calls in md/api
camillaol Jul 2, 2020
b892cc0
updates for compass 3l model translation
camillaol Jul 3, 2020
e9153e4
fix tests
yourheropaul Jul 6, 2020
3e6d382
Merge branch 'thumbnail3l' into modeltranslate3l
yourheropaul Jul 6, 2020
a615756
fixed tests; removed duplicated code
yourheropaul Jul 6, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -65,3 +65,6 @@ crashlytics.properties
crashlytics-build.properties
fabric.properties
.idea/

# Mac
.DS_Store
1 change: 1 addition & 0 deletions assets/TestFile.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Test test 1 2 3
41 changes: 18 additions & 23 deletions dm/bucket.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,9 @@ package dm
import (
"bytes"
"encoding/json"
"errors"
"io/ioutil"
"net/http"
"strconv"

"github.com/apprentice3d/forge-api-go-client/oauth"
"github.com/outer-labs/forge-api-go-client/oauth"
)

// BucketAPI holds the necessary data for making Bucket related calls to Forge Data Management service
Expand Down Expand Up @@ -46,6 +43,11 @@ type BucketDetails struct {
// ErrorResult reflects the body content when a request failed (g.e. Bad request or key conflict)
type ErrorResult struct {
Reason string `json:"reason"`
StatusCode int
}

func (e *ErrorResult) Error() string{
return "[" + strconv.Itoa(e.StatusCode) + "] " + e.Reason
}

// ListedBuckets reflects the response when query Data Management API for buckets associated with current Forge secrets.
Expand All @@ -61,7 +63,6 @@ type ListedBuckets struct {

// CreateBucket creates and returns details of created bucket, or an error on failure
func (api BucketAPI) CreateBucket(bucketKey, policyKey string) (result BucketDetails, err error) {

bearer, err := api.Authenticate("bucket:create")
if err != nil {
return
Expand Down Expand Up @@ -106,10 +107,6 @@ func (api BucketAPI) GetBucketDetails(bucketKey string) (result BucketDetails, e
return getBucketDetails(path, bucketKey, bearer.AccessToken)
}





/*
* SUPPORT FUNCTIONS
*/
Expand All @@ -132,13 +129,13 @@ func getBucketDetails(path, bucketKey, token string) (result BucketDetails, err
}
defer response.Body.Close()

decoder := json.NewDecoder(response.Body)
if response.StatusCode != http.StatusOK {
content, _ := ioutil.ReadAll(response.Body)
err = errors.New("[" + strconv.Itoa(response.StatusCode) + "] " + string(content))
err = &ErrorResult{StatusCode:response.StatusCode}
decoder.Decode(err)
return
}

decoder := json.NewDecoder(response.Body)
err = decoder.Decode(&result)

return
Expand Down Expand Up @@ -175,14 +172,13 @@ func listBuckets(path, region, limit, startAt, token string) (result ListedBucke
return
}
defer response.Body.Close()

decoder := json.NewDecoder(response.Body)
if response.StatusCode != http.StatusOK {
content, _ := ioutil.ReadAll(response.Body)
err = errors.New("[" + strconv.Itoa(response.StatusCode) + "] " + string(content))
err = &ErrorResult{StatusCode:response.StatusCode}
decoder.Decode(err)
return
}

decoder := json.NewDecoder(response.Body)
err = decoder.Decode(&result)

return
Expand Down Expand Up @@ -217,14 +213,13 @@ func createBucket(path, bucketKey, policyKey, token string) (result BucketDetail
}
defer response.Body.Close()

decoder := json.NewDecoder(response.Body)
if response.StatusCode != http.StatusOK {
content, _ := ioutil.ReadAll(response.Body)
err = errors.New("[" + strconv.Itoa(response.StatusCode) + "] " + string(content))
err = &ErrorResult{StatusCode:response.StatusCode}
decoder.Decode(err)
return
}

decoder := json.NewDecoder(response.Body)

err = decoder.Decode(&result)

return
Expand All @@ -248,10 +243,10 @@ func deleteBucket(path, bucketKey, token string) (err error) {
return
}
defer response.Body.Close()

decoder := json.NewDecoder(response.Body)
if response.StatusCode != http.StatusOK {
content, _ := ioutil.ReadAll(response.Body)
err = errors.New("[" + strconv.Itoa(response.StatusCode) + "] " + string(content))
err = &ErrorResult{StatusCode:response.StatusCode}
decoder.Decode(err)
return
}

Expand Down
25 changes: 22 additions & 3 deletions dm/bucket_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,22 @@ package dm_test

import (
"fmt"
"github.com/apprentice3d/forge-api-go-client/dm"
"log"
"os"
"testing"

"github.com/outer-labs/forge-api-go-client/dm"
)

func TestBucketAPI_CreateBucket(t *testing.T) {

// prepare the credentials
clientID := os.Getenv("FORGE_CLIENT_ID")
clientSecret := os.Getenv("FORGE_CLIENT_SECRET")

if clientID == "" || clientSecret == "" {
t.Skipf("No Forge credentials present; skipping test")
}

bucketAPI := dm.NewBucketAPIWithCredentials(clientID, clientSecret)

t.Run("Create a bucket", func(t *testing.T) {
Expand Down Expand Up @@ -53,6 +58,11 @@ func TestBucketAPI_GetBucketDetails(t *testing.T) {
// prepare the credentials
clientID := os.Getenv("FORGE_CLIENT_ID")
clientSecret := os.Getenv("FORGE_CLIENT_SECRET")

if clientID == "" || clientSecret == "" {
t.Skipf("No Forge credentials present; skipping test")
}

bucketAPI := dm.NewBucketAPIWithCredentials(clientID, clientSecret)

testBucketKey := "my_test_bucket_key_for_go"
Expand Down Expand Up @@ -95,6 +105,11 @@ func TestBucketAPI_ListBuckets(t *testing.T) {
// prepare the credentials
clientID := os.Getenv("FORGE_CLIENT_ID")
clientSecret := os.Getenv("FORGE_CLIENT_SECRET")

if clientID == "" || clientSecret == "" {
t.Skipf("No Forge credentials present; skipping test")
}

bucketAPI := dm.NewBucketAPIWithCredentials(clientID, clientSecret)

t.Run("List available buckets", func(t *testing.T) {
Expand All @@ -106,7 +121,9 @@ func TestBucketAPI_ListBuckets(t *testing.T) {
})

t.Run("Create a bucket and find it among listed", func(t *testing.T) {

testBucketKey := "just_for_testing"

_, err := bucketAPI.CreateBucket(testBucketKey, "transient")

if err != nil {
Expand All @@ -122,6 +139,7 @@ func TestBucketAPI_ListBuckets(t *testing.T) {
found := false

for _, bucket := range list.Items {

if bucket.BucketKey == testBucketKey {
found = true
break
Expand All @@ -144,12 +162,13 @@ func ExampleBucketAPI_CreateBucket() {
// prepare the credentials
clientID := os.Getenv("FORGE_CLIENT_ID")
clientSecret := os.Getenv("FORGE_CLIENT_SECRET")

bucketAPI := dm.NewBucketAPIWithCredentials(clientID, clientSecret)

bucket, err := bucketAPI.CreateBucket("some_unique_name", "transient")

if err != nil {
log.Fatalf("Failed to create a bucket: %s\n", err.Error())
// handle error
}

fmt.Printf("Bucket %s was created with policy %s\n",
Expand Down
66 changes: 66 additions & 0 deletions dm/bucket_three_legged.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
package dm

import (
"github.com/outer-labs/forge-api-go-client/oauth"
)

// BucketAPI holds the necessary data for making Bucket related calls to Forge Data Management service
type BucketAPI3L struct {
Auth oauth.ThreeLeggedAuth
Token TokenRefresher
BucketsAPIPath string
}

// NewBucketAPIWithCredentials returns a Bucket API client with default configurations
func NewBucketAPI3LWithCredentials(auth oauth.ThreeLeggedAuth, token TokenRefresher) *BucketAPI3L {
return &BucketAPI3L{
Auth: auth,
Token: token,
BucketsAPIPath: "/oss/v2/buckets",
}
}

// CreateBucket creates and returns details of created bucket, or an error on failure
func (api BucketAPI3L) CreateBucket3L(bucketKey, policyKey string) (result BucketDetails, err error) {
if err = api.Token.RefreshTokenIfRequired(api.Auth); err != nil {
return
}

path := api.Auth.Host + api.BucketsAPIPath
result, err = createBucket(path, bucketKey, policyKey, api.Token.Bearer().AccessToken)

return
}

// DeleteBucket deletes bucket given its key.
// WARNING: The bucket delete call is undocumented.
func (api BucketAPI3L) DeleteBucket3L(bucketKey string) error {
if err := api.Token.RefreshTokenIfRequired(api.Auth); err != nil {
return err
}

path := api.Auth.Host + api.BucketsAPIPath

return deleteBucket(path, bucketKey, api.Token.Bearer().AccessToken)
}

// ListBuckets returns a list of all buckets created or associated with Forge secrets used for token creation
func (api BucketAPI3L) ListBuckets3L(region, limit, startAt string) (result ListedBuckets, err error) {
if err = api.Token.RefreshTokenIfRequired(api.Auth); err != nil {
return
}

path := api.Auth.Host + api.BucketsAPIPath

return listBuckets(path, region, limit, startAt, api.Token.Bearer().AccessToken)
}

// GetBucketDetails returns information associated to a bucket. See BucketDetails struct.
func (api BucketAPI3L) GetBucketDetails3L(bucketKey string) (result BucketDetails, err error) {
if err = api.Token.RefreshTokenIfRequired(api.Auth); err != nil {
return
}

path := api.Auth.Host + api.BucketsAPIPath
return getBucketDetails(path, bucketKey, api.Token.Bearer().AccessToken)
}
101 changes: 101 additions & 0 deletions dm/common.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
package dm

type ForgeResponseObject struct {
JsonApi JsonAPI `json:"jsonApi`
Links Links `json:"links"`
Data Data `json:"data"`
Included *[]Data `json:"included, omitempty"`
}

type ForgeResponseArray struct {
JsonApi JsonAPI `json:"jsonApi`
Links Links `json:"links"`
Data []Data `json:"data"`
Included *[]Data `json:"included, omitempty"`
}

type JsonAPI struct {
Version string `json:"version"`
}

type Links struct {
Self *Href `json:"self, omitempty"`
Related *Href `json:"related, omitempty"`
First *Href `json:"first, omitempty"`
Prev *Href `json:"prev, omitempty"`
Next *Href `json:"next, omitempty"`
}

type Data struct {
Type string `json:"type"`
Id string `json:"id"`
Attributes *Attributes `json:"attributes, omitempty"`
Relationships *Relationships `json:"relationships, omitempty"`
Links *Links `json:"links, omitempty"`
}

type Attributes struct {
Name string `json:"name"`
Extension Extension `json:"extension"`
Region *string `json:"region, omitempty"`
Scopes *[]string `json:"scopes, omitempty"`
DisplayName *string `json:"displayName, omitempty"`
ObjectCount *int `json:"objectCount, omitempty"`
CreateTime *string `json:"createTime, omitempty"`
CreateUserId *string `json:"createUserId, omitempty"`
CreateUserName *string `json:"createUserName, omitempty"`
LastModifiedTime *string `json:"lastModifiedTime, omitempty"`
LastModifiedUserId *string `json:"lastModifiedUserId, omitempty"`
LastModifiedUserName *string `json:"lastModifiedUserName, omitempty"`
Hidden *bool `json:"displayName, omitempty"`
VersionNumber *int `json:"versionNumber, omitempty"`
Mimetype *string `json:"mimeType, omitempty"`
FileType *string `json:"fileType, omitempty"`
StorageSize *int `json:"storageSize, omitempty"`
Reserved *bool `json:"reserved, omitempty"`
ReservedTime *string `json:"reservedTime, omitempty"`
ReservedUserId *string `json:"reservedUserId, omitempty"`
ReservedUserName *string `json:"reservedUserName, omitempty"`
PathInProject *string `json:"pathInProject, omitempty"`
}

type Relationships struct {
Hub *RelatedLinks `json:"hub, omitempty"`
Projects *RelatedLinks `json:projects, omitempty"`
RootFolder *RelatedLinks `json:"rootFolder, omitempty"`
TopFolders *RelatedLinks `json:"topFolders, omitempty"`
Parent *RelatedLinks `json:"parent, omitempty"`
Tip *RelatedLinks `json:"tip, omitempty"`
Versions *RelatedLinks `json:"versions, omitempty"`
Contents *RelatedLinks `json:"contents, omitempty"`
Refs *RelatedLinks `json:"refs, omitempty"`
Links *RelatedLinks `json:"links, omitempty"`
Item *RelatedLinks `json:"item, omitempty"`
Storage *RelatedLinks `json:"storage, omitempty"`
Derivatives *RelatedLinks `json:"derivatives, omitempty"`
Thumbnails *RelatedLinks `json:"thumbnails, omitempty"`
DownloadFormats *RelatedLinks `json:"downloadFormats, omitempty"`
}

type Extension struct {
Type string `json:"type"`
Version string `json:"version"`
Schema Href `json:"schema"`
Data struct{} `json:"data"`
}

type RelatedLinks struct {
Meta *Meta `json:"meta, omitempty"`
Links *Links `json:"links, omitempty"`
Data *Data `json:"data, omitempty"`
}

type Meta struct {
Link Href `json:"href"`
}

type Href struct {
Href string `json:"href"`
}

// Note on use of omitempty: https://www.sohamkamani.com/blog/golang/2018-07-19-golang-omitempty/
Loading