Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add composedb correctness test that perfroms parallelization #115

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 89 additions & 1 deletion suite/src/__tests__/fast/composedb-model-sync.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,15 @@ import { beforeAll, describe, test, expect } from '@jest/globals'
import { Composite } from '@composedb/devtools'
import { newCeramic } from '../../utils/ceramicHelpers.js'
import { createDid } from '../../utils/didHelper.js'
import { BasicSchema } from '../../graphql-schemas/basicSchema'
import { BasicSchema, MultiQuerySchema } from '../../graphql-schemas/basicSchema'
import { StreamID } from '@ceramicnetwork/streamid'
import { waitForDocument } from '../../utils/composeDbHelpers.js'
import { CommonTestUtils as TestUtils } from '@ceramicnetwork/common-test-utils'

const ComposeDbUrls = String(process.env.COMPOSEDB_URLS).split(',')
const adminSeeds = String(process.env.COMPOSEDB_ADMIN_DID_SEEDS).split(',')
const parallelIterations = process.env.CDB_ITRS_PARALLEL ? parseInt(process.env.CDB_ITRS_PARALLEL, 10) : 20;
const transactionIterations = process.env.CDB_TXN_ITRS ? parseInt(process.env.CDB_TXN_ITRS, 10) : 20;
const timeoutMs = 60000

describe('Sync Model and ModelInstanceDocument using ComposeDB GraphQL API', () => {
Expand All @@ -28,12 +30,18 @@ describe('Sync Model and ModelInstanceDocument using ComposeDB GraphQL API', ()
const ceramicInstance2 = await newCeramic(ComposeDbUrls[1], did2)

const composite = await Composite.create({ ceramic: ceramicInstance1, schema: BasicSchema })
const composite2 = await Composite.create({ ceramic: ceramicInstance1, schema: MultiQuerySchema })
composeClient1 = await new ComposeClient({
ceramic: ceramicInstance1,
definition: composite.toRuntime(),
})
composeClient1.setDID(did1)

composeClient2 = await new ComposeClient({
ceramic: ceramicInstance1,
definition: composite2.toRuntime(),
})

// CACAO resources URLs for the models the client interacts with
const resources = composeClient1.resources
// Fetch themodelId of the model created by the node
Expand Down Expand Up @@ -109,4 +117,84 @@ describe('Sync Model and ModelInstanceDocument using ComposeDB GraphQL API', ()
const queryResponseid = queryResponseObj?.data?.node?.id
expect(queryResponseid).toBeDefined()
})

test('Create 20 model instance documents in a second and query them from the same node using mult query in the same node, query 20 times ina second', async () => {

for(let i = 0; i < transactionIterations; i++) {
// TODO : Add whatever mutation gitcoin was doing
const createDocumentMutation = `
mutation MultiQuerySchema($input: CreateBasicSchemaInput!) {
MultiQuerySchema(input: $input) {
document {
id
myData
}
}
}
`;


// TODO : Generate 1 kb data and create 20 documents in a second
const createDocumentPromises = [];
for (let i = 0; i < parallelIterations; i++) {
createDocumentPromises.push((async () => {
const createDocumentVariables = {
input: {
content: {
myData: 'my data ' + Date.now() + ' ' + i,
},
},
};

const response = await composeClient2.executeQuery(
createDocumentMutation,
createDocumentVariables,
);
const responseObject = await JSON.parse(JSON.stringify(response));
const documentId = responseObject?.data?.createBasicSchema?.document?.id;
expect(documentId).toBeDefined();
return documentId;
})());
}

const documentIds = await Promise.all(createDocumentPromises);
documentIds.forEach(id => expect(id).toBeDefined());


// TODO: Create a multi query instead of simple query
const getDocumentsByStreamIdQuery = `
query MultiQuerySchemaById($ids: [ID!]!) {
nodes(ids: $ids) {
... on MultiQuerySchema {
id
myData
}
}
}
`;

const getDocumentsByStreamIdVariables = {
ids: documentIds,
};

// Generate simultaneous query load of 20
const queryPromises = [];
for (let i = 0; i < parallelIterations; i++) {
queryPromises.push(
composeClient2.executeQuery(
getDocumentsByStreamIdQuery,
getDocumentsByStreamIdVariables,
)
);
}

const queryResponses = await Promise.all(queryPromises);
queryResponses.forEach(queryResponse => {
const queryResponseObj = JSON.parse(JSON.stringify(queryResponse));
const nodes = queryResponseObj?.data?.nodes;
expect(nodes).toHaveLength(parallelIterations);
});

}
})
})
6 changes: 6 additions & 0 deletions suite/src/graphql-schemas/basicSchema.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,9 @@ type BasicSchema @createModel(accountRelation: LIST, description: "A set of uniq
myData: String! @string(maxLength: 500)
}
`

export const MultiQuerySchema = `
type MultiQuerySchema @createModel(accountRelation: LIST, description: "A set of unique numbers")
@createIndex(fields: [{ path: "myData" }]){
myData: String! @string(maxLength: 500)
}`
Comment on lines +7 to +12
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems to be the same schema as the BasicSchema above, any reason to duplicate please?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we want to plug in the schema that gitcoin uses or something similar. This is a TODO

Loading