From 33ff76e754760767356ab8117e4cc20f30eb5840 Mon Sep 17 00:00:00 2001 From: Tristen Harr Date: Mon, 16 Oct 2023 20:42:15 -0500 Subject: [PATCH] check in Qdrant to registry --- registry/qdrant/README.md | 150 ++++++++++++++++++++++++++++++++++ registry/qdrant/logo.png | Bin 0 -> 9284 bytes registry/qdrant/metadata.json | 27 ++++++ 3 files changed, 177 insertions(+) create mode 100644 registry/qdrant/README.md create mode 100644 registry/qdrant/logo.png create mode 100644 registry/qdrant/metadata.json diff --git a/registry/qdrant/README.md b/registry/qdrant/README.md new file mode 100644 index 00000000..4959b280 --- /dev/null +++ b/registry/qdrant/README.md @@ -0,0 +1,150 @@ +## Qdrant Connector Overview + +The Qdrant Data Connector allows for connecting to a Qdrant instance giving you an instant GraphQL API that supports querying on top of your data. This uses the [Typescript Data Connector SDK](https://github.com/hasura/ndc-sdk-typescript) and implements the [Data Connector Spec](https://github.com/hasura/ndc-spec). + +In order to use this connector you will need a Qdrant database setup. This connector currently only supports querying. + +## Before you get started + +It is recommended that you: + +* Setup a [Qdrant Database instance](https://qdrant.tech/) +* Install the [Hasura3 CLI](https://github.com/hasura/v3-cli#hasura-v3-cli) +* Log in via the CLI +* Install the [connector plugin](https://hasura.io/docs/latest/hasura-cli/connector-plugin/) +* Install [VSCode](https://code.visualstudio.com) +* Install the [Hasura VSCode Extension](https://marketplace.visualstudio.com/items?itemName=HasuraHQ.hasura) + +## Deployment For Hasura Users + +To deploy a connector and use it in a Hasura V3 project, follow these steps: + +1. Create a Hasura V3 project (or use an existing project) + +2. Generate a configuration file for your Qdrant Database, there are 2 ways to get the configuration file. + + First you'll need to clone [this repo](https://github.com/hasura/ndc-qdrant), and run ```npm install``` + i. The easiest way to generate a configuration file is to run the generate-config script using ts-node. + + When running this script specify: + + --url The URL where Qdrant is hosted + + --key The API key for connecting to the Qdrant Client. + + --output The name of the file to store the configuration in + + Example Usage: + + ```ts-node generate-config --url https://qdrant-url --key QdrantApiKey --output config.json``` + + ii. You can also run the connector in configuration mode and generate the config file using CURL. + + ```ts-node ./src/index.ts configuration serve``` + + You can then send a CURL request specifying the qdrant_url and qdrant_api_key to get the configuration file. + + Example: + + ```curl -X POST -H "Content-Type: application/json" -d '{"qdrant_url": "https://link-to-qdrant.cloud.qdrant.io", "qdrant_api_key": "QdrantApiKey"}' http://localhost:9100 > config.json``` + +3. Once you have a configuration file, you can deploy the connector onto Hasura Cloud + +Ensure you are logged in to Hasura CLI + +```hasura3 cloud login --pat 'YOUR-HASURA-TOKEN'``` + +From there, you can deploy the connector: + +```hasura3 connector create qdrant:v1 --github-repo-url https://github.com/hasura/ndc-qdrant/tree/main --config-file ./config.json``` + +## Usage + +Once your connector is deployed, you can get the URL of the connector using: +```hasura3 connector list``` + +``` +my-cool-connector:v1 https://connector-9XXX7-hyc5v23h6a-ue.a.run.app active +``` + +In order to use the connector once deployed you will first want to reference the connector in your project metadata: + +```yaml +kind: "AuthConfig" +allowRoleEmulationFor: "admin" +webhook: + mode: "POST" + webhookUrl: "https://auth.pro.hasura.io/webhook/ddn?role=admin" +--- +kind: DataConnector +version: v1 +definition: + name: my_connector + url: + singleUrl: 'https://connector-9XXX7-hyc5v23h6a-ue.a.run.app' +``` + +If you have the [Hasura VSCode Extension](https://marketplace.visualstudio.com/items?itemName=HasuraHQ.hasura) installed +you can run the following code actions: + +* `Hasura: Refresh data source` +* `Hasura: Track all collections / functions ...` + +This will integrate your connector into your Hasura project which can then be deployed or updated using the Hasura3 CLI: + +``` +hasura3 cloud build create --project-id my-project-id --metadata-file metadata.hml +``` + +## Service Authentication + +If you don't wish to have your connector publically accessible then you must set a service token by specifying the `SERVICE_TOKEN_SECRET` environment variable when creating your connector: + +* `--env SERVICE_TOKEN_SECRET=SUPER_SECRET_TOKEN_XXX123` + +Your Hasura project metadata must then set a matching bearer token: + +```yaml +kind: DataConnector +version: v1 +definition: + name: my_connector + url: + singleUrl: 'https://connector-9XXX7-hyc5v23h6a-ue.a.run.app' + headers: + Authorization: + value: "Bearer SUPER_SECRET_TOKEN_XXX123" +``` + +While you can specify the token inline as above, it is recommended to use the Hasura secrets functionality for this purpose: + +```yaml +kind: DataConnector +version: v1 +definition: + name: my_connector + url: + singleUrl: 'https://connector-9XXX7-hyc5v23h6a-ue.a.run.app' + headers: + Authorization: + valueFromSecret: BEARER_TOKEN_SECRET +``` + +NOTE: This secret should contain the `Bearer ` prefix. + + +## Default Collection Parameters: + +You'll find that each collection on your graph is parameterized, and that you have the ability to pass in the following parameters as collection arguments: + +vector +positive +negative + +These will allow you to perform vector searches, or to get recommendations. + +You can pass in a search vector to the vector parameter, which is a flat list of floats. This will typically be the output from some embedding model, and it will return results ordered by closest match. You'll likely want to ensure that you are passing a limit on all your queries. + +You can also pass in an array of ID's to the positive and negative parameters to provide example data-points. This is an easy way to get recommendations without having to manage or deal with passing around entire vectors. If you know the ID of some positive and negative data-points, you can simply pass the ID's. You must provide at least 1 positive example when using this. You can provide a list of positive examples, a list of positive and a list of negative, but you cannot provide only a list of negative examples. + +You can read more about these parameters [here](https://qdrant.tech/documentation/concepts/search/) \ No newline at end of file diff --git a/registry/qdrant/logo.png b/registry/qdrant/logo.png new file mode 100644 index 0000000000000000000000000000000000000000..12cbdad24f4120100fe206448f201846aabd0f69 GIT binary patch literal 9284 zcmX|n1z42b6YuWQy|j`Gh|(z_y)=?i0wN(PQc5>V2n$F{OSm*hNjD;$(y@e;G)Onx z*Z+6#efD{_&U@y}Z)VP!I1~OtLx~8V1|I|h5j|Iy*9L*Woxt}LTx{Uo+cNbD@PTcv zsw5A(|M$vnDM$c;ST3K-%jkN}?PcJ2>gvt)1Y0>YmH6-1HEwh834Zlp0?Uxz!UY7G zJ|o94Nik&$13^L2u-sn?3K2*)BtjM!#@hD_Q#F>EI~Rnl4AO6}L4&k8iYT?=LHnQf^KYW-q$B=^|1I4^->FyLA$|3v084>swTUDJQG38yyTVO!OzaXV3 zeV-;Hb{qEFn1?ZRdXHpIVK~~~?&lJ}H!vQ1*kzEYb!gjy``#-N^9?@>TjsKVcP}NU zW4Dsujpq$91q#ql_)kN>_LO~S zTd(h9KVRL416)Rw+&7;Xd8Go>*8~ww`9Fe;18(Q2;H^qXh zTIBBXDcXhQ!}Pz^-k@?uX)seyyFqX8A~n%uv{^54JV=9JxJQN-EAI2O39&SqW)0@b z52)BNl!Tquo^D9L*Jr{F%2pXzem}p)Q|CVX8{qSP`A+k5;6F8S=&HSbM+@okm&SPW z{C?$)n4&{i6A=J-A-liE>PVdLaZI>sQFIdt@E4`Sz=@C9kH3w^%a@;X$ldRVs=$Dr zp{!Nh35;{Li`Eg>n*5>wQHVUts&?Ag;U*8fLibu&SgHog8V|t5sQRN;?ao>|y=dlw zJH~7Z5XB`WuE)X;w7{tBy`b4f;%&!m331r5`yev_Hh5qmi5=&Cf|azgL;gSXtG4p< zwTcvy4Mcq8w!=^RP3;l0STayc&{hSv&PQSQBWspGt#^V7TO9YD_5ixEuP2L1MNes^ zKVTS5dNGT?5)5KR_f+CJeG?&j!27=K+t11s-V{GT3K!>H5%RO$miaBU+K9H3@DDY0+6H6|Jo`BeD$xHdJw5fEcP_>} zm8yxE@8rS#?L4FChZ`D}z(bM!bA@#FgG3$fn>)3tQm>>whB(_=#U!iQA5%7bub^w1 zD-Nk-IxMzeN!#4ctGtDk*|=936j*PbuU158mVyIorNfe2LZsb32-agBjN~sX;6dY0xK~rEW!SuOg?W>^r$l;AFzD15s~F# zWfo2G(C0Nq4Ob~dL3GT>e9eO)Y$FI*dTLojYaE>r)b9@Y7t9KYksHLu7^`8q)mt;n3ivq{- zTNFKQa);EuR3d%Zf?+biK-jjcHKW$U`k8v}Mn*JgLLHw^mWM%05(_iNsbO!~-Qa@j z((pj@0f-s%pwWe{vypG|MvnukPI^o%t8%E>iGODPO#XhQ*^@-n0m5Ov zB*6hRfC+nvD<8xCy#GzKXj2I9l#`lr{1^oUFXUZK>bCU03zfN>E@1~^iAjah79@`g zfpX%=_l+IRbR;Ux)^DMOoCemM{@4l%?W-j^ZJVx*EFplXp(ueabmeczB!wImCHTWx0fZ$i-Ax$CX@*menNy~ zFuLn{P9<`XuEIML&OcDdtrLJK7jVc=JGUBF)NLKoJ%}4HT&-R8#b83<{)5j(f(*E# zuA1F(@-zHnetWgk$2BXZ!C{9z;G>V3JiJnfVi8;X2SUt zTXaOC(drh~yE>~Q5zf+~i3$;s$_f(~gZGh4=4QAs2ojo$ktbs7%qf{*+$)uDv;974 z*1r;Eqp8fR9S1*|u`wznoX7E*od+LOwh%Re-$M4%y^Uw9sckBI0}ra^G@~hS{*wwl z8`^wN9yEA#HapG@m2~ffDb!0|A`}~@T9V!$!Z4wLEsVY3>is~zj`8H`YIlnPsZ>N0 zNs8|@67Fp(7W75PlK(p2kP#3uAT4F9zQC_ch!idTw)JV^2GxS7XifGwvUuh^3A%6B zbbxUFu|CIN)NG@94wuN6gISZQc~ckumK2uZn@_`gqm@&pk{ill;2{k41H zfn5G4_~m2^1)q$%#xiLL0Xo1oV?W+?BR$*&BNM9+XwG9fR(sh$cGKP*En)mOj^}rl z1VI+B?YlKId1yv=bvwk}M2FdVKmb(0fq{t(vW}MGPr2lpZJrWCqGD9!@_NvW3Kx;oCsuBA| zy*FQ3rzP^AG;rY)#K`9EjpAYQk0;R2)94RXYvmHJsxba%B$%*b0{Ohg!GR;*)}&5h zDe-L%W7Gfa2NTAD-x++`l$ptY!(tp*kvl|pw(O+zPH_$N54H>lWe&e_p6GBn@bk`n zzdPa37_yA)GfF8v7yno;wCVOgT>$$d5o4uq}?VJ%2~V39-qr4jT-&6j3}rY2?5 zt-18Btu&7~`bXD+yAq6OuBy`)NnN$UcjvxW zsc)3txi@QW**Vub`Vq91aV4B8G%^xj?UHP@I_5}^8&6vfuwIAn2Si^VY~3vNNujc0 za(3vHm)K7{C#ULg@YODktd zjc#z8yCp$v+5qPN{q)I9u1DJ#MJR8~J(KY{p9OpsH zFE#hTp|i*oAG7;l#2t4Fo%(NH#YQhYyxNx=e5lG#JO?I6J|?3F{dl!P*;yGfOClpZ z9~!$&*)E#JE|I4uHQm48M*VHGO6|zZvkTa|ns+mfMA|sXc&)yQTc5x!U#155Uzs$t zzxhUA9Q3}zLelQ|HM5{dOe>fNgyJR0t-d(WF`FMAU!l!1UbsUp2BuhTUueF1X5)1! z5Nmu^kFEFh;7;In&@ihl84j~f?O-feI%jAp{G!F89-hT!X=LvwFtq11{wL(;VlvDx z`^dr}pyno{h1>A&gWLX%^!O%4n&9pe(dU-Qr*jI(Y28y!n6puWcW6t~E;KTuR!<8` zob~;q+opK(q_3|*>U%J>MXZD1>p`v2Ww_IQaB$^Xv=%RXuy&?t^x$GPY=oN814BvG^(MCak+4 zdDG&(Sc|7>U!f1Ek`C}k>wjJgJ1l#wnNpDa^Bz%rmNcY>-O^?^V20D6fhkAndgeawsHpEek zezZF)@WH1Zh3d!0$Owz4S#_52;@Dx^@^>HIH_}gtI}jjLPg0rq@Z;7hBV63;sM{|e zU(qLq5+DplM^kIcy2V3}T|?%yN#MC6$7KRbs4CT64F-3dEIw|;rd0mk4)N}6hk_!gCywjuFSUPuB3Yp74DT>@fs+VtuF2uA;m4%nK~PUm(;w z{F_D^MRkX=sq|xZ`eqDwZR0Z;v-j;C*|{EYoT*X_`g%gvZW;@LARUd=@47@814hir zIg|5A$&)+8PSZ!;)zbVSe+}vscrznoGwjBIl2YP?kxAs^Ud-<@JN;3mG@vumb=Lo- zFs?A~)6~I|rZBYk<=yhB)%5@_QesG9lXz*vI6-KvM>j%J5ra8^pO_6$Vi&b^MF`dw zn-$dF(P8l$5>HZ?Fd6~3CxKB_?vFC!h5=SHs3|N7GTRh_ zdZh|ja6O>BhO)99;q!O5cocZi#C}PZ3doo~xoItX^&)QD1zp=!u2Kg^USXrepX|^e z47O)ZPN`weR+#AfWj(#5qvC@4-&aflsqyd9)9k~`Uij~5=z9q(Q1$!}aCvv@X0q|J zR+x`ox}5LV@Lx5>iajR4Kj~s2ZvK6dyGTxEzzYhDgQq`tJ! zt~}|MjB^j7Jz_vm)dh9+6LJ{Uk%h5;+J~QA7)TIG8~)intnB4ekU0W8s>5A?8vEdi zdF5Y3kdW-0SVXRS+H0AP2}^49(ctHU5*U|O6qXhD&E*b8;s8!J57CR}VVCG5czwIH zo8Xm9$iUtm#mCGAG3*tuqO2GRpT>Tj@e2c$@Z-*%_wbq$FJq*U#bUxmx1Sk4t9 zsYcs&@Q~W&%8?GbRw%Q}(W<7yDl-30rI6%CxnMKU=u+UO31-(5YMpVj20!$(2bUV` zAcR404AN6)3KrGn!7ou7$Y^vZ;?61^>7I$_+!L*@J4Ol>ytwc()tMBN0mEwl1ume` z9TB%3+71H#^im83D*P90rYcY1xyF17Z?LzNeG9yrt>yLdq?NYi*xHU|K5HO6iw#~O z&cRBYA0b-d=9?g9#O49~q=>&*_VSGt&h1~vKOIdmI}oG_*K5W2-AkMALz7t0BD;`g zvz)&;3RGhS8&+KZi|I_1s3XLA)}0zuxWDXQWzG6y$v<@Pxxqy`d)(8yfjSptyE`n6 zAOG!g<@w+9i*(}#_I~m&1=|&>|!I*@H zzMSSm3{7N(Px4z|*cCLY(CdVKeWJm`wMbhfN+Z6(%j;PwKvNmde_5RQZCE`ye1aV+ z*cUyy**@mOT5(DnZHM34$fRZeIyiGTyQ}OXxY!1E)M?zuIT4GqRgbtcn;pZG5tf%x-jyd>jJ*(Z%Y@cJ!<-x*23=!}xJEW)@lxTLxqtJ!9ds0-{C?Xc zS*T$?@%_un5+u#-{hZw&x!f`qSq~%?8U!55pAi=pdmFP|V!Rbq+$L|tZ3;ZGI;Yf4 z-weD{+TNC#6orQ2US);}%`Dt>wZpmAesI3}5xYSgL9jQi`XY*3jT5HLS!QjnDP1T@ z18gBGtd)QrH=DM}gTAA=?h5WKe^VYPtq^Tz{;v^6A++0h_BJ|#ZbI`x}x_HQMhXbF4yi$$x$?UGplPes9sc*+V z-nBMZD=kPDUM(^m`3hQGb$VK61qP-RU=gw-{;(@tnY?`Qa%e$R3cWCCF2c+HauAck z9`FI32n=Zqjdme0Yz^*l&Y)I1 zo|qe5n-3%+1m1iYI7q5%^T6XPd|v&e#oWHqAontm`EPJ#`aV$7Au5Tfkd%C)6D1{% z*t}ohIU~4koxqdHTUdq!do;bEL`wV)mpZOav8%HgjeBccsnjqf0_ucgpd1iy5Sx|U zk*(%Sb+q`fD?w;cZ*_ZUZrJDH^3@&}h(`?=+-dP+YCGskT%y0QWhw8R?OCt z!3p_bX?x?J@Z{@Qoy7XwNwP)+?A$P+x46h2J4Ic}n~d^xgN}R6uOD3+f7=n^!pM42 zm^B98YF}69UxWDDqt%Aa973^CJ;MIp2*2}Y3^))EG|E)z|2=R0Om6SMN)>v>k3XLU z2d@~&G~@@6`(tVxV}@S_*2`7jYV;i3+j+vgx!m?&x>uZnK%=~fw*l&b2mT-FHssc8 zKRUZP)cI%sy2bm%q7!+lPCCsk7GW?)-$-^0?rg_kmJaf`AcC^0N)EXa;J%J$J zn!y@l)CoR6#Mr5m*yuIlQxOs|UU;U#=@|gA3}cgy7_X75)+cZJOHh(W04v7nJ-MFENIfb%`IkIo{L92h9(hTj!(Z*) zb*0~pwUMB*jMO#GBRK$wc*Ue+rZEQ>SPZMe8(JkG)H1?T<~?*#Ax+1E@j`xmcH}6a?e=HXp`v0BvTuPF z5xt#BH;iZpsviPdMf$1EwhtP=N73v-JxYk=*`vDb#W%{&$Bfv*F`3V!w#ksX^`>9P zLnRwAQ52#528p?)UUAhH61Eg-5D?_+Qh%T@y@=e6PuPj5jyhm0#;P4P^jGwnMN zY^c*Hl9kI$S+rkxB|&w4X*}Ah3tb7t?JVukd>cZa`bvp|KPH#?+8QmRu8GNYHk{Zv zLo}>Iq7xRuiupx02k`+t7`1EJs~Q3Nz_KUHl6F!81@gMB=M^XPe>RB$=f);dL14K% z<}Yd@w8B7y@ch_6D)}HcE*FJsjV>2!Nd8DsI;Z6rY{oE=^AM^&^)>@iiKx5P=1J^X zjF(z1e(BP)gAMtN8cj2+8sW30VYivdWICvHIrYH>uXtdhQbJn{qGJ1h07^M191@e6 zOWgXIy3h7K_|z{<*%m{JjIL`JCX& zM_?Nl-V7wBpblFlm6ol`d1sc*L|6+l_7v{AszRTS8zc`2Wcmc(s+pZ&As>ETBw7NP%L7n9P%3r+7I4ibfn7PtVeJjH` zzgLJ^(n+5mu3r;?%$bI8tb*p&AaN*FFLwFFF|X=pGGeSdD1C9>`Kl=|;Y@_fETB&n zTN2ARvpZ6bN!4={rBaNYd<59Y{RlyMlBcOSsPWm_bD0&7F2N zKtR9$*1X2lnJ3w|@{kf)O^~*{B{<7&$}g7G(47AaQ-? z7>ryRI4tP}nLj>Xz}k8tA(jxNJylqpLeI6A*Ny|D^1SJ#0UhGUeAhj>1S+Rbs=c>X z>>&5Plz(2d@+z&ZbFg!ds1(sW>}JPE9uT-zv%yloqQY2NG)vCnd-haC7^vSdLL3V@ z@aUW9U*vDrIk@GEHfv$^2qj{zcrkqFaXGXMP4^+_ByjaTEhh7SJh~!-k3Us3_}f^+ z;KkaI)dh;qWH6FQSD>tQ{$K|KO5oY}sl0#yALEd0s486L>H9NpwJimOA39+J+Q|g0 zPkA#I`Ul!s#gBPuus)^h(_3EdS_qYU*#LqHDHIYhiFfJDV9D z>2>9MVDon{V3Rv7%M#4I_BIna7O=9yboyI3>3dbjy9o}6x=SzMc=e1)ND~xPTz{oA zrx@$ED}`6D+bwaQxFOqA?E?kkm;dc(t|hS6||wf2>s7S{5(7SK(X&o#s{XC zj^Rs64E!Z^3wV~f$z{Fil6H`#+vEGv)5-%PmBlotC!7Db>iweKuB&Wbd`j%t98XFb zs3icE@E7fzw#Sz^r!&_0|GTXTdOA8+>ZQKLcB%@5p1@@57v-27z_}fkOG(R5Hv?S>R5NRWF^`JA2dZ zh{1irIk&CwaWW(@<&7ijHv?K3U~QNdCW4Sf9vS$& zT+_FgDzBfgm^H9y3u1u}^%=Ha)wms?`3e5D3sBn^4t>_e9!|~LpoQj)7|CLj5Jh}y zHr%pPcN$xJ=^_6vGX~D}$VjVHVIrsg7-05A5l8uu?nC&^PYN0#qe#os+f(#nqC;&r z8Hm>l-09lS?n~!E14ym-bNrW$!M@=t&VCGL&6jl{#Kn21HP%2WEzauu&||0xHvuyL zPw+^JEcSUp{HDnA{=eoF6FY}@{