Relay for Frontend and Sdw

Since the aiweb3 frontend is on VPS and the SDW is on the local machine with private IP, we need to utilize this project as the relay on both sides to connect them.

Part I: Relay at the AI side

Before running the relay at the AI side, we need to install the SDW and run it as a service.

Install stable diffussion webui

1. Since there are some difference with respect to the platform, plz find the best way to install SDW https://github.com/AUTOMATIC1111/stable-diffusion-webui The python version I used is 3.10.6
2. For now, we use the model darkSushiMixMix_225D , which can be download at https://civitai.com/models/24779/dark-sushi-mix-mix ,where the model should be put in the folder /models/Stable-diffusion (if using lora, so should be put in Lora folder)

Run SDW as a service

conda activate xxx (this is the name of the conda environment)

bash webui.sh --nowebui --gradio-auth admin:admin123   

(more args: https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Command-Line-Arguments-and-Settings)

Run the relay (this project)

Now, we can run the relay.

Install

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh | bash
source ~/.bashrc

nvm install node
npm install -g typescript
npm install --global yarn

yarn install

❌ !!! The package 'stable-diffusion-sdk' has some issue, so we need to modify the its code in node_modules/stable-diffusion-api/dist/lib/StableDiffusionApi.js around line 460

❌const modelNames = models.map((model) => model.name);

to

❌const modelNames = models.map((model) => model.model_name);

✅ For now, I have remove the package 'stable-diffusion-sdk' from package.json, and add an revised one in my_modules/stable-diffusion-sdk. So, u dont need to revise the sdk code!

Add customer parameter into /config

aiweb3.ts is the config file for managing base prompts, i.e., the NFT style.

general.ts is the config file for network setting such as the port number and the IP address.

sdw.ts is the config file for the SDW setting such as security token, model name, network parameters.

You may need to update the ip address in the src/config/general.ts for the ai part before you compile it

Compile

tsc

Run at the AI side

node dist/index.js --relaySide ai

Run at the frontend side

node dist/index.js --relaySide frontend

😆😆😆

How the frontend interact with the relay

User login (verfity the user's signature)

The frontend should post data to http://127.0.0.1:1985/verifyUserSignature with the following format

{   
    "message": "0xaaaaa is signing in to Aiweb3 at timestamp [current_timestamp] with nonce [random_nonce]", // the message to be signed
    "signature": "0x...", // user's signature for the message
    "userAddress": "0x..."  // user's metamask address
}

Here, the port number 1985 is defined in the config file general.ts. Current_timestamp and random_nonce are generated by the frontend, in order to prevent replay attacks (this is important).

The backend will reply 'correct'/'wrong' to the frontend.

Example code:

    const urlVerifyUserSignature = "http://127.0.0.1:1985/verifyUserSignature" 
    const data = JSON.stringify({
        message: "0xaaaaa is signing in to Aiweb3 at timestamp [current_timestamp] with nonce [random_nonce]",
        signature: "0x...",
        userAddress: "0x..."
    })
    const response = await fetch(urlVerifyUserSignature, {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: data
    })
    const result = await response.json()   

Here, result should be 'correct'/'wrong'.

❌ Mint NFT (only for test, not for production)

✅ Mint NFT with Code

The frontend should post data to http://127.0.0.1:1985/mintNFTwithCode with the following format

{   
    "feature": "{\"feature1\":\"red hat\", \"feature2\":\"blue t-shirt\"}, ..."
    "userAddress": "0x..."  // user's metamask address,
    "NFTCode": "ABCD" // the NFT code

}

Here, the feature indicates the users' bais on their NFT. The NFTCode is for verify if the User is eligible to mint the NFT. If not, u will get "The NFT code is not invalid". If yes, the NFT task will be created at the backend. After a while (should < 20 seconds), u can use /getUserAllImg to get all the IMG paths of the target user.

Fetch NFT paths

Example code:

    const urlVerifyUserSignature = "http://127.0.0.1:1985/getUserAllImg" 
    const data = JSON.stringify(
        {
            "accountType":"substrate",
            "userAddress":"5xxxx"
        }
    )
    const response = await fetch(urlVerifyUserSignature, {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: data
    })
    const result = await response.json()   

The result should be a json object with the following format (absolute path):

{
    "paths": [
        "/Users/cao/CAO/github/relay_frontend_sdw/DB_backend/IMG/15_10.jpg",
        "/Users/cao/CAO/github/relay_frontend_sdw/DB_backend/IMG/15_11.jpg",
        "/Users/cao/CAO/github/relay_frontend_sdw/DB_backend/IMG/15_12.jpg"
    ]
}