Introducing: Plugin Mechanism

[Yeuoly]

Introduce Plugin Mechanism

We'm glad to share that we're launching Plugins beta.
Plugins are modular components that extend AI applications with plug-and-play simplicity. Now you can assemble external services and custom functionalities with your Dify apps effortlessly.

What have we done?

• We've developed a plugin system, a standard for Dify that lets users create plugins with the suffix .difypkg. By installing different plugin packages, you can add various capabilities to Dify.
• Through this system, we've decoupled tools and models that can be expanded from Dify, enabling you to install them in a truly plug-and-play manner.
• We've introduced a new Agent node in Workflow, where the agent's behavior can now be defined by plugins. This means Dify is no longer limited to built-in strategies like CoT; you can define your own agent strategies.
• We've also introduced a new type of plugin called Endpoint, which is a custom service hosted within Dify. You can customize this service's code through plugins to handle external Webhook events.
• We've launched the Dify Marketplace, a centralized platform where plugins can be freely shared, allowing users to publish and download different plugins.

Importants

• !! This is a beta version of Dify v1.0.0. Please do not upgrade your production environment directly to this version.
• !! In this plugin version, some tools are too closely integrated with Dify and haven't been fully migrated to v1.0.0. This might cause certain tools to become unavailable during migration. Tools developed through custom development on Dify also cannot be migrated. You can manually migrate your existing implementations to plugins by referring to the plugin development documentation.

Docker Deployment

v1.0.0 supports one-click Docker compose deployment. You can start the Dify service using the following commands:

git checkout 1.0.0-beta1
cd ./docker
./generate_docker_compose
docker compose -f docker-compose.yaml up -d

Plugin Migration

• !! Note, if you are upgrading from an older version to v1.0.0, you need to execute some basic migration steps.
  The first step in upgrading to v1.0.0 is to install the tools and models used in your current environment into the new plugin environment, ensuring plugin installation is completed before database migration. During installation, run the following command:

poetry run flask extract-plugins --workers=20

This command extracts all models and tools used in the current environment. The workers parameter determines the number of parallel processes during extraction; adjust it according to your needs.
The final result will be saved in the plugins.jsonl file, which contains the plugin information for all workspaces in the current Dify instance.
Ensure your network is functioning properly and can access https://marketplace.dify.ai, then run the following command:

poetry run flask install-plugins

This command will download and install all necessary plugins into the latest environment.

Database Migration

After completing the basic plugin installation process, follow the standard Dify upgrade procedure to update the database schema to the latest version.

poetry run flask db upgrade

Dify v1.0 includes compatibility for old data. You should be able to operate normally, but for better future compatibility, run the following data migration command once you're sure you won't revert to an older version:

poetry run flask migrate-data-for-plugin

This command will prefix existing model and tool vendors in the database with langgenius, such as converting openai to langgenius/openai/openai. This operation may affect the functioning of the older Dify version, so back up your database before running this command to prevent unwanted issues.

[ddrcrow]

where to run poetry run flask extract-plugins --workers=20?
I ran it under dify folder, it reported

when I ran it under dify/app
it reported

[Yeuoly]

where to run poetry run flask extract-plugins --workers=20? I ran it under dify folder, it reported

when I ran it under dify/app it reported

Under /app/api