Skip to content

v1.22.0
Compare
Choose a tag to compare
@LexLuthr LexLuthr released this 12 Jun 19:24
· 161 commits to main since this release
v1.22.0
633c576
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.

Overview

We are excited to announce the initial General Availability (GA) release of Curio, our innovative solution to enhance the efficiency and reliability of Filecoin data onborading. This first release lays the groundwork for Curio, providing a stable and robust platform with high availability, efficient task management, and seamless configuration capabilities. 🚀

Key Features

  1. High Availability:

    • Curio is designed for high availability, allowing multiple instances of Curio nodes to handle similar tasks.
    • The distributed scheduler and greedy worker design ensure task completion despite partial outages.
    • You can update one Curio machine without disrupting the operation of others.

  2. Node Heartbeat:

    • Each Curio node in a cluster posts a heartbeat message every 10 minutes in HarmonyDB, updating its status.
    • If a heartbeat is missed, the node is considered lost, and all tasks are rescheduled on remaining nodes.

  3. Task Retry:

    • Each task in Curio has a retry limit before being declared lost, preventing indefinite retries of bad tasks.
    • This feature safeguards against lost computation time and storage.

  4. Polling System:

    • Curio uses a polling system to avoid overloading nodes, with nodes checking for tasks they can handle.
    • Idle nodes are prioritized to ensure even workload distribution.

  5. Simple Configuration Management:

    • Configuration is stored in the database as layers that can be stacked to create a final configuration.
    • Users can reuse these layers to control multiple machines' behavior without maintaining each node's configuration.
    • Start the binary with the appropriate flags to connect with YugabyteDB and specify configuration layers.

  6. Running Curio with Multiple GPUs:

    • Curio can handle multiple GPUs simultaneously without running multiple instances of the Curio process.
    • Curio can be managed as a single systemd service without concerns about GPU allocations.

Curio vs. Lotus Miner:

Feature Curio Lotus-Miner
Scheduling Collaborative (Prioritized Greedy) Single point of failure
High Availability Available Single control process
Redundant Post Available Not Available
Task Retry Control Task retry with a cutoff limit (per task) Unlimited retry leading to resource exhaustion
Multiple Miner IDs Curio cluster supports multiple Miner IDs Single Miner ID per Lotus-Miner
Shared Task Nodes Curio nodes handle tasks for multiple Miner IDs Attached workers handle tasks for a single Miner ID
Distributed Configuration Management Configuration stored in highly-available Yugabyte Database All configuration in a single file

Getting Started: 📄

To begin your journey with Curio, please visit our official documentation for detailed guides, tutorials, and support resources. For more information on this release, check out our release page on GitHub.

Thank you for choosing Curio. We look forward to supporting your storage needs and hearing your feedback!

--- Curio Team ❤️

What's Changed

Full Changelog: https://github.com/filecoin-project/curio/commits/v1.22.0

Contributors

snadrus, magik6k, and LexLuthr
Assets 2
Loading