v2.13.1

What's Changed

• Update SN19 JSON by @bkb2135 in #464

Full Changelog: v2.13.0...v2.13.1

v2.13.0

What's Changed

• Implement Epistula by @bkb2135
• Switch from vllm to HF Transformers by @bkb2135 and @richwardle
• Update Bittensor version to 8.3.1 by @bkb2135
• Implement CICD precommits by @dbobrenko

Full Changelog: v2.12.1...v2.13.0

v2.12.1

What's Changed

• Sn1 325 remove typo in inference problem by @bkb2135 in #459

Full Changelog: v2.12.0...v2.12.1

v2.12.0

What's Changed

• delete old documentation by @richwardle in #440
• Update configuration instructions in validator.md and update .env.validation.example by @richwardle in #441
• Sn1 308 re enable programming task by @bkb2135 in #444
• Merge generate and chat_generate into a single generate method by @richwardle in #443
• Remove bt.config and deploy metagraph from subtensor by @bkb2135 in #445
• Add estimate_block property method to Validator class by @richwardle in #447
• SN1-311 & SN1-284: add blocking call for miner response collection + increasing p by @Hollyqui in #446
• Bump version to 2.12.0 by @bkb2135 in #455
• Fix keyword argument error by upgrading packages by @richwardle in #456
• v2.12.0 by @bkb2135 in #450

New Contributors

• @richwardle made their first contribution in #440 :)

Full Changelog: v2.11.0...v2.12.0

v2.11.0

Changes

• Add SN19 for inference task;
• Add ChatGPT API as a fallback for inference task.

Stats

PR link: #435

v2.10.0

Changes

• Improve Multi-Choice;
• Increased steepness of reward landscape (p=0.6);
• Add SN13 dataset for Inference Task;
• Remove LMSys dataset from Inference Task.

v2.9.2

Changes

• Hot-fix: Multi-choice with logits and length exploit fix (#409).

Statistics

W&B;

Miner's reward statistics with current change:

v2.9.1

Changes

• Hot-fix: Force Suspension of Loops to Allow for Other Loops to Execute.

What's Changed

• v2.9.1 by @bkb2135 in #420

Full Changelog: v2.9.0...v2.9.1

v2.9.0

Changes

• Modify reward landscape that allows for the adjustment of the network dynamics towards a winner-takes-all configuration. This enhancement provides greater flexibility in incentivizing desired behaviors within the network.
• Revisit Web Retrieval task, the expected format for the miner’s response to ensure consistency and clarity. The miner’s response must now be a JSON object containing the following fields:
  • url: URL of the website. The direct link to the relevant website.
  • content: Full website page content. Comprehensive text and data extracted from the entire webpage.
  • relevant: Most relevant section of the website page. The specific section of the webpage that directly addresses and satisfies the query requirements.

W&B

Run: https://wandb.ai/macrocosmos/prompting-validators/runs/d4i2jalp

v2.8.4

What's Changed

• Add Logit Scoring and Multi-Choice Post Processing.

Full Changelog: v2.8.3...v2.8.4