Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add claude caching #4

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Conversation

teilomillet
Copy link

I have added Claude caching functionality with the following changes:

  1. Added caching configuration options in settings.js:

    • Enable/disable caching via claudeOptions.cacheOptions.enabled
    • Configurable minimum tokens via claudeOptions.cacheOptions.minTokens
  2. Modified ClaudeClient.js to support intelligent caching:

    • Added cache-specific headers in getHeaders()
    • Implemented strategic cache placement in buildApiParams():
      • Caches at branching points in conversations
      • Maintains last two branch points for efficient context switching
      • Uses remaining cache blocks for periodic checkpoints
      • Respects Claude's 4-block cache limit
    • Each cache block preserves full conversation context up to that point
  3. Cache block distribution strategy:

    • System message (when present)
    • Two most recent conversation branch points
    • Periodic checkpoints every 10 messages (using remaining blocks)
  4. Run 'npm audit fix' to fix vulnerabilities

Note: While the functionality is implemented following Anthropic's documentation, it requires testing to verify proper real-world performance, particularly around branch point detection and cache block distribution.

Testing needed:

  • Branch point detection
  • Cache block count limit
  • Cache persistence (5-minute TTL)
  • Performance impact on long conversations

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant