Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: maintain the sort order of the server sync #3748

Merged
merged 4 commits into from
Nov 4, 2024

Conversation

UnderKoen
Copy link
Member

@UnderKoen UnderKoen commented Oct 28, 2024

Makes actualbudget/actual-server#470 actually work.

Testing:

  • using the bank sync, sync
  • if you had 2 or transactions on the same day they should be order on the time the happend

@actual-github-bot actual-github-bot bot changed the title fix: maintain the sort order of the server sync [WIP] fix: maintain the sort order of the server sync Oct 28, 2024
Copy link

netlify bot commented Oct 28, 2024

Deploy Preview for actualbudget ready!

Name Link
🔨 Latest commit c9266cb
🔍 Latest deploy log https://app.netlify.com/sites/actualbudget/deploys/671fcc6d0f15f800089a4c79
😎 Deploy Preview https://deploy-preview-3748.demo.actualbudget.org
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

Copy link
Contributor

github-actions bot commented Oct 28, 2024

Bundle Stats — desktop-client

Hey there, this message comes from a GitHub action that helps you and reviewers to understand how these changes affect the size of this project's bundle.

As this PR is updated, I'll keep you updated on how the bundle size is impacted.

Total

Files count Total bundle size % Changed
9 5.32 MB 0%

Changeset

No files were changed

View detailed bundle breakdown

Added

No assets were added

Removed

No assets were removed

Bigger

No assets were bigger

Smaller

No assets were smaller

Unchanged

Asset File Size % Changed
static/js/indexeddb-main-thread-worker-e59fee74.js 13.5 kB 0%
static/js/usePreviewTransactions.js 1.64 kB 0%
static/js/resize-observer.js 18.37 kB 0%
static/js/BackgroundImage.js 122.29 kB 0%
static/js/AppliedFilters.js 21.3 kB 0%
static/js/narrow.js 82.57 kB 0%
static/js/wide.js 232.16 kB 0%
static/js/ReportRouter.js 1.5 MB 0%
static/js/index.js 3.33 MB 0%

Copy link
Contributor

github-actions bot commented Oct 28, 2024

Bundle Stats — loot-core

Hey there, this message comes from a GitHub action that helps you and reviewers to understand how these changes affect the size of this project's bundle.

As this PR is updated, I'll keep you updated on how the bundle size is impacted.

Total

Files count Total bundle size % Changed
1 1.27 MB → 1.27 MB (+56 B) +0.00%
Changeset
File Δ Size
packages/loot-core/src/server/accounts/sync.ts 📈 +151 B (+0.61%) 24.16 kB → 24.3 kB
View detailed bundle breakdown

Added

No assets were added

Removed

No assets were removed

Bigger

Asset File Size % Changed
kcab.worker.js 1.27 MB → 1.27 MB (+56 B) +0.00%

Smaller

No assets were smaller

Unchanged

No assets were unchanged

@UnderKoen UnderKoen changed the title [WIP] fix: maintain the sort order of the server sync fix: maintain the sort order of the server sync Oct 28, 2024
Copy link
Contributor

coderabbitai bot commented Oct 28, 2024

Walkthrough

The recent changes in the pull request focus on enhancing the reconcileTransactions function within the sync.ts file of the loot-core package. A new feature has been implemented to maintain the sort order of transactions that are added during the reconciliation process. This is achieved by introducing code that sets the sort_order property for each transaction in the added array. The sorting is based on a timestamp calculated from the current time, adjusted by the index of each transaction in the array.

The core functionality and error handling of the reconcileTransactions function remain unchanged, ensuring that existing operations continue to function as before. The new sorting logic is added towards the end of the function, prior to a conditional check related to creating new payees and updating transactions. Importantly, there are no alterations to the declarations of exported or public entities, nor are there changes to the parameters or return types of the function.


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Outside diff range and nitpick comments (1)
packages/loot-core/src/server/accounts/sync.ts (1)

415-419: LGTM! Consider using higher timestamp precision for better uniqueness.

The implementation effectively maintains the sort order of transactions by using timestamp-based ordering with index offsets. The nullish coalescing operator ensures we don't override any existing sort orders.

For even better uniqueness guarantee in high-frequency scenarios, consider using higher precision timestamps:

-  const now = Date.now();
+  const now = process.hrtime.bigint();
   added.forEach((t, index) => {
-    t.sort_order ??= now - index;
+    t.sort_order ??= Number(now) - index;
   });
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Files that changed from the base of the PR and between b349edd and c9266cb.

⛔ Files ignored due to path filters (2)
  • packages/loot-core/src/server/accounts/__snapshots__/parse-file.test.ts.snap is excluded by !**/*.snap
  • upcoming-release-notes/3748.md is excluded by !**/*.md
📒 Files selected for processing (1)
  • packages/loot-core/src/server/accounts/sync.ts (1 hunks)
🔇 Additional comments (1)
packages/loot-core/src/server/accounts/sync.ts (1)

415-419: Verify sort order usage in queries.

Let's ensure that the sort order is consistently used in transaction queries throughout the codebase.

✅ Verification successful

Let me analyze the SQL usage further to ensure we have a complete picture.


Based on the gathered information, I can now provide a final response:

The sort order implementation is consistent and properly maintained

The codebase shows that:

  • sort_order is properly defined as a REAL column in relevant tables
  • There's a dedicated index trans_sorted on transactions that includes sort_order
  • The implementation maintains consistency with existing sorting patterns in the database

The changes in sync.ts that set sort_order for new transactions align with the database schema and existing query patterns, ensuring proper transaction ordering.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check how sort_order is used in SQL queries and table definitions

# Test 1: Check table definition
echo "Checking table definition for sort_order column..."
rg -l "sort_order" --type sql

# Test 2: Check SQL queries using sort_order
echo "Checking SQL queries using sort_order..."
rg "ORDER BY.*sort_order" --type ts

Length of output: 2449


Script:

#!/bin/bash
# Check table definitions and migrations for sort_order
echo "Checking sort_order in table definitions..."
rg -A 5 "sort_order" packages/loot-core/src/server/sql/init.sql

# Check if sort_order is used in transaction-related tables/queries
echo "Checking transaction-related sort_order usage..."
rg -A 5 "transactions.*sort_order|sort_order.*transactions" --type sql --type ts

# Check if there are any other sort_order manipulations in the codebase
echo "Checking sort_order manipulations..."
rg "\.sort_order\s*=" --type ts

Length of output: 1507

@UnderKoen UnderKoen merged commit 29fc22a into actualbudget:master Nov 4, 2024
20 checks passed
@UnderKoen UnderKoen deleted the fix/server-sort branch November 4, 2024 16:56
@youaintnobody
Copy link

Hi, I'm running the following:
Client version: v24.11.0
Server version: v24.11.0

I'm still seeing this issue. Happens both on local and on server. Is there a fix for this or an option to easily re-order the transaction?
I've using export/import via CSV or from bank ofx file.

image

image

@UnderKoen
Copy link
Member Author

Hi @youaintnobody, could you make a new bug issue for this? As this PR only fixxed the order of transactions for bank sync. Alternatively you could upvote #1165

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants