Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fetch older messages on scrolling up in message list #264

Merged
merged 14 commits into from
Aug 14, 2023

Conversation

gnprice
Copy link
Member

@gnprice gnprice commented Aug 11, 2023

Fixes: #78

Most of the changes here are in MessageListView, making use of the tests added in #260. We also make some tweaks to the message-list widget tests, before the main commit.

gnprice added 14 commits August 10, 2023 18:16
Also make the comment a bit more specific and more concise.
…ynamic

This doesn't use any data beyond what it's passed, so marking as `static`
makes that explicit for the reader.  That will also be helpful when we
potentially go to move this logic elsewhere in the future.
This change is NFC if you include a well-behaved server in the
system.  That is, this request should produce the exact same
response as before.
These tests were relying, for the consistency of their data, on the
test messages lying within the test stream they created.  That's
currently true but might not remain so.

Instead, use the all-messages narrow, so that there's no such
constraint.  As a bonus the test code gets slightly simpler.
…ests

In particular this will allow individual test cases to call
`connection.prepare` for setting up further API responses.
We'll soon want to add some tests where this detail varies.
Copy link
Collaborator

@chrisbobbe chrisbobbe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exciting! Small comments below.

Comment on lines +68 to +70
int _findItemWithMessageId(int messageId) {
return binarySearchByKey(items, messageId, _compareItemToMessageId);
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to check my understanding: when items starts having multiple items with the same message ID (a date separator and recipient header, in addition to MessageListMessageItem), this method will need to be replaced, right, because it assumes there is just one item per message ID.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question. It'll need some adjustments and I suppose some renamings for clarification, but I think the core logic will actually continue to work fine: the key is that

  • it's really looking for the message item for the message ID;
  • there'll still be exactly one of those per message;
  • and the other items with a given message ID will all be in a predictable direction relative to the message item (specifically, date separator and recipient header will be before the message, and if we later dream up some item that goes after its associated message then that'll fit into this scheme too).

So when the comparison function sees some item that has the right message ID but isn't a message item, it can just say those compare before the key.

Comment on lines 140 to 146
switch ((items.firstOrNull, haveOldest)) {
case (MessageListHistoryStartItem(), true): break;
case (MessageListHistoryStartItem(), _ ): items.removeFirst();

case (_, true): items.addFirst(const MessageListHistoryStartItem());
case (_, _): break;
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool! This is nice and compact!

TestZulipBinding.ensureInitialized();
await TestZulipBinding.instance.globalStore.add(eg.selfAccount, eg.initialSnapshot());
store = await TestZulipBinding.instance.globalStore.perAccount(eg.selfAccount.id);
connection = store.connection as FakeApiConnection;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

msglist test [nfc]: Use shared store/connection variables in widget tests

In particular this will allow individual test cases to call
`connection.prepare` for setting up further API responses.

Hmm, couldn't this make it easier for problematic state leakage between tests to happen?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In principle this sort of thing can. The key is that all of the shared variables get reset at the start of each test, by doing so in this one central helper function that each test calls at the start.

This is similar to what was done in #260. Like there, the alternative for getting similar functionality but with full encapsulation between tests would be to put things on a class and have each test make its own instance of the class.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense.

Comment on lines +137 to +140
// TODO parse/process messages in smaller batches, to not drop frames.
// On a Pixel 5, a batch of 100 messages takes ~15-20ms in _insertAllMessages.
// (Before that, ~2-5ms in jsonDecode and 0ms in fromJson,
// so skip worrying about those steps.)
Copy link
Collaborator

@chrisbobbe chrisbobbe Aug 11, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this strategy become counterproductive with extremely long messages/contents? Those are just Lists, so I expect inserting at index 0 (as in fetchOlder) to be linear time in the length of those lists.

Ah, though I guess this doesn't say inserting in smaller batches, just parsing/processing. So maybe this means still doing one large insert for each list, but of unparsed/unprocessed placeholders, and then processing those placeholders in batches.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, or perhaps better yet: don't insert until we have fully-processed items to insert, but do the parsing/processing in smaller batches and then insert them all when we're done.

If processing the whole overall batch takes a frame or two, the extra wait of a frame or two isn't too bad — the network request will often/typically have more latency than that, and we compensate for the latency by just starting the request early before the user scrolls all the way to the edge. So probably best not to add the complexity of having a concept of placeholder items.


Or really probably the right answer at that point is to just turn messages and contents into QueueLists too :-)

@chrisbobbe
Copy link
Collaborator

Thanks for the replies above! Please merge when ready.

@gnprice gnprice merged commit bccaead into zulip:main Aug 14, 2023
@gnprice
Copy link
Member Author

gnprice commented Aug 14, 2023

Thanks for the review! Merged.

@gnprice gnprice deleted the pr-fetch-older branch August 14, 2023 20:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fetch older messages on scrolling to top
2 participants