Skip to content

Commit

Permalink
test: Add initial integration tests
Browse files Browse the repository at this point in the history
Refer to `docs/integration_tests.md` for
notes.
  • Loading branch information
sirpengi committed Oct 24, 2023
1 parent c28dc7e commit e1bbb52
Show file tree
Hide file tree
Showing 3 changed files with 147 additions and 0 deletions.
56 changes: 56 additions & 0 deletions docs/integration_tests.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Integration Tests

Integration tests in Flutter allow self-driven end-to-end
tests of app code.

The focus in this document are notes for using integration
tests to capture performance metrics on physical devices.
For more information on that topic see
[Flutter cookbook on integration profiling][profiling-cookbook].

For more background on integration testing in general
see [Flutter docs on integration testing][flutter-docs].

[flutter-docs]: https://docs.flutter.dev/testing/integration-tests
[profiling-cookbook]: https://docs.flutter.dev/cookbook/testing/integration/profiling

## Writing tests

Writing an integration test involves two parts: test code
that runs on a device and driver code that runs on the host.

Integration test code is written in a similar style as
widget test code, using a `testWidgets` function as well as
a `WidgetTester` instance to arrange widgets and run
interactions. A difference is the usage of
`IntegrationTestWidgetsFlutterBinding` which provides a
`traceAction` method used to record a Dart VM timeline.

Driver code runs on the host and is useful to configure
output of captured timeline data. There is a baseline driver
at `integration_test/perf_driver.dart` that also creates a
summary of timeline data to extract metrics such as widget
build times and frame rendering performance.

## Running tests

The command to run an integration test:

```
$ flutter drive \
--driver=integration_test/perf_driver.dart \
--target=integration_test/unreadmarker_test.dart \
-d <device_id> \
--profile \
--no-dds
```

Obtain the `device_id` using `flutter devices`.

From this command timeline data will be produced in
`build/trace_output.timeline.json`. This is a raw data file
that contains all manner of captured event timings.

A more readily consumable file will also be produced in
`build/trace_output.timeline_summary.json`. This contains
widget build times and frame timings.
25 changes: 25 additions & 0 deletions integration_test/perf_driver.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
// This file is derived from BSD licensed example code in
// Flutter documentation about integration tests profiling:
// https://docs.flutter.dev/cookbook/testing/integration/profiling#3-save-the-results-to-disk
//
// This integration driver configures output of timeline data
// and a summary thereof from integration tests. See
// [our docs on integration tests](../docs/integration_tests.md)
// for background.

import 'package:flutter_driver/flutter_driver.dart' as driver;
import 'package:integration_test/integration_test_driver.dart';

Future<void> main() {
return integrationDriver(
responseDataCallback: (data) async {
if (data != null) {
final timeline = driver.Timeline.fromJson(data['timeline']);
final summary = driver.TimelineSummary.summarize(timeline);
await summary.writeTimelineToFile(
'trace_output',
pretty: true,
includeSummary: true);
}
});
}
66 changes: 66 additions & 0 deletions integration_test/unreadmarker_test.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@

import 'package:flutter/material.dart';
import 'package:flutter_test/flutter_test.dart';
import 'package:integration_test/integration_test.dart';
import 'package:zulip/api/model/events.dart';
import 'package:zulip/api/model/model.dart';
import 'package:zulip/model/narrow.dart';
import 'package:zulip/model/store.dart';
import 'package:zulip/widgets/message_list.dart';
import 'package:zulip/widgets/store.dart';

import '../test/api/fake_api.dart';
import '../test/example_data.dart' as eg;
import '../test/model/binding.dart';
import '../test/model/message_list_test.dart';

void main() {
final binding = IntegrationTestWidgetsFlutterBinding.ensureInitialized();
TestZulipBinding.ensureInitialized();

late PerAccountStore store;
late FakeApiConnection connection;

Future<List<Message>> setupMessageListPage(WidgetTester tester, int messageCount) async {
addTearDown(testBinding.reset);
await testBinding.globalStore.add(eg.selfAccount, eg.initialSnapshot());
store = await testBinding.globalStore.perAccount(eg.selfAccount.id);
connection = store.connection as FakeApiConnection;

// prepare message list data
final messages = List.generate(messageCount, (index) {
return eg.streamMessage(id: index, flags: [MessageFlag.read]);
});
connection.prepare(json:
newestResult(foundOldest: true, messages: messages).toJson());

await tester.pumpWidget(
MaterialApp(
home: GlobalStoreWidget(
child: PerAccountStoreWidget(
accountId: eg.selfAccount.id,
child: const MessageListPage(narrow: AllMessagesNarrow())))));
await tester.pumpAndSettle();
return messages;
}

testWidgets('_UnreadMarker animation performance test', (tester) async {
final messages = await setupMessageListPage(tester, 500);

// short pause for buffer during video-capture
await tester.pump(const Duration(seconds: 2));

await binding.traceAction(() async {
store.handleEvent(eg.updateMessageFlagsRemoveEvent(
MessageFlag.read,
messages));
await tester.pumpAndSettle();
store.handleEvent(UpdateMessageFlagsAddEvent(
id: 1,
flag: MessageFlag.read,
messages: messages.map((e) => e.id).toList(),
all: false));
await tester.pumpAndSettle();
});
});
}

0 comments on commit e1bbb52

Please sign in to comment.