Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add seed tools. #1026

Merged
merged 10 commits into from
Jul 9, 2024
Merged

Add seed tools. #1026

merged 10 commits into from
Jul 9, 2024

Conversation

goodov
Copy link
Member

@goodov goodov commented Apr 26, 2024

This PR adds seed_tools CLI tool to support seed manipulation stored in per-file format.

Related: brave/brave-browser#33654

@goodov goodov force-pushed the add-seed-tools branch 5 times, most recently from 235dde7 to 482d95e Compare April 26, 2024 14:23
@goodov goodov force-pushed the add-seed-tools branch 3 times, most recently from b9f85c5 to 5482696 Compare May 27, 2024 15:43
@goodov goodov force-pushed the add-seed-tools branch 22 times, most recently from e314b08 to daa11e3 Compare June 7, 2024 15:27
const intValue = Number.isInteger(value) ? value : parseInt(value);
return new Date(intValue * 1000).toISOString();
}
case 'channel':
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we pass a param isBrave: boolean = true to make the tools work for Finch seed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added Options.isChromium to disable the channel modification if set to true.

// You can obtain one at https://mozilla.org/MPL/2.0/.

import { program } from '@commander-js/extra-typings';
import * as execa from 'execa';
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, does execa have advantages over child_process for our usage?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, execa is capable of launching scripts from node_modules (with preferLocal: true) the same way as npx <...> works. This way we don't need to create a custom path like node_modules/bin/... manually.

src/seed_tools/commands/check_study.test.ts Show resolved Hide resolved
src/seed_tools/commands/check_study.test.ts Show resolved Hide resolved
src/seed_tools/commands/check_study.test.ts Show resolved Hide resolved
src/seed_tools/commands/check_study.test.ts Show resolved Hide resolved
src/seed_tools/commands/check_study.test.ts Show resolved Hide resolved
src/seed_tools/commands/create_seed.test.ts Show resolved Hide resolved
src/seed_tools/commands/create_seed.test.ts Show resolved Hide resolved
src/seed_tools/commands/create_seed.test.ts Show resolved Hide resolved
src/seed_tools/commands/create_seed.test.ts Show resolved Hide resolved
src/seed_tools/commands/create_seed.ts Show resolved Hide resolved
@goodov goodov force-pushed the add-seed-tools branch 2 times, most recently from 69152bb to de85ea3 Compare June 26, 2024 07:11
@goodov goodov requested a review from iefremov June 27, 2024 08:40
@goodov
Copy link
Member Author

goodov commented Jul 3, 2024

@atuchin-m @iefremov ptal

src/base/result.ts Outdated Show resolved Hide resolved
src/base/containers/default_map.ts Show resolved Hide resolved
src/base/result.ts Outdated Show resolved Hide resolved
src/scripts/generate_proto_ts.ts Outdated Show resolved Hide resolved
src/proto/README.md Outdated Show resolved Hide resolved
src/seed_tools/utils/seed_validation.ts Outdated Show resolved Hide resolved
src/seed_tools/utils/seed_validation.ts Show resolved Hide resolved
Copy link
Contributor

github-actions bot commented Jul 9, 2024

[puLL-Merge] - brave/brave-variations@1026

Description

This PR introduces significant changes to the Brave Variations project, including new tools for managing and validating study files, improved seed generation, and various utility functions. The changes aim to enhance the robustness and functionality of the variations system, particularly in handling study files and generating seed data.

Possible Issues

  1. The new seed generation process may produce slightly different output compared to the previous system, which could potentially affect existing studies or experiments.
  2. The stricter validation rules might flag issues in existing study files that were previously overlooked.

Security Hotspots

  1. The parseStudyArray function in study_json_utils.ts parses JSON input, which could be a potential vector for injection attacks if not properly sanitized.
  2. The generateSerialNumber function in create_seed.ts uses MD5 for hashing, which is considered cryptographically weak. While not directly a security risk in this context, using a stronger hash function would be advisable.
Changes

Changes

  1. .gitattributes:

    • Added new file to enforce LF line endings and mark binary files.
  2. .prettierignore:

    • Added ignore rule for unformatted test data files.
  3. package.json:

    • Updated build scripts and dependencies.
    • Added new seed_tools script.
  4. seed/serialize.py:

    • Added support for mocking serial numbers.
  5. src/base/:

    • Added new utility classes and functions, including DefaultMap, path utilities, and a Result type.
  6. src/jest.config.js:

    • Updated configuration to support absolute imports from src/.
  7. src/proto/:

    • Added README explaining the use of protobuf-ts and handling of required fields.
    • Added patch file for study.proto to work with protobuf-ts.
  8. src/scripts/:

    • Updated existing scripts and added new ones for generating Protobuf TS files.
  9. src/seed_tools/:

    • Added new directory with various tools for seed and study management, including:
      • Commands for checking studies, creating seeds, comparing seeds, and splitting seed JSON.
      • Utility functions for study validation, version comparison, and JSON handling.
  10. src/test/data/:

    • Added test data for invalid and valid seeds and studies.

These changes significantly enhance the project's capabilities in managing and validating study files, generating seeds, and overall improving the robustness of the variations system.

@goodov goodov merged commit f48f335 into main Jul 9, 2024
6 checks passed
@goodov goodov deleted the add-seed-tools branch July 9, 2024 10:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants