Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Replace WebDriverIO with Cypress #831

Merged
merged 19 commits into from
Dec 17, 2023
Merged

Replace WebDriverIO with Cypress #831

merged 19 commits into from
Dec 17, 2023

Conversation

HarelM
Copy link
Collaborator

@HarelM HarelM commented Dec 17, 2023 •
edited
Loading

This does the following:

  1. Moves the WDIO code from javascript to typescript
  2. Moves to use files that are cy.ts instead of index.js
  3. Replace e2e to use cypress
  4. Introduce back some skipped tests

This is in continue to the conversation here:
HarelM#3

Before:

 "spec" Reporter:
------------------------------------------------------------------
[chrome 120.0.6099.71 linux #0-0] Running: chrome (v120.0.6099.71) on linux
[chrome 120.0.6099.71 linux #0-0] Session ID: ee9a87bcfce007ac7721929c6e6234d0
[chrome 120.0.6099.71 linux #0-0]
[chrome 120.0.6099.71 linux #0-0] » /test/functional/index.js
[chrome 120.0.6099.71 linux #0-0] maputnik
[chrome 120.0.6099.71 linux #0-0]     history
[chrome 120.0.6099.71 linux #0-0]        - undo/redo
[chrome 120.0.6099.71 linux #0-0]
[chrome 120.0.6099.71 linux #0-0]     layers
[chrome 120.0.6099.71 linux #0-0]         ops
[chrome 120.0.6099.71 linux #0-0]            ✓ delete
[chrome 120.0.6099.71 linux #0-0]            ✓ duplicate
[chrome 120.0.6099.71 linux #0-0]            ✓ hide
[chrome 120.0.6099.71 linux #0-0]
[chrome 120.0.6099.71 linux #0-0]         background
[chrome 120.0.6099.71 linux #0-0]            ✓ add
[chrome 120.0.6099.71 linux #0-0]
[chrome 120.0.6099.71 linux #0-0]             modify
[chrome 120.0.6099.71 linux #0-0]                 layer
[chrome 120.0.6099.71 linux #0-0]                    - expand/collapse
[chrome 120.0.6099.71 linux #0-0]                    ✓ id
[chrome 120.0.6099.71 linux #0-0]                    ✓ min-zoom
[chrome 120.0.6099.71 linux #0-0]                    ✓ max-zoom
[chrome 120.0.6099.71 linux #0-0]                    ✓ comments
[chrome 120.0.6099.71 linux #0-0]                    - color
[chrome 120.0.6099.71 linux #0-0]
[chrome 120.0.6099.71 linux #0-0]                 filter
[chrome 120.0.6099.71 linux #0-0]                    - expand/collapse
[chrome 120.0.6099.71 linux #0-0]                    - compound filter
[chrome 120.0.6099.71 linux #0-0]
[chrome 120.0.6099.71 linux #0-0]                 paint
[chrome 120.0.6099.71 linux #0-0]                    - expand/collapse
[chrome 120.0.6099.71 linux #0-0]                    - color
[chrome 120.0.6099.71 linux #0-0]                    - pattern
[chrome 120.0.6099.71 linux #0-0]                    - opacity
[chrome 120.0.6099.71 linux #0-0]
[chrome 120.0.6099.71 linux #0-0]                 json-editor
[chrome 120.0.6099.71 linux #0-0]                    - expand/collapse
[chrome 120.0.6099.71 linux #0-0]                    - modify
[chrome 120.0.6099.71 linux #0-0]                    - parse error
[chrome 120.0.6099.71 linux #0-0]
[chrome 120.0.6099.71 linux #0-0]         fill
[chrome 120.0.6099.71 linux #0-0]            ✓ add
[chrome 120.0.6099.71 linux #0-0]            - change source
[chrome 120.0.6099.71 linux #0-0]
[chrome 120.0.6099.71 linux #0-0]         line
[chrome 120.0.6099.71 linux #0-0]            ✓ add
[chrome 120.0.6099.71 linux #0-0]            - groups
[chrome 120.0.6099.71 linux #0-0]
[chrome 120.0.6099.71 linux #0-0]         symbol
[chrome 120.0.6099.71 linux #0-0]            ✓ add
[chrome 120.0.6099.71 linux #0-0]
[chrome 120.0.6099.71 linux #0-0]         raster
[chrome 120.0.6099.71 linux #0-0]            ✓ add
[chrome 120.0.6099.71 linux #0-0]
[chrome 120.0.6099.71 linux #0-0]         circle
[chrome 120.0.6099.71 linux #0-0]            ✓ add
[chrome 120.0.6099.71 linux #0-0]
[chrome 120.0.6099.71 linux #0-0]         fill extrusion
[chrome 120.0.6099.71 linux #0-0]            ✓ add
[chrome 120.0.6099.71 linux #0-0]
[chrome 120.0.6099.71 linux #0-0]         groups
[chrome 120.0.6099.71 linux #0-0]            ✓ simple
[chrome 120.0.6099.71 linux #0-0]
[chrome 120.0.6099.71 linux #0-0]     map
[chrome 120.0.6099.71 linux #0-0]         zoom level
[chrome 120.0.6099.71 linux #0-0]            - via url
[chrome 120.0.6099.71 linux #0-0]            - via map controls
[chrome 120.0.6099.71 linux #0-0]
[chrome 120.0.6099.71 linux #0-0]     modals
[chrome 120.0.6099.71 linux #0-0]         open
[chrome 120.0.6099.71 linux #0-0]            ✓ close
[chrome 120.0.6099.71 linux #0-0]            - upload
[chrome 120.0.6099.71 linux #0-0]            ✓ load from url
[chrome 120.0.6099.71 linux #0-0]
[chrome 120.0.6099.71 linux #0-0]         shortcuts
[chrome 120.0.6099.71 linux #0-0]            ✓ open/close
[chrome 120.0.6099.71 linux #0-0]
[chrome 120.0.6099.71 linux #0-0]         export
[chrome 120.0.6099.71 linux #0-0]            ✓ close
[chrome 120.0.6099.71 linux #0-0]            - download
[chrome 120.0.6099.71 linux #0-0]
[chrome 120.0.6099.71 linux #0-0]         sources
[chrome 120.0.6099.71 linux #0-0]            - active sources
[chrome 120.0.6099.71 linux #0-0]            - public source
[chrome 120.0.6099.71 linux #0-0]            - add new source
[chrome 120.0.6099.71 linux #0-0]
[chrome 120.0.6099.71 linux #0-0]         inspect
[chrome 120.0.6099.71 linux #0-0]            ✓ toggle
[chrome 120.0.6099.71 linux #0-0]
[chrome 120.0.6099.71 linux #0-0]         style settings
[chrome 120.0.6099.71 linux #0-0]            ✓ name
[chrome 120.0.6099.71 linux #0-0]            ✓ owner
[chrome 120.0.6099.71 linux #0-0]            ✓ sprite url
[chrome 120.0.6099.71 linux #0-0]            ✓ glyphs url
[chrome 120.0.6099.71 linux #0-0]            ✓ maptiler access token
[chrome 120.0.6099.71 linux #0-0]            ✓ thunderforest access token
[chrome 120.0.6099.71 linux #0-0]            ✓ style renderer
[chrome 120.0.6099.71 linux #0-0]
[chrome 120.0.6099.71 linux #0-0]         sources
[chrome 120.0.6099.71 linux #0-0]            - toggle
[chrome 120.0.6099.71 linux #0-0]
[chrome 120.0.6099.71 linux #0-0]     screenshots
[chrome 120.0.6099.71 linux #0-0]        ✓ front_page
[chrome 120.0.6099.71 linux #0-0]        ✓ open
[chrome 120.0.6099.71 linux #0-0]        ✓ export
[chrome 120.0.6099.71 linux #0-0]        ✓ sources
[chrome 120.0.6099.71 linux #0-0]        ✓ style settings
[chrome 120.0.6099.71 linux #0-0]        ✓ inspect
[chrome 120.0.6099.71 linux #0-0]
[chrome 120.0.6099.71 linux #0-0]     accessibility
[chrome 120.0.6099.71 linux #0-0]         skip links
[chrome 120.0.6099.71 linux #0-0]            ✓ skip link to layer list
[chrome 120.0.6099.71 linux #0-0]            ✓ skip link to layer editor
[chrome 120.0.6099.71 linux #0-0]            ✓ skip link to map view
[chrome 120.0.6099.71 linux #0-0]
[chrome 120.0.6099.71 linux #0-0]     keyboard
[chrome 120.0.6099.71 linux #0-0]         shortcuts
[chrome 120.0.6099.71 linux #0-0]            ✓ ESC should unfocus
[chrome 120.0.6099.71 linux #0-0]            ✓ '?' should show shortcuts modal
[chrome 120.0.6099.71 linux #0-0]            ✓ 'o' should show open modal
[chrome 120.0.6099.71 linux #0-0]            ✓ 'e' should show export modal
[chrome 120.0.6099.71 linux #0-0]            ✓ 'd' should show sources modal
[chrome 120.0.6099.71 linux #0-0]            ✓ 's' should show settings modal
[chrome 120.0.6099.71 linux #0-0]            - 'i' should change map to inspect mode
[chrome 120.0.6099.71 linux #0-0]            ✓ 'm' should focus map
[chrome 120.0.6099.71 linux #0-0]            ✓ '!' should show debug modal
[chrome 120.0.6099.71 linux #0-0]
[chrome 120.0.6099.71 linux #0-0] 44 passing (58.8s)
[chrome 120.0.6099.71 linux #0-0] 23 skipped

After:

accessibility
    skip links
      - skip link to layer list
      - skip link to layer editor
      - skip link to map view

history
    ✓ undo/redo (4894ms)

keyboard
    shortcuts
      ✓ ESC should unfocus (1912ms)
      ✓ '?' should show shortcuts modal (458ms)
      ✓ 'o' should show open modal (710ms)
      ✓ 'e' should show export modal (692ms)
      ✓ 'd' should show sources modal (588ms)
      ✓ 's' should show settings modal (894ms)
      ✓ 'i' should change map to inspect mode (804ms)
      ✓ 'm' should focus map (837ms)
      ✓ '!' should show debug modal (607ms)

layers
    ops
      ✓ delete (4313ms)
      ✓ duplicate (1780ms)
      ✓ hide (1862ms)
    background
      ✓ add (1675ms)
      modify
        layer
          - expand/collapse
          ✓ id (3735ms)
          ✓ min-zoom (2209ms)
          ✓ max-zoom (2127ms)
          ✓ comments (2515ms)
          ✓ color (2022ms)
        filter
          - expand/collapse
          - compound filter
        paint
          - expand/collapse
          - color
          - pattern
          - opacity
        json-editor
          - expand/collapse
          - modify
          - parse error
    fill
      ✓ add (1831ms)
      - change source
    line
      ✓ add (1844ms)
      ✓ groups (687ms)
    symbol
      ✓ add (2035ms)
    raster
      ✓ add (1814ms)
    circle
      ✓ add (1867ms)
    fill extrusion
      ✓ add (1963ms)
    groups
      ✓ simple (2653ms)

map
    zoom level
      ✓ via url (2279ms)
      ✓ via map controls (733ms)

modals
    open
      ✓ close (2519ms)
      - upload
      ✓ load from url (1557ms)
    shortcuts
      ✓ open/close (1136ms)
    export
      ✓ close (755ms)
      - download
    sources
      - active sources
      - public source
      - add new source
    inspect
      ✓ toggle (1020ms)
    style settings
      ✓ name (1085ms)
      ✓ owner (1060ms)
      ✓ sprite url (1214ms)
      ✓ glyphs url (1553ms)
      ✓ maptiler access token (1111ms)
      ✓ thunderforest access token (1102ms)
      ✓ style renderer (922ms)
    sources
      - toggle



      Spec                                              Tests  Passing  Failing  Pending  Skipped  
  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ ✔  accessibility.cy.ts                       52ms        3        -        -        3        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  history.cy.ts                            00:06        1        1        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  keyboard.cy.ts                           00:10        9        9        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  layers.cy.ts                             00:39       28       17        -       11        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  map.cy.ts                                00:04        2        2        -        -        - │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  modals.cy.ts                             00:16       18       12        -        6        - │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘
    ✔  All specs passed!                        01:18       61       41        -       20        -

HarelM and others added 17 commits December 14, 2023 18:14
@HarelM
Use driver pattern for e2e tests (#2)
73f7798 
* Initial commit

* Fix spec

* Move driver

* Fix config location

* Fix helper location

* More usage of driver

* Add click

* Fix click

* Migrate more tests

* Add setValue to driver

* Move more code to driver

* add isExisting to driver

* Change modal tests to use driver

* Fix tests

* Fix test

* Fix invalid alert wait

* Fix missing wd

* Fix tests

* Fix missing fs

* Fix test

* Fix path

* Move screenshort to driver

* Migrate keyboard

* Migrate skiplinks to driver

* Fix tests

* Try fix skip-links

* add config

* Add helper

* Fix driver?

* remove helper

* remove wd-helper

* Remove redundant file

* Remove webdriver extsions
@HarelM
Alot of the cypress code and tests
865ea72
@HarelM
remove old tests
2ad4993
@HarelM
Finish migration of tests
9c9d6bc
@HarelM
Add open command to package.json
abfed99
@HarelM
Add to CI
f7de7ff
@HarelM
Remove leftovers
5c7d6f2
@HarelM
Fix build
5d8041f
@HarelM
remove all async part, fix tests
9fb06d1
@HarelM
Fix history test for linux
2b10213
@HarelM
Add firefox to browser tests
2c45b5a
@HarelM @nyurik
Update cypress/e2e/map.cy.ts
a1c2670 
Co-authored-by: Yuri Astrakhan <[email protected]>
@HarelM @nyurik
Update cypress/e2e/driver.ts
2f62c20 
Co-authored-by: Yuri Astrakhan <[email protected]>
@HarelM @nyurik
Update config/webpack.production.config.js
c6bfd96 
Co-authored-by: Yuri Astrakhan <[email protected]>
@HarelM @nyurik
Update config/webpack.profiling.config.js
4654219 
Co-authored-by: Yuri Astrakhan <[email protected]>
@HarelM
Merge branch 'maputnik:master' into master
d4c9b99
@HarelM
Merge branch 'master' into cypress
011fb77
@codesandbox-ci CodeSandbox CI
Copy link

codesandbox-ci bot commented Dec 17, 2023 •
edited
Loading

This pull request is automatically built and testable in CodeSandbox.

To see build info of the built libraries, click here or the icon next to each commit SHA.

Latest deployment of this branch, based on commit 9c15ce8:

Sandbox Source
maputnik Configuration

nyurik
nyurik reviewed Dec 17, 2023
View reviewed changes
Copy link
Member

@nyurik nyurik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good, thx! Is it required to remove the coverage report from the CI?

cypress/support/commands.ts Show resolved Hide resolved
@HarelM
Copy link
Collaborator Author

HarelM commented Dec 17, 2023

I'll work on CI with codecov, thanks for the reminder!

@HarelM
Copy link
Collaborator Author

HarelM commented Dec 17, 2023

I meant code coverage won't be a part of this PR but a layer one.
Let me know if you want any changes or I'll merge this and continue working on other parts.

@nyurik
Copy link
Member

nyurik commented Dec 17, 2023

Look great! Coverage can be done as a separate PR. One thing that must go in as part of this PR is readme/docs update related to the new testing setup

@HarelM
Copy link
Collaborator Author

HarelM commented Dec 17, 2023

Right, I'll update those in the next hour or so.

@HarelM
Copy link
Collaborator Author

HarelM commented Dec 17, 2023

I've updated the readme, is there anywhere else I should be updating?

@nyurik
Copy link
Member

nyurik commented Dec 17, 2023

webdriverio is still listed in the package.json and the lock - is it still needed there?

@nyurik
Copy link
Member

nyurik commented Dec 17, 2023

also there is a hack for webdriver in src/styles/index.scss ?

@HarelM
Copy link
Collaborator Author

HarelM commented Dec 17, 2023

THANKS! Removed now.

nyurik
nyurik approved these changes Dec 17, 2023
View reviewed changes
Copy link
Member

@nyurik nyurik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks! Tested locally, seem to pass. Feel free to squash & merge.

@HarelM HarelM merged commit 1df2e36 into maplibre:master Dec 17, 2023
8 checks passed
@HarelM HarelM deleted the cypress branch December 20, 2023 20:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nyurik nyurik nyurik approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@HarelM @nyurik