-
Notifications
You must be signed in to change notification settings - Fork 3.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature]: single aria snapshot for different engines/browsers #34483
Comments
You are getting separate snapshots because you are using separate projects for each browser, which will create separate files by default. You can use |
Thanks for suggesting this. I tried it, however we do have some visual diff testing using |
You're right. This is something we need to improve on. For now, you can use the explicit inline Aria snapshot mode: await expect(page.locator('body')).toMatchAriaSnapshot(`
- heading "todos"
- textbox "What needs to be done?"
`); |
Thank you!! |
This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [@playwright/test](https://playwright.dev) ([source](https://github.com/microsoft/playwright)) | devDependencies | minor | [`1.49.1` -> `1.50.1`](https://renovatebot.com/diffs/npm/@playwright%2ftest/1.49.1/1.50.1) | --- ### Release Notes <details> <summary>microsoft/playwright (@​playwright/test)</summary> ### [`v1.50.1`](https://github.com/microsoft/playwright/releases/tag/v1.50.1) [Compare Source](microsoft/playwright@v1.50.0...v1.50.1) ##### Highlights microsoft/playwright#34483 - \[Feature]: single aria snapshot for different engines/browsers microsoft/playwright#34497 - \[Bug]: Firefox not handling keepalive: true fetch requests microsoft/playwright#34504 - \[Bug]: update snapshots not creating good diffs microsoft/playwright#34507 - \[Bug]: snapshotPathTemplate doesnt work when multiple projects microsoft/playwright#34462 - \[Bug]: updateSnapshots "changed" throws an error #### Browser Versions - Chromium 133.0.6943.16 - Mozilla Firefox 134.0 - WebKit 18.2 This version was also tested against the following stable channels: - Google Chrome 132 - Microsoft Edge 132 ### [`v1.50.0`](https://github.com/microsoft/playwright/releases/tag/v1.50.0) [Compare Source](microsoft/playwright@v1.49.1...v1.50.0) #### Test runner - New option [`timeout`](https://playwright.dev/docs/api/class-test#test-step-option-timeout) allows specifying a maximum run time for an individual test step. A timed-out step will fail the execution of the test. ```js test('some test', async ({ page }) => { await test.step('a step', async () => { // This step can time out separately from the test }, { timeout: 1000 }); }); ``` - New method [test.step.skip()](https://playwright.dev/docs/api/class-test#test-step-skip) to disable execution of a test step. ```js test('some test', async ({ page }) => { await test.step('before running step', async () => { // Normal step }); await test.step.skip('not yet ready', async () => { // This step is skipped }); await test.step('after running step', async () => { // This step still runs even though the previous one was skipped }); }); ``` - Expanded [expect(locator).toMatchAriaSnapshot()](https://playwright.dev/docs/api/class-locatorassertions#locator-assertions-to-match-aria-snapshot-2) to allow storing of aria snapshots in separate YAML files. - Added method [expect(locator).toHaveAccessibleErrorMessage()](https://playwright.dev/docs/api/class-locatorassertions#locator-assertions-to-have-accessible-error-message) to assert the Locator points to an element with a given [aria errormessage](https://w3c.github.io/aria/#aria-errormessage). - Option [testConfig.updateSnapshots](https://playwright.dev/docs/api/class-testconfig#test-config-update-snapshots) added the configuration enum `changed`. `changed` updates only the snapshots that have changed, whereas `all` now updates all snapshots, regardless of whether there are any differences. - New option [testConfig.updateSourceMethod](https://playwright.dev/docs/api/class-testconfig#test-config-update-source-method) defines the way source code is updated when [testConfig.updateSnapshots](https://playwright.dev/docs/api/class-testconfig#test-config-update-snapshots) is configured. Added `overwrite` and `3-way` modes that write the changes into source code, on top of existing `patch` mode that creates a patch file. ```bash npx playwright test --update-snapshots=changed --update-source-method=3way ``` - Option [testConfig.webServer](https://playwright.dev/docs/api/class-testconfig#test-config-web-server) added a `gracefulShutdown` field for specifying a process kill signal other than the default `SIGKILL`. - Exposed [testStep.attachments](https://playwright.dev/docs/api/class-teststep#test-step-attachments) from the reporter API to allow retrieval of all attachments created by that step. - New option `pathTemplate` for `toHaveScreenshot` and `toMatchAriaSnapshot` assertions in the [testConfig.expect](https://playwright.dev/docs/api/class-testconfig#test-config-expect) configuration. #### UI updates - Updated default HTML reporter to improve display of attachments. - New button for picking elements to produce aria snapshots. - Additional details (such as keys pressed) are now displayed alongside action API calls in traces. - Display of `canvas` content in traces is error-prone. Display is now disabled by default, and can be enabled via the `Display canvas content` UI setting. - `Call` and `Network` panels now display additional time information. #### Breaking - [expect(locator).toBeEditable()](https://playwright.dev/docs/api/class-locatorassertions#locator-assertions-to-be-editable) and [locator.isEditable()](https://playwright.dev/docs/api/class-locator#locator-is-editable) now throw if the target element is not `<input>`, `<select>`, or a number of other editable elements. - Option [testConfig.updateSnapshots](https://playwright.dev/docs/api/class-testconfig#test-config-update-snapshots) now updates all snapshots when set to `all`, rather than only the failed/changed snapshots. Use the new enum `changed` to keep the old functionality of only updating the changed snapshots. #### Browser Versions - Chromium 133.0.6943.16 - Mozilla Firefox 134.0 - WebKit 18.2 This version was also tested against the following stable channels: - Google Chrome 132 - Microsoft Edge 132 </details> --- ### Configuration 📅 **Schedule**: Branch creation - "* 0-3 1 * *" (UTC), Automerge - "* 0-3 * * *" (UTC). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xMzYuMSIsInVwZGF0ZWRJblZlciI6IjM5LjEzNi4xIiwidGFyZ2V0QnJhbmNoIjoiZm9yZ2VqbyIsImxhYmVscyI6WyJkZXBlbmRlbmN5LXVwZ3JhZGUiLCJ0ZXN0L25vdC1uZWVkZWQiXX0=--> Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/6756 Reviewed-by: Earl Warren <[email protected]> Co-authored-by: Renovate Bot <[email protected]> Co-committed-by: Renovate Bot <[email protected]>
This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [@playwright/test](https://playwright.dev) ([source](https://github.com/microsoft/playwright)) | devDependencies | minor | [`1.49.1` -> `1.50.1`](https://renovatebot.com/diffs/npm/@playwright%2ftest/1.49.1/1.50.1) | --- ### Release Notes <details> <summary>microsoft/playwright (@​playwright/test)</summary> ### [`v1.50.1`](https://github.com/microsoft/playwright/releases/tag/v1.50.1) [Compare Source](microsoft/playwright@v1.50.0...v1.50.1) ##### Highlights microsoft/playwright#34483 - \[Feature]: single aria snapshot for different engines/browsers microsoft/playwright#34497 - \[Bug]: Firefox not handling keepalive: true fetch requests microsoft/playwright#34504 - \[Bug]: update snapshots not creating good diffs microsoft/playwright#34507 - \[Bug]: snapshotPathTemplate doesnt work when multiple projects microsoft/playwright#34462 - \[Bug]: updateSnapshots "changed" throws an error #### Browser Versions - Chromium 133.0.6943.16 - Mozilla Firefox 134.0 - WebKit 18.2 This version was also tested against the following stable channels: - Google Chrome 132 - Microsoft Edge 132 ### [`v1.50.0`](https://github.com/microsoft/playwright/releases/tag/v1.50.0) [Compare Source](microsoft/playwright@v1.49.1...v1.50.0) #### Test runner - New option [`timeout`](https://playwright.dev/docs/api/class-test#test-step-option-timeout) allows specifying a maximum run time for an individual test step. A timed-out step will fail the execution of the test. ```js test('some test', async ({ page }) => { await test.step('a step', async () => { // This step can time out separately from the test }, { timeout: 1000 }); }); ``` - New method [test.step.skip()](https://playwright.dev/docs/api/class-test#test-step-skip) to disable execution of a test step. ```js test('some test', async ({ page }) => { await test.step('before running step', async () => { // Normal step }); await test.step.skip('not yet ready', async () => { // This step is skipped }); await test.step('after running step', async () => { // This step still runs even though the previous one was skipped }); }); ``` - Expanded [expect(locator).toMatchAriaSnapshot()](https://playwright.dev/docs/api/class-locatorassertions#locator-assertions-to-match-aria-snapshot-2) to allow storing of aria snapshots in separate YAML files. - Added method [expect(locator).toHaveAccessibleErrorMessage()](https://playwright.dev/docs/api/class-locatorassertions#locator-assertions-to-have-accessible-error-message) to assert the Locator points to an element with a given [aria errormessage](https://w3c.github.io/aria/#aria-errormessage). - Option [testConfig.updateSnapshots](https://playwright.dev/docs/api/class-testconfig#test-config-update-snapshots) added the configuration enum `changed`. `changed` updates only the snapshots that have changed, whereas `all` now updates all snapshots, regardless of whether there are any differences. - New option [testConfig.updateSourceMethod](https://playwright.dev/docs/api/class-testconfig#test-config-update-source-method) defines the way source code is updated when [testConfig.updateSnapshots](https://playwright.dev/docs/api/class-testconfig#test-config-update-snapshots) is configured. Added `overwrite` and `3-way` modes that write the changes into source code, on top of existing `patch` mode that creates a patch file. ```bash npx playwright test --update-snapshots=changed --update-source-method=3way ``` - Option [testConfig.webServer](https://playwright.dev/docs/api/class-testconfig#test-config-web-server) added a `gracefulShutdown` field for specifying a process kill signal other than the default `SIGKILL`. - Exposed [testStep.attachments](https://playwright.dev/docs/api/class-teststep#test-step-attachments) from the reporter API to allow retrieval of all attachments created by that step. - New option `pathTemplate` for `toHaveScreenshot` and `toMatchAriaSnapshot` assertions in the [testConfig.expect](https://playwright.dev/docs/api/class-testconfig#test-config-expect) configuration. #### UI updates - Updated default HTML reporter to improve display of attachments. - New button for picking elements to produce aria snapshots. - Additional details (such as keys pressed) are now displayed alongside action API calls in traces. - Display of `canvas` content in traces is error-prone. Display is now disabled by default, and can be enabled via the `Display canvas content` UI setting. - `Call` and `Network` panels now display additional time information. #### Breaking - [expect(locator).toBeEditable()](https://playwright.dev/docs/api/class-locatorassertions#locator-assertions-to-be-editable) and [locator.isEditable()](https://playwright.dev/docs/api/class-locator#locator-is-editable) now throw if the target element is not `<input>`, `<select>`, or a number of other editable elements. - Option [testConfig.updateSnapshots](https://playwright.dev/docs/api/class-testconfig#test-config-update-snapshots) now updates all snapshots when set to `all`, rather than only the failed/changed snapshots. Use the new enum `changed` to keep the old functionality of only updating the changed snapshots. #### Browser Versions - Chromium 133.0.6943.16 - Mozilla Firefox 134.0 - WebKit 18.2 This version was also tested against the following stable channels: - Google Chrome 132 - Microsoft Edge 132 </details> --- ### Configuration 📅 **Schedule**: Branch creation - "* 0-3 1 * *" (UTC), Automerge - "* 0-3 * * *" (UTC). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xMzYuMSIsInVwZGF0ZWRJblZlciI6IjM5LjEzNi4xIiwidGFyZ2V0QnJhbmNoIjoiZm9yZ2VqbyIsImxhYmVscyI6WyJkZXBlbmRlbmN5LXVwZ3JhZGUiLCJ0ZXN0L25vdC1uZWVkZWQiXX0=--> Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/6756 Reviewed-by: Earl Warren <[email protected]> Co-authored-by: Renovate Bot <[email protected]> Co-committed-by: Renovate Bot <[email protected]>
🚀 Feature Request
In the latest release v1.50 aria-snapshot can be generated using
toMatchAriaSnapshot({ name: 'snapshot.yml })
but this expects different snapshot for chromium, firefox and webkit on windows and linux.The aria snapshot will not be different for the above different scenarios, can aria-snapshot work with single yml file please. Thanks
Example
No response
Motivation
This will reduce the effort of copy pasting same regex in different files
The text was updated successfully, but these errors were encountered: