Documentation

Learn how to install the plugin, configure tests, manage notifications, review results, and integrate visual testing into your CI/CD workflow via API.

Plugin Installation

To install the plugin on your website, choose one of the following approaches.

Note: The website must be publicly accessible in order to set up and run the tests. Password protection or any kind of firewall might prevent the plugin from working correctly. You may whitelist the screenshotter IP.

—

Install from your WordPress backend

Log in to your WordPress admin dashboard.
Navigate to Plugins > Add New.
In the search bar, type “vrts”.
Locate the Visual Regression Tests plugin in the results.
Click Install Now, then Activate once installation is complete.

—

Download from the Plugin Directory

Visit the Visual Regression Tests plugin page.
Click the Download button to get the plugin’s ZIP file.
In your WordPress dashboard, go to Plugins > Add New.
Click on Upload Plugin.
Choose the downloaded ZIP file and click Install Now.
Activate the plugin after installation.

—

Install via Composer

If you use Composer for dependency management, you can install the plugin via wpackagist:

composer require wpackagist-plugin/visual-regression-tests

After installation, ensure the plugin is activated in the WordPress backend or via WP Cli.

Test Configuration

Proper test configuration is crucial for accurate visual regression testing. Below are the key configuration options.

—

Adding Tests

You can add any public post or page, including custom post types, to VRTs. There are two ways to do this:

From within the VRTs plugin: Go to Tests > Add New, select the post or page you want to test from the list and add it to VRTs.
From within a post type: Navigate to the posts or pages list of your desired post type. Use the Bulk Actions dropdown menu, select Add to VRTs, and apply it to add multiple items to VRTs at once.
From within a post or page: Navigate to the VRTs meta box in the sidebar and activate the option “Add to VRTs”.

—

Click Element

Define an element that should be clicked before taking a screenshot. This is useful for closing cookie banners or modals.

How to configure:

In the Settings, find the Click Element field.
Enter a CSS selector for the element to be clicked.

Note: If multiple elements are found for the specified selector, only the first occurrence of the element will be clicked.

—

Hide Elements

Hide dynamic or irrelevant elements to prevent false positives in your tests.

Tips on what to hide:

Animations: Auto-playing animations, such as Lottie files, sliders, or animated icons.
Dynamic Content: Elements like rotating banners, live feeds, or related content.
Advertisements: Ad blocks that change content regularly.

How to configure:

Find the Hide Elements textarea in the post meta box, in a tests quick edit options, or in the context menu of an alert.
Enter one or multiple CSS selectors of elements to hide, separated by commas.

Note: The elements will be rendered invisible by applying the CSS declaration visibility: hidden.

—

Testing Post Archives

The VRTs plugin primarily supports WordPress pages and posts. Automated visual testing of pages with dynamically changing content, such as archive pages, can lead to false positives. However, you can still test these pages by following these steps:

Create a new blank page or post in WordPress.
Set up a redirect from this new page to your desired archive or dynamic URL.
Configure a test for this new page. The screenshotter will follow the redirect, capturing the target page.

Note: To minimize false positives, exclude elements that change frequently, such as titles, excerpts, post dates, and featured images, by adding their CSS selectors to the Hide Elements field. you may still encounter false positives if the layout shifts.

—

Whitelisting IPs

If your site restricts access with a firewall, you may need to whitelist our screenshot service IP to ensure that it can browse and screenshot your pages. The VRTs screenshot service uses the following IP address:

49.13.14.240

Example: Whitelisting IPs in Cloudflare

To whitelist the VRTs IP in Cloudflare, follow these steps:

Log in to your Cloudflare account.
Navigate to Security > WAF (Web Application Firewall).
Click on Tools.
In the IP Access Rules box, enter the IP address.
Select Allow from the action dropdown.
Optionally, add “VRTs” as a note to indicate this rule’s purpose.
Click Add to apply the rule.

—

Multiple Environments

Using the VRTs plugin across different environments, like staging and production, allows you to test changes before pushing them live. However, when syncing environments, the plugin’s test data can be affected if not handled correctly. Here’s how to manage VRTs data effectively during deployment:

Steps for syncing environments

Sync VRTs options to staging: Export the VRTs options from production and import them to staging to ensure settings consistency. You can do this by selecting all VRTs-related entries from the options table:
```
SELECT * FROM wp_options WHERE option_name LIKE 'vrts_%';
```
Exclude VRTs tables from production sync: To retain tests on production and avoid overwriting them, exclude the following tables when syncing:
- vrts_alerts
- vrts_tests
- vrts_test_runs
This prevents production tests from being overridden, allowing you to compare screenshots before and after the deployment.

Note: Syncing options back to staging is only necessary if plugin settings, like notification email, click element, or the license key, are changed.

Test Automation

There are multiple ways to trigger tests, ensuring you can monitor changes across different scenarios. Tests can be initiated automatically or manually, depending on your needs.

Note: Screenshots and comparisons are performed on an external server and sent to your WordPress website. All generated images are stored on AWS. Only required meta data is stored in your database.

—

Schedule

All tests are automatically scheduled to run every 24 hours, providing a daily snapshot and comparison of your site’s appearance. This continuous testing helps you promptly catch unexpected visual changes and ensures that the reference screenshot remains relevant for accurate comparisons.

—

Update

Tests are triggered automatically whenever updates occur, such as changes to plugins, themes, translations, or WordPress core. This ensures that any visual changes resulting from updates are detected, helping you maintain stability across upgrades.

How it works

Tests automatically run after updates to plugins, themes, translations or WordPress core.
The plugin uses WordPress hooks to ensure tests run whether you use auto-updates or external management tools.
Multiple updates are combined to minimize redundant tests. You may notice a brief delay before the update test starts, as it waits for any subsequent updates to be included.

Note: If you update dependencies directly on the file system, such as by using Composer or CI/CD, you’ll need to trigger tests via the API.

—

Manual

You can run tests manually on demand, either for all configured pages or for specific individual pages. Manual testing is ideal for verifying changes immediately after global edits, giving you instant feedback on visual integrity.

Run All Tests:

Navigate to Tests.
Click on Run All Tests to initiate a test run for all configured pages.

Run Individual Tests:

In the tests list, find the specific page.
Click the Run Test action to initiate a test run for this page.

—

API

Trigger tests via PHP scripts or WP CLI for integration with other tools, deployment pipelines or custom automations.

Capabilities:

Trigger all active tests.
Add optional notes (to label actions, versions, or deployment identifiers).

Triggering Tests via PHP

To trigger all tests from a PHP script on a site where the plugin is installed, use the following WordPress action:

do_action('vrts_run_tests', 'Your optional notes');

Triggering Tests via WP-CLI

If WP-CLI is available on your server, you can also trigger tests directly from the command line:

wp eval 'do_action("vrts_run_tests", "Your optional notes");'

User Notification

Specify email addresses for each trigger, to set up notifications for various teams or stakeholders.

—

Example scenario for Triggers:

Schedule -> Client: Send test results directly to clients, allowing them to monitor site stability and handle alerts independently.
Update -> Support: Notify your support team of changes introduced by new versions, enabling them to quickly identify and address any issues.
API -> Devs & PM: Alert developers and PMs on deployments to verify that code changes are intentional and free of unintended side effects.
Manual -> User: Notifications for manual tests are automatically sent to the user who initiated the test, making it easy for any stakeholder to review their test results directly.

Note: If no email address is provided in the settings field for a specific trigger, no email notification will be sent. The tests will still run, and test results will appear in the plugin backend.

Test Review

Reviewing test results helps you identify and address visual changes efficiently. Below are tools and options to streamline your review process.

—

Read / Unread

Keep track of test results that require further attention by marking alerts as unread.

How it works:

Alerts are automatically marked as read when viewed.
A run is considered “read” when it contains no unread alerts.
You can mark individual alerts as unread from the alert’s context menu.

—

Flag as False Positive

Identify acceptable changes or non-issues to minimize unnecessary notifications.

How it works:

When an alert is flagged as a false positive, identical alerts will be suppressed in future test runs.
Only pixel-perfect duplicates of the flagged alert will be ignored, ensuring accuracy in future alerts.
To mark an alert as a false positive, open the alert’s context menu and select Flag as false positive.

—

Hiding Elements

Adjust your test configuration based on review findings by hiding elements that frequently cause false positives. This is achieved by adding a CSS selector for the elements to the Hide Elements list, which is then saved to the test.

How it works:

In the context menu of an alert, select Hide elements.
Add the CSS selectors of elements to hide, which will be stored with the test for future runs.

For more information, see Hide Elements.

Subscription Management

To upgrade, purchase a license key through the Upgrade menu item in the plugin. Once you have your license key, enter it to activate your subscription.

If your license key becomes inactive, all but three tests will be paused. You can reactivate the paused tests by re-entering a valid license key.

For further assistance or inquiries, visit the plugin’s support forum or contact us.