Refactoring environments and introducing a new look

The keyword of our latest release is simplification. At Frontend Robot we believe that improving the User Experience is as important as adding new features. What’s the point of adding new features, if the user can’t find their way to using them? A simple to use product is a powerful product.

The problem with environments

Test environments allowed a user to run the same test suite with a different set of variables and platforms. The main use case for environments was to allow a user to run the same test suite on different product deployments - such as production or staging - without the need to duplicate tests or suites.

However, during the last few months we noticed that users were generally confused by the feature. First of all, environments were optional, which created a fork in the workflow of the user. A test suite with environments would look different than a test suite without environments. The UI elements for accessing the same functionality (e.g. running the suite) were placed in different locations depending on the presence/absence of environments in the suite.

This difference in the user experience between the two use cases was already an indicator that something could be improved.

But the ultimate factor that contributed to our decision to rethink environments, was the fact that often we don’t want to run exactly the same test cases on two different deployments. For example, the practice is to run a more comprehensive set of tests on staging compared to production. Also, we often can’t perform some operations on a certain deployment, think about testing a payment on production.

More powerful test suites

So after some intense designing sessions, we came up with what we consider the perfect solution; we removed completely the concept of test environment and we moved its functionality directly into suites. This means that now you can attach a set of variables and platforms directly to a test suite rather than to an environment.

As you can see from the screenshots above, we can now assign variables and platforms directly to a test suite. The beauty of this simple change is that we removed environments but we kept the same level of functionality. For example, now if we want to run the same tests on different deployments all we have to do is creating two different test suites containing the same tests, and assigning them different variables - for example a different starting URL for the tests. This is incredibly simple and intuitive, comparing to having different environments within the same suite. Also, we can now run different tests on different deployments, while sharing just a few between them.

MIGRATION NOTE: All the data of your existing test environments will be migrated into separate test suites. For each test suite/test environment pair you will now find a new test suite named <Test Suite> [<Test Environment>] containing all the test of the original suite and the variables/platforms of the environment.

Sharing tests between suites

To enable suites to mimic the role that environments had in the past, we had to allow a test case to be shared between different test suites. This way the same test case can be run with a different set of variables and platform, without the need of re-writing it.

To import a test case from another test suite, all you have to do is to go to the Tests tab and click on Add Existing.

A dialog will open showing all other test suites, from where we can import other test cases.

A new UI

As you may already have noticed from the screenshots above, with this new release we also introduced a new, simplified UI.

The new UI is also the result of our commitment to providing an outstanding user experience driven by simplicity.

First of all in the new UI we rely less on icons and more on colors and words. Icons are sometimes subject to interpretation, basic colors - such as green and red - and words provide instead a clearer and more immediate meaning.

The toolbar has been moved from the top to the left, for an optimal management of the screen real estate on modern wide-screen monitors.

All features that need to be accessed less frequently such as webhooks, test scheduling, variables and platforms are now all under the Settings tab of a test suite.

Closing remarks

This new release also introduces many other improvements under the hood, such as a better CSS selector generator engine and countless bug fixes.

We hope that this new release resonates with you as it did with us, and as always if you have any questions or feedback you can send me an email at mario@frontendrobot.com or chat to us on our website.