Is Selenium Dead? Learn About Automated Testing With Cypress.io

Just a few years ago Selenium was the go-to tool for creating automated functional tests for your web application. Now a new automation framework called Cypress has made its way onto the scene and has surpassed Selenium in downloads for JavaScript usage. We will walk you through the advantages of Cypress and then let you know whether we believe Selenium is indeed dead.

Is Cypress better than Selenium?

Cypress is an automation-as-code tool using JavaScript. One of its major advantages is that one download gives you all the browser engines, frameworks and assertion libraries needed to get started with automation. With Selenium, you must separately download the Selenium engine, chromedriver, multiple test frameworks and assertion libraries. Cypress features easier debugging and extensive documentation on how to best use their API. Test runs are more reliable, and its code is easier to read. We were also impressed with the size of the active community for Cypress and how many useful NPM packages and add-ons exist. Cypress is an open-source tool, meaning it can be downloaded for free. They do offer a licensed version that adds email support and a test summary dashboard, which might be useful to some but is certainly not a must-have. 

The Cypress Test Runner, included in the free version, has a unique, user-friendly, and interactive interface that displays helpful information about your tests. The Test Runner gives you a visual step-by-step breakdown of each test as it executes which remains on-screen for diagnosis after your test run is complete. The Test Runner automatically re-runs your tests every time they are modified and saved. You can also see before and after statuses of your application when doing actions like cy.click(). Cypress automatically produces screenshots and videos for each of your test runs. With Selenium, screenshots and videos require extra work to set up or possibly a third-party paid tool.

Finding elements easily 

One of the advantages of the Cypress library is that you do not have to identify the element to use the element in the test. Using the cy.contains() command, you can select plain or button text on the webpage and Cypress will find that text in the test without you having to inspect the DOM and retrieve the element from the HTML. Elements often get changed with code updates to a web page causing tests to break, whereas with retrieving the text on the page the tests will not break unless the page text itself is changed. An example of how this looks with Cypress is below:

cy.contains('World Wide Technology').should('be.visible').click() 

A slight limitation to this is that if the text is repeated multiple times on the page, the command will fail, or it will take the first element. Cypress has a solution for this by allowing the element type to be added to the text such as with the H3 below. 

cy.contains('h3', 'World Wide Technology').should('be.visible').click() 

If you still have multiple text of the same element type, Cypress will take all of the elements found in the cy.contains() text and turn those elements into an array based on the order found in the HTML. In the example below, you can choose which element to select by using the .eq(#), the 1 is the second element with text of H3 World Wide Technology on the page. 

cy.contains('h3', 'World Wide Technology').eq(1).should('be.visible').click() 

This is not an option with Selenium. You still must retrieve the element by an attribute like id and are not able to simply utilize the text from the webpage.

driver.findElement(By.id('page-title')).should('be.visible').click() 

Even if you want to select by text in Selenium, you still have to inspect the DOM and find what type of element the text is affiliated with.

driver.findElement(By.Xpath("//h3[text()='World Wide Technology'")).should('be.visible').click()

If your code does not have id attributes this can be even more challenging. You would have to find the element by XPath which could result in having to go through multiple layers of the HTML. This also makes the automation code very fragile as one simple update to a web page's structure can break all the tests that interact with that page. 

driver.findElement(By.Xpath("(//*[contains (@class, 'titlebar-container')]//*[contains (@class, 'page-title')])").should('be.visible').click() 

You can see how using cy.contains() can be very advantageous.

The downfalls of Selenium

Selenium test runs can be slower and more brittle than Cypress. There is a higher likelihood of failures even when there is nothing wrong with your application. It can take a highly experienced automation engineer to get around the brittle nature of Selenium. Promises are handled less gracefully thus the code is harder to read because of the reliance on async/await and .then. The order of execution within a test file is alphabetical meaning the last test could run first if it starts with 'A'. Creating independent tests is the proper way to avoid that but we realize some may not follow that discipline. With Cypress, the test execution is from top to bottom of the test file in the order tests are written. We mentioned how Selenium is more challenging and time-consuming to set up. Another pitfall is that the community support and documentation around Selenium is becoming stale. We recently used Selenium for a client project here at WWT and were astonished by how many of the online support articles were outdated.

Does that mean Selenium is dead?

No, we do not believe Selenium is dead. In fact, while we love the advantages of Cypress, there are many times where Selenium may be more appropriate because of things Cypress cannot do. Selenium can handle iFrames whereas with Cypress it is very challenging. We recently completed a test automation project for a client where we had to shift from Cypress to Selenium because the client's third-party authentication took over the Cypress Test Runner and could not stay within the iFrame. Selenium can switch between browser tabs and handle external links, something Cypress cannot do. Selenium can also be written in multiple languages like Java, C# and Python in addition to JavaScript whereas Cypress is only JavaScript or Typescript. You can use Selenium to move the mouse to specific coordinates within the browser. It also simulates keypresses and clicks whereas Cypress does not provide true user simulations for these actions.  

One more advantage is that Selenium has add-ons like Winium where you can access the desktop in your browser tests. On the Insitu project, WWT created a two-hundred test automation suite for the client using Selenium WebDriver. One of the tests focused on a drag-and-drop feature. We were able to use Winium to control the desktop, grab a file, drag it into the web browser and verify the file was uploaded to the webpage, all in one test. Cypress does not have this capability with the desktop.

Authentication — a common issue

Authentication can be challenging with automation tools in general. The larger the automation suite, the more complex it can become. With Selenium and Cypress, you must authenticate for each class or test file, adding considerable time to your test run assuming you authenticate several or dozens of times for each test user.   In our original version of this article we had detailed a custom solution WWT created to read, preserve and set cookies in order to solve the issue of avoiding user login for each test file.  With Cypress 12 this solution no longer worked due to the deprecation of Cypress' cy.preserve() command.  Cypress replaced this behavior with their new session command.  The cy.session() command, when configured properly, will cache and restore cookies for an authenticated user between test files, accomplishing the same result that WWT created with our custom cookie functionality previously.  

Utilizing Cy.Session() to Preserve Cookies

Cypress.Commands.add('loginExternalUser', () => {
  cy.session('loginExternalUser', () => {
    cy.visit('/')
    cy.get(login.loginButton).click()
    cy.get('input[name=username]').type(Cypress.env('cypressExternalUserName'))
    cookiesCheck()
    cy.get('button[aria-label="next"]').click()
    
    cy.get('body').then(($body) => {
      if ($body.find('button[aria-label="Log In With Password"]').length) {
        cy.get('button[aria-label="Log In With Password"]').click()
      }
    })
    
    cy.get('input[name=password]').type(Cypress.env('cypressExternalUserPassword'))
    cy.get('input[type=submit]').click()
    },
    
    {cacheAcrossSpecs: true}
  )
})

Incorporating cy.session() with a login function as shown above will store the cookies for that user's authentication session for use later in the test suite.  The first time Cypress comes to the login command using cy.session() a given user in the test suite, it will perform the full login function. The next time the login command is called for that user (see cy.loginExternalUser() below), the cookies will be read from the user's previous login session and the login function will not execute any longer for the duration of the suite. The user will be authenticated each time their login command is read for the remainder of the suite run.  

describe('Follow Topics', function () {
  before(function () {
    cy.loginExternalUser()
  })

  it('can follow the topic then change preferences in account settings', function () {
    cy.visit('/topic/primary-storage')

    cy.get('body').then(($body) => {
      if ($body.find(topicCategoryPage.unfollowTopicCategory).length) {
        cy.get(topicCategoryPage.unfollowTopicCategory).click()
      }
    })

    cy.get(topicCategoryPage.followTopicCategory).should('be.visible').click()
    cy.get(topicCategoryPage.unfollowTopicCategory).should('be.visible')

    cy.visit('/my/profile-settings/notifications')
    cy.contains('h3', 'Data Center').click()
    cy.contains('label', 'Primary Storage').click()

    cy.visit('/topic/cloud-networking')
    cy.get(topicCategoryPage.followTopicCategory).should('be.visible')
  })
})

You can switch between users by having separate cy.session() commands for each user login within your script. This could mean users in separate test files or even the same test file.  You can also run a test file with no authentication by using the cy.clearAllCookies() command at the start of that specific test file.  Then you could go back to a user again in the next test file by calling the login command from the cy.session().

There are two keys to making this work across multiple users and test files within a Cypress test automation suite.  First, the login function where the cy.session() is incorporated must have the cacheAcrossSpecs setting set to true (see the first code snippet above).  This allows the cookie and session data to be called for the same user across different test or spec files.  Second, the testIsolation configuration in the cypress.config file must be set to false.  

const { defineConfig } = require('cypress')

module.exports = defineConfig({
  e2e: {
    testIsolation: false
  }
})

The Cypress Test Isolation setting has a default of true which clears all cookies and session storage between each test file.  Setting testIsolation to false will allow cookies and storage to remain between tests in your suite.  It will clear the current browser context but not the page.   

Which one has a future in testing Cypress or Selenium?

Selenium still has its place because there could be times when Cypress may not be suitable for your automation needs. We believe that Cypress is a great tool and is worth exploring if you have not experimented with it yet. You will see how easy it is to get started writing tests and the benefit of their Test Runner. Hopefully, you can utilize the cy.session() command to cut some time from your test run the same way it did for us.