Supercharging e2e Testing with Cypress and Airbyte’s Config API

During this round of Hack Days at Airbyte, my two-day objective was clear - boost the speed of our front-end end-to-end (e2e) test suite. We use Cypress for frontend e2e testing, running an OSS instance in CI and letting Cypress simulate user operations in a headless browser. As useful as this is, some areas of our suite ended up slow and flaky due to redundant operations.

Teaming up with a fellow engineer, Ella Rohm-Ensing, we optimized our e2e suite using Airbyte’s Config API to generate fixtures for sources, destinations, and connections. This is the same API our web app uses to communicate with the Platform. We aimed to reduce our slowest suite’s runtime by about 30%.

Luckily, a previous engineer already started some groundwork for a Cypress to Airbyte Config API integration, enabling us to make significant progress in two days. We ended up being able to leverage the config API in three suites, and while we were in there we managed to remove redundant tests from three suites and increase coverage in others.

The payoff was substantial:

• Overall decrease in the frontend e2e runtime: ~60%
• Overall decrease in runtime within the slowest suite: ~96%

Before:

After:

Note that some tests moved (connection.cy.ts now lives at connection/configuration.cy.ts), and a handful of tests were added/removed in various suites to increase coverage and reduce redundancy.

Overall, the frontend chapter is thrilled that our PRs can pass CI much more quickly now.  And while this project did not reduce the backend Platform PR build time due to parallelization, the tests are more reliable and easier to debug if something does fail.

While the goal of Hack Days is to create a prototype, not a pull request… we did end up merging this one. 

You can check out the changes we made here!