Data driven testing with the IDE


This article demonstrates one way to achieve real-time, complex data-driven testing with the Selenium IDE. Yeah, you heard me; real-time, complex data-driven testing with the Selenium IDE. When I say real-time, I mean queried data, not data stored in an XML or CSV file (or any other static data method that has to be refreshed to be considered real-time and valid).

For this scenario, let's make a few assumptions that I believe are actually typical to most companies:

  1. You are a tester (as I was) with the need to perform real-time data-driven testing.
  2. You have a familiarity with programming, but would definately need help with more advaced techniques like creating methods for database connections
  3. You will need to support any automation scripts that are created in the process and that the programming resources are always leveraged to the hilt and might not be available to help you do what you need to do to support the code they write (I know, the dog chasing its tail ... but we all know it's real life)

I make these assumptions because I feel they are very typical and help to eliminate the following options:

  • Using Selenium RC or Webdriver and writing the code you need to make the database connections as well as the rest of the functional scripts that use the data. Yes, its possible, and maybe you are already an advanced automation scripter/programmer, but this is really something that could not easily be supported by the rest of the testing team. Many companies get themselves into the position where they have a *specialized* automation testing team, because they are at least realistic in understanding that a majority of testers are NOT developers even if they do dabble). However, because creating the automation scripts required specialized resources, and there are often far fewer of them then regular testers, their auotmation team becomes a bottleneck waiting for scripts to be written that the testing teams need at a much faster rate then they can be produced.
  • Assuming the developers will be able to afford the time to *help* the tester by writing some or all of the scripts required as well as maintaining them in the future. Again, a bottleneck ... which really just means the developers are writing the automation tests ... not the testers.
  • Not doing as much complex automation work as you could because of either of the above conditions. Many testing managers just make the assumption they will get what they can, abandon thier aspirations (if they even ever had any) of real-time data-driven testing with a tool that allows midlevel and advanced testiing resources to accomplish the goal without being or leaning hard on the programmers.

What does that leave us with? Testers writing automation scripts. Weird, right? Well there are a couple of different approaches, and one of them might be to download the data-driven user-extension that was contributed to the Selenium community However, this is NOT real-time data. It relies on XML static based data. That means you would have to constantly update or maintian these data documents. Data becomes state, burned, out of sync, etc... and a true dynamic, dependant test might not even make using these static lists very practical since something you want to test in routine 5, was dependant on something you did way back in routine 2. You would then have to have multiple lists of state-based data.  What then can we do?

The solution is actually easy, it just means the length of the test might be longer. My solution was to implement an open-source web-based database report generator. This middleware application would basically connect to the AUT (Application Under Test) in your development or test enviornment and allow the testers to either setup canned reports that would server up real-time data for any query requested, or allow for new ad-hoc reports to be created during the testing process.

An example workflow for one of these scripts might be:

  1. The script starts and loads any test setup data
  2. The script executes a few commands irrelevant to the data-driven tests
  3. The script opens the page for the web-based report generator for the canned report that queries the options that should be available for the AUT on samplepage.aspx in select list XYZ.
  4. The scripts stores the table data on the web page as a stored variable array (i.e., option1,option2,option3,option4,etc.)
  5. The script opens samplepage.aspx for the AUT and executes a verifySelectOptions command using the stored variable in step 4

Done. You just performed real-time, data-driven tests with the Seleium IDE, and probably only added 2-3 seconds for the script to read the data from a third party web-based report generator. More importanty, you enabled the testers the ability to perform data-driven tests without programming. The canned reports can be easily updated, shared, etc ... The order the options appear in the select list can be controlled in the canned query. In fact, as long as you have verified the SQL is correct, you can simply copy and paste the query being used to populate the list in the first place.

This is only a simplified example to demonstrate how this can be done. You could use any web-based database report generator. Point is, the programmer is not required!

Related Articles: