datadriven

Data driven testing with Selenium and your XML test data. Your test suite must be running under the Selenium IDE or Selenium Test Runner.

The usage of Selenium-RC is typically suggested as a solution to data driven testing with Selenium. While Selenium-RC is a fine and powerful application the tester must be reasonably comfortable with a programming language to realise their data driven testing goals. This extension adds data driven to the Selenium IDE and Test Runner, enabling the tester to build automated tests using Selenium, to quickly and simply make an existing test case data driven, or to create data driven tests avoiding the use of Selenium-RC for whatever reason.

required extensions

  • flowControl Andrey Yegorov and Darren DeRidder. - Note that you must use Darren DeRidder's version if you're intending on running your scripts within the IDE or TestRunner. I'm choosing to link to Darren's version as I suspect that the majority of testers wishing to use datadriven will be doing exactly that. The Selenium Core version can be found here.
  • include by Jerry Qian, Robert Zimmermann, Paul Hammant, Andy Lawrence and Alex. (Make sure you use the include4IDE version if you are using Selenium IDE).

With appreciation to the authors of the above.

The order in which you configure all three extensions must be;

  1. flowControl
  2. include
  3. datadriven

features

  • Logs each item of test data. This may be useful for tracing why a test case would fail with a particular item (row) of test data.
  • Some rudimentary checks of test data consistency.

example of use

Seven steps toward making a data driven test case.

  1. Create your test case as you normally would.
  2. Write your XML test data file. There must be one 'test' element per required iteration through your test case. Specify your test data as attributes within each 'test' element. All test elements must have the same number of attributes and all attribute names must match throughout the test elements.
  3. At the start of your test case, add a loadTestData command in column one and the location of your XML file in column two. The XML location may be specified using either the http or file protocols.
  4. The next command after loadTestData should be a while statement. The condition of this while statement MUST be !testdata.EOF().
  5. The next command after the while statement must be nextTestData. This will initialise variables as specified in the first/next test data row.
  6. Your test case commands should be specified from this point.
  7. The final command in your data driven test case must be endWhile.

Remember, your XML attribute name and value pairs become Selenium variables. You may reference these variables as you normally would within your test case.

In summary the structure of your data driven test case will be;

Command Target Value
loadTestData <file path>  
while !testdata.EOF()  
nextTestData    
testcommand1    
testcommand...    
testcommandn    
endWhile    

You may find the included exampletest.xml useful. Be aware that you may have to copy the included googlesearches.xml to the root of your C: drive (if you have one) otherwise you must edit the path supplied to the loadTestData command to the file's location of your choice.

limitations

  • Test data will not apply to each test case in your suite. You must configure each test case as described above if you wish for multiple test cases to be data driven.
  • You cannot specify each test data row's expected outcome. The assumption is that the expected outcome will be specified in the test case.
  • Selenium Test Runner's statistics aren't updated. E.g. if there are five rows of test data used within a single test case, as far as the statistics are concerned the number of tests run will be 1.
  • This extension is intended for use under the Selenium IDE or Selenium Test Runner. Do not be surprised if your test script starts to behave oddly if you include this extension under a Selenium-RC test particularly with IEHTA or IExplore as your web client. In any case with the full power of your programming language of choice and Selenium-RC at your fingertips, why on Earth would you want to run this extension under Selenium-RC?
  • Source data in XML format only. If you require data driven testing by connecting to databases this extension will be of little to no use to you. One solution may be to export your data as XML and go from there. On the other hand if you need to connect to a database and can't format the test data how this extension requires it your best bet may be to use Selenium-RC after all.

known issues

  • Using the Selenium Core version of the flowControl extension will result in your being alerted to this error;
    Failed to load user-extensions.js!
    ...
    error=ReferenceError: HtmlRunnerTestLoop is not defined.

This message can be safely ignored as flowControl's primary intended use is within Selenium Core. I strongly recommend the use of Darren DeRidder's version of flowControl which safely works under the Selenium IDE.

help & support

My email address is within the comments of the datadriven extension's Javascript. You may contact me for help and support with this extension but please (since I have a real life to live), only after you've done the following:

  1. Read all of the above documentation carefully and thoroughly.
  2. Exhausted all possible means at your disposal of resolving the issue yourself.
  3. Read all of the above documentation carefully and thoroughly again.
  4. Made further efforts to resolve the issue by yourself.
  5. Searched the Selenium IDE forums for similar issues to your own in an attempt to resolve your issue.

I'll welcome your mail if you've done the above and absolutely require help. In this case;

  1. Keep it relevant to Selenium and the datadriven extension.
  2. It's a shame this has to be said but please exercise some courtesy and professionalism. Being rude and abrupt won't get you anywhere other than my spam senders list.
  3. Please be detailed with regard to what you're trying to achieve.
  4. Please be detailed with regard to the problem you're facing and the actual outcome you're experiencing.
  5. Provide as many relevant resources and code as possible. E.g. all user extensions files in use, your test scripts and test data. Please note that I am not interested in your employer's actual data, active logins or anything of this nature. Any mails containing what I suspect to be material sensitive to your employer will be returned with a polite request to re-read these notes. Repeat offenders will be permanently added to my spam senders list.
  6. Finally, when you've done all that... Please be patient. You will get a response.

File: 

AttachmentSize
Plain text icon datadriven.js.txt2.83 KB

Type: 

User-Extension