fbpx

Importing Automated Results – Scheduled Import

Duette Schedules allow you to create schedules where individual or multiple sets of result files can be imported into Enterprise Tester, with the associated automated test, automated test assignment and runs being created automatically.

Features

Duette Schedules supports the following features:

  • Ability to schedule importing on an ad-hoc, periodic or daily basis.
  • Support for multiple schedules.
  • Support for record summary and details history of imports.
  • Ability to manually trigger an import schedule.
  • Ability to import multiple files using a file name “capture” within the source path of the tests.
  • Ability to import multiple files using a folder name “capture” within the source path of the tests (can not be combined with filename capture).
  • Ability to combine multiple source files into a single automated test result.
  • Ability to either always import results on scheduled run, or only import results if the source files have changed since last time.
  • Ability to duplicate an existing import configuration.
  • Ability to enable or disable import configurations and schedules.
  • Ability to automatically maintain only a certain number of result files (automatically purging the oldest runs over the maximum number of retained results).

Creating an Import Schedule

You can configure Duette Schedules from the Resources tab of the tree view navigator.  When the Duette Plugin is enabled, a section called “Duette Schedules” is available.

If Duette Plugin is enabled and Duette Schedules is not available in the Resources tab, contact your Admin to ensure you have the correct permissions for Duette Plugin.

To create a new schedule, expand Duette Schedules to list all projects. 

Right click on the project you wish to create the schedule for.
Select Add Schedule from the menu. 
The Edit Automated Test Schedule screen will be displayed where you can add one or more configurations and set schedules for imports ( adhoc, periodic or daily intervals).

On the Configurations tab, select the Add icon.
Complete the details:

FieldDescription
Configuration NameEnter a name for your Configuration
TypeSelect the test type from the drop down list. HP Quick Test Professional, IBM Rational Functional Tester,Selenium or Unit Test Results
Sub-TypeOnly applicable for Unit Test Results. Select from the dropdown list the type of results you are importing
Default PathIf you also plan on manually importing results enter the pathway. Otherwise leave this field blank.
Source PathEnter the path to the results file to upload e.g. c:\testdata\{Name}.xml
Name TemplateEnter a name for the results files e.g.{name}
Combine ResultsCheck to combine all results in the path into a single run. This can be useful for unit test where there maybe many xml files which comprise of a single run.
Skip files if unchangedCheck to skip importing files if they have not changed since the last import. e.g. the tests have not run since the last schedule import.
Root Script PackageSelect the package/ folder in the Script Library where you would like the Automated Test Script created.
Root Execution PackageSelect the execution package to import the run results to.
Retained Max # ResultsEnter the maximum number of runs to retain ( in the run history of the automated test assignment). Runs outside of the retention period will be removed.

Using the Source Path and Name Template 


Creating a Single Automated Test from a Single File
In its simplest form a schedule can be created to import a single file as a result into an automated test assignment. 

Creating a Single Automated Test from a Single File

Source Path: c:\testdata\results.xml
Name Template: my result

Creating an Automated Test from a Folder of Results

Creating an Automated Test from a Folder of Results

Source Path: c:\testdata\
Name Template: my results

These both behave in the same manor as manually importing an automated test result and filling in the “Results Path” field on screen. In addition, zip files are also supported, which effectively works similarly to configuring a folder (with the contents of the zip file treated the same as the contents of the folder). 


Capturing part of the path into the “name” variable
It is also possible to “capture” part of the path into a special variable called “name” which can then be referenced within the “Name template” to generate a unique name for each test. 
Below is an example using files:

Examples – Capturing part of the path into the “name” variable

Source Path: c:\testdata\{name}.xml
Name Template: {name}

For a directory of files, here is what would be generated 

FileGenerated NameComments
c:\testdata\report-results.xmlreport-resullts
c:\testdata\report.pngskipped, as it doesn’t match source path
c:\testdata\ui-results.xmlui-results

Source Path: c:\testdata\{name}\

Name Template: {name}

FileGenerated NameComments
c:\testdata\build123\build123
c:\testdata\builder456\build456 
c:\testdata\test.xmlskipped, as file is not a folder

Wildcards are also supported.  In the Source path, you can replace the name with “*” . Wildcards are not supported in the Name Template field 

Example – Using wildcards

Source Path: c:\testdata\*\
Name Template: {name}

FileGenerated NameComments
c:\testdata\build123\build123
c:\testdata\builder456\build456 
c:\testdata\test.xmlskipped, as file is not a folder

Limitations

 There some limitation so the Source Path and Name Template fields.  These include the following:

  •  A folder name with a file cannot be captured e.g. “c:\testdata\{name}\input.xml” is not supported.
  • The combine results option is not supported for zip files e.g. “c:\testdata\{name}.zip”.
  • The combine results option is not supported when capturing folders e.g. “c:\testdata\{name}\” you can not use the “combine results” option.
  • If capturing files which include zip and non-zip files, the zip files will not be automatically expanded/traversed (but you can use the “combine results” option).  In this case zip files are treated like any other file making up a single result.

Configuring the Schedule

Now that your configuration is complete you are ready to set up your Import Schedule.  The import frequency can be configured from the Schedules tab. There are three options that can be configured:

  1. Adhoc
  2. Periodic
  3. Daily
TypePeriodTime
AdhocN/AN/A
PeriodicSpecify the synchronization frequency in minutesN/A
DailyN/ASpecify the time using the (24hr clock) when the synchronization will occur daily.

Once you have configured your import schedule, a summary of the configured import schedules is available.  You can see the time of the Last Run, the Next Run (if applicable), whether the schedule is enabled or not and the current import status.

You can use the tool bar to add new import schedules, delete an existing configuration, enable or disable an existing schedule, configure an existing schedule or manually initiate an import.

Import History

You can view the import history from Import History tab.
From the Import History screen you can do the following;

  • View all synchronization events
  • Select to only view errors
  • Export the synchronization events to a csv file
  • Clear the history.

Step 4 – Compiler Suite Preview tab

Step 4 of 4

Compiler Suite Preview tab displays the Test Cases that can be generated.

The Test Script Names are displayed. As the Test Script Names had variables in the Steps screen, you can see the Test Script Names are all different depending on the variables. This makes the Test Script Names more descriptive.

If the Test Steps contain a variable that is unique in the Expected Results column, you can also enter Expected Result data in this screen.

If you have a lot of test cases that can be generated you can choose to Filter the Test Scripts which makes it easier to select only the Test Scripts that you want to generate

(You can export the data to a CSV file if you want to use the dataset in other Compiler Suite Templates)

The total number of test cases generated is shown in the bottom left of the screen

You cannot generate more than 400 Test Cases.

Select all or some of the Test Cases by clicking the checkbox to the left of each Test Case or the top left box to select all test scripts.

Once the Test Cases are selected, click Generate.

Select the Script Library that the Test Cases need to be saved in.

Click OK and a successfully generated message will be displayed.

A new folder will be created and displayed under the Script Library that was selected.

All Test Cases are now created and ready to be organised in Explorer for Test Execution.

If you open one of the Test Scripts that was generated, you will see all the variable data is displayed.

Step 3 – Compiler Suite Data tab

Step 3 of 4

Compiler Suite Data tab lets you build the dataset required for each of your variables.

Data can be manually entered or imported.  A variety of functions such as insert, copy, paste and delete are also available.

Import Data

To import data you must have a CSV file prepared, containing relevant data in columns.

Simply click the import button and select your CSV file, 400 rows maximum can be imported.

Perform mappings (Compiler Suite variables are in the left-hand column and the right-hand side is the CSV column headers) and click OK to import.

Generation Options

Using the Test Case template containing variables and data, there are now three options available to generate Test Cases.

Generation options are located at the bottom of the Compiler Suite Data tab.

Option 1: Row by Row – Data in each row will be generated as a Test Case.
Option 2: Minimum – Minimum number of Test Cases will be generated, covering each piece of data at least once.
Option 3: Pairwise – a Combinatorial method where Test Cases will be generated for each discrete pair combination.

Option 1: Row by Row

Data in each row will be generated as a single Test Case, for example, the data in the screenshot here will generate four test cases.

If the Test Steps contain a variable that is unique in the Expected Results column, you can also enter Expected Result data in this screen.

  • You can only enter Expected Results in Option 1: Row by Row as each row will be generated so you will know what the Expected Result is.
  • For the other 2 options you don’t know what data will be generated so you cannot know the Expected Result

You cannot generate more than 400 Test Cases.

Option 2: Minimum Number

The minimum number of Test Cases will be generated, covering each piece of data at least once, for example, the data in the screenshot here will generate six test cases.

In the top right-hand corner of the screen it shows the minimum number of Test Scripts that will be generated. 

You cannot generate more than 400 Test Cases.

Option 3: Pairwise 

The combinatorial method where Test Cases will be generated for each discrete pair combination.  Based on the observation that most faults are caused by interactions of at most two factors, Pairwise-generated test suites cover all combinations of two therefore are much smaller than exhaustive ones yet still very effective in finding defects. 

For example, the data in the screenshot here will generate twelve test cases.

You cannot generate more than 400 Test Cases.

Once you have added data and selected the generation method, click Next

Step 2 – Compiler Suite Steps tab

Step 2 of 4

Compiler Suite Steps tab is consistent with the Test Script Steps screen. 

This is where you will build your Test Script using variables.

Add variables using Square brackets. [ ] you can use variables in the following fields:

  • Test Script Name
  • Description 
  • Expected Result

Each variable will become a column in the Compiler Template Data screen (this is the next step of the process – Step 3).

The variables created in the Test Script Template must be unique. This means that if you create a variable i.e. [age] wherever you use this in the Test Script Template it will have the same value. 

If the square brackets contain a space, this will not be recognised as a variable, e.g [application url] will not be recognised, but [application_url] will.

Variables are case sensitive, e.g [age] would be treated separately to [Age].

If a variable only relates to an Expected Result, only add the variable in the Expected Result column of the Steps screen

Using Variables in Test Script Name field

Using variables in the Test Script Name field helps to make the Test Scripts more descriptive when they are generated.

Depending on the data set used, the above example would create the Test Script Name as below.

Premiums for 250,000, Female, 30 year old paying Weekly.

The variables will change for each Test Script Name depending on the data table

Variables in Description and Expected Result fields

The variables created in the Test Script Template must be unique.This means that if you create a variable i.e. [age] wherever you use this in the Test Script Template it will have the same value.

If you want to have a generated value for your Expected Result step, you need to add a unique variable that is only in the Expected Result field and not in Test Script Name, or Description field.

In the example above [Premium] is only in the Expected Result field for Step 5.

The other variables used are in the Test Script Name, Description and Expected Result fields.

You can have more than one Expected Result data column.

The Expected Result generated values can only be used in Option 1: Row by Row for Test Script generation ( this is in the Data screen, which is the next step in the process – Step 3). 

Step 1 – Compiler Suite Details tab

Step 1 of 4

Compiler Suite Details tab is consistent with the Test Script details screen.  

Any details you record here will be included in the generated Test Scripts, this includes Custom fields.

 Once you have added information in the Details tab as required, click the Next button.

By default, the below fields are available, fields may differ if there are added custom fields or hidden fields as updated by your Enterprise Tester Administrator.

NumberScript number
NameA short name for the test script
PrioritySelect from the pick list
StatusSelect from the pick list
TypeSelect from the pick list
Est. DurationAn estimate of how long the script should take to run 
Assigned ToWho is assigned the Test Script
DescriptionA description of the test script
ObjectiveThe objective of the test
Pre-conditionsAny conditions that must exist before the test script can be run
Post-conditionsAny conditions that should result once the test script has completed
NotesNotes related to the test script

Step 2

Getting started with Compiler Suite

Once Compiler Suite is enabled, an additional folder is displayed for each project:

Right click on the Compiler Suite folder, select Add Compiler Template:

Compiler Suite Template is added under the Compiler Suite folder, the Details tab is displayed:

Installing Compiler Suite

Please contact our Customer Team for a Compiler Suite License.

 Important Notes:

  • Compiler Suite is only compatible with Enterprise Tester 6.1 and above.
  • You must ensure you have upgraded to the .Net 4.6 Framework on your Enterprise Tester server before you apply the Compiler Suite license.  This should have been prompted in the upgrade process to Enterprise Tester 6.1.  If you need to upgrade .Net, downloads can be found here: www.microsoft.com/download/

Once you have Compiler Suite License text, enable Compiler Suite by performing the following:

  • From the Admin tab, expand the Extensions folder and expand the Plugins folder.
  • Double click on Compiler Suite, the License details screen will open.

  • Paste your Compiler Suite license text to the screen and click Save. A message will now display indicating you must restart Enterprise Tester for the changes to take place.

  • Click the Restart link and wait for Enterprise Tester to restart.

  • Head over to the user guide for detailed instructions on using Compiler Suite.

Compiler Suite – Data Driven Testing

What is Compiler Suite?

Compiler Suite enables automatic generation of a number of Test Cases that contain defined variables and data.  Once variables and data are defined, there are three generation methods:

1) Row by row – where the data in each row is generated as a single test case.

2) Minimum number – where the minimum number of tests are generated to cover each piece of data once.

3) Pairwise method – based on the observation that most faults are caused by interactions of at most two factors, Pairwise-generated test suites cover all combinations of two therefore are much smaller than exhaustive ones yet still very effective in finding defects. 

A step by step process is available:

Why would I use it?

Smarter test practices mean time is gained and quality is increased, data generation methods can result in up to 85% time savings when creating test suites.  Benefits are:

  • Creating combinatorial data for data load tests.
  • Quickly creating hundreds of Test Cases where repetition is required.
  • Data driven test planning.
  • Creating test cases for an already implemented module without any documented specification. 
  • During review or walk through of specifications or code.