Understanding Insight Objects in UFT 11.5!!

With Unified Functional Testing, there were a lot of features like integrating GUI testing and API testing, introducing File Content Checkpoint, and create insight object or recording in Insight mode. In this article I will try to explain what this feature of Insight recording is and how to implement same.

Insight is an image-based identification ability, to recognize objects in application based on what they look like instead of using properties that are part of design. UFT stores an image of the object with insight test objects and uses this image as main description property to identify object in the application. Together with insight property, UFT can use ordinal identifier to uniquely identify the object. Another property we can use is Similarity - Specifies how similar a control in the application has to be to the test object image for it to be considered a match. And Visual relation identifier can also be used to identify the object based on its relative position with respect to other objects in the application. 

Low points of insight object are we cannot add insight object from object spy and snapshots occupy a lot of disk space.

Adding Insight Objects :

UFT allow adding Insight object either in recording Mode or manually in object repository:

1.  Insight Recording: When we record in UFT, There is option to select recording mode. The recording mode in UFT is default, analog, Low Level recording, and Insight recording. When we select recording mode as Insight recording, UFT records object as insight object for images and identifies rest of objects like edit box as window object WinObject and perform action on them.

2.Insight Objects can be added through Object Repository. Go to Resources>Object Repository. This will open Object Repository window. In Object Repository window, Navigate to Object>Add Insight objects to Local. This can also be added by icon as shown below.

Once we click on Add Insight objects to local, UFT asks to select Learn Mode which can be manual and automatic. Manual Mode provides magnifier to select minute objects in the application.

Insight object is always added to the object repository as a child of the test object that represents its containing application, such as a Window or Browser object. 

In the Editor, test object image is displayed in the step instead of the test object name. Enlarge Image is displayed on hovering over the object.

Options available to work with Insight in UFT 11.5:

UFT 11.5 provides following options for working with insight objects. Options for Insight Object are at Tools>Options>GUI Testing>Insight. This pane enables you to define options that customize how UFT handles Insight test objects when creating test object, and during record and run sessions.

Options to customize the insight object are as follows:

 a.   Save the clicked coordinates as the test object’s Click Point when recording a test object.

 b.   When recording a test, display mouse operations.

c.When editing shows test object image in steps and displaying Select learn mode dialog box

d. Snapshots for insight object takes a lot of disk space, we can provide option to limit test object image, maximum pixels around image, and maximum number of snapshots to save when recording a test object

--Sudhakar.Mangi

Using Page Factories with Loadable Component!!!

Continues with my previous Post!!!

LoadableComponent is another way to approach PageObjects. LoadableComponent 
is a base class that all of the pages need to extend. The base class has the following 
methods on the interface
             
                   1) get()
                   2) isLoaded()
                   3) load()


 Instead of the usual public class PageObject, we change it:

     public class PageObject extends LoadableComponent<PageObject>

We will have to add overrides for the load() and isLoaded() method. The load method
will load the page for us and the isLoaded() method can allow us to check if the page 
has been loaded correctly.

@override
Protected void load() {
selenium.get("http://book.theautomatedtester.co.uk");
}

@protected void isLoaded() {
String url = selenium.getCurrentUrl();
If (url != "http://book.theautomatedtester.co.uk"){
throw new Exception("The wrong page has loaded");
}
}

As we can see this is just a simple bit of code, but we can make sure that we start 
on the right page when we need to.

Now that we have learnt about LoadableComponents, we should have a look at 
seeing it in action. We need to make changes to our Java Class.


1. The following is how the code should look so far:

public class Chapter2 {
WebDriver selenium;
@FindBy(how= How.NAME, using="verifybutton")
WebElement verifybutton;
public Chapter2(WebDriver selenium){
this.selenium = selenium;
if (!"Chapter 2".equalsIgnoreCase(this.selenium.getTitle())){
selenium.get("http://book.theautomatedtester.co.uk/chapter2");
}
}
public boolean isButtonPresent(String button){
return selenium.findElements(By.xpath
("//input[@id='"+button+"']")).size()>0;
}
}


2. If we have a look at our Chapter 2 Java class, we can see that we need to extend
LoadableComponent. Since this takes generics we will have to pass in our
PageObject class. It should look like:

public class Chapter2 extends LoadableComponent<Chapter2> {

3. In our constructor, we will have to initialize our page factory. We can remove the
rest of the code in there since that will be moved to load(). It should look like the
following:

public Chapter2(WebDriver selenium){
this.selenium = selenium;
PageFactory.initElements(selenium, this);
}


4. We now need to add our override methods. These will allow us to check that we are
on the right page when we load this component:

@override
Protected void load() {
selenium.get("http://book.theautomatedtester.co.uk/chapter2");
}
@protected void isLoaded() {
String url = selenium.getCurrentUrl();
If (url != "http://book.theautomatedtester.co.uk/chapter2"){
throw new Exception("The wrong page has loaded");
}
}

5. Now we need to have a look at updating our test to load everything for us.
 To do this we need to change:

@Test
public void ShouldLoadTheHomePageAndThenCheckButtonOnChapter2() {
selenium.get("http://book.theautomatedtester.co.uk");
HomePage hp = new HomePage(selenium);
Chapter2 ch2 = hp.clickChapter2();
assertTrue(ch2.isButtonPresent("but1"));
}

6. To look like this:

@Test
public void ShouldLoadTheHomePageAndThenCheckButtonOnChapter2(){
Chapter2 cht = new Chapter2(selenium).get();
ch2.isButton("but1");
}


7. Run your test. Everything should look like the following:

public class Chapter2 extends LoadableComponent<Chapter2>{
WebDriver selenium;
@FindBy(how= How.NAME, using="verifybutton")
WebElement verifybutton;
public Chapter2(WebDriver selenium){
this.selenium = selenium;
PageFactory.initElements(selenium, this);
}
@override
Protected void load() {
selenium.get("http://book.theautomatedtester.co.uk/chapter2");
}
@protected
public void isLoaded() {
String url = selenium.getCurrentUrl();
If (url != "http://book.theautomatedtester.co.uk/chapter2"){
throw new Exception("The wrong page has loaded");
}
}
public boolean isButtonDisplayed(String button){
return selenium.findElement(By.id("button")).isDisplayed();
}
}

What just happened?

We have just converted our page object to use the LoadableComponent class 
that comes with the Selenium Project. We saw how we simplified on constructors 
and then just moved this into somewhere easy to maintain. We have seen that 
we can move a lot of the boiler plate code out of our class and rely on it 
being pulled in via LoadableComponent. This means that we no 
longer need to maintain it or we add those items.

Imagine how you have to work with a flow that takes you through a number of pages.
LoadableComponent allows us to set up a workflow. To get this right we need to pass
one in like the following when doing your test setup:

@Before
public void prepareComponents() {
WebDriver selenium = new FirefoxDriver();
HomePage homePage = new HomePage(selenium);
Chapter2 chapter2 = new SecuredPage(selenium, homePage);

}

Using Page Factories with Page Objects!!

Continues with my previous Post!!!

The code that we have learnt to write earlier can be quite verbose.
To clean up our code, we can start to use Page Factories.
This allows us to annotate variables in our page objects with how to search the page.
This means that we don't have to have full
WebElement element = driver.findElement(…); code all over the file.

We can change it to:
@FindBy(how=How.ID, using="foo")
WebElement foo;

As you can see this can make our code slightly easier to read and therefore more
maintainable. If you regularly use other languages like Ruby or Python,
you will notice that they don't have the PageFactory support project.
This is because those languages don't have Factory constructs in the language.
They are not idiomatic and therefore not in the language. To use the
PageFactory project in WebDriver, we will have to make sure that the
we have added it as a dependency.

Let us now update my previous post code from with an example of the PageFactory.

See my  the previous post   and go to Chapter2.java. It should look like the
following example:

Chapter2.java

import org.openqa.selenium;
import junit.framework.Assert;
public class Chapter2 {
WebDriver selenium;
WebElement verifybutton;

public Chapter2(WebDriver selenium){
this.selenium = selenium;
verifybutton = selenium.findElement(By.id("verifybutton"));
if (!"Chapter 2".equalsIgnoreCase(this.selenium.getTitle())){
selenium.get("http://book.theautomatedtester.co.uk/chapter2");
}
}
public boolean isButtonPresent(String button){
return selenium.findElements(By.xpath("//input[@id='"+button+"']")).size()>0;
}
}

We can then change the line that looks for verify button so that it is not in 
the constructor.  This then changes to:

public class Chapter2 {
WebDriver selenium;

@FindBy(how= How.NAME, using="verifybutton")
WebElement verifybutton;

public Chapter2(WebDriver selenium){
this.selenium = selenium;
if (!"Chapter 2".equalsIgnoreCase(this.selenium.getTitle())){
selenium.get("http://book.theautomatedtester.co.uk/chapter2");
}
}
public boolean isButtonPresent(String button){
return selenium.findElementByXpath('//button[@id='+button+']');
}
}

If you run your test now, you will see it do the same thing but we have not 
called the findElement() method available to WebDriver.

In the test we need to initialize the factory by calling initElements():

TestChapter2.java

import org.openqa.selenium.*;
import org.junit.*;
public class TestChapter2 {
WebDriver selenium;

@Before
public void setUp(){
selenium = new FirefoxDriver();
}

@After
public void tearDown(){
selenium.quit();
}
public Chapter2 clickChapter2(){
clickChapter("2");
return PageFactory.initElements(selenium, Chapter2.class);
}

@Test
public void ShouldLoadTheHomePageAndThenCheckButtonOnChapter2() {
selenium.get("http://book.theautomatedtester.co.uk");
HomePage hp = new HomePage(selenium);
Chapter2 ch2 = hp.clickChapter2();
assertTrue(ch2.isButtonPresent("but1"));
}

}

We can seen how we can get rid of a line of code from a constructor or a method
by adding a decorator to the variable. When our code is compiled, the variable will get
populated at the right time so that we can make sure that it gets the right bit of the
DOM. It will look like our element hasn't been instantiated. When we initialize the
PageFactory, by calling initElements() it will populate the variables with the right data.

Sudhakar.Mangi

Design Patterns In Selenium

‹‹ Page Object design
‹‹ Using Page Factory in Page Objects
‹‹ Using Loadable Components

Important preliminary points
In this chapter it will be assumed that all files will have the following import statements:
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.support.FindBy;

Page Objects
In this section ,we are going to have a look at how we can apply some best practices to tests. You will learn how to make maintainable test suites that will allow you to update tests in seconds. We will have a look at creating your own DSL so that people can see intent. We will create tests using the Page Object Pattern.

Create a new Java class :
1. Import the relevant Selenium Packages.
2. Create the setup() and  teardown()  method. I prefer the JUnit 4 style of tests and
will show code samples with the annotations.
3. We need to check that the page is on the correct page. For this we will use the
selenium.getTitle to see the page title and then if incorrect move to the
chapter 2 link. We do this because navigating to page is slower than checking the
page's title or any other calls to the page already loaded.
4. We need to then validate that it is correct and then work accordingly. The following
is a code snippet of how we can do this:

if (!"Page 2".equals(selenium.getTitle())){
selenium.get(
"http://book.theautomatedtester.co.uk/chapter2");
}

5. Create the rest of the test to check that items are on the page.

Time for action – moving Selenium steps into private methods
to make tests maintainable

Imagine that you just need to test one page on your site and you have quite a few tests
for this page. A lot of the tests will be using the same code over and over again. This can
be quite annoying to maintain if something changes on the page meaning we have to go
through all the tests to fix this one issue

tests as follows:

@Test
public void shouldCheckButtonOnChapter2Page(){
selenium.get("http://book.theautomatedtester.co.uk");
selenium.findElement(By.link, "Chapter2").click();
Assert.assertEqual(selenium.findElements(
By.id"but1").getSize(), 1);
}
@Test
public void shouldCheckAnotherButtonOnChapter2Page(){
selenium.get("http://book.theautomatedtester.co.uk");
selenium.findElement(By.link, "Chapter2").click();
Assert.assertEqual(selenium.findElements(
By.id,"verifybutton").getSize(), 1);

}

Time for action – using the Page Object Pattern to design tests

Imagine that you have a site that has a number of different pages that you need to test. This
is quite common for a number of sites. We can create an object that represents the page and
then pass the Selenium object in the programming language. So let us now create our first
Page Object against the home page.
1. Create a new Java class in IDEA called HomePage.
2. Import the relevant packages for the tests to run.

3. We will now need a constructor to handle Selenium. You may want to make it go to
the home page when it is instantiated too. An example of this can be seen as follows:

HomePage.java

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
public class HomePage{
WebDriver selenium;
public HomePage(WebDriver selenium){
this.selenium = selenium;
}
public Chapter2 clickChapter2(){
clickChapter("2");
return new Chapter2(selenium);
}
private void clickChapter(String number){
selenium.findElement(By.linkText("Chapter"+number)).click();
}
}

Chapter2.java

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
public class Chapter2 {
WebDriver selenium;
public Chapter2(WebDriver selenium){
this.selenium = selenium;
if (!"Chapter 2".equalsIgnoreCase(
this.selenium.getTitle())){
selenium.get(
"http://book.theautomatedtester.co.uk/chapter2");
}
}
public boolean isButtonPresent(String button){
return selenium.findElements(By.xpath("//input[@id='" +
button + "']")).size()>0;
}
}

BestPractises3.java

import org.junit.After;
import org.junit.Before;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
public class BestPractises3 {
WebDriver selenium;
@Before
public void setUp(){
selenium = new FirefoxDriver();
}
@After
public void tearDown(){
selenium.quit();
}
@Test
public void
ShouldLoadTheHomePageAndThenCheckButtonOnChapter2(){
selenium.get("http://book.theautomatedtester.co.uk");
HomePage hp = new HomePage(selenium);
Chapter2 ch2 = hp.clickChapter2();
assertTrue(ch2.isButtonPresent("but1"));
}
}

If you create these three files you will see it pass. The test is a lot more succinct and
easier to maintain.

Test data model: Table overview and TEST_CASE

I have been very busy the last months both at work and at home (I  switched from a Mumbai  to  Hyderabad)...

...but now I thought I should try to explain my test data model which I use for administrating tests and test environments.

First out is an overview picture of all tables and perhaps the most important table, TEST_CASE including some brief information. I will gather all table information in an RTF-document which should be available for download here: http://dl.dropbox.com/u/8538167/TestFiles/TEST_DATA_MODEL.zip

 A snapshot document of my test framework data model is now available for download:

http://dl.dropbox.com/u/8538167/TestFiles/TEST_DATA_MODEL_090420.zip

What to use it for?

I think it mainly can serve as an inspiration source on how to (or how not to...) model your test framework as it is difficult to use it "as is" since it requires SQL logic (in code or in store procedures) and a bunch of GUIs (some has been published earlier).

When I read the document I realize that some things should be refactored (bit instead of varchar for True/False columns for example) but all design decisions made sense to me at the time of creation. The database is "hand-made" from scratch using SQL Server GUI to create and change tables. Columns and tables have been added from time to time ever since.

One of the reason why I choosed to use SQL Server instead of excel was that my AUT required of lot of parameters...if I do select distinct test_case_parameter_name from test_case_parameter the answer is 192...

Agile Application Lifecycle Management (ALM)

Agile Application Lifecycle Management (ALM) gains more and more momentum.with Agile, or finding a pragmatic approach to ALM at all. Meanwhile, more and more tool vendors find it helpful to label their tools to be Agile tools
or even Agile ALM tools. But what is Agile ALM? In my opinion, ALM synthesizes technical and functional elements to provide a comprehensive approach to common project activities and phases, addressing
build, configuration, deployment, release, test, quality, integration, and requirements management, see Figure 1.
With its interdisciplinary approach, Agile ALM integrates project roles, project phases and artifact types.
An Agile ALM enriches an ALM with agile values and strategies. An agile approach to ALM improves product quality, reduces time to market, and makes for
happier developers. My definition of Agile ALM results in processes and tool chains that are flexible, open to change and high in quality.
This is one of the ways in which ALM helps to provide structure for Agile.

Figure 1. ALM addresses different disciplines and development phases.

Some underlying aspects of Agile ALM are not completely new, and you should respect all the different struggles from previous decades, and put results together to get the best solution for today: in my opinion, ALM evolved from software configuration management (SCM), that in turn has its root in basic version control. You should define your processes and your requirements first, before selecting the tools that best
fit the given task.

Individuals and interactions over processes and tools

Above all, Agile ALM is a discipline and a mental approach. Working with Agile ALM should start with values and people as
well as concepts behind it. An Agile ALM tool is an ALM tool that fosters an agile process.

An Agile ALM tool must be able to add value to the system and improve the collaboration of the stakeholders.
In my opinion, an Agile ALM tool chain must implement the building blocks of Agile ALM, such as continuous integration (including continuous inspection and
continuous deployment), functional/technical releasing, stakeholder focus (and collaborative development) and task-based development.

Many projects feel comfortable with an orchestration of single best of breed tools. Integrating lightweight, configurable tools into flexible tool chains results in a mashup of tools that exactly offer
the features that are needed to solve a given task.

Agile ALM tools should have an open architecture that enables you to add further tools or functionality. Relying on lightweight tool chains can improve flexibility dramatically
because you can replace small units of the overall infrastructure easily without questioning other parts of that infrastructure. Now let's discuss some of the important building blocks of Agile ALM, and start with
task-based development.

Task-based development

With a task-based approach, the task is the unit of interaction and the base of work. Task-based development is the technique of traceably linking work items
to the specific set of changes that was made to complete the work item. One example use case can look like this: you are working on a task that
is listed in your ticket system and has the unique identifier AGILEALM-9. Your IDE (e.g. Eclipse, with Mylyn) is integrated with the
ticket system (e.g. JIRA). The CI server Jenkins integrates with JIRA, with the version control system (VCS) and with the component repository (e.g.
Artifactory) to make the progress of work and the dependencies of artifacts and work items transparent. With Jenkins, together with Artifactory, you can drive staged builds in order to deploy the release
to higher staging environments, without re-building the release ("build once, run everywhere"). Figure 2 shows how Jenkins integrates with
other tools. Zoomed in to a build result page in Jenkins, it is easy to navigate to the VCS (to look at the underlying changes), to the ticket system (to work on the
task) and to the component repository (to work on the binaries).

Figure 2. The CI server Jenkins integrates with VCS, ticket system and component repository.

Collaborative development

Software development is all about implementing requirements. The requirement is the central unit and the driver of a software release. Approaches like unit testing (to
validate that the right thing is developed correctly) and acceptance tests (to validate that the right thing is developed) are not new. But in the past, these approaches have often been
handled in an isolated or puristic way. Instead, a comprehensive, pragmatic solution should be prefered that focuses on the requirement itself, always having all stakeholders in mind.
You can use dedicated, lightweight tools to write acceptance tests, such as Fit. Or you can use specialized languages. Scala and Groovy, both languages offer interesting features for
setting up a polyglot ecosystem, leveraging existing platforms by providing solutions that involve special purpose languages. With Scala and Groovy, you can write tests, which helps
to overcome various barriers:

• Barriers between project phases and project activities (because coding and testing move together more closely)
• Barriers between artifact types (because code and executable specifications are written on the same unified infrastructure)
• Barriers between project roles (because tests are written collaboratively, with mechanisms to use terms close to the problem domain)
• Barriers between tools (because the same tools are used for programming and testing)

The following simple example gives you an impression of how it can look like to write acceptance tests with Scala and the specs2 library.

package alm

import org.specs2._

class AccSpec extends Specification { def is =

  "This is a specification to check the 'Agile ALM' string"                ^
                                                                           p^
  "The 'Agile ALM' string should"                                          ^
    "start with 'Agile'"                                                   ! e1^
    "end with 'ALM'"                                                       ! e2^
                                                                           end
  def e1 = "Agile" must startWith("Agile")
  def e2 = "Agile ALM" must endWith("ALM")
}

The defined is method lists specification fragments which are either simple text (description of the target system), an example (including exectuable code that
returns a result) or a formatting fragment (the p for adding a blank line and starting a new block). Fragments are separated and linked by the ^ character.
For more details on specs2, see specs2.org.

Release management

Release management comprises producing software artifacts and releasing those artifacts according to a defined process. Release management can be differentiated into a functional and a
technical part. To deliver software successfully, both parts are important and should be integrated with each other. Automation and continuous integration are crucial facets of the software
release and delivery process.

Functional release management

Functional release management involves picking the customer's requirements, assigning them to releases and delivering the functionality to the customer, in
high quality. Often agile practices are used to support this process, and many projects achieve good results from using the management template Scrum. While defining a thin set
of rules, Scrum fosters discipline and makes defects (in software as well as in the process) visible. But Scrum is too abstract to apply it "out of the
book". You must implement Scrum and adopt it for software engineering. Implementation practices, for instance, may include distinguishing between special development phases,
on a micro level, inside a Scrum release: during a release, you may think about closing the develoment phase with a frozen zone which allows developers to only work on bug fixes,
instead of new features. Another helpful option is to use code freeze intervals, to complete and ship the final release.

Technical release management

Technical releasing consists of building the software and providing the final product to the user. Build management (comprised of compiling scripts, and packaging and distributing components)
is essential for Agile ALM. Technical release management describes activities to identify configuration items, track and audit changes on requirements and configuration items
and integrate and deliver the implementation. In software engineering change is more the rule than the exception. Because requirements change, it is important to keep the requirements in
sync with their implementations. Possible gaps between functional and technical release management should be bridged. Strategies like VCS hooks help to marry both parts of release management.

Continuous integration (including continuous inspection and continuous deployment)

To automate manual steps means results are delivered in an objective and reproducible way.

Automating the most error-prone, most repetitive and most time-consuming activities is essential. Continuous integration (CI) is the automation of the build, test
and release process and has the goal of integrating the activities of colleagues and the work items others produce.
This can result in a build ecosystem where a new code commit directly triggers a continuous build including compile, technical tests, audits, package, functional tests and deployment.
All different artifact types, platforms and languages, should be integrated, e.g. Java (Groovy, Scala, ..), .NET, PHP and Cobol, using an unified tool infrastructure, see Figure 3. If no native build system exists for a respective language/platform, non-native build technologies can be used to include these artifact types in the CI farm.

Figure 3. A comprehensive CI ecoystem that integrates different artifact types, on an unified infrastructure.

In a continuous integration process, build reports and notifiers should be in place, and information should be shared and aggregated. Aggregating information means that the integrated tool chain
spans the whole, heterogenous ecosystem. This enables the stakeholders to "zoom in" where needed, to get more information where necessary, and make knowledge out of information.
Examples for this are: given a specific build, you can browse to the underlying changes in the VCS, or: collect all produced binaries in the component repository that belong together
semantically, in order to perform operations on them as a group.

Conclusion

Agile ALM spans many disciplines in software engineering. Agile ALM is about people and strategies, and implementing these strategies with lightweight tool chains.
Agile ALM helps to provide structure for Agile and helps to approach ALM in a determined, pragmatic way. Using an Agile approach to ALM, you'll profit from a quicker win and better results.
There is a lot more to say, but I hope this article gave you an impression about what Agile ALM is, and what I desribe in much more depth in my book "Agile ALM".

Evolution of test automation frameworks!!!!!!!!!!!!

Implementing automated tests is something that everybody wants to do. If you ask any tester - test automation is her/his aim. It is the golden target that every tester aims for, but only a few of them take pains to assess the required knowledge, being under the illusion that a programming language or expensive tool will suffice to cope with all the problems that are likely to arise. This is not true. Writing good automated tests is much harder than that. 
1. Dimensions
The first thing that you must consider when setting out test automation efforts is framework or architecture (sometimes called a test harness) in which tests will be placed. By these terms, I mean all code that will support and facilitate test cases development and execution, but also the project structure and any management that will take place (e.g. test data management or documentation). Let's run through the different framework concepts. Perhaps you will recognise the path that you took during your test automation adventure (if you are an experienced tester). Perhaps, if you have just entered the path, it will help you make some decisions. This trip will also consider tester development skills and their evolution that accompanies the evolution of framework. 

Thinking about automation framework, one can think about it in many dimensions. I borrowed these dimensions from Brian Marick’s “Testing quadrant” concept (business facing, technology facing, supporting the team, critique the product) and used it to create frames, a starting point for our weigh ups. We will consider automation frameworks in the following dimensions:
Supporting the code - we will see what coding means to automated testing and how code complexity grows with our needs
Supporting the users - shows different styles and approaches of implementing tests that focus on other individuals then a tester (business, customers)
In Supporting the tests we will talk about what else is needed for framework to be useful and see in what way the context is important

Diagram 1 - Test automation dimensions

2. Supporting the code

When reading testing press, blog posts comments, various forum threads on testing, one special question tops all the others in the context of test automation. This question is usually posed by manual testers lacking technical skills or testing beginners and it reads - how much development skill do I need to proceed with test automation? or do I need to be a developer to start out with automating tests? This is an important question and the supporting the code dimension tries to clarify these doubts.

Linear tests

We usually start with the simplest approach - record and play. A tool records our actions and then makes it possible to have them played back. Tests usually look like linear code - one line after another and so on. Easy to create but leads us to a maintenance nightmare. Often these linear scripts need to be re-factored to run them more than once. Also, the recording feature can be used to speed up test creation in much more complicated frameworks instead of typing it from scratch each time. This approach is easy to use for a tester who does not possess any development skills.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

@Test public void linearTest() {   selenium.open(url);   selenium.type(text);   selenium.click(button);   selenium.verifyTitle(title);   selenium.click(link);   selenium.wait(time);   selenium.open(url2);   selenium.type(text2);   selenium.click(button2);   selenium.verifyTitle(title2);   selenium.click(link2);   selenium.wait(time); }

Example of linear test code.

Structured tests

With higher complexity of test cases that we want to implement, the need for more sophisticated test scripts emerges. We start to familiarize with a programming or script language that the tool supports and start to use its features - conditional statements, variables and loops. We start to call one script from another creating a chains of scripts. With this, our scripts can do more, more extensively and more reliably. On the other hand, lack of development skills can easily lead to creating spaghetti code with the complex and tangled control structure. We still suffer from maintenance issues and going into raptures about new possibilities we do not realise that we are coming closer to a dead end.

Reusable tests

Finally, when maintenance issues hit us it might be too late. But we see that some portions of code repeat and understand that we can reduce the maintenance impact by creating reusable blocks of code. We start using functions or procedures to extract code and create libraries to gather it in one place. We share it between tests and projects. Tests get more clear and readable. At this point, the tester has made a big progress with her/his development skills. She/He knows the basics of the language that she/he uses and starts to explore more advanced features.

1 2 3 4 5 6 7 8 9 10 11

@Test public void reusableTest() {   CQ55 cq = new CQ55();   cq.openWcm();   cq.open(INTEGRATION_AUTHOR_INSTANCE);   PresentationTemplatePage presentation = new PresentationTemplatePage();   presentation.runExportPdfWorkflow();   presentation.confirmWorkflowInfoDialog();   presentation.waitUntillWorkflowNotFinish(); }

Example of reusable test code in Java.

Design Patterns

Sometimes we can go much further. We might bravely step into the development world and take little bites from anything that can help us. Design patterns are that thing. A design pattern is a general reusable solution to a problem occurring commonly. With design patterns we can improve some tests capabilities for example Selenium’s Page Object pattern reduces the maintenance impact and fosters order in source code. In Selenium alsoSingleton can be used to provide a driver instance to tests instead of inheritance. If you need to verify a database, the Entity can be useful to reduce the quantity of database connections. One connection fills up the entity and from now, actual values for assertions can be fetched from that object. Using the design patterns in automated tests is not necessary but recommended. It requires some dedication from a tester as it can be a giant leap for her/him in terms of her/his development skills. I think it is worth trying at least.

3. Supporting the user

The aforementioned architectures fits best when technical engineers are the individuals that implement tests. But what if we want a customer to be involved in the process of writing tests or we have more non-technical testers? Is such participation possible and what can be derived from it? What if we want to separate a test design from implementation? A code from tests? 

Keyword-driven tests

In the Keyword-Driven framework tests are created in an external file - spreadsheet, text or xml outside of code. Tests consist of keywords that correspond to the actions that user can take upon the application under a test e.g.: click, type, drag and drop. These actions can take arguments like object to be clicked on or typed in. To improve maintainability one can move to the higher level of actions that can be undertaken and define more complex keywords like “login”, “add document”, “create an invoice” - it only depends on the application under the test. The keywords are defined in code and adding a new one or changing the old one involves fixing a code base. Also, since the test cases are on a different level, one can change the execution tool beneath without changing test cases itself. Playing this game makes sense only if a non-technical person will be involved in test automation because this approach has some serious shortcomings. The fact that tests are defined in a text file, outside code and “outside” a programming language brings about problems with debugging. We lose a test case abstraction staying over only with the keyword abstraction. Since tests that we want to implement can be really complicated and GUI testing tends to be flaky, the loss of debugging on the test level can be painful. 

The concept of the keyword-driven framework may seem to be complicated especially to implement. Fortunately there are tools/frameworks already implemented that can be integrated with yours. For example Google’s Robot Framework can be easily integrated with Selenium/Webdriver. Also commercial tools have such capabilities. If not, they provide sets of helper objects that can facilitate this task. But still the tester needs to demonstrate development skills in order to get all of these working.

keyword	argument 1	argument 2
open	www.mypage.com
click	addButton
type	editName	John
type	editAge	30
click	submitButton
verifyText	header	Thank John!

Table 1 - an example of keyword driven test in a spreadsheet manner.

Data-Driven tests

When Keyword-Driven tests focus on the logical flow of actions, Data-driven tests focus on data and inputs/outputs. Usually, we have a test case defined and set of test data that will be passed into it as arguments. This way we can execute one scenario with different inputs or outputs for example when testing some sort of calculations or algorithms. Test data can be defined in external file - in this way we separate test data from test code. But this is not necessary. TheTestNG framework, allowing implementation of unit tests  (and not only) for Java applications, comes with the @DataProvider feature letting us to create data sets that can be used and shared among the test method. Using this framework let us remove redundancy from tests when we have many similar tests differing with data only. Moving the data to external source facilitates users, customer, business to create tests and a better collaboration.

BDD tests

But if one need to emphasis a business point of view and requirements then one could go BDD - Behaviour Driven Development. Originating from TDD and eXtreme programming,  Given-When-Then style tests focus on the system behaviour understandable to people and allows non technical individuals to participate in test creation. This style focuses on collaboration between business, QA and developer and this collaboration is facilitated with a ubiquitous language, understandable to everyone. BDD is about examples and there are tools that help us express them in a different manner. Tools like FIT, Fitness, Cucumber allows to create examples in spreadsheets or wiki tables or text files. No development skills are required for creating tests (defining feature or scenario) but a mediation layer called fixture or step definitions must be implemented in a programming language to make them all work.

Model Based tests

On the other hand, sometimes we don’t need to support a non-technical user but we might need to conduct a more technical analysis of the system and therefore test in a more pragmatic way. In such case, the Model Based testing might be a good choice. In this framework, test cases are derived from prepared model automatically so changes in the model are reflected in changes in test cases. With such an approach, one can manage requirements via managing models. Like in aforementioned frameworks, a test design is separated from implementation. Models in this case are an abstraction layer. These models can be created and maintained by testers but also business analysts or whoever who can handle a modelling tool. Since the most often used model is a finite state machine reflecting system behaviour, the modeller must understand fundamentals of software engineering at least. Besides the model, code that executes tests must be implemented and this is usually done by devs. When looking for MBT (Model Based Testing) tools, one can find Spec Explorer or MISTA - a Model Based Testing tool ready for Selenium Integration. Model Based Testing evangelists claim that testing is always based on models. If not as a tangible, genuine form then as a mental one. I think there is something in this statement. Thinking about flows of actions, the structure or behaviour is much easier when visualised. And even if we do not transcribe it, we have always such a model in mind in one or another form.

4. Hybrids

In most cases our needs go beyond one solution. Usually we need to combine nearly all of them or parts of them to accomplish our work. This is what we call aHybrid and implementing such a hybrid on your own may require serious development skills from the tester. But tools like Fit/Fitnesse, Robot Framework or Test Complete has all that features available ready to use so there is no need to implement it from scratch. A hybrid may consist of a mix of BDD with Data-driven tests and Page Objects so in the real world we will rather meet conjunctions then the exact implementation of one framework. These framework does not exclude using them both.

What is the best approach and the golden solution? There are no golden solution, one might say, but this is determined by one’s needs. The most important questions that determines that can be as follows:

• who will use the system and who will develop the tests?
• does the user possess the right technical skills?
• do we need to separate tests from code?
• do we need to emphasise collaboration?
• do we have the time and budget to implement it on our own? or do we emphasise learning  by implementing it from scratch?

 
Diagram 2 - the Hybrid

5. Supporting the tests

The architecture construction is not done yet. While most of the commercial tools provide many helper objects to support tests, using tools like Selenium, one must develop many of the themselves. Or use already created ones like Robot Framework. Nevertheless, this may denote that the code supporting tests will be the main effort in the implementation.

 What do I mean by “Supporting the tests”? I mean that in tests, we need to conduct comparisons, analysis, set up, tear down and all of this code must be implemented to help us achieve that. Features supporting the tests generally vary as our needs do in terms of the application under a test. Depending on the context, Web applications might need, for example, verification of JavaScript errors. Using the Oracle database causes that we might need to implement the mechanism to import a database dump file before every test runs or develops a set of SQL queries to clear up database tables. When testing analytics, we must have means to check that proper requests are sent by the application and extract desired properties from it. So as you see, this effort might require, not only development skills, but also decent domain knowledge (databases, telecommunication etc). Let’s try to name some issues supporting tests:

• logging - to console, to a text file, to an HTML file with a timestamp allowing us to analyse and then precisely conduct a root cause analysis of “red” or flaky tests,
• comparisons and checkpoints - comparing images, XML files, text file, database tables, application under test control’s values, strings, numbers,
• statistics - created at the end of tests, the number of successful, failed tests, execution time, brief of failed tests,
• abstraction layers - at the test run level, test suite level, test level, action level (“add document”, “edit user”), operation level (click, type). It may be needed to inject probes for a better analysis of some issues that happen during tests,
• email support - creating an email to be sent after tests, connect with an email box to verify the email content,
• working with files,
• working with databases,
• test evidence - integrate with a video or screen-shot capture tool to gather the evidence of conducted tests,
• API’s - use tools/libraries (like DefaultHttpClient) to be able to submit HTTP requests in order to communicate with API’s,
• test execution parallelization - this can be developed as a solution in code but also can be brought out of code e.g.: by splitting tests to different test suites and run with different jobs in some CI tool (like Jenkins),

...and many more according to one’s needs.

 Beside all the things that were described here, implementing a test case is the most important issue anyway. You can have sophisticated code with many design patterns, BDD, keywords, data, models but pointless and imprecise tests ruins all the effort. Hence, implementing well balanced tests itself is the most important thing to achieve. That is a subject for another post, though.