Agile Application Lifecycle Management (ALM)

Agile Application Lifecycle Management (ALM) gains more and more momentum.with Agile, or finding a pragmatic approach to ALM at all. Meanwhile, more and more tool vendors find it helpful to label their tools to be Agile tools
or even Agile ALM tools. But what is Agile ALM? In my opinion, ALM synthesizes technical and functional elements to provide a comprehensive approach to common project activities and phases, addressing
build, configuration, deployment, release, test, quality, integration, and requirements management, see Figure 1.
With its interdisciplinary approach, Agile ALM integrates project roles, project phases and artifact types.
An Agile ALM enriches an ALM with agile values and strategies. An agile approach to ALM improves product quality, reduces time to market, and makes for
happier developers. My definition of Agile ALM results in processes and tool chains that are flexible, open to change and high in quality.
This is one of the ways in which ALM helps to provide structure for Agile.

Figure 1. ALM addresses different disciplines and development phases.

Some underlying aspects of Agile ALM are not completely new, and you should respect all the different struggles from previous decades, and put results together to get the best solution for today: in my opinion, ALM evolved from software configuration management (SCM), that in turn has its root in basic version control. You should define your processes and your requirements first, before selecting the tools that best
fit the given task.

Individuals and interactions over processes and tools

Above all, Agile ALM is a discipline and a mental approach. Working with Agile ALM should start with values and people as
well as concepts behind it. An Agile ALM tool is an ALM tool that fosters an agile process.

An Agile ALM tool must be able to add value to the system and improve the collaboration of the stakeholders.
In my opinion, an Agile ALM tool chain must implement the building blocks of Agile ALM, such as continuous integration (including continuous inspection and
continuous deployment), functional/technical releasing, stakeholder focus (and collaborative development) and task-based development.

Many projects feel comfortable with an orchestration of single best of breed tools. Integrating lightweight, configurable tools into flexible tool chains results in a mashup of tools that exactly offer
the features that are needed to solve a given task.

Agile ALM tools should have an open architecture that enables you to add further tools or functionality. Relying on lightweight tool chains can improve flexibility dramatically
because you can replace small units of the overall infrastructure easily without questioning other parts of that infrastructure. Now let's discuss some of the important building blocks of Agile ALM, and start with
task-based development.

Task-based development

With a task-based approach, the task is the unit of interaction and the base of work. Task-based development is the technique of traceably linking work items
to the specific set of changes that was made to complete the work item. One example use case can look like this: you are working on a task that
is listed in your ticket system and has the unique identifier AGILEALM-9. Your IDE (e.g. Eclipse, with Mylyn) is integrated with the
ticket system (e.g. JIRA). The CI server Jenkins integrates with JIRA, with the version control system (VCS) and with the component repository (e.g.
Artifactory) to make the progress of work and the dependencies of artifacts and work items transparent. With Jenkins, together with Artifactory, you can drive staged builds in order to deploy the release
to higher staging environments, without re-building the release ("build once, run everywhere"). Figure 2 shows how Jenkins integrates with
other tools. Zoomed in to a build result page in Jenkins, it is easy to navigate to the VCS (to look at the underlying changes), to the ticket system (to work on the
task) and to the component repository (to work on the binaries).

Figure 2. The CI server Jenkins integrates with VCS, ticket system and component repository.

Collaborative development

Software development is all about implementing requirements. The requirement is the central unit and the driver of a software release. Approaches like unit testing (to
validate that the right thing is developed correctly) and acceptance tests (to validate that the right thing is developed) are not new. But in the past, these approaches have often been
handled in an isolated or puristic way. Instead, a comprehensive, pragmatic solution should be prefered that focuses on the requirement itself, always having all stakeholders in mind.
You can use dedicated, lightweight tools to write acceptance tests, such as Fit. Or you can use specialized languages. Scala and Groovy, both languages offer interesting features for
setting up a polyglot ecosystem, leveraging existing platforms by providing solutions that involve special purpose languages. With Scala and Groovy, you can write tests, which helps
to overcome various barriers:

• Barriers between project phases and project activities (because coding and testing move together more closely)
• Barriers between artifact types (because code and executable specifications are written on the same unified infrastructure)
• Barriers between project roles (because tests are written collaboratively, with mechanisms to use terms close to the problem domain)
• Barriers between tools (because the same tools are used for programming and testing)

The following simple example gives you an impression of how it can look like to write acceptance tests with Scala and the specs2 library.

package alm

import org.specs2._

class AccSpec extends Specification { def is =

  "This is a specification to check the 'Agile ALM' string"                ^
                                                                           p^
  "The 'Agile ALM' string should"                                          ^
    "start with 'Agile'"                                                   ! e1^
    "end with 'ALM'"                                                       ! e2^
                                                                           end
  def e1 = "Agile" must startWith("Agile")
  def e2 = "Agile ALM" must endWith("ALM")
}

The defined is method lists specification fragments which are either simple text (description of the target system), an example (including exectuable code that
returns a result) or a formatting fragment (the p for adding a blank line and starting a new block). Fragments are separated and linked by the ^ character.
For more details on specs2, see specs2.org.

Release management

Release management comprises producing software artifacts and releasing those artifacts according to a defined process. Release management can be differentiated into a functional and a
technical part. To deliver software successfully, both parts are important and should be integrated with each other. Automation and continuous integration are crucial facets of the software
release and delivery process.

Functional release management

Functional release management involves picking the customer's requirements, assigning them to releases and delivering the functionality to the customer, in
high quality. Often agile practices are used to support this process, and many projects achieve good results from using the management template Scrum. While defining a thin set
of rules, Scrum fosters discipline and makes defects (in software as well as in the process) visible. But Scrum is too abstract to apply it "out of the
book". You must implement Scrum and adopt it for software engineering. Implementation practices, for instance, may include distinguishing between special development phases,
on a micro level, inside a Scrum release: during a release, you may think about closing the develoment phase with a frozen zone which allows developers to only work on bug fixes,
instead of new features. Another helpful option is to use code freeze intervals, to complete and ship the final release.

Technical release management

Technical releasing consists of building the software and providing the final product to the user. Build management (comprised of compiling scripts, and packaging and distributing components)
is essential for Agile ALM. Technical release management describes activities to identify configuration items, track and audit changes on requirements and configuration items
and integrate and deliver the implementation. In software engineering change is more the rule than the exception. Because requirements change, it is important to keep the requirements in
sync with their implementations. Possible gaps between functional and technical release management should be bridged. Strategies like VCS hooks help to marry both parts of release management.

Continuous integration (including continuous inspection and continuous deployment)

To automate manual steps means results are delivered in an objective and reproducible way.

Automating the most error-prone, most repetitive and most time-consuming activities is essential. Continuous integration (CI) is the automation of the build, test
and release process and has the goal of integrating the activities of colleagues and the work items others produce.
This can result in a build ecosystem where a new code commit directly triggers a continuous build including compile, technical tests, audits, package, functional tests and deployment.
All different artifact types, platforms and languages, should be integrated, e.g. Java (Groovy, Scala, ..), .NET, PHP and Cobol, using an unified tool infrastructure, see Figure 3. If no native build system exists for a respective language/platform, non-native build technologies can be used to include these artifact types in the CI farm.

Figure 3. A comprehensive CI ecoystem that integrates different artifact types, on an unified infrastructure.

In a continuous integration process, build reports and notifiers should be in place, and information should be shared and aggregated. Aggregating information means that the integrated tool chain
spans the whole, heterogenous ecosystem. This enables the stakeholders to "zoom in" where needed, to get more information where necessary, and make knowledge out of information.
Examples for this are: given a specific build, you can browse to the underlying changes in the VCS, or: collect all produced binaries in the component repository that belong together
semantically, in order to perform operations on them as a group.

Conclusion

Agile ALM spans many disciplines in software engineering. Agile ALM is about people and strategies, and implementing these strategies with lightweight tool chains.
Agile ALM helps to provide structure for Agile and helps to approach ALM in a determined, pragmatic way. Using an Agile approach to ALM, you'll profit from a quicker win and better results.
There is a lot more to say, but I hope this article gave you an impression about what Agile ALM is, and what I desribe in much more depth in my book "Agile ALM".

Evolution of test automation frameworks!!!!!!!!!!!!

Implementing automated tests is something that everybody wants to do. If you ask any tester - test automation is her/his aim. It is the golden target that every tester aims for, but only a few of them take pains to assess the required knowledge, being under the illusion that a programming language or expensive tool will suffice to cope with all the problems that are likely to arise. This is not true. Writing good automated tests is much harder than that. 
1. Dimensions
The first thing that you must consider when setting out test automation efforts is framework or architecture (sometimes called a test harness) in which tests will be placed. By these terms, I mean all code that will support and facilitate test cases development and execution, but also the project structure and any management that will take place (e.g. test data management or documentation). Let's run through the different framework concepts. Perhaps you will recognise the path that you took during your test automation adventure (if you are an experienced tester). Perhaps, if you have just entered the path, it will help you make some decisions. This trip will also consider tester development skills and their evolution that accompanies the evolution of framework. 

Thinking about automation framework, one can think about it in many dimensions. I borrowed these dimensions from Brian Marick’s “Testing quadrant” concept (business facing, technology facing, supporting the team, critique the product) and used it to create frames, a starting point for our weigh ups. We will consider automation frameworks in the following dimensions:
Supporting the code - we will see what coding means to automated testing and how code complexity grows with our needs
Supporting the users - shows different styles and approaches of implementing tests that focus on other individuals then a tester (business, customers)
In Supporting the tests we will talk about what else is needed for framework to be useful and see in what way the context is important

Diagram 1 - Test automation dimensions

2. Supporting the code

When reading testing press, blog posts comments, various forum threads on testing, one special question tops all the others in the context of test automation. This question is usually posed by manual testers lacking technical skills or testing beginners and it reads - how much development skill do I need to proceed with test automation? or do I need to be a developer to start out with automating tests? This is an important question and the supporting the code dimension tries to clarify these doubts.

Linear tests

We usually start with the simplest approach - record and play. A tool records our actions and then makes it possible to have them played back. Tests usually look like linear code - one line after another and so on. Easy to create but leads us to a maintenance nightmare. Often these linear scripts need to be re-factored to run them more than once. Also, the recording feature can be used to speed up test creation in much more complicated frameworks instead of typing it from scratch each time. This approach is easy to use for a tester who does not possess any development skills.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

@Test public void linearTest() {   selenium.open(url);   selenium.type(text);   selenium.click(button);   selenium.verifyTitle(title);   selenium.click(link);   selenium.wait(time);   selenium.open(url2);   selenium.type(text2);   selenium.click(button2);   selenium.verifyTitle(title2);   selenium.click(link2);   selenium.wait(time); }

Example of linear test code.

Structured tests

With higher complexity of test cases that we want to implement, the need for more sophisticated test scripts emerges. We start to familiarize with a programming or script language that the tool supports and start to use its features - conditional statements, variables and loops. We start to call one script from another creating a chains of scripts. With this, our scripts can do more, more extensively and more reliably. On the other hand, lack of development skills can easily lead to creating spaghetti code with the complex and tangled control structure. We still suffer from maintenance issues and going into raptures about new possibilities we do not realise that we are coming closer to a dead end.

Reusable tests

Finally, when maintenance issues hit us it might be too late. But we see that some portions of code repeat and understand that we can reduce the maintenance impact by creating reusable blocks of code. We start using functions or procedures to extract code and create libraries to gather it in one place. We share it between tests and projects. Tests get more clear and readable. At this point, the tester has made a big progress with her/his development skills. She/He knows the basics of the language that she/he uses and starts to explore more advanced features.

1 2 3 4 5 6 7 8 9 10 11

@Test public void reusableTest() {   CQ55 cq = new CQ55();   cq.openWcm();   cq.open(INTEGRATION_AUTHOR_INSTANCE);   PresentationTemplatePage presentation = new PresentationTemplatePage();   presentation.runExportPdfWorkflow();   presentation.confirmWorkflowInfoDialog();   presentation.waitUntillWorkflowNotFinish(); }

Example of reusable test code in Java.

Design Patterns

Sometimes we can go much further. We might bravely step into the development world and take little bites from anything that can help us. Design patterns are that thing. A design pattern is a general reusable solution to a problem occurring commonly. With design patterns we can improve some tests capabilities for example Selenium’s Page Object pattern reduces the maintenance impact and fosters order in source code. In Selenium alsoSingleton can be used to provide a driver instance to tests instead of inheritance. If you need to verify a database, the Entity can be useful to reduce the quantity of database connections. One connection fills up the entity and from now, actual values for assertions can be fetched from that object. Using the design patterns in automated tests is not necessary but recommended. It requires some dedication from a tester as it can be a giant leap for her/him in terms of her/his development skills. I think it is worth trying at least.

3. Supporting the user

The aforementioned architectures fits best when technical engineers are the individuals that implement tests. But what if we want a customer to be involved in the process of writing tests or we have more non-technical testers? Is such participation possible and what can be derived from it? What if we want to separate a test design from implementation? A code from tests? 

Keyword-driven tests

In the Keyword-Driven framework tests are created in an external file - spreadsheet, text or xml outside of code. Tests consist of keywords that correspond to the actions that user can take upon the application under a test e.g.: click, type, drag and drop. These actions can take arguments like object to be clicked on or typed in. To improve maintainability one can move to the higher level of actions that can be undertaken and define more complex keywords like “login”, “add document”, “create an invoice” - it only depends on the application under the test. The keywords are defined in code and adding a new one or changing the old one involves fixing a code base. Also, since the test cases are on a different level, one can change the execution tool beneath without changing test cases itself. Playing this game makes sense only if a non-technical person will be involved in test automation because this approach has some serious shortcomings. The fact that tests are defined in a text file, outside code and “outside” a programming language brings about problems with debugging. We lose a test case abstraction staying over only with the keyword abstraction. Since tests that we want to implement can be really complicated and GUI testing tends to be flaky, the loss of debugging on the test level can be painful. 

The concept of the keyword-driven framework may seem to be complicated especially to implement. Fortunately there are tools/frameworks already implemented that can be integrated with yours. For example Google’s Robot Framework can be easily integrated with Selenium/Webdriver. Also commercial tools have such capabilities. If not, they provide sets of helper objects that can facilitate this task. But still the tester needs to demonstrate development skills in order to get all of these working.

keyword	argument 1	argument 2
open	www.mypage.com
click	addButton
type	editName	John
type	editAge	30
click	submitButton
verifyText	header	Thank John!

Table 1 - an example of keyword driven test in a spreadsheet manner.

Data-Driven tests

When Keyword-Driven tests focus on the logical flow of actions, Data-driven tests focus on data and inputs/outputs. Usually, we have a test case defined and set of test data that will be passed into it as arguments. This way we can execute one scenario with different inputs or outputs for example when testing some sort of calculations or algorithms. Test data can be defined in external file - in this way we separate test data from test code. But this is not necessary. TheTestNG framework, allowing implementation of unit tests  (and not only) for Java applications, comes with the @DataProvider feature letting us to create data sets that can be used and shared among the test method. Using this framework let us remove redundancy from tests when we have many similar tests differing with data only. Moving the data to external source facilitates users, customer, business to create tests and a better collaboration.

BDD tests

But if one need to emphasis a business point of view and requirements then one could go BDD - Behaviour Driven Development. Originating from TDD and eXtreme programming,  Given-When-Then style tests focus on the system behaviour understandable to people and allows non technical individuals to participate in test creation. This style focuses on collaboration between business, QA and developer and this collaboration is facilitated with a ubiquitous language, understandable to everyone. BDD is about examples and there are tools that help us express them in a different manner. Tools like FIT, Fitness, Cucumber allows to create examples in spreadsheets or wiki tables or text files. No development skills are required for creating tests (defining feature or scenario) but a mediation layer called fixture or step definitions must be implemented in a programming language to make them all work.

Model Based tests

On the other hand, sometimes we don’t need to support a non-technical user but we might need to conduct a more technical analysis of the system and therefore test in a more pragmatic way. In such case, the Model Based testing might be a good choice. In this framework, test cases are derived from prepared model automatically so changes in the model are reflected in changes in test cases. With such an approach, one can manage requirements via managing models. Like in aforementioned frameworks, a test design is separated from implementation. Models in this case are an abstraction layer. These models can be created and maintained by testers but also business analysts or whoever who can handle a modelling tool. Since the most often used model is a finite state machine reflecting system behaviour, the modeller must understand fundamentals of software engineering at least. Besides the model, code that executes tests must be implemented and this is usually done by devs. When looking for MBT (Model Based Testing) tools, one can find Spec Explorer or MISTA - a Model Based Testing tool ready for Selenium Integration. Model Based Testing evangelists claim that testing is always based on models. If not as a tangible, genuine form then as a mental one. I think there is something in this statement. Thinking about flows of actions, the structure or behaviour is much easier when visualised. And even if we do not transcribe it, we have always such a model in mind in one or another form.

4. Hybrids

In most cases our needs go beyond one solution. Usually we need to combine nearly all of them or parts of them to accomplish our work. This is what we call aHybrid and implementing such a hybrid on your own may require serious development skills from the tester. But tools like Fit/Fitnesse, Robot Framework or Test Complete has all that features available ready to use so there is no need to implement it from scratch. A hybrid may consist of a mix of BDD with Data-driven tests and Page Objects so in the real world we will rather meet conjunctions then the exact implementation of one framework. These framework does not exclude using them both.

What is the best approach and the golden solution? There are no golden solution, one might say, but this is determined by one’s needs. The most important questions that determines that can be as follows:

• who will use the system and who will develop the tests?
• does the user possess the right technical skills?
• do we need to separate tests from code?
• do we need to emphasise collaboration?
• do we have the time and budget to implement it on our own? or do we emphasise learning  by implementing it from scratch?

 
Diagram 2 - the Hybrid

5. Supporting the tests

The architecture construction is not done yet. While most of the commercial tools provide many helper objects to support tests, using tools like Selenium, one must develop many of the themselves. Or use already created ones like Robot Framework. Nevertheless, this may denote that the code supporting tests will be the main effort in the implementation.

 What do I mean by “Supporting the tests”? I mean that in tests, we need to conduct comparisons, analysis, set up, tear down and all of this code must be implemented to help us achieve that. Features supporting the tests generally vary as our needs do in terms of the application under a test. Depending on the context, Web applications might need, for example, verification of JavaScript errors. Using the Oracle database causes that we might need to implement the mechanism to import a database dump file before every test runs or develops a set of SQL queries to clear up database tables. When testing analytics, we must have means to check that proper requests are sent by the application and extract desired properties from it. So as you see, this effort might require, not only development skills, but also decent domain knowledge (databases, telecommunication etc). Let’s try to name some issues supporting tests:

• logging - to console, to a text file, to an HTML file with a timestamp allowing us to analyse and then precisely conduct a root cause analysis of “red” or flaky tests,
• comparisons and checkpoints - comparing images, XML files, text file, database tables, application under test control’s values, strings, numbers,
• statistics - created at the end of tests, the number of successful, failed tests, execution time, brief of failed tests,
• abstraction layers - at the test run level, test suite level, test level, action level (“add document”, “edit user”), operation level (click, type). It may be needed to inject probes for a better analysis of some issues that happen during tests,
• email support - creating an email to be sent after tests, connect with an email box to verify the email content,
• working with files,
• working with databases,
• test evidence - integrate with a video or screen-shot capture tool to gather the evidence of conducted tests,
• API’s - use tools/libraries (like DefaultHttpClient) to be able to submit HTTP requests in order to communicate with API’s,
• test execution parallelization - this can be developed as a solution in code but also can be brought out of code e.g.: by splitting tests to different test suites and run with different jobs in some CI tool (like Jenkins),

...and many more according to one’s needs.

 Beside all the things that were described here, implementing a test case is the most important issue anyway. You can have sophisticated code with many design patterns, BDD, keywords, data, models but pointless and imprecise tests ruins all the effort. Hence, implementing well balanced tests itself is the most important thing to achieve. That is a subject for another post, though.