Blog

Posts about technology and arts.

Discussing the flexibility of TAP to cover information generated by TestNG

In today's post I am expanding the previous one regarding the use of TAP with TestNG. Let's discuss about the flexibility of TAP to cover information generated by TestNG, more specifically:

  • Expected Exceptions
  • Data Providers
  • Groups
  • Tests that get skipped
  • Dependencies

We will be addressing these bullet points using tap4j, a TAP implementation for Java. The integration between TAP and TestNG is done through the use of TestNG Listeners developed in tap4j project (a big thanks here to Cesar Fernandes de Almeida for all his help).

There are two listeners in tap4j, br.eti.kinoshita.tap4j.ext.testng.TestTAPReporter and br.eti.kinoshita.tap4j.ext.testng.SuiteTAPReporter. The first was created based on org.testng.reporters.TestHTMLReporter and the latter was created based on org.testng.reporters.SuiteHTMLReporter.

In this figure you have testing tools as TAP producers (left) and different types of applications as consumers (right)

Getting the maximum from TestNG with TAP

Strangely enough, today I decided start to write about TAP (Test Anything Protocol) and received a mail from a Linkedin Group with a white paper about data integration in information systems. Well, let me first explain what is TAP, or Test Anything Protocol. This protocol was created in the beginning of Perl to log test results and is still being used by Perl and other languages (PHPUnit can output TAP too, for instance).

Now, if you already know about TAP and YAML, or if you are like me and like trying things out before reading a long text, then go to the hands on part of this post.

An output in TAP, or a TAP stream, is written in pure text in a human readable way. Like follows:

TAP version 13
1..2
ok 1 Test 1
not ok 2 Missing test parameter: url

Belgium Testing Days 2011: Manage automated tests with TestLink and Hudson/Jenkins

Slides used at Belgium Testing Days 2011: Manage automated tests with TestLink and Hudson/Jenkins

Belgium Testing Days 2011: Manage automated tests with TestLink and Hudson/Jenkins by brunodepaulak

Cheers

Belgium Testing Days (BTD) 2011 How To Automate Tests Using Testlink And Hudson

How To Automate Tests Using Testlink And Hudson

Event
Belgium Testing Days 2011
Where ?
Brussels, Belgium
When ?
2011-02-14

Code coverage helps, but it's not enough

Today I talked to two QA Engineers about code coverage and we all agreed that code coverage helps, but is not enough. And by coincidence today I had to fix a bug in tap4j that reminded me the same thing. My cobertura (the coverage tool that I was using at moment) report said that my code was covered by my tests. However there was a bug in the covered code.

If you are curious about the code you can see this issue here. However if you really would like to know more about code coverage, I suggest you to read this fantastic article written by Brian Marick in 99 with the title “How to misuse code coverage”.

I will finish this post now quoting him:

“(…) they (code coverage tools) are only helpful if they’re used to enhance thought, not to replace it”

Cheers

Subscribe