Introduction to Writing Tests with ScalaTest

To be able to quickly start visualizing some of the tests that can be written with ScalaTest, we can take advantage of the test-patterns-scala template from the Typesafe Activator. It consists of a number of examples that essentially target the ScalaTest framework. The easiest way to get the code is to download the template bundle for test-patterns-scala. An archive contains a bootstrap script that can simplify our life’s by starting Activator automatically.

Setting up the test-patterns-scala activator project only requires you to go to the directory where you installed the Typesafe Activator and then, either start the GUI through the > activator ui command, or type > activator new to create a new project and select the appropriate template when prompted.

• Download the bundle
• Extract the downloaded zip file
• cd to test-patterns-scala directory from your console
• Run ./activator ui

You can also generate the project by running:

./activator new PROJECTNAME test-patterns-scala

The template project already contains the sbteclipse plugin; therefore, you can generate eclipse-related files by simply entering from a command prompt in the root directory of the project, as follows:

./activator eclipse

You can look into the various test cases in src/test/scala. As some of the tests use frameworks such as Akka, Spray, or Slick, we will skip these for now to concentrate on the most straightforward ones. In its simplest form, a ScalaTest class (which, by the way, might also test Java code and not just Scala code) can be declared by extending org.scalatest.FunSuite. Each test is represented as a function value, and this is implemented in the Test01.scala class, as shown in the following code:

package scalatest
import org.scalatest.FunSuite

class Test01 extends FunSuite {
  test("Very Basic") {
    assert(1 == 1)
  }
  test("Another Very Basic") {
    assert("Hello World" == "Hello World")
  }
}

To execute only this single test class, you should enter the following command in the command prompt:

./activator
> test-only 

In our case, this command will be as follows:

> test-only scalatest.Test01   (or scalatest.Test01.scala)
[info] Test01:
[info] - Very Basic (38 milliseconds)
[info] - Another Very Basic (0 milliseconds)
[info] ScalaTest
[info] Run completed in 912 milliseconds.
[info] Total number of tests run: 2
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 2, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[info] Passed: Total 2, Failed 0, Errors 0, Passed 2
[success] Total time: 9 s, completed Nov 11, 2013 6:12:14 PM

The example given under src/test/scala/scalatest/Test02.scala within the test-patterns-scala project is very similar, but the extra === instead of == will give you additional info when the test fails. This is shown as follows:

class Test02 extends FunSuite {
  test("pass") {
    assert("abc" === "abc")
  }
  test("fail and show diff") {
    assert("abc" === "abcd") // provide reporting info
  }
}

Once again running the test can be done by entering the following command:

> test-only scalatest.Test02
[info] Test02:
[info] - pass (15 milliseconds)
[info] - fail and show diff *** FAILED *** (6 milliseconds)
[info]   "abc[]" did not equal "abc[d]" (Test02.scala:10)
[info] …
[info] *** 1 TEST FAILED ***
[error] Failed: Total 2, Failed 1, Errors 0, Passed 1

Before fixing the failing test, this time, we can execute the test in the continuous mode, using the ~ character in front of test-only (from the activator prompt), as follows:

>~test-only scalatest.Test02

The continuous mode will make SBT rerun the test-only command each time the Test02 class is edited and saved. This feature of SBT can make you save a significant amount of time by running in the background tests or just programs without having to explicitly write the command. On the first execution of Test02, you can see some red text indicating «abc[]» did not equal «abc[d]» (Test02.scala:10).

As soon as you correct the abdc string and save the file, SBT will automatically re-execute the test in the background, and you can see the text turning green.

The continuous mode works for the other SBT commands as well, such as ~run or ~test.

Test03 shows you how to expect or catch exceptions:

class Test03 extends FunSuite {
  test("Exception expected, does not fire, FAIL") {
    val msg = "hello"
    intercept[IndexOutOfBoundsException] {
      msg.charAt(0)
    }
  }
  test("Exception expected, fires, PASS") {
    val msg = "hello"
    intercept[IndexOutOfBoundsException] {
      msg.charAt(-1)
    }
  }
}

The first scenario fails as it was expecting an IndexOutOfBoundsException, but the code is indeed returning a valid h, the character at index 0 of the hello string.

To be able to run ScalaTest test suites as JUnit test suites (for example, to run them within the IDE or when extending an existing JUnit-based project that is already built in Maven, or when reporting to a build server), we can use the available JUnitRunner class along with the @RunWith annotation, as shown in the following sample:

import org.junit.runner.RunWith
import org.scalatest.junit.JUnitRunner
import org.scalatest.FunSuite
@RunWith(classOf[JUnitRunner])
class MyTestSuite extends FunSuite {
  // ...
}

BDD-style testing

Test06 is an example of a test written in a different style, namely BDD. In short, you specify some kind of a user story in almost plain English that describes the behavior of the scenario you want to test. This can be seen in the following code:

class Test06 extends FeatureSpec with GivenWhenThen {

  feature("The user can pop an element off the top of the stack") 
  {
info("As a programmer")
  info("I want to be able to pop items off the stack")
  info("So that I can get them in last-in-first-out order")

  scenario("pop is invoked on a non-empty stack") {

    given("a non-empty stack")
    val stack = new Stack[Int]
    stack.push(1)
    stack.push(2)
    val oldSize = stack.size

  when("when pop is invoked on the stack")
  val result = stack.pop()

  then("the most recently pushed element should be returned")
  assert(result === 2)

  and("the stack should have one less item than before")
  assert(stack.size === oldSize - 1)
  }

  scenario("pop is invoked on an empty stack") {

    given("an empty stack")
    val emptyStack = new Stack[Int]

    when("when pop is invoked on the stack")
    then("NoSuchElementException should be thrown")
    intercept[NoSuchElementException] {
    emptyStack.pop()
    }

  and("the stack should still be empty")
  assert(emptyStack.isEmpty)
  }
}
}

BDD-style tests represent a higher level of abstraction than JUnit tests, and are more suitable for integration and acceptance testing as well as documentation, for people knowledgeable about the domain. You just need to extend the FeatureSpec class, optionally with a GivenWhenThen trait, to describe acceptance requirements. More details about BDD-style tests can be found at http://en.wikipedia.org/wiki/Behavior-driven_development. We just want to illustrate here that it is possible to write the BDD-style tests in Scala, but we won’t go further into their details as they are already largely documented for Java and other programming languages.

ScalaTest provides a convenient DSL to write assertions in a way close to plain English. The org.scalatest.matchers.Matchers trait contains many possible assertions and you should look at its ScalaDoc documentation to see many usage examples. Test07.scala expresses a very simple matcher, as shown in the following code:

package scalatest

import org.scalatest._
import org.scalatest.Matchers

class Test07 extends FlatSpec with Matchers {
"This test" should "pass" in {
    true should be === true
  }
}

Let’s write a few more assertions using a Scala Worksheet. If you are using Eclipse, right-click on the scalatest package that contains all the test files that were previously reviewed and then select new | Scala Worksheet. We will name this worksheet as ShouldWork. We can then write and evaluate matchers by extending a FlatSpec specification with the Matchers trait, as shown in the following code:

package scalatest
import org.scalatest._
object ShouldWork extends FlatSpec with Matchers {

  true should be === true

}

Saving this worksheet will not produce any output as the matcher passes the test. However, try to make it fail by changing one true to false. This is shown in the following code:

package scalatest
import org.scalatest._

object ShouldWork extends FlatSpec with Matchers {

  true should be === false

}

This time, we get a full stack trace as part of the evaluation, as shown in the following screenshot:

We can start evaluating many more should matchers, as shown in the following code:

package scalatest
import org.scalatest._

object ShouldMatchers extends FlatSpec with Matchers {

  true should be === true

  List(1,2,3,4) should have length(4)

  List.empty should be (Nil)

  Map(1->"Value 1", 2->"Value 2") should contain key (2)
  Map(1->"Java", 2->"Scala") should contain value ("Scala")

  Map(1->"Java", 2->"Scala") get 1 should be (Some("Java"))

  Map(1->"Java", 2->"Scala") should (contain key (2) and not contain value ("Clojure"))

  3 should (be > (0) and be 

The evaluation of the worksheet stops whenever we encounter a test failure. Therefore, we have to fix it in order to be able to progress in the test. This is identical to running the whole testsuite with the SBT test command. See the following code:

object ShouldMatchers extends FlatSpec with Matchers {

"Hello" should be ("Hello")

"Hello" should (equal ("Hej")
               or equal ("Hell")) //> org.scalatest.exceptions.TestFailedException:

"Hello" should not be ("Hello")
}

In the previous example, the last statement (which is the opposite of the first one) should fail; instead, it is not evaluated.

Functional testing

ScalaTest is well integrated with Selenium by providing a complete DSL, making it straightforward to write functional tests. Test08 is a clear example of this integration:

class Test08 extends FlatSpec with Matchers with WebBrowser {

  implicit val webDriver: WebDriver = new HtmlUnitDriver
go to "http://www.amazon.com"
click on "twotabsearchtextbox"
textField("twotabsearchtextbox").value = "Scala"
submit()
pageTitle should be ("Amazon.com: Scala")
pageSource should include("Scala Cookbook: Recipes")
}

Let’s try to run a similar invocation directly into a worksheet. As worksheets give feedback on every statement evaluation, they are very convenient to directly identify what the problem is, for instance, if a link, a button, or content is not found as expected.

Just create another worksheet called Functional next to the ShouldWork worksheet that is already present. Right-click on the scalatest package and select New | Scala Worksheet.

The worksheet can be filled as follows:

package scalatest
import org.scalatest._
import org.scalatest.selenium.WebBrowser
import org.openqa.selenium.htmlunit.HtmlUnitDriver
import org.openqa.selenium.firefox.FirefoxDriver
import org.openqa.selenium.WebDriver
object Functional extends FlatSpec with Matchers with WebBrowser {
implicit val webDriver: WebDriver = new HtmlUnitDriver
  go to "http://www.packtpub.com/"
  textField("keys").value = "Scala"
  submit()
  pageTitle should be ("Search | Packt Publishing")
  pageSource should include("Akka")
}

Upon the save operation, the worksheet will be evaluated and should probably display some output information for every statement, except for the last two lines with should matchers, as they should evaluate to true.

Try to change («Search | Packt Publishing») to a different value, such as Results or just Packt Publishing, and notice how the console output provides handy information on what does not match. This is shown in the following screenshot:

This functional test just scratches the surface of what’s possible. As we are using the Java Selenium library, in Scala, you can inherit the power of the Selenium framework that is available in Java.

References

• Scala for Java Developers By Thomas Alexandre