Renaming imports

One useful thing if you're spending a lot of time interfacing with Java from Scala is renaming imports.

I use scala ArrayLists and java.util.ArrayList a lot. Here's an easy way to not get confused:

import java.util.{ArrayList => JArrayList}

Testing in Scala using a Java tool

This is my first post on Graceless Failures, I plan to write here about my continued experiences learning Scala developing a BDD framework, a build tool and a grid-based gene sequencing tool. Hopefully I'll be able to post a few good code snippets, feel free to comment on whether you'd like to see more or less code.

So here goes...

Scala, like a lot of other languages these days, ships with a unit testing framework - SUnit - built in. Many other Scala specific "testing" frameworks have sprung up in recent times that contain similar or vastly different feature sets to the traditional xUnit tools. These include Reductio, ScalaCheck, Specs, ScalaTest, and SUnit (built into the Scala distribution).

And as Scala is "just Java" you can also use Java frameworks such as JUnit and TestNG. Having only used Reductio, I can't vouch for any others, though ScalaTest is getting good airplay on Artima and Specs seems to have the Scala BDD mindshare.

These tools can be loosely categorised as traditional unit testing tools, ala xUnit, or automated specification testing tools, ala QuickCheck. Reductio and ScalaCheck are incarnations of automated specification testing, while Specs, ScalaTest and SUnit are more your traditional xUnit frameworks.

However, I'm not to write about any of these frameworks, instead, I'm going to write about Instinct, a Java BDD framework that I've been developing for around 18 months, and for which I've recently started to add specific support for Scala into the codebase. Good fodder for blog posts!

The process of getting Instinct running Scala code proved quite trivial, most issues related to typical Ant shenanigans and issues with Scala annotation syntax. The good news is that as advertised, Scala specification classes compiled and were run using the standard Java runners without modification. I was pleasantly surprised by this, it's some of the first (semi-complicated) Java integration I've done with Scala, the only real deal killer I've encountered so far is that class level variables in Scala need to be initialised, something Instinct does automatically for Java specs (there's probably a way around this, but I couldn't find it in the two minutes I spent on it).

There's plenty of literature on BDD out there so I won't speak about it here; let's get to some code. Here's the canonical BDD example, the stack (sans imports and copious line wrapping to satisfy blogger):

final class AnEmptyStackSpeccedUsingScala {
    @Stub var element: Int = 1

    @Specification {
      val expectedException = classOf[RuntimeException], 
      val withMessage = "Cannot pop an empty stack"}
    def failsWhenPopped {
        EmptyStack.pop
    }

    @Specification {
      val expectedException = classOf[RuntimeException], 
      val withMessage = "Nothing to see"}
    def failsWhenPeeked {
        EmptyStack.peek
    }

    @Specification
    def returnsNoneWhenSafelyPopped {
        expect.that(EmptyStack.safePop).isEqualTo(None)
    }

    @Specification
    def isNoLongerEmptyAfterPush {
        val stack = EmptyStack.push(element)
        expect.that(stack.peek).isEqualTo(element)
    }
}

And here's the corresponding code it's speccing out:

sealed trait Stack[+A] {
    def push[B >: A](element: B): Stack[B]
    def pop: Stack[A]
    def safePop: Option[Stack[A]]
    def peek: A
    def safePeek: Option[A]
}

final case object EmptyStack extends Stack[Nothing] {
    override def push[B](element: B) = 
        NonEmptyStack(element, this)
    override def pop = error("Cannot pop an empty stack")
    override def safePop = None
    override def peek = error("Nothing to see")
    override def safePeek = None
}

final case class NonEmptyStack[+A](
        element: A, elements: Stack[A]) 
        extends Stack[A] {
    override def push[B >: A](element: B) =
        NonEmptyStack(element, this)
    override def pop = elements
    override def safePop = Some(elements)
    override def peek = element
    override def safePeek = Some(element)
}

The first thing to notice about the specification is that it's too verbose, we can and should do better with Scala. Some things to focus on are implicit conversions, which would give us RSpec like state expectations, and removing some of the cruft on annotations. Instinct will let us use naming conventions (ala JUnit, etc.) here, but at the expense of flexibility, but we can do better. One approach is the one Specs takes, using closures, another approach, backing off a little, would be closer to RSpec. I'm not sure what is the best option here, hopefully I'll be able to share my learning here, comments are most welcome.

Scala for Rubyists

Over at Coderspiel, n8han has implemented _why's semi-canoncial Ruby tutorial adventure game Dwemthy's Array in Scala.

I like this for two reasons:

1. There aren't enough non-trivial side-by-side Ruby vs Scala examples for Rubyists like myself to learn from.

2. The author has a number in his name, and that's how you know he's a righteous dude.

Also, for gems like this: "And finally, code is data or whatever."

actors must exeunt, not exit

Just found out, while writing unit tests, that the exit method on Actor is for use only within the actor, not externally. That is, a unit test cleanup script can't just call

    myActor.exit

but instead needs to send a message to the actor and let it exit from within:

    case Stop => exit

?~

Scala loves it some overloaded operators.

When working with the highly useful Can class from net.liftweb.util, the ?~ operator is equivalent to asking "is this Can empty?", but has the added bonus of returning a Failure object if it is.

This makes for pleasantly terse expressions that handle failure cases inline.

using call-by-name parameters

At David's suggestion, I stole a logging idea from lift for configgy: using call-by-name parameters to avoid evaluating a log expression until we decide that some handler is actually going to log it (in other words, the formatter is called). Under the hood, scala is turning this:

    log.ifDebug("got " + getCount + " of them")

into something like this:

    log.ifDebug(new Function0[String] { def apply = { "got " + getCount + " of them" } })

I was worried for a few minutes that this might be a lot of overhead for a logging method, but convinced myself that the compiler is doing most of the work, and all that's happening at runtime is a "new", which would happen for almost any logging.

What I wanted, and eventually worked around, was a way to turn the call-by-name parameter "message" from the method name:

    def ifDebug(message: => AnyRef)

into the function object that I knew scala was passing around. I tried casting, but the compiler was way too smart for me. Eventually I just wrote a method that would return the contents of message, and therefore evaluate the function. But I still think it would be nice if I could somehow sweet-talk the compiler into giving me the function object. Call-by-name is mostly for the syntactical-sugar convenience of the caller.

compacting java code, and style

An un-named library included this joyous piece of sample code (please ignore the gratutious style violations -- they were clearly C++ coders):

  public String[] getPrefixes(String prefix) {
      Vector<string> v = new Vector<string>();
      for(String s : map.keySet())
      {
          if(s.startsWith(prefix) && !s.equals(prefix))
          {
              v.add(s);
          }
      }

      if(v.size() > 0)
      {
          return (String[])v.toArray(new String[0]);
      }
      else
      {
          return null;
      }
  }

I thought, "This is a place where scala can shine. Let's try to remove the 99% of this code that's just boilerplate." Here's what I came up with:

    def getPrefixes(prefix: String): Array[String] = {
        val keyList = (for (val key <- map.keys if key.startsWith(prefix) && (key != prefix)) yield key).toList
        if (keyList.size > 0) keyList.toArray else null
    }

But wait. That's really terse, but doesn't look awesome. The first line is too long to completely absorb when skimming, and the last line is questionable style -- I wanted a ruby-style "keyList.toArray unless keyList.empty?" but scala uses inline if/else for the "a ? b : c" operator so it looks weird. Second try, let's use matching instead:

    def getPrefixes(prefix: String): Array[String] = {
        (for (val key <- map.keys if key.startsWith(prefix) && (key != prefix)) yield key).toList match {
            case Nil => null
            case list => list.toArray
        }
    }

That works. The "(...).toList" construct still bothers me a bit though. Is there some better way to do that?

Class file '...' is broken

Nope. Nothing is broken. Just missing. Scala 2.7.1 emits this slightly misleading error message when a dependency is missing. I suppose this then "breaks" the dependent class, but only from the point-of-view of the compiler, not from the point-of-view of the developer.

For example, I'm using net.spy.memcached, but I only placed memcached-2.1.jar in my classpath, and not the sub-dependency spy-2.4.jar. The error returned is:

error: error while loading MemcachedClient, class file '../third-party/java/memcached-2.1.jar(net/spy/memcached/MemcachedClient.class)' is broken
(class net.spy.SpyThread not found.)
SomeClass.scala:13: error: net.spy.memcached.MemcachedClient does not have a constructor
val c = new MemcachedClient(AddrUtil.getAddresses(h, p));

The actual problem is described sotto voce: "(class net.spy.SpyThread not found.)". But, instead I foolishly focused on "class file '...' is broken". Broken is much stronger than not found, and these jars were new to my repository. So, I went around verifying the integrity of the .jar file, missing the main point of the message.

This has been a common theme in my experience thus far: Scala compiler errors need a bit more honing. They sometimes lead my Java and gcc eyes astray. I have to slow down more and re-read the compiler output.

pattern matching and extraction

Pattern matching is cool:

val lst = List(1,2,3,4)

lst match {
case 1 :: _ => // match anything with the first element as 1
}

lst match {
case _ :: 2 :: _ => // match anything with the second element as 2
}

But it gets fun, because you can "extract" values (put them out) and match at the same time:

lst match {
case first :: 2 :: _ => // matches if the second item in the list is 2 and assigns the first element to "first"
}


:: == cons

The ::operator is more or less equivalent to cons in Lisp, ML, and similar languages. When you're pattern matching, it extracts. When you're declaring or assigning variables (particularly Lists), it concatenates.

lazy val

If you want a field or local variable to be initialized only when it's required (that is, accessed/used for the first time), you want a lazy val.

Details and debate on this feature's implementation can be found on the Scala User mailing list, but perhaps a better resource is this post that preceded the feature's release from a developer at EPFL (Scala's home).

Scala Classloader Hijinx

Today I found that Scala's classloader is a bit of a comprise. The details are murky, but JDK libraries that subsequently load non-JDK libraries can have troubles. My guess is that there is some conflict between the bootloader and the regular classloader.

In my case, I have a custom java.util.logging.Formatter class that I have used for years of pawing through gigabytes of server logs and test output. I got one previous coworker hooked enough that he extended this Formatter with a few options, and then wrote a filter to color code and correlate threads.

When moving over to Scala, I couldn't get the scala(1) invoked JVM to load my Java class. I was always left with the ugly and nearly unusable SimpleFormatter output. From verdant wilds of a rain forest island, David suggested that I invoke the whole app from java(1) rather than scala(1), and include the scala jar in the classpath. Works just fine.

My tcsh wrapper went from:

setenv JAVA_OPTS "-ea -esa -Djava.util.logging.config.file=trace.properties"
scala -classpath classes EntryClassName

to

java -ea -esa -classpath /Users/john/scala/lib/scala-library.jar:classes -Djava.util.logging.config.file=trace.properties EntryClassName

Singletons and companion objects

Scala's syntax for singleton objects and the methods thereupon is one my favorite first-blush features of the language:

object MySingletonObject {
def singletonMethod() = println("What a fancy method, and so singular!")
}

So now if you want that supremely valuable console message, just call MySingletonObject.singletonMethod().

Say, though, that you want the equivalent to this bit of Ruby code:

class MyClass
def say_hi
  print "hi"
end

def self.say_yo_from_the_class
  print "yo"
end
end

>> m = MyClass.new
=> #
>> m.say_hi
hi=> nil
>> MyClass.say_yo_from_the_class
yo=> nil

You've got MyClass defined in your Scala code, but how to get that badass say_yo_from_the_class class method? Like this, kid:

object MyClass {
def sayYoFromTheClass() = println("yo")
}

The above singleton is now the companion object of MyClass. Companion objects have to be defined the same source file as the class they accompany. Other than that, pretty intuitive.

map/flatMap/foreach -> for comprehension

Scala has some nifty syntactic sugar that makes operations on collections prettier.