Moving Day

This site now lives at http://gracelessfailures.com/. I mean, it always has, but now it's hosted on another blogging platform. The posts made it over there, but the comments didn't. Old content will live here on Blogspot for posterity.

Parser surprise

1.toString works fine

-1.toString saz " error: ';' expected but '.' found."

need to use (-1).toString

class what?

I'm a little annoyed that this doesn't work:

scala> class Foo
defined class Foo

scala> object Thing extends Foo
defined module Thing

scala> val set = scala.collection.immutable.Set[Class[_ <: Foo]](Thing.getClass) 
<console>:6: error: type mismatch;
 found   : java.lang.Class[?0] where type ?0
 required: Class[_$1] forSome { type _$1 <: Foo }
       val set = scala.collection.immutable.Set[Class[_ <: Foo]](Thing.getClass)
                                                                       ^

I'm sure it's probably a java compatibility thing, but shouldn't the class of an object of type T return a class of type Class[T]?

Confused by cons

while learning how to use lists:

val oneTwo = List(1, 2)
val threeFour = List(3, 4)
val oneTwoThreeFour = oneTwo :: threeFour //forgot one ":", I really meant to ":::" (concatenate)
val filteredList = oneTwoThreeFour.filter(n => n > 1)

scalac informed me:

Error:Error:line (16)error: value > is not a member of scala.this.Any
val l2 = oneTwoThreeFour.filter(n => n > 1)

Commenting out the offending line and replacing it with

println("oneTwoThreeFour " + oneTwoThreeFour)

yields

oneTwoThreeFour List(List(1, 2), 3, 4)

which clearly shows that the first element is not an Int.

However, the following line compiles:

val filteredList = oneTwoThreeFour.filter(n => n == 1)

probably because '==' is defined for scala.this.Any

Pattern Guards fooled by null

A sadness:

Pattern guards don't handle nulls well, and, also, introducing a pattern guard appears to make a default _ not match on null values:

scala> def guard_fails(s: String) {
| s match {
| case s: String if s == null => println("null")
| case s: String if s != null => println("s=" + s)
| case _ => println("default")
| }
| }
guard_fails: (String)Unit

scala> guard_fails("test")
s=test

scala> guard_fails(null)
scala.MatchError
at .guard_fails(:5)
at .(:6)
at .()
at RequestResult$.(:3)
at RequestResult$.()
at RequestResult$result()
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMeth...
scala>

scala> def guard_fails2(s: String) {
| s match {
| case s: String if s != null => println("s=" + s)
| case _ => println("default")
| }
| }
guard_fails2: (String)Unit

scala> guard_fails2("test")
s=test

scala> guard_fails2(null)
scala.MatchError
at .guard_fails2(:5)
at .(:6)
at .()
at RequestResult$.(:3)
at RequestResult$.()
at RequestResult$result()
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMet...
scala>

To code this match defensively, you have to define a null pattern explicitly:

scala> def ok(s: String) {
| s match {
| case s: String if true => println("s=" + s)
| case null => println("null")
| case _ => println("default")
| }
| }
ok: (String)Unit

scala> ok("test")
s=test

scala> ok(null)
null

scala>

Scala RichString is not comparable to String

A sadness:
val r: scala.runtime.RichString = "1"
val s: String = "1"
if (r == s) {
println("ok1")
} else {
println("sad1")
}

if (s == r) {
println("ok2")
} else {
println("sad2")
}

if (s.equals(r)) {
println("ok3")
} else {
println("sad3")
}

if (r.equals(s)) {
println("ok4")
} else {
println("sad4")
}

Will print:
sad1
sad2
sad3
sad4

This isn't as sad as:
sad1
ok2
sad3
ok4
which would precipitate Robey's "ultimate sadness".

I got into this state by adding a .drop to a String, and chaos resulted. The details are tangentally described in: http://www.nabble.com/String-and-RichString-equality-td14888607.html

Still, this is another thing to keep in your head when coding along. I suppose it's best to coerce RichStrings back to Strings immediately, so as not to allow RichStrings to propagate too far.

NumberFormatException

scala> "true".toBoolean
res1: Boolean = true

That's pretty cool. Does it work for "false" too?

scala> "false".toBoolean
res1: Boolean = false

Yeah! Awesome! How does it handle other values, I wonder?

scala> "1".toBoolean
java.lang.NumberFormatException: For input string: "1"

... Err, what?

Handling nulls

There's been a bunch of posts about handling nulls in Scala (and even Java), so let's throw one more into the mix.

This morning, I've been writing some code to read test data from the filesystem, using resources. Following David's lead on showing Java-Scala code, here's a typical Java take on it, note that I need to handle the case of reading the file in IntelliJ differently than I do in the build (IntelliJ now places resources in a subdirectory per module):

def dataFile(path: String): FilePath = {
  val resource = getResource(path)
  if (resource == null) {
    val intellijResource = getResource("furnace/" + path)
    if (intellijResource == null) {
      error("No resource found at '" + path + "' or 'furnace/" + path + "'")
    } else {
      intellijResource
    }
  } else {
    resource
  }
}

def getResource(path: String) = Foo.getClass.getResource(path)
 

Now, here's the code rewritten using Option for handling a null returned from getResource (apologies for formatting). I'm using Scalaz's OptionW, as it provides a nice (implicit) conversion from null to an Option (onull) and for throwing an error on None (err). As an aside, these would make a fine addition to the standard API.

def dataFile(path: String): FilePath = dataFileFromAntBuild(path)

def dataFileFromAntBuild(path: String) = getResource(path).getOrElse(dataFileFromIntelliJ(path))

def dataFileFromIntelliJ(path: String) = {
  val intellijPath = "furnace" + path
  onull(getResource(intellijPath)).err("No resource found at '" + path + "' or '" + intellijPath + "'")
}

def getResource(path: String) = Foo.getClass.getResource(path)
 

This code is nicer than the first as it explicitly shows that you're sequencing through the nullness and is much more concise (we could make it more concise too if we wish).

For those still stuck in Javaland, there's always Functional Java's Option to achieve a similar thing (though with more verbosity).

Accumulation

It's a pretty common operation to take a list of stuff and accumulate the items. For example, if I have a collection of word frequencies and I want to aggregate the counts by word.

The "Java" imperative way of doing this is to create a HashMap of stuff and populating the HashMap. For example (this is Java-style Scala code):

def accumulate(in: Seq[(String, Int)]): Seq[(String, Int)] = {
  val map = new HashMap[String, Int]
  
  for ((str, cnt) <- in) {
    if (map.isDefinedAt(str)) map(str) = map(str) + cnt
    else map(str) = cnt
  }

  map.toList
}

What are we doing? We're creating some state and iteratively updating the state.

My preferred way of doing this is thinking about the issue as "transformative". We're transforming the input to the output, without mutating anything.

def acc(in: Seq[(String, Int)]): Seq[(String, Int)] =
  in.foldLeft[Map[String, Int]](Map.empty){
    case (map, (str, cnt)) => map + (str -> (map.getOrElse(str, 0) + cnt))
  }.toList

The transformative approach is shorter and I believe, sweeter.

Invoking Java varargs methods from Scala

A quick little example of invoking a Java varags method from Scala. Suppose you've code like the following:

object MyApplication {
  private lazy val javaApi = new JavaApi

  def main(args: Array[String]) {
    val numbers = Array(1, 2, 3)
    javaApi.methodWithVarArgs(numbers)
  }
}

This will generate a nice error message telling you exactly what to do:

.../MyApplication.scala:6: warning: I'm seeing an array passed into a Java vararg.
I assume that the elements of this array should be passed as individual arguments to the vararg.
Therefore I wrap the array in a `: _*', to mark it as a vararg argument.
If that's not what you want, compile this file with option -Xno-varargs-conversion.
    javaApi.methodWithVarArgs(numbers)
                              ^
one warning found
Compile suceeded with 1 warning; see the compiler output for details.

What this is telling you is that you need to help the inferencer by type annotating the argument:

object MyApplication {
  private lazy val javaApi = new JavaApi

  def main(args: Array[String]) {
    val numbers = Array(1, 2, 3)
    javaApi.methodWithVarArgs(numbers: _*)
  }
}

Nice one! Type annotating is something you'll get used to doing with Scala...

importing collections

I'd like to propose the following convention when importing classes from scala's collection library:

If you're using more than one class from the collections library, or might, import the packages themselves, like so:

import scala.collection.immutable
import scala.collection.mutable

They can then be unambiguously used throughout:

val cache = new mutable.HashMap[String, String]

What say ye?

Function Pointers

Scala treats functions declared without a parameter list differently from those with a parameter list. It seems that

def f : Int = 1

isn't a first-class function, but

def fn(): Int = 2

is a first-class function. You cannot assign f to a value or pass it as a parameter. You can pass fn as a parameter trivially, but you need a bit of syntax to assign fn to a value.

class F {
  def f: Int = 1
  def fn(): Int = 2
}

class G {
  def g(func: () => Int) : Int = func()
  def yup: Int = g((new F).fn)
  def yay: Int = {
    val fn = () => (new F).fn()
    g(fn)
  }
  //def nope1: Int = g((new F).f)
  //def nope2: Int = {
  //  val fn: () => Int = (new F).fn()
  //  g(fn)
  //}
  //def nope3: Int = {
  //  val fn = (new F).fn()
  //  g(fn)
  //}
}

None of the nope functions will compile, as fn() is taken as an immediate that returns an Int, not as a function object. Even if you try to "cast" it to () => Int, as in nope2, the compiler still takes fn as an immediate. The yup function works without any fuss which, to this naif, appears at odds with nope2. Both seem to be "casts" of the same ilk. Mmm.

Robey came up with the syntax in the function yay this afternoon. I arrived at the same place via a different path, by emulating how anonymous functions can be assigned to values. In polite company, I might say this is "idiomatic."

So, to assign a previously defined parameterless function to a value, you must declare it as:

def fn(): Int = {...}

and assign it thus:

val function = () => fn()

Introduction to High-Level Programming With Scala

Tony Moris gave a nice introduction to Scala talk at the local Java User's Group last week:

In this presentation, Tony introduces some of the essential tenets of functional programming and why they are important. He draws on the existing Java knowledge of the audience to relate foreign concepts. Tony then goes on to introduce the Scala programming language to the audience.

Full details are available: Introduction to High-Level Programming With Scala (duration 106 mins). Oh, and that's me who keeps interrupting in the background...

Disclaimer: Tony and I work together at Workingmouse.

The Seductions of Scala, Part I

A nice introductory Scala post on Scala by Dean Wampler:

However, I decided to learn Scala first, because it is a JVM language that combines object-oriented and functional programming in one language. At ~13 years of age, Java is a bit dated. Scala has the potential of replacing Java as the principle language of the JVM, an extraordinary piece of engineering that is arguably now more valuable than the language itself. (Note: there is also a .NET version of Scala under development.)

Here are some of my observations, divided over three blog posts.

Source: The Seductions of Scala, Part I

Renaming imports

One useful thing if you're spending a lot of time interfacing with Java from Scala is renaming imports.

I use scala ArrayLists and java.util.ArrayList a lot. Here's an easy way to not get confused:

import java.util.{ArrayList => JArrayList}

Testing in Scala using a Java tool

This is my first post on Graceless Failures, I plan to write here about my continued experiences learning Scala developing a BDD framework, a build tool and a grid-based gene sequencing tool. Hopefully I'll be able to post a few good code snippets, feel free to comment on whether you'd like to see more or less code.

So here goes...

Scala, like a lot of other languages these days, ships with a unit testing framework - SUnit - built in. Many other Scala specific "testing" frameworks have sprung up in recent times that contain similar or vastly different feature sets to the traditional xUnit tools. These include Reductio, ScalaCheck, Specs, ScalaTest, and SUnit (built into the Scala distribution).

And as Scala is "just Java" you can also use Java frameworks such as JUnit and TestNG. Having only used Reductio, I can't vouch for any others, though ScalaTest is getting good airplay on Artima and Specs seems to have the Scala BDD mindshare.

These tools can be loosely categorised as traditional unit testing tools, ala xUnit, or automated specification testing tools, ala QuickCheck. Reductio and ScalaCheck are incarnations of automated specification testing, while Specs, ScalaTest and SUnit are more your traditional xUnit frameworks.

However, I'm not to write about any of these frameworks, instead, I'm going to write about Instinct, a Java BDD framework that I've been developing for around 18 months, and for which I've recently started to add specific support for Scala into the codebase. Good fodder for blog posts!

The process of getting Instinct running Scala code proved quite trivial, most issues related to typical Ant shenanigans and issues with Scala annotation syntax. The good news is that as advertised, Scala specification classes compiled and were run using the standard Java runners without modification. I was pleasantly surprised by this, it's some of the first (semi-complicated) Java integration I've done with Scala, the only real deal killer I've encountered so far is that class level variables in Scala need to be initialised, something Instinct does automatically for Java specs (there's probably a way around this, but I couldn't find it in the two minutes I spent on it).

There's plenty of literature on BDD out there so I won't speak about it here; let's get to some code. Here's the canonical BDD example, the stack (sans imports and copious line wrapping to satisfy blogger):

final class AnEmptyStackSpeccedUsingScala {
    @Stub var element: Int = 1

    @Specification {
      val expectedException = classOf[RuntimeException], 
      val withMessage = "Cannot pop an empty stack"}
    def failsWhenPopped {
        EmptyStack.pop
    }

    @Specification {
      val expectedException = classOf[RuntimeException], 
      val withMessage = "Nothing to see"}
    def failsWhenPeeked {
        EmptyStack.peek
    }

    @Specification
    def returnsNoneWhenSafelyPopped {
        expect.that(EmptyStack.safePop).isEqualTo(None)
    }

    @Specification
    def isNoLongerEmptyAfterPush {
        val stack = EmptyStack.push(element)
        expect.that(stack.peek).isEqualTo(element)
    }
}

And here's the corresponding code it's speccing out:

sealed trait Stack[+A] {
    def push[B >: A](element: B): Stack[B]
    def pop: Stack[A]
    def safePop: Option[Stack[A]]
    def peek: A
    def safePeek: Option[A]
}

final case object EmptyStack extends Stack[Nothing] {
    override def push[B](element: B) = 
        NonEmptyStack(element, this)
    override def pop = error("Cannot pop an empty stack")
    override def safePop = None
    override def peek = error("Nothing to see")
    override def safePeek = None
}

final case class NonEmptyStack[+A](
        element: A, elements: Stack[A]) 
        extends Stack[A] {
    override def push[B >: A](element: B) =
        NonEmptyStack(element, this)
    override def pop = elements
    override def safePop = Some(elements)
    override def peek = element
    override def safePeek = Some(element)
}

The first thing to notice about the specification is that it's too verbose, we can and should do better with Scala. Some things to focus on are implicit conversions, which would give us RSpec like state expectations, and removing some of the cruft on annotations. Instinct will let us use naming conventions (ala JUnit, etc.) here, but at the expense of flexibility, but we can do better. One approach is the one Specs takes, using closures, another approach, backing off a little, would be closer to RSpec. I'm not sure what is the best option here, hopefully I'll be able to share my learning here, comments are most welcome.

Scala for Rubyists

Over at Coderspiel, n8han has implemented _why's semi-canoncial Ruby tutorial adventure game Dwemthy's Array in Scala.

I like this for two reasons:

1. There aren't enough non-trivial side-by-side Ruby vs Scala examples for Rubyists like myself to learn from.

2. The author has a number in his name, and that's how you know he's a righteous dude.

Also, for gems like this: "And finally, code is data or whatever."

actors must exeunt, not exit

Just found out, while writing unit tests, that the exit method on Actor is for use only within the actor, not externally. That is, a unit test cleanup script can't just call

    myActor.exit

but instead needs to send a message to the actor and let it exit from within:

    case Stop => exit

?~

Scala loves it some overloaded operators.

When working with the highly useful Can class from net.liftweb.util, the ?~ operator is equivalent to asking "is this Can empty?", but has the added bonus of returning a Failure object if it is.

This makes for pleasantly terse expressions that handle failure cases inline.

using call-by-name parameters

At David's suggestion, I stole a logging idea from lift for configgy: using call-by-name parameters to avoid evaluating a log expression until we decide that some handler is actually going to log it (in other words, the formatter is called). Under the hood, scala is turning this:

    log.ifDebug("got " + getCount + " of them")

into something like this:

    log.ifDebug(new Function0[String] { def apply = { "got " + getCount + " of them" } })

I was worried for a few minutes that this might be a lot of overhead for a logging method, but convinced myself that the compiler is doing most of the work, and all that's happening at runtime is a "new", which would happen for almost any logging.

What I wanted, and eventually worked around, was a way to turn the call-by-name parameter "message" from the method name:

    def ifDebug(message: => AnyRef)

into the function object that I knew scala was passing around. I tried casting, but the compiler was way too smart for me. Eventually I just wrote a method that would return the contents of message, and therefore evaluate the function. But I still think it would be nice if I could somehow sweet-talk the compiler into giving me the function object. Call-by-name is mostly for the syntactical-sugar convenience of the caller.