Rough Notes on Scala Existential

Summary

This article is the summary of my understanding of the articles written by others. I might have reproduced some of the paragraphs as they were. Please refer to the references section for the full articles.

Existential types are ways of abstracting over types where they are not specified but knowing they are there. In other words, variables as references to the unknown type . Most importantly, existential types would not have existed if it is not for Java wildcards.

Iterator[T] forSome { type T } is equivalent to Iterator<?> in Java.

Iterator[T] forSome { type T <: Number } is equivalent to Iterator<? extends Number> in Java.

Raw types, as in pre-Java 5 libraries, had no type parameter. Existential types are not raw types. They are safer because they stop the compiler instead of letting the compiled code to blow up during execution by throwing runtime exception.

In Scala, Array[T] forSome { type T } is an existential type. It is an array of T where T is some completely unknown type. All that is to T is that it exists at all. This assumption is weak but it means Array[T] at the very least indeed an Array and not an Apple.

Array[T] forSome { type T } matches Array of any type. This is to say foo(arr: Array[T] forSome { type T }) can be written as foo(arr: Array[_]) which also can be written as foo[T](arr: Array[T])

Array[T forSome { type T }], on the other hand, is equivalent to Array[Any] which takes only Array[Any] because Array[A] is defined with an invariant type parameter unlike List[+A] is defined with a covariant type parameter.

More Examples

Map[Class[T forSome { type T }], String] is equivalent to Map[Class[Any], String]. This map only accepts classOf[Any] as keys.

Map[Class[T] forSome { type T }, String] is probably what we want. It accepts keys of Class type i.e. classOf[String], classOf[Int],....

Map[Class[T], String] forSome { type T } matches a map such that once T is defined the rest of the keys must be of same type e.g. if T is String, the rest of the keys must be classOf[String].

// Tested working as of Scala 2.11.x

var a: List[Class[_]] = List(classOf[Int],classOf[String])
var b: List[Class[_]] = List(classOf[String],classOf[String])
var c: List[Class[_]] = List(classOf[Any],classOf[Any])

// var d: List[Class[T forSome{type T}]] = List(classOf[Int],classOf[String]) // compile error
// var e: List[Class[T forSome{type T}]] = List(classOf[String],classOf[String]) // compile error
var f: List[Class[T forSome{type T}]] = List(classOf[Any],classOf[Any])

var g: List[Class[T] forSome{type T}] = List(classOf[Int],classOf[String])
var h: List[Class[T] forSome{type T}] = List(classOf[String],classOf[String])
var i: List[Class[T] forSome{type T}] = List(classOf[Any],classOf[Any])

// var j: List[Class[T]] forSome{type T} = List(classOf[Int],classOf[String]) // compile error
var k: List[Class[T]] forSome{type T} = List(classOf[String],classOf[String])
var l: List[Class[T]] forSome{type T} = List(classOf[Any],classOf[Any])

References

  1. Scala’s existential types. Further down the tread Martin mentioned Scala existential would not have existed if not for Java wildcards.
  2. Existential types in Scala
  3. Existential types are not raw types
  4. Returning the “Current” Type in Scala. Using F-bound type effectively in Scala and why Typeclass is a better solution than F-bound in Scala.

Splitting Hair With Scala Trait

When comes to trait there are Object Oriented (OO) and Functional Programming (FP) ways to look at it. From the OO perspective, trait is treated similar to Java interface where it is implemented with a class. On the other hand, in FP perspective , trait is a type class. Some books did not go further after showing how trait is used in an OO setup. It raises questions when it is used as type class in other examples.

For example, in OO way,

trait Foo[T] {
  def plus(a: T, b: T): T
}

class Bar extends Foo[String] {
  override def plus(a: String, b: String) = a + b
}

val bar = new Bar
bar.plus("10", "aa")

For the same use case in type class approach we have,

implicit object FooString extends Foo[String] {
  override def plus(a: String, b: String) = a + b
}

def add[T: Foo](a: T, b: T)(implicit f: Foo[T]) = 
  f.plus(a, b)

add("aa", "bb")

I would prefer type class pattern approach over interface implementation because type class is more flexible.

Use Try Instead of try-catch Block

Java and Scala have try-catch-finally or try-catch, for short, block to manage exceptions and errors. But, unlike Java, Scala has no checked exceptions and treats all Java exceptions as unchecked exceptions a.k.a runtime exceptions.

The try-catch syntax and usage are about the same for both languages mentioned. Nothing difficult here. However, for most of the part, it is preferably or idiomatic to use Try or Try[+T] in Scala than the try-catch block. Try has more benefits compare to the try-catch block because it is easily chained together and malleable.

Take for example,

Try{foo(10)}.flatMap(_ + 1).map(_.toString)

The integer returns by foo(...) is increased by 1 and then converts to a string. Yet, the value in Try can be further be manipulated on and on as it passes from function to function. Value returns by function inside Try is contained inside Success. Use get to extract the value.

Even if there is an exception thrown by foo(...), the whole chain is still intact. The final result at this point is an instance of Failure holding the exception thrown by foo(...).

Try is the super class of Success and Failure.

The better part of it is the whole chain can be assigned to a value,

val t: Try[String] = Try{foo(10)}.flatMap(_ + 1).map(_.toString)

and pass to the next function or return to the caller.

Let’s look at a longer example of how try-catch is managed in Java compares to Scala. The requirements and the Java implemention are copied from my previous article with the class name changed,

Example 1: foo(...) should throw an exception if r is more 60 and bar(...) should throw an exception if r is more than 70. If r is less than 60, the sum of bar(...) and foo(...) and 20 shall be printed. In the event that foo(…) or bar(…) throws an exception, 20 shall be printed instead.

public class TypicalJavaExceptionHandler {
public static void main(String[] args) {
int r = Integer.parseInt(args[0]);
TypicalJavaHandling handling = new TypicalJavaHandling();
System.out.println(handling.fuzz(r));
}

public int fuzz(int r) {

// version 1
try {
return handling.bar(r) + handling.foo(r) + 20;
} catch (Exception e) {
return 20;
}
}

public int foo(int r) {
if (r > 60) throw new IllegalArgumentException("foo value more than 60");
return 100 / r;
}

public int bar(int r) {
if (r > 70) throw new IllegalArgumentException("bar value more than 70");
return 200 / r;
}
}

And the Scala version with Try[T] instead of the try-catch block,

import scala.util.Try

object ScalaExceptionHandler {

def main(args: Array[String]) {
val r = args(0).toInt
println(fuzz(r))
}

def fuzz(r: Int) =
// version 1
Try{bar(r)}
.flatMap(x => Try{foo(r)}
.map(y => x + y + 20))
.recover{ case t => 20}

def foo(r: Int) = {
if (r > 60) throw new Exception("foo value more than 70")
100 / r
}

def bar(r: Int) = {
if (r > 70) throw new Exception("bar value more than 70")
200 / r
}
}

In this example, there is no obvious advantage over the try-catch block other than the visual aesthetic. Unfortunately, there is a change to the original requirement,

Example 2: foo(...) should throw an exception if r is more 60 and bar(...) should throw an exception if r is more than 70. If r is less than 60, the sum of bar(...) and foo(...) and 20 shall be printed. In the event of an exception is thrown from foo(…), 20 is assumed and for bar(…), 10 is assumed.

With this new requirement, Java method fuzz(…) is rewritten as,

public int fuzz(int r) {

// version 1
/*
try {
return handling.bar(r) + handling.foo(r) + 20;
} catch (Exception e) {
return 20;
}
*/

// version 2
int bar = 0;
int foo = 0;

try {
bar = bar(r);
} catch (Exception e) {
bar = 10;
}

try {
foo = foo(r);
} catch (Exception e) {
foo = 20;
}

return foo + bar;
}

and similarly in Scala,

// version 2
def fuzz(r: Int) =
// version 1
// Try{bar(r)}
// ...

// version 2
Try{bar(r)}
.recover{case e => 10}
.flatMap(x => Try{foo(r)}
.recover{case e => 20}
.map(y => x + y + 20))

Again, using Try is still clear and crispy. When an exception is thrown, recover with 10 and pass on. Again, when there is an exception thrown in the second Try, recover with 20 and finally sum all these values as if there is no exception has occurred. Obviously, when there is no exception, the values x and y are what return by the functions bar(…) and foo(…).

The previous Try chain can be rewritten as,

for {
x 10}
y 20}
} yield x + y + 20

This syntactical sugar code block will be converted to the previous flatMap and map equivalents.

Summary

Try allows the developers to compose the value transformations and exception recoveries. They are also assignable to a value unlike the plain try-catch block which is rigid. However, do not take this as an insinuation that the try-catch block is evil and avoid using them. They are the building blocks for greater things like Try and serve different purposes and it is especially true for library and framework developers. As developers at a higher level, Try should be the first choice.

To Checked or To Unchecked Exceptions

Java classes java.lang.Error and java.lang.Exception manage errors and exceptions. They are direct subclasses of java.lang.Throwable. Errors are thrown when serious problems occur and the application is not supposed to capture them and let the JVM terminates. Exceptions, on the other hand, are recoverable errors most of the time. Developers should catch them and try an alternate logic flow to mitigate the errors. Exceptions are divided into 2 groups of checked and unchecked exceptions, java.lang.Exception and java.lang.RuntimeException respectively. Future exceptions extends java.lang.Exception are checked exceptions and likewise, unchecked exceptions a.k.a runtime exceptions extends java.lang.RuntimeException. Ironically, java.lang.RuntimeException extends java.lang.Exception.

This article is not about handling exceptions but to show why unchecked exceptions are preferred over checked exceptions. There are lots of material available for those who wants to learn more in-depth about Java exceptions.

Checked Exceptions

Java introduced checked exceptions to make sure that exceptions are properly addressed to during compilation and developers are obliged to catch them with the try-catch block or re-throw them to the caller to manage. A typical Java class handling checked exceptions,

Example 1: foo(...) should throw an exception if r is more 60 and bar(...) should throw an exception if r is more than 70. If r is less than 60, the sum of bar(...) and foo(...) and 20 shall be printed. In the event that foo(…) or bar(…) throws an exception, 20 shall be printed instead.

public class Checked {
    public static void main(String[] args) {
        int r = Integer.parseInt(args[0]);

        if (r > 60) {  // preflight check
            System.out.println(20);
        } else { 
            Checked handling = new Checked();
            System.out.println(handling.fuzz(r));
        }
    }

    public int fuzz(int r) {
        try {
            return bar(r) + foo(r) + 20;
        } catch (Exception e) {
            return 20;
        }
    }

    public int foo(int r) throws Exception {
        if (r > 60) throw new Exception("foo value more than 60");
        return 100 / r;
    }

    public int bar(int r) throws Exception {
        if (r > 70) throw new Exception("bar value more than 70");
        return 200 / r;
    }
}

Although there is a preflight check, fuzz(...) has to include the unnecessary try-catch boilerplate or otherwise, it will not compile.

Runtime Exceptions

Developers are not obliged to catch runtime exceptions. The technique to catch runtime exceptions is the same as catching checked exceptions. Uncaught runtime exceptions behave like checked exceptions, they get thrown from one method to another until it reaches the “top” and the JVM terminates e.g. the infamous uncaught NullPointerException. Runtime exception is implicitly thrown from one method to another unless there is a try-catch block waiting to catch it.

Example 2: Same requirements as Example 1

public class Unchecked {
    public static void main(String[] args) {
        int r = Integer.parseInt(args[0]);

        if (r > 60) {  // preflight check
            System.out.println(20);
        } else { 
          Unchecked handling = new Unchecked();
          System.out.println(handling.fuzz(r));
        }
    }

    public int fuzz(int r) {
        return bar(r) + foo(r) + 20;

        /* previous try-block is made obsolete because of preflight check
        try {...}
        */
    }

    public int foo(int r) {
        if (r > 60) throw new RuntimeException("foo value more than 60");
        return 100 / r;
    }

    public int bar(int r) {
        if (r > 70) throw new RuntimeException("bar value more than 70");
        return 200 / r;
    }
}

fuzz(...) method has become much simple. The preflight check removes the necessity of the cumbersome try-catch block if the input is already been verified and ready to use. Preflight checks are mechanisms to vet raw input before passing on to the other internal methods verified.

Can you spot the bug in Examples 1 and 2?

Conclusion

Using runtime exceptions frees the developer from repeating the try-catch blocks when the input is known to be safe and usable.

Methods that deal directly with external sources can use the try-catch block as one of the many ways to manage random inputs from external sources.

Why Does A Function With 2 Parameters Is Defined Function2[-T1, -T2, +R]?

A two-parameter Scala function is defined with contravariant parameter type and covariant return type,

Function2[-T1, -T2, +R](t1: T1, t2: T2) : R

instead of,

Function2[+T1, +T2, -R](t1: T1, t2: T2) : R

Why? Let’s assume a higher order function accepts a function with covariant type parameters. There is no guarantee that the given function can find the methods in the instances provided by the higher order function.

For example, Fruit is the parent of classes Apple and Orange. If the function is going to work with Fruit, it can work with Apple or Orange instances from the higher order function. The function will crash when the function invoked a method specific to Orange with an instance of Apple.

Do not confuse this with the actual instances pass to the function by the higher function and the function instance. The rule applies to the function instance pass to the higher function.

Next, the contravariant return type does not go well with the caller because the function could return Fruit or Plant instances, assuming Plant is the parent of Fruit, if the function return type is Apple. This is because there is no guarantee that the method available in Fruit or Apple is also available in Plant. and it will cause the higher function to blow up if the method is not there.

Going back to the original definition. Remember, we can pass subclasses but not the parents of the intended parameter type regardless of how the parameter types are defined i.e. +T or -T. When the higher order function invokes the given function, it knows for sure that the instances it passes can handle what the function demands because the function is expecting the parent or the exact type declared in the parameter.

And finally, the return type is covariant is because the instance returned can manage what is asked by the higher function. The higher function can only invoke methods defined in the return type of the function it accepts. Say, the higher function accepts a function with the return type of Fruit, the higher function can only invoke methods available in Fruit. Therefore, the function can return instances of Fruit, Apple or Orange without breaking the contract.

Type Classes And Interfaces

What interface in Java is what trait in Scala. Types are not classes. Types are interfaces and they are abstract. As such,

  • A type is an abstract interface.
  • A class is an implementation of a type or interface.
  • An instance or object is the state a class holds at any point in time.

Say, I have a Driveable (interface / trait) for anything that can move forth and back. Next, classes Car and Bus implement Driveable. Each has similar and distinct properties but both can move forth and back. Finally, we C200, CX9 and Tata which are Car and Bus instances that hold the state for each class separately at any given time.

However, this article is not about trait, it is about the Type Class pattern which uses trait and implicit to achieve its objective.

Type Class Pattern

Type class is a pattern, an alternative to Strategy and Adapter patterns. It has its root in Haskell. Scala uses implicit and trait as shown here to implement type class pattern.

case class Alcohol(liters:Double)
case class Water(liters:Double)

case class Fire(heat:Double)
trait Flammable[A] {
  def burn(fuel:A): Fire
}

implicit object AlcoholIsFlammable extends Flammable[Alcohol] {
  def burn(fuel:Alcohol) = Fire(120.0)
}

def setFire[T](fuel:T)(implicit f:Flammable[T]) = f.burn(fuel)

setFire(Alcohol(1.0)) // ok
setFire(Water(1.0)) // FAIL

Using import creatively I could have have different Flammable without recompiling setFire.

There is a Java version which I mimicked the Scala version,

// AlcoholIsFlammable.java
public class AlcoholIsFlammable implements Flammable<Alcohol> {
    public Fire burn(Alcohol fuel) {
        return new Fire(120.0);
    }
}

// Alcohol.java
public class Alcohol {
    public double liters;

    public Alcohol(double liters) {
        this.liters = liters;
    }   
}

// Fire.java 
public class Fire {
    public double heat;

    public Fire(double heat) {
        this.heat = heat;
    }   
}

/* Flammable.java */
public interface Flammable<A> {
    Fire burn(A fuel);
}

/** uncomment this to make water flammable :)
// WaterIsFlammable.java
public class WaterIsFlammable implements Flammable<Water> {
    public Fire burn(Water fuel) {
        return new Fire(60.0);
    }
}

// Water.java
public class Water {
    public double liters;

    public Water(double liters) {
        this.liters = liters;
    }   
}
*/

// RunMe.java
public class RunMe {
    public static void main(String[] args) {    
        System.out.println(setFire(new Alcohol(1.0), 
            new AlcoholIsFlammable()).heat);
        // compilation will fail if the next line is uncommented
        // System.out.println(setFire(new Water(1.0), 
        //    new AlcoholIsFlammable()).heat);          
    }

    public static <T> Fire setFire(T fuel, Flammable<T> f) {
        return f.burn(fuel);
    }
}

Without implicit it makes no sense in Java because all types and classes must be present during compilation unless dynamic class injection like Guice or Spring IoC is used. Therefore, IoC makes less sense in Scala.

Summary

This article is a summary from a few random resources including Wikipedia. Out of them, I find this paragraph,

A crucial distinction between type classes and interfaces is that for class A to be a “member” of an interface it must declare so at the site of its own definition. By contrast, any type can be added to a type class at any time, provided you can provide the required definitions, and so the members of a type class at any given time are dependent on the current scope. Therefore we don’t care if the creator of A anticipated the type class we want it to belong to; if not we can simply create our own definition showing that it does indeed belong, and then use it accordingly. So this not only provides a better solution than adapters, in some sense it obviates the whole problem adapters were meant to address.

and this,

Type classes define a set of contracts that the adaptee type needs to implement. Many people misinterpret type classes synonymously with interfaces in Java or other programming languages. With interfaces the focus is on subtype polymorphism, with type classes the focus changes to parametric polymorphism. You implement the contracts that the type class publishes across unrelated types.

stands out because it is clear and simple to understand

Scala Puzzlers

An Artima published book by Andrew Phillips and Nermin

I just completed Scala Puzzlers and found it good for intermediate Scala programmers. I consider myself an intermediate Scala programmer. Beginners to Scala might not find some it useful just yet because they do not understand the language and the way how ?Scala works behind the scene. Once you have gained sufficient Scala experience, the reader will find lots of “aha” moments and fun too with this book. When I started with Scala, I encountered lots of peculiar errors and warnings during compilation or run-time. These messages or explanations were not easily understood. “Googling” helped to resolve some, some were not found and shelved until I come back for them for second time round. Hopefully, by then, someone would explain them in a blog or book (well, this is the book).

I tend to forget as of what causes the errors if I seldom see them. So, I make quick references to some of the chapters for the explanations and solutions to these errors.

There is a companion website to this book.

Arg Arrgh!! (Puzzler 6)

A good pointer for applying a function repeatedly,

val result = (1 to n).foldLeft(arg) {
    (acc, _) => f(acc) // f(f(...(f(arg))))
}

This chapter answered why there was a compile error: could not find implicit value for parameter Ops:Numeric[N] as N was clearly bound to Double,

def nextNumber[N](n: N)(implicit numericOps: Numeric[N]) =
    numericOps.plus(numericOps.times(n, n), numericOps.one)
def applyNMulti(3)(2.0, nextNumber)

where applyNMulti was defined,

def applyNMulti[T](n: Int)(arg: T, f: T => T) = ...

No, the compiler looks for each type requirement “individually” and not within the same parameter list. Unlike in curried function,T was evaluated to Double in the earlier part of the parameter list, arg, in def applyNMulti[T](n: Int)(arg: T)(f: T => T) = .... That’s why it worked in curried function.

However, the functions can be used as uncurried variant by explicitly putting the type,

  1. applyNMulti(3)(2.0, nextNumber[Double]) -or-
  2. applyNMulti[Double](3)(2.0, nextNumber)It might not be practical solution if the first parameter type differs. It has nothing to do with Typeclass Pattern.

A Case of Equality (Puzzler 10)

This chapter explained the inner working of case classes implicit overrides with the methods hashCode() & equals(...) where their short hands were ## & == respectively. It showed the replacement mechanisms for new and case class factory method. For example, how to new case class without causing exception or replacing the standard factory method generated by the compiler in default case class declaration. It also showed to use abstract case class to replace the the default case class factory method.

As a result, structure equality was maintained i.e. 2 instances are equal if they have the same type and equal constructor arguments and types.

hashCode() and equals(...) are independent. Implement them separately.

To Map or Not to Map (Puzzler 12)

Words of advice, convert the original to Seq if stable iteration is required.

Return to Me! (Puzzler 14)

For Scala versions 2.11.7 or later, this might not happen because the compiler will throw and error when the statement is defined.

val two = (x: Int) => { return x; 2 }

will throw an exception NonLocalReturnControl exception to imitate return x and therefore the whole expression 1 + one(2) + two(3) will return 3. Also, this will happen to val but not def.

Count Me Now, Count Me Later (Puzzler 15)

Highlighted the difference between add(counter)(_) and add(counter) _. Answer, counter in the former would NOT be evaluated immediately but the later would be evaluated immediately.

From http://scalapuzzlers.com/#pzzlr-022, f(a) _ and f(a)(_) have different meanings, governed by different sections of the Scala language specification.

  1. f(a)(_) is the placeholder syntax for anonymous functions, described in SLS §6.23. Evaluation is deferred.
  2. f(a) _ is eta expansion, described in SLS §6.26.5. Arguments given are evaluated eagerly; only the method call itself is deferred.

It is worth noting a couple of points about this program. First, the use of vars does not reflect idiomatic Scala style. Even though Scala supports mutable state, it encourages a functional style of programming, i.e., preferring vals and immutability.

One Bound, Two to Go (Puzzler 16)

TODO Try out

What’s in A Name? (Puzzler 19)

TODO Try val adder = new AdderWithBonus

Irregular Expression (Puzzler 20)

  1. Side effect from print caused the first statement succeed. This is because MatchIterator initialized the matcher via toString() call.
  2. hasNext() is a more reliable approach unless it is known the iterator is non-empty.
  3. use findAllMatchIn(...) instead of findAllIn(...). Otherwise, use findAllIn(...) with matchData to avoid accidental access to start.

Cast Away (Puzzler 22)

A wrapped value stored in a collection will be unboxed when compared against another value type. Of the declared type of the collection is a wrapper type, by contrast, no unboxing will occur and the value type will be boxed instead.

Because of performance reasons, compiler will always try to carry out operations on primitive types instead of wrapper type. Refer to the examples.

A Case of Strings (Puzzler 27)

  • Null is just above Nothing
  • null is the one and only instance of Null
  • null cannot be tested with isInstanceOf[Null]

Pick A Value, Pick AnyVal! (Puzzler 28)

The default value of type T as follow:

  • 0 if T is Int
  • 0 if T is Long
  • 0.0f if T is Float
  • 0.0d if T is Double
  • false if T is Boolean
  • () if T is Unit
  • null for the rest of T types for given
    trait NutritionalInfo {
        type T <: AnyVal
        var value: T = ...
    }
    
    val containsSugar = new NutritionalInfo { 
        type T = Boolean 
    }
    
    println(containsSugar.value)     // null
    println(!containsSugar.value)   // false
    

    because when attempt is made to negate the value, the compiler knows that it is a Boolean to be able to invoke the unary_! method. Automatic unboxing i.e. scala.runtime.BoxesRuntimeUnboxToBoolean was called.

The compiler refuses to unbox if the value can be treated as an Any i.e. println(...), accepts Any as parameter type. Hence, there is no unboxing and underlying null value is retrieved.

Implicit Kryptonite (Puzzler 29)

Overriding implicit of different name but of same type will cause ambiguity and compilation error.

Follow a pattern to avoid this and find similar patterns provided by the library. Case in point,

class OperatingMode {
    implicit val ConsoleRender: Scanner.Console {...}
    implicit val DefaultAlarmHandler: Scanner.AlarmHandler {...}

object NormalMode extends OperatingMode 
object TestMode extends OperatingMode {
    override implicit val ConsoleRender: Scanner.Console {...}
    override implicit val TestAlarmHandler: Scanner.AlarmHandler {...}
}

Quite The Outspoken Type (Puzzler 30)

Function vals e.g. val stringToInt: String => Int = _.toInt take precedence over defs during implicit search. Therefore, it caused StackOverflowException in this puzzler because it kept referring to itself instead of augmentString method in object Predef. Had it has been defined using def type mismatch error would have thrown.

However, if regular defs and function vals were treated equally, an ambiguous reference to the overloaded definition would cause an exception.

Solution: prefer defs over function vals when defining implicit to have ambiguity exception thrown during compilation to catch easy to miss bug.

A View to Shill (Puzzler 31)

mapValues return a new iterator every time a value is retrieved as in the example. Therefore, force each with view before iterate through the sequence using next().

Set The Record Straight (Puzzler 32)

Methods define without parenthesis may accidentally invoke the apply(...) method. Therefore, it is suggested that include empty parenthesis only for method invocations with side-effects.

Beware of unintended type widening caused by methods on collections that allow the element type of a returned collection to be wider than original type.

The Devil Is in The Detail (Puzzler 33)

Use Map.withDefaultValue for immutable defaults only other use Map.withDefault. withDefaultValue returns the same instance across all map entries. withDefault { _ => Buffer(100)} to create new instances as in this example.

A Listful of Dollars (Puzzler 35)

type Dollar = Int
final val Dollar: Dollar = 1
val a: List[Dollar] = List(1, 2, 3)

println(a map { a: Int => Dollar })
println(a.map(a: Int => Dollar))  // (2)

This entry is to highlight the difference of using curly brackets and brackets within a function e.g. map(...). So, a type declaration in an “Expr” that is not enclosed in parentheses, as in our case, is treated as a type ascription to the entire expression. In other words:

  • { a: Int => Dollar } is parsed as { (a: Int) => Dollar }, with Int being the type of a and Dollar the constant value 1
  • (a: Int => Dollar) is parsed as (a: (Int => Dollar)}, with a being a function of type Int => Dollar and Dollar a type alias for Int

Therefore, the second a in (2) is a List which was declared earlier takes an index of type Int and returns an Int (alias Dollar). That is why an IndexOutOfBoundsException was thrown at (2) when it was asked for the 4th element with ‘3’.

Additional Notes

Hard to Reason

Why is side effect is “hard to reason”? When functions with side effect are passed around it is hard to know when and how many times it is called. Basically, it is hard to debug and understand. So it is more predictable if the functions have no side effect.

Eta Expansion

Eta expansion is the operation of automatically coercing a method into an equivalent function, for example,

val f = foo _ // or
val f: A => B = foo

Single Method Call

The compiler can turn consecutive innovations into a single method call if all arguments are provided, for example,

curried(x: Int)(y: Int)(z: Int) == 
    singleCurried(x: Int, y: Int, z: Int)

So basically the JVM will invoke curried(1)(2)(3) as curried(1, 2, 3) in one go instead of 3 times.

Of Scala Functions and Methods

Of Scala Functions and Methods

“What is the difference between a function and a method?” I have been doing Java for so long that I have to paused to think for a moment to answer. Java has no functions, at least before version 8. As far as Scala goes, you can define function using either ways,

scala> def plusOne(i: Int) = i + 1
plusOne: (i: Int)Int

scala> val plusOne = (i: Int) => i + 1
plusOne: Int => Int = <function1>

Or variation of definitions using def and val.

What got me to write this article was not the original question I was asked. It was clearly explained in Programming in Scala Second Edition. Martin Odersky – Lex Spoon – Bill Venners

function A function can be invoked with a list of arguments to produce a result. A function has a parameter list, a body, and a result type. Functions that are members of a class, trait, or singleton object are called methods. Functions defined inside other functions are called local functions. Functions with the result type of Unit are called procedures. Anonymous functions in source code are called function literals. At run time, function literals are instantiated into objects called function values.

I wrote this article to remind myself why functions cannot be overloaded while methods can. For example,

scala> object PlusOneUtils {
         def plusOne(i: Double) = i + 1
         def plusOne(i: Int) = i + 1
       }
defined object PlusOneUtils

scala> PlusOneUtils.plusOne(1)
res2: Int = 2

scala> PlusOneUtils.plusOne(1.0)
res3: Double = 2.0

And,

scala> object SumUtils {
         val sum = (a:Double, b:Double) => a + b
         val sum = (c:Int, d:Int) => c + d
       }
<console>:13: error: sum is already defined as value sum
     val sum = (c:Int, d:Int) => c + d

Methods (defined with def) can be converted to functions,

scala> val p1 = PlusOneUtil.plusOne _
p1: Int => Int = <function1>

But it cannot be overloaded,

scala> val p1:Double = PlusOneUtil.plusOne _ 
<console>:11: error: type mismatch;
 found   : Int => Int
 required: Double
       val p1:Double = plusOne _

While a standalone function will show its type when executed without its parameters,

scala> val p2 = (a:Int, b:Int) => a+b
p2: (Int, Int) => Int = <function2>

scala> p2
res14: (Int, Int) => Int = <function2>

An error would be thrown for function define with def ,

scala> val p1 = PlusOneUtil.plusOne
<console>:11: error: missing arguments for method plusOne in object PlusOne; follow this method with `_' if you want to treat it as a partially applied function
       val p1 = plusOne
                ^

A Curious Case of Method Overloading

There is a curious case resulted in overloading methods of same parameter after type erasure. Usually, scalac, will complaint and terminate the compilation. However, if the return type is different for the methods, scalac will compile and there is no exception thrown during execution. For example,

object Overload{
  def foo(xs : String*) = "foo";
  def foo(xs : Int*) = 3;  // returns an Int type
}

compiles but this would fail the compilation,

object Overload{
  def foo(xs : String*) = "foo";
  def foo(xs : Int*) = "bar";  // returns String type
}

Of Clojure Lists And Vectors

I converted a minute Scala assignment to Clojure to experience the differences in solving the assignment. There were possibilities that I solved the issues the Scala way and not knowing there are better Clojure methods to resolve them. I tried to be as objective as possible while I was completing the quest.

Long story short, unlike Scala and Java, Clojure does not do classes, obviously, and uses the only four (4) fundamental structures lists, maps, vectors and sets to build bigger and more complex structures, e.g. maps of maps of lists and sets, to represent artefacts from the real world.

Second, I want to highlight these confusing commands I worked with during my assignment. The results from the same command were different depending on the type of structure used. Before we go into the example to show you the behaviour I am referring to, let me give you a short background on

  1. list e.g. ‘(1 2 3) or (list 1 2 3); in Clojure, you must declare your list with a ‘ (single quote) otherwise it will treat the first element as a command instead of an element.
  2. vector e.g. [1 2 3] or (vector 1 2 3)
  3. cons returns a new collection of a new element and the given collection
  4. conj returns a new collection of a new element and the given collection
  5. into returns a new collection that combines two given collections, think “union”

() in Clojure is a linked list (LinkedList in Java) whereas [], a vector, is an array (ArrayList in Java). There are subtle differences between them and I find them bit annoying using them and easy to make mistakes. Lots of practices required here.

Let us go to the examples, the result for each command is preceded with “=>”.

;; cons and conj with vector
(conj [1 2 3] 4)
; => [1 2 3 4]
 ​
(conj 4 [1 2 3]) 
; => java.lang.ClassCastException: java.lang.Long cannot be cast to clojure.lang.IPersistentCollection …
 ​
(cons [1 2 3] 4)
; => java.lang.IllegalArgumentException: Don't know how to create ISeq from: java.lang.Long …
 ​
(cons 4 [1 2 3])
; => (4 1 2 3)
 ​
;; cons and conj with list
(conj '(1 2 3) 4) 
; => (4 1 2 3)
 ​
(conj 4 '(1 2 3))
; => java.lang.ClassCastException: java.lang.Long cannot be cast to clojure.lang.IPersistentCollection …
 ​
(cons '(1 2 3) 4)
; => java.lang.IllegalArgumentException: Don't know how to create ISeq from: java.lang.Long …
 ​
(cons 4 '(1 2 3))
; => (4 1 2 3)
 ​
(into '(4) '(1 2 3))
; => (3 2 1 4)
 ​
(into '(1 2 3) '(4))
; => (4 1 2 3)
 ​
(into [4] [1 2 3])
; => [4 1 2 3]
 ​
(into [1 2 3] [4])
; => [1 2 3 4]
 ​
(rest '(1 2 3 4))
; => (2 3 4)
 ​
(rest [1 2 3 4])
; => (2 3 4)

​Depending on the structure (list or vector), (cons …) or (conj …) operates on, the new element could be added as the first or the last element in the new collection. This could pose a serious problem if the position of the new element added to the collection matters.

Finally, if you want to combine two vectors together where the elements in second vector are added after the last element in the first vector and the first element in the first vector is removed. On top of this, the order of the elements is utter important. We can achieve this using the command (into …) and (rest …) i.e.

(into (rest [1 2 3 4]) [5])
; =>  (5 2 3 4)

However, the result returned is (5 2 3 4) and not the expected result of [2 3 4 5]. What has happened?

(rest …) will return a list of elements minus the first element regardless what is the given collection type it operates on. Subsequently, when the result is passed to (into …) as in (into (rest [1 2 3 4]) [5]),  the result is (5 2 3 4) instead of the expected [2 3 4 5]. Please refer to the examples above for clarity on (into …) operates on list and vector. (rest …) drops the first element and silently converts the vector to a list before passing the result to (into …). The remedy is to convert the list to vector before passing to (into …) i.e. (into (vec (rest [1 2 3 4])) [5]) and you have the expected result.

(into (vec (rest [1 2 3 4])) [5])
; => [2 3 4 5]

This is a hard to find bug. There was no error reported or exception thrown because Clojure is a dynamic typing language. So, coming from a statically typed language world, this is something really unexpected. I have no other recommendation other than writing sufficient test cases to cover all the outcome to nail this bug.