The Developer’s Guide to Scala Implicit Values (Part I)

Implicit parameters and conversions are powerful tools in Scala increasingly used to develop concise, versatile tools such as DSLs, APIs, libraries…

When used correctly, they reduce the verbosity of Scala programs thus providing easy to read code. But their most important feature is that they offer a way to make your libraries functionality extendible without having to change their code nor needing to distribute it.

A great power comes with a great responsibility however. For new comers, as well as for relatively experienced Scala users, they can become a galaxy of confusions and pitfalls derived from the fact that the use of implicit values imply the compiler making decisions not obviously described in the code and following a set of rules with some unexpected results.

This post pretends to shed some light on the use of implicit values. Its content isn’t 100% original, it is just a tourist guide through this full of marvels, and sometimes dangerous, code jungle.
As most of those monstrous things that make us shiver, implicit values are mostly harmless once you get to know them.

Meeting the implicit family: Implicit Parameters and Implicit Conversions

In essence, implicit values are no more than a tool for providing the compiler with a way to continue when the code it is compiling lacks some elements which should had been explicitly specified. These elements can only be:

• A conversion of an element from a type (T) to another (S) when the element is used in a place where a S is required.
• An actual parameter (value passed to a function in a function call).

And are fulfilled with those implicit values present in the context of the incomplete code.
Therefore, there is one type of implicit value for each kind of element above: Implicit conversions and implicit parameters.

The goal of implicit conversions

The straight line is the shortest path from one point to another, similarly the shortest path from a type to another is a function. What is then the most obvious Scala language tool to describe conversions from a type to another?

Yes, functions! A conversion is just a function from T to S

def conversion(x: T): S = ???

Ok, but just defining a function doesn’t imply that all places where S is expected will be also be able to receive a parameter of type T. Let’s solve that:

implicit def conversion(x: T): S = ???

Making the definition as implicit tells the compiler that it should try to apply conversion when S is expected but T is passed.

The provided type is known as the conversion source and the type the conversion provides is the target.

 
case class A(x: Int)
case class B(m: String, v: Double) {
 def method = println(s"hello world $m $v")
}
 
val a = A(10)
var b: B = B("what is the meaning of life?", 42.0)

It is obvious that trying something like:

b = a

Won’t even compile because a type mismatch. However, once an implicit conversion from A (source) to B (target) is available (in context; Don’t worry, this concept will be explained soon), the compiler will try to apply it in order to fix the type error:

implicit def conv(v: A): B = B("Some String", v.x.toDouble)
 
b = a //Now, this is equivalent to write
b = conv(a) // so it works! (A is the source type and B the target)

Double dval = a.v // A doesn't have a v attribute, but B does...
// so it is translated to: Double dval = conv(a).v
String m = a.m // m = conv(a).m
a.method // conv(a).method

Finally, A elements become also valid actual parameters for formal parameters of type B:

def f(x: B): Double = x.m.size * x.v
f(a) == f(conv(a))

And implicit parameters’s?

The concept behind implicit parameters is simple: A method (or function) parameter which can be omitted when calling the method.

As any Scala developer should know, it allows the declaration of functions or methods with 0, 1 or more than 1 parameter lists (this last case is, in fact, the way the language enables currying). Any function implicit parameter must be in the last parameter list and this list must be marked as implicit. Any parameter within this list will be therefore considered implicit which means that can be potentially provided by an implicit value within the function call context (again this term! It’s claiming its own sub-section here!)

Lets see this in action:

 
def f(implicit x: Int) = x*2
 
//f can be used as any other function ...
val res = f(42)   //.. and receive  'x' explicitly
 
//Or, with an implicit 'Int' in context, ...
implicit val v: Int = 42 //Implicit value (implicit actual param)
 
// Be called without any actual parameter:
val resb = f

def g(x: Int, y: Int)(implicit base: Int): Int = (x+y)%base
 
g(41,1) //Will return 0, (41+1)%42, but ...
g(41,1)(100)//will return 42 because of the higher value for 'base'

Implicit values are hugely used to express contexts. e.g:
In a physics simulator for different planets, wouldn’t it be uncomfortable having to explicitly pass the gravitational constant to each calculation method? Why not make it an implicit parameter with a value when working on Earth an another when working on Mars? This observation drives to to concept of implicit context, again!

It is worth of mentioning that the compiler will try those implicit actual parameters (implicit values) which are of the same type than the one required by the formal parameter. The value identifier, its name in the program, doesn’t make any different for the compiler, however it is desirable to follow the same naming rules than any other identifier.

Implicit Conversions and Implicit Parameters: Two sides of the same coin

Before throwing ourselves into the sea of the implicit context, it would be nice to make an stop and think about the relation between the two, so seemingly different, implicit tools.

Being Scala a language with such a functional programming load,  its functions can be considered values.  We have also seen that implicit conversions are functions marked implicit. Following this reasoning one could try to write an implicit conversion as follows:

case class B(x: Int)
case class A(x: String)
 
implicit val a2b  = (v: A) => B(v.x.toInt)
 
val b: B = A("12")

And, as expected, that would work. Implicit conversions are functions, therefore values.

On the other hand, actual implicit parameters (the ones used to complete incomplete function calls) are just that: values.

So, at the end of the day, we have that both implicit conversions and implicit actual parameters are values, special values so marked by the implicit keyword. This should be the first clue to understand the concept of implicit context.

Context?

At last! Lest’s talk about the infamous implicit context. Last section ended providing some insight on what may be this concept so many times referenced in this post.

The implicit context at a given point of your source code is the set of implicit values available for the compiler to perform type conversions or complete function calls when that point of code is providing an unexpected type object or missing some parameter in a function call.

The same way the Hitchhiker’s Guide to the Galaxy describes a towel as … the most massively useful thing an interstellar hitch hiker can have … – an item you should under no circumstances loose – the knowledge of your current implicit context is the most important think to keep in mind when developing and working with code which uses implicit values.

At a given point the implicit context is determined by a fairly concrete set of rules. Instead enumerating all of them here (for that you can always use Google for that), I’ll try to give a guide to understand what implicit values are available at each moment.

All implicit values (both conversions and actual parameters) which are declared in the same scope than a given code snippet are available for the compiler to complete what it my by missing in that snippet

The easiest way of understanding this affirmation is by visualizing implicit elements as regular mutable or immutable variables. If you understand the rules governing the following example:

{
 val x = 3
 var y = 40;
 {
  val x = 7
  y = 42
  println(x*y)
 }
 println(x*y)
}
//Printing:
// 294
// 126

def f(implicit a: Int, b: String): Unit = println(s"a=$a\tb=$b")

This function is defined in the same scope than the following block:

{
 implicit val x: Int = 3; //Implicit value of type Int
 implicit val y: String = "humans"; /* This implicit value will be
 available at this scope level as well as at any sub-level */
 {
  f; //The implicit actual parameter for strings here is "humans"
  {
   //But this implicit declaration for strings eclipses the latter
   implicit val y: String = "dolphins";
   f
  }
 }
 f
}

a=3 b=humans
a=3 b=dolphins
a=3 b=humans

• The first line corresponds to the first call being performed at depth 1. At that point, implicit values x and y are in the implicit context as they are declared at depth 0.
• The second is printed by the call at depth 2. The implicit context in this level has changed, the value “humans” for  y has been hidden by a new one which will be only available at this scope since it has not more sub-levels.
• Therefore, the last call will print “humans” again, the scope here is the one for level 0.

Yes,  I said the compiler wouldn’t consider the identifier as a criteria for picking implicit values. Then, why has the implicit value at level 2 the same value as at level 0? Easy, if both identifiers were different they then the implicit context at level 2 would contain two different values for strings. Precisely because the compiler doesn’t take into consideration the implicit identifiers for picking the gap fillers, it wouldn’t know which of them use:

{
 implicit val x: Int = 3; //Implicit value of type Int
 implicit val y: String = "humans"; /* This implicit value will be
 available at this scope level as well as at any sub-level */
 {
  f; //The implicit actual parameter for strings here is "humans"
  {
   implicit val z: String = "dolphins";
   /* The implicit context at this point contains 2
      implicit values of type String because they are in scope
      with different identifiers.
   */
   f
  }
 }
 f
}

: error: ambiguous implicit values:
 both value z of type String
 and value y of type String
 match expected type String

This is just an example for implicit parameters but the same ruling pattern applies for implicit conversions. So far, implicit parameters behave the same way implicit conversions do. Let’s explore their differences.

Formal implicit parameters become implicit values within the function body

Any parameter in a function implicit parameter list is an implicit value in the scope of the function body. It doesn’t matter whether the actual parameter has been filled by the compiler or explicitly passed. As long as it was declared within the implicit parameter list, it will be implicit within the function scope.

For the following examples, consider the function below:

def implicitly[T](implicit e: T) = e

It expects a parameter of the parametrized type and use it as its return value. This kind of function is useful to “capture” implicit values references and make them explicit references. That isn’t only useful for debugging purposes but also increases the scalability of your code. e.g:

• What if you need to pass your implicit physics simulation engine context to a method for which that context parameter isn’t declared as implicit?
• What if you are using Java libraries?
• What is the easiest way to check how an implicit value is being resolved?

This function importance is such that it is included within Scala’s Predef object.

Knowing the existence of implicitly when can now use it at the promised examples:

def f(implicit x: Int) = println(
  s"Implicit value for integers within f's body: ${implicitly[Int]}"
)
 
f(3) /* Despite 3 being explicitly passed to f,
        it is an implicit value within f's body */
 
implicit val vOutsideF = 42
f /* Obviously, that's also true when the actual parameter is
     filled by the compiler. */

Implicit conversions defined at a target class companion object are available for the compiler to use source ‘s instances where target’s are expected

When the compiler finds a type mismatch it will not only try to apply implicit conversions in scope but also those defined in the companion object of the expected type (target type) class:

case class A(x: Int)
 
case class B(x: Double)
 
object B { //B class companion object
  implicit def fromA(o: A): B = B(o.x.toDouble)
}
 
val b: B = A(3) //B is the target class
 
//B, formal parameter class, is the target class
def show(x: B) = println(x)
show(A(42))

Implicit conversions defined at a source class companion object are available for the compiler to use source’s instances where target’s are expected

Likewise the case above, the compiler will also try implicit conversions defined in the companion object of the passed instance class (source class).

case class A(x: Int)
case class B(x: Double)
 
object A { //A class companion object
  implicit def toB(o: A): B = B(o.x.toDouble)
}
 
val b: B = A(3) //B is the target class
 
//B, formal parameter class, is the target class
def show(x: B) = println(x)
show(A(42))

This subspace of the implicit context is great for expanding old libraries without modifying their source code since it allows you to make your own types compatible with them by just adding the convenient implicit conversions in your classes companion objects.

Mapping your Scala developer’s adventure satchel

Yes, your implicit context is a big satchel. So far we have been exploring its pockets so it seems a good a idea to have its image as a whole:

It ins’t that the scope is more important than any other implicit context part but it is drawn a little bit bigger because of its dynamism: It can mutate at any time by the means of new declarations of the use of import.

This is just a simplification of the principles determining the current implicit context at a Scala program point. These principles can interoperate resulting in not so intuitive effects. I find one  them particularly interesting because it highlights the nature of implicit parameters and conversions as different flavours of the same concept: Implicit value.

Consider the following pair of functions:

def show(x: B) = println(x) //Expects B instances

def f(x: A)(implicit a2b: A => B) = show(x) //Receives A instances

It may result strange that the code above compiles and even more so that the following snippet works:

f(A(42))( v => B(v.x.toDouble))

But there is no magic here once we analyse what the a2b parameter is: It is of the type A=>B which means that it is a function from A to B, the same type an implicit conversion for these types would be. On the other hand, it is an implicit parameter and that implies a2b to be marked as implicit within f body. Both the type and the mark as implicit are the ingredients for an implicit conversion so when the call to show expects a B instance the compiler will be able to use the awkwardly declared implicit conversion a2b.

To be continued…

So far we have reviewed the basis of implicit parameters and conversions. This is like describing what roads and cars are and how the work but not talking about how to exploit them by asking good intentioned people to drive you to your destination in exchange of contemplating your shiny and friendly thumb.

The next part (hopefully posted soon) will explain how to utilize these powerful tools in a higher level of abstraction as well as warning you of the dangerous singularities and pitfalls you may encounter in their use.