Here in Stratio we use Scala massively; we enjoy doing so and the most important thing is: we try to improve our functional-skills every single day. If you really want to learn and soak up every bit of Scala’s powerful functional features try not to learn them all at once, pick one and try to think of parts of your current code where this feature might fit in.
From Java to Scala
Like many people, before programming in Scala I was a Java guy with quite the experience. In my first professional project with the new language we built a brand new web content delivery platform from scratch for a German publisher based in London. Within that project I did pair programming which helped me out a lot in grasping the basic concepts of the language. Then I moved to Madrid and started working for Stratio where I’ve been involved in four quite different Scala projects: in the first one we built an ad-hoc fraud detection system for a Spanish Bank, in the second one we created a lightweight API to provide an interface to connect to the powerful open-source Stratio Streaming CEP engine, then I worked in a very interesting Big Data project for a Saudi Arabian telecom company using Apache Spark and currently I am working in Stratio Datavis, a data visualization tool which is being built using the Play framework and will easily allow the user to create dashboards and reports with multiple widgets (pie-charts, bars, line-charts, time-lines…), filtering data on-the-fly and querying multiple datasources, such as mongoDB, Cassandra, Spark SQL, and ElasticSearch among others (we are about to launch the beta version of the product so…keep an eye on this one guys!). Ok, let’s stop talking about myself and start with the interesting stuff…
As you know, Scala is not a purely functional language. This feature might be a double-edged sword because when you struggle to write code in a “functional way” you might give up and wind up writing your Scala code in a java-like fashion, please, try to not do this.
If you really want to learn about Scala, spend some time understanding the awesome pattern matching, play with the amazing Scala API collections, use Scala’s Try or Either to change the way you are currently handling exceptions, do not return and handle nulls; change your mind and start using Options, find out the infinite uses of the for-comprehension and the most important stuff: try not using mutable variables anymore!
Writing Scala Code without mutable variables
When I am writing Scala code I strongly avoid using mutable variables. Why? Well the most important reason for me is because it is FUN. Yes, it is fun, because it forces you to change your mind and start thinking in a proper, functional way. It’s fun because each piece of code or method that you write becomes a small challenge. Sometimes you might think: “Shit! I cannot implement this stuff without using a mutable var” but believe me: you can.
Let’s see some code to show it. Last week while I was working on the Stratio Datavis module I faced a pretty simple problem. We needed to find String duplicates within a list of objects and if we found some duplicates we needed to append a sequential id to the string. For instance the following List:
List(DataResult(“myId”, 0), DataResult(“yourId”, 1), DataResult(“myId”,2))
Should be turned into this one:
List(DataResult(“myId[1]”, 0), DataResult(“yourId”, 1), DataResult(“myId[2]”,2))
How would we resolve this in Java? I guess we would need to create some auxiliar mutable structures to find out the DataResult objects with a duplicated value, then we might loop through the list again, we should look for duplicated elements for each element in our auxiliar structure, keeping one mutable count variable for each duplicated element, appending it to the string id and incrementing that variable, mmmm….it doesn’t seem like a neat solution, does it?
Let’s see how I solved this puzzle in Scala. First of all I took advantage of the API collections to find out duplicated elements, see following the method:
def getDuplicatedElements[A](listOfElements: List[(String, A)]): Map[String, List[Int]] = listOfElements .groupBy(_._1) .mapValues(_.size) .filter(_._2 > 1) .map(nameAndNumber => (nameAndNumber._1, (1 to nameAndNumber._2).toList))
The operations over a Scala collection can be put together so first I group the elements by its first element (the id), this way I get tuples of ids and an iterator with its repeated elements, then I get the size of the iterator using the mapValues method after that I filter the elements with more than one repetead element and finally, for convenience, I transform the number of elements into a list containing elements from 1 to the number of elements (for instance 3 would be transformed into List(1,2,3)). This kind of one-liners might seem intimidating at first but when you get used to the API collections, you discover infinite options that allow you to perform very powerful operations in a simple way which IMHO leads you to write code that is both easy to read and to understand.
The next step in this small “challenge” was to append the sequential ids to the strings. I asked myself, how would I do that without using mutable vars? Let’s take advantage of another FP tool and implement a recursive method:
private def assignSequentialIds[A](repeatedValues: Map[String, List[Int]], listOfElements: List[(String, A)]): List[(String, A)] = {
listOfElements match {
case Nil => List.empty
case head :: tail if (isDuplicate(head._1, repeatedValues)) => {
(appendSequentialId(head._1, repeatedValues), head._2) ::
assignSequentialIds(removeElementFromList(repeatedValues, head._1), tail)
}
case head :: tail => head :: assignSequentialIds(repeatedValues, tail)
}
}
In Scala this type of recursive methods over a collection are pretty common. Scala lists allow us to apply pattern matching over them and take its head and tail using the :: operator. The first case of the matching is the “exit” case: if the list is empty we return an empty List. The second case takes care of lists with head and tail and check whether the first element of the list is a duplicated element (exists within the repeatedValues map worked out previously) if so we assign the sequential id using the list calculated in the first step and call the recursive method again getting rid of the sequential id that we have just used and also removing the head of the list. Finally the third case of the pattern matching handles not repeated elements for which we need to do nothing but append to the head of the list the result of calling the recursive method using the list’s tail.
This is a pretty simple example but I think it shows quite well how you can take advantage of Scala’s FP features to change your mind and start writing your code in a different and IMHO neater way. If you want to go into details about FP in Scala, have a look at the following presentation I gave at Stratio: FP in Scala
To conclude, if you are a passionate developer and would like to take your programming skills to the next level writing great functional code…join us! We are hiring and…we love Scala 😉
