Working with European Central Bank Data in Scala

It seems my Working with ECB Data in Python post is popular. But I noticed that the code in there stopped working.

I have been trying since morning, but I can’t get the urllib.request library work nicely. Then, as much as I love common lisp, playing with XML under CL is nauseating, and unfortunately clojure isn’t any better. One option is to convert XML to JSON but I’d like to stick with XML.

So, I’ll re-write the code in scala this time :)

Libraries

I am going to use sttp for the http requests. Scala plays really nicely with XML data: far better than python, clojure, and even common lisp. I am going to exploit that as well.

So, let us load the required libraries.

import $ivy.`com.softwaremill.sttp::core:1.0.5`
import com.softwaremill.sttp._
import scala.xml.XML.loadString

import $ivy.$                                  
import com.softwaremill.sttp._
import scala.xml.XML.loadString

By the way, I am using jupyter’s scala kernel for this post. But the first import statement I wrote above also works in ammonite.

I am going to need a backend to handle connections.

implicit val backend = com.softwaremill.sttp.HttpURLConnectionBackend()

backend: SttpBackend[Id, Nothing] = com.softwaremill.sttp.FollowRedirectsBackend@7d4124a1

Now, the function that handles connections and returns the result.

def ECBRequest(series: String) = {
    val raw = sttp.get(uri"https://sdw-wsrest.ecb.europa.eu/service/data/EXR/$series.EUR.SP00.A")
                  .header("Accept","application/vnd.sdmx.structurespecificdata+xml;version=2.1")
                  .send()
    val res = loadString(raw.unsafeBody)
    val left = (res \ "@TIME_PERIOD").map(_.text)
    val right = (res \ "@OBS_VALUE").map(_.text.toDouble)
    left.zip(right).toMap
}

defined function ECBRequest

The series string has 2 components:

1. The frequency: this could be annual (‘A’), monthly (‘M’), quarterly ('Q’), or daily (‘D’)
2. The currency.

Let us test:

ECBRequest("M.TRY")

res3: Map[String, Double] = Map(
  "2017-10" -> 4.323390909090909,
  "2010-02" -> 2.07559,
  "2013-09" -> 2.695233333333333,
  "2012-08" -> 2.229082608695652,
  "2003-11" -> 1.7267813,
  "2001-08" -> 1.265762304347826,
  "2015-07" -> 2.970504347826088,
  "2000-03" -> 0.559335521739131,
  "2010-11" -> 1.971663636363636,
  "2012-01" -> 2.375854545454545,
  "2014-01" -> 3.029704545454546,
...

And let us be nice and close the backend :)

backend.close