Using Quandl with Common Lisp

Description of the problem

I do give quite a number of senior graduation projects on data analysis. For these projects my students need good reliable sources of data. I have used Quandl few times as a source of financial data, and I am quite happy about it. Its API has python and R bindings. But today, I am going to write a simple common lisp function to fetch data from Quandl.

Implementation

In order to fetch data from quandl, we are going to need an api-key. I am going to assume you already got an api-key from quandl. Once you get that, using the web API is quite simple.

For the purposes of this implementation, I am going to need a JSON decoder and an http-client.

(mapc #'require '(:drakma :cl-json))

(DRAKMA CL-JSON)

I am going to need a utility macro.

(defmacro ->> (x &rest forms)
  (dolist (f forms x)
    (if (listp f)
        (setf x (append f (list x)))
        (setf x (list f x)))))

->>

Next, the api call function:

(defun get-quandl (api-key data-set start end)
  (->> (concatenate 'string
                    "https://www.quandl.com/api/v3/datasets/"
                    data-set 
                    ".json"
                    "?api_key=" api-key 
                    "&start_date=" start 
                    "&end_date=" end)
       drakma:http-request
       octets-to-string
       cl-json:decode-json-from-string
       (assoc :dataset)
       cdr))

GET-QUANDL

And here is a utility function that returns the selected portion of the data:

(defun get-column (data column)
  (let ((n (position column
                     (assoc :column--names data)
                     :test #'string-equal)))
    (mapcar (lambda (x) (elt x n))
            (cdr (assoc :data data)))))

GET-COLUMN

I am going to calculate the correlation between EUR to CAD exchange rate to EUR to USD exchange rate. First let us get the data:

(defparameter CAD (get-quandl YOUR-API-KEY "ECB/EURCAD" "2016-01-01" "2016-12-31"))
(defparameter USD (get-quandl YOUR-API-KEY "ECB/EURUSD" "2016-01-01" "2016-12-31"))

CAD
USD

Let us define some basic statistics functions I need below:

(defun mean (data)
  (->> (reduce #'+ data)
       (* (/ (length data)))))

(defun cov (xs ys)
  (let* ((x (mean xs))
         (y (mean ys)))
     (mean (mapcar (lambda (u v) (* (- u x) (- v y))) xs ys))))

(defun std (xs)
  (let ((x (mean xs)))
    (sqrt (mean (mapcar (lambda (u) (* (- u x) (- u x))) xs)))))

(defun cor (xs ys)
  (/ (cov xs ys)
     (* (std xs) (std ys))))

MEAN
COV
STD
COR

and finally the correlation between EURCAD and EURUSD.

(cor (get-column CAD "Value") (get-column USD "Value"))

0.21443476