I do give quite a number of senior graduation projects on data analysis. For these projects my students need good reliable sources of data. I have used Quandl few times as a source of financial data, and I am quite happy about it. Its API has python and R bindings. But today, I am going to write a simple common lisp function to fetch data from Quandl.
In order to fetch data from quandl, we are going to need an api-key. I am going to assume you already got an api-key from quandl. Once you get that, using the web API is quite simple.
For the purposes of this implementation, I am going to need a JSON decoder and an http-client.
(mapc #'require '(:drakma :cl-json))
(DRAKMA CL-JSON)I am going to need a utility macro.
(defmacro ->> (x &rest forms)
(dolist (f forms x)
(if (listp f)
(setf x (append f (list x)))
(setf x (list f x)))))
->>Next, the api call function:
(defun get-quandl (api-key data-set start end)
(->> (concatenate 'string
"https://www.quandl.com/api/v3/datasets/"
data-set
".json"
"?api_key=" api-key
"&start_date=" start
"&end_date=" end)
drakma:http-request
octets-to-string
cl-json:decode-json-from-string
(assoc :dataset)
cdr))
GET-QUANDLAnd here is a utility function that returns the selected portion of the data:
(defun get-column (data column)
(let ((n (position column
(assoc :column--names data)
:test #'string-equal)))
(mapcar (lambda (x) (elt x n))
(cdr (assoc :data data)))))
GET-COLUMNI am going to calculate the correlation between EUR to CAD exchange rate to EUR to USD exchange rate. First let us get the data:
(defparameter CAD (get-quandl YOUR-API-KEY "ECB/EURCAD" "2016-01-01" "2016-12-31"))
(defparameter USD (get-quandl YOUR-API-KEY "ECB/EURUSD" "2016-01-01" "2016-12-31"))
CAD
USDLet us define some basic statistics functions I need below:
(defun mean (data)
(->> (reduce #'+ data)
(* (/ (length data)))))
(defun cov (xs ys)
(let* ((x (mean xs))
(y (mean ys)))
(mean (mapcar (lambda (u v) (* (- u x) (- v y))) xs ys))))
(defun std (xs)
(let ((x (mean xs)))
(sqrt (mean (mapcar (lambda (u) (* (- u x) (- u x))) xs)))))
(defun cor (xs ys)
(/ (cov xs ys)
(* (std xs) (std ys))))
MEAN
COV
STD
CORand finally the correlation between EURCAD and EURUSD.
(cor (get-column CAD "Value") (get-column USD "Value"))
0.21443476