Few days ago I described a machine learning algorithm that learns the coefficients of a regression. The setup I gave is far too general. If you read it carefully, you see that I described a perceptron
Today, I am going to redo what I did in scala.
Here is my implementation:
import $ivy.'org.scalanlp:breeze_2.11:1.0-RC2'
import breeze.linalg._
import math.random
import scala.util.Random.nextInt
def logit(x:Double):Double = 1.0/(1.0 + math.exp(-x))import $ivy.'org.scalanlp:breeze_2.11:1.0-RC2'
import breeze.linalg._
import math.random
import scala.util.Random.nextInt
defined function logitcase class perceptron(n: Int, fn: Double=>Double, eta: Double){
private var derivative = 0.0
private var input = DenseVector.zeros[Double](n+1)
var weights = DenseVector.zeros[Double](n+1)
def forward(xs:DenseVector[Double]):Double = {
input = DenseVector.vertcat(DenseVector(1.0),xs)
val calc = input.dot(weights)
derivative = (fn(calc + eta/2) - fn(calc - eta/2))/eta
return fn(calc)
}
def train(ys:Double, xs:DenseVector[Double]) {
val delta = ys - forward(xs)
weights += delta*eta/(derivative + eta*(2.0-math.random))*input
}
}defined class perceptronThe data I am going to use is Skin Segmentation Data Set from UCI. The data consists of 4 columns each is an integer from 0 to 255 indicating pixel color (RGB) values, and the last is either 1 or 2.
Let us read the data:
val raw = scala.io.Source.fromURL("https://archive.ics.uci.edu/ml/machine-learning-databases/00229/Skin_NonSkin.txt")
.mkString
.split("\n")
.map(_.split("\t").map(_.toDouble).reverse)
val n = raw.length-1raw: Array[Array[Double]] = Array(
Array(1.0, 123.0, 85.0, 74.0),
Array(1.0, 122.0, 84.0, 73.0),
Array(1.0, 121.0, 83.0, 72.0),
Array(1.0, 119.0, 81.0, 70.0),
Array(1.0, 119.0, 81.0, 70.0),
Array(1.0, 118.0, 80.0, 69.0),
Array(1.0, 119.0, 81.0, 70.0),
Array(1.0, 119.0, 81.0, 70.0),
Array(1.0, 125.0, 87.0, 76.0),
Array(1.0, 125.0, 87.0, 76.0),
Array(1.0, 126.0, 88.0, 77.0),
...-(77%)
n: Int = 245056There are approximately 250K data points. I am going to use only a random sample of 10K points to build a model. Since I am using the sigmoid function, the class labels 1 and 2 h as to be normalized to 0 and 1.
val node = perceptron(3,logit,4e-2)
(1 to 10000).foreach(i=>{
val x = raw(nextInt(n))
node.train(x(0)-1.0, DenseVector(x.slice(1,x.length)))
})node: perceptron = perceptron(3, , 0.001)Let us test on randomly chosen 1K data points.
def experiment(m: Int, data:Array[Array[Double]]): Double = {
val n = data.length
var res = 0.0
(1 to m).foreach(i=>{
val x = data(nextInt(n))
val y = x(0)
val u = 1.0 + node.forward(DenseVector(x.slice(1,4)))
if(math.abs(u-y)>1e-2) res += 1
})
return 1.0 - res/m
}
(1 to 100).map(i=>experiment(100,raw)).reduce(_+_)/100defined function experiment
res97_1: Double = 0.9128999999999999A success of 91.3%. Very nice!