In this example we will try to learn, usign a neural network, the behavior of a deterministic XOR function. The input is a vector of two bits, and the correct output is 1 if exactly one of them is 1, and 0 otherwise.

To do this, we will use the dataset available in the examples folder. So, the first thing to do is load the needed libraries and load the data:

# Remember to install HDF5 and JLD packages
using HDF5, JLD, NeuralNetsLite

# Loading our examples
@load Pkg.dir("NeuralNetsLite", "examples", "xor", "data.jld")

To learn from this dataset we will use a neural network with 2 units in the first layer (input), 2 neurons in the hidden layer and 1 neuron in the output layer:

net = FFNNet(2, 2, 1)

Before training, check the error that our model makes in the dataset:

println("In-Sample Mean Error before training: ", meanerror(net, ins, outs))
println("In-Sample Classification Error before training: ", classerror(net, ins, outs))

Now, we just need to train our network using the dataset as example:

train!(net, ins, outs, α=0.5, η=0.1, epochs=500, batchsize=1)

Here we used a learning rate of 0.5 and momentum rate of 0.1. We trained the network 500 times with the same dataset, one example at a time (batchsize=1).

Now, checkout how the error decreased after training:

println("In-Sample Mean Error before training: ", meanerror(net, ins, outs))
println("In-Sample Classification Error before training: ", classerror(net, ins, outs))