Extending this library
Users can define new cost and activation functions, given that they preserve the expected interface of these kinds of functions.
New cost function
To define a new cost function, you should define a function with the following signature:
function newcost(output::Vector{Real}, target::Vector{Real})
This function must return a cost of type Float64
. Alongside this function you must define it's gradient () with respect to the output vector, like this:
function newcostprime(output::Vector{Real}, target::Vector{Real})
This function must return a Vector{Float64}
with the derivatives of the error with respect to each of the output
's coordinates. Check the conceptual PDF more details.
After defining both these functions, you must add newcost
and newcostprime
to the derivatives dictionary:
derivatives[newcost] = newcostprime
New activation function
To define a new activation function, you should define a function with the following signature:
function newactivation(l::FFNNLayer)
This function must return a variable of type Vector{Float64}
containing the activation of each neuron. Alongside this function you must define a function that differentiate the activation of a layer with respect to its neurons, like this:
function newactivationprime(l::FFNNLayer)
This function must return the Jacobian Matrix of the activation of the layer with respect to the neurons of the layer. Check the conceptual PDF more details.
After defining both these functions, you must add newactivation
and newactivationprime
to the derivatives dictionary:
derivatives[newactivation] = newactivationprime