# Network

Copyright | (c) 2017 Christian Merten
Maintainer | c.merten@gmx.net
Stability | experimental
Portability | GHC
Safe Haskell | None

Network

Contents

- Network
- Learning functions
- Activation functions
- Network serialization

Description

An implementation of artifical feed-forward neural networks in pure Haskell.

An example is added in XOR.hs

Synopsis

- data Network a = Network {
  - layers :: [Layer a]
  }
- data Layer a = Layer {
  - weights :: Matrix a
  - biases :: Vector a
  }
- newNetwork :: [Int] -> IO (Network Double)
- output :: (Numeric a, Num (Vector a)) => Network a -> ActivationFunction a -> Vector a -> Vector a
- trainShuffled :: Int -> (Network Double -> Int -> String) -> Network Double -> CostFunction -> Lambda -> Samples Double -> Int -> Double -> IO (Network Double)
- trainNTimes :: Int -> (Network Double -> Int -> String) -> Network Double -> CostFunction -> Lambda -> Samples Double -> Int -> Double -> Network Double
- data CostFunction
  - = QuadraticCost
  - | CrossEntropyCost
- getDelta :: Floating a => CostFunction -> a -> a -> a -> a
- type LearningRate = Double
- type Lambda = Double
- type TrainingDataLength = Int
- type Sample a = (Vector a, Vector a)
- type Samples a = [Sample a]
- (-->) :: Vector a -> Vector a -> Sample a
- type ActivationFunction a = a -> a
- type ActivationFunctionDerivative a = a -> a
- sigmoid :: Floating a => ActivationFunction a
- sigmoid' :: Floating a => ActivationFunctionDerivative a
- saveNetwork :: (Element a, Binary a) => FilePath -> Network a -> IO ()
- loadNetwork :: (Element a, Binary a) => FilePath -> IO (Network a)

# Network

data Network a

The generic feedforward network type, a binary instance is implemented. It takes a list of layers with a minimum of one (output layer). It is usually constructed using the `newNetwork` function.

Constructors

Network |

Fields

layers :: [Layer a]

Instances

(Show a, Element a) => Show (Network a) |
(Element a, Binary a) => Binary (Network a) |

data Layer a

One layer of a network, storing the weights matrix and the biases vector of this layer.

Constructors

Layer |

Fields

weights :: Matrix a
biases :: Vector a

Instances

(Show a, Element a) => Show (Layer a) |
(Element a, Binary a) => Binary (Layer a) |

newNetwork :: [Int] -> IO (Network Double)

Initializes a new network with random values for weights and biases in all layers.

    net <- newNetwork [2, 3, 4]

output :: (Numeric a, Num (Vector a)) => Network a -> ActivationFunction a -> Vector a -> Vector a

Calculate the output of the network based on the network, a given `ActivationFunction` and the input vector.

# Learning functions

trainShuffled :: Int -> (Network Double -> Int -> String) -> Network Double -> CostFunction -> Lambda -> Samples Double -> Int -> Double -> IO (Network Double)

The most used training function, randomly shuffling the training set before every training epoch

    trainShuffled 30 (\n e -> "") net CrossEntropyCost 0.5 trainData 10 0.1

trainNTimes :: Int -> (Network Double -> Int -> String) -> Network Double -> CostFunction -> Lambda -> Samples Double -> Int -> Double -> Network Double

Pure version of `trainShuffled`, training the network n times without shuffling the training set, resulting in slightly worse results.

data CostFunction

Cost Function Enum

Constructors

QuadraticCost |
CrossEntropyCost |

Instances

Eq CostFunction |
Show CostFunction |

getDelta :: Floating a => CostFunction -> a -> a -> a -> a

getDelta based on the raw input, the activated input and the desired output results in different values depending on the CostFunction type.

type LearningRate = Double

The learning rate, affects the learning speed, lower learning rate results in slower learning, but usually better results after more epochs.

type Lambda = Double

Lambda value affecting the regularization while learning.

type TrainingDataLength = Int

Wrapper around the training data length.

type Sample a = (Vector a, Vector a)

Training sample that can be used for the training functions.

    trainingData :: Samples Double
    trainingData = [ fromList [0, 0] --> fromList [0],
                     fromList [0, 1] --> fromList [1],
                     fromList [1, 0] --> fromList [1],
                     fromList [1, 1] --> fromList [0]]

type Samples a = [Sample a]

A list of `Sample`s

(-->) :: Vector a -> Vector a -> Sample a

A simple synonym for the (,) operator, used to create samples very intuitively.

# Activation functions

type ActivationFunction a = a -> a

Activation function used to calculate the actual output of a neuron. Usually the `sigmoid` function.

type ActivationFunctionDerivative a = a -> a

The derivative of an activation function.

sigmoid :: Floating a => ActivationFunction a

The sigmoid function

sigmoid' :: Floating a => ActivationFunctionDerivative a

The derivative of the sigmoid function.

# Network serialization

saveNetwork :: (Element a, Binary a) => FilePath -> Network a -> IO ()

Saves the network as the given filename. When the file already exists, it looks for another filename by increasing the version, e.g mnist.net becomes mnist1.net.

loadNetwork :: (Element a, Binary a) => FilePath -> IO (Network a)

Load the network with the given filename.