Thread: Neural Network method not working

I'm currently trying to make a neural network in java. Most of my code is working, but for some reason the network isn't learning.

I'm having problems implementing backpropagation in the neural net and I don't really understand how to use backpropagation. Could anyone explain it to me?
Here's the code.

Is there a better way to create a neural network than the way I'm using?

(also, this is my first time using this forum, so sorry if i'm not doing it right.)

import java.lang.Math;
import java.util.ArrayList;
import java.util.Scanner;

class Network {

static class Neuron {
double value;
double weight;
int[] connections;
int x, y;
double bias;

public Neuron(double pvalue, double pweight, int[] pconnections, int px, int py, double pbias) {
value = pvalue;
weight = pweight;
connections = pconnections;
x = px;
y = py;
bias = pbias;
}
}

// activation methods:

static double sigmoid(double x) {
return 1/(1+(Math.exp(-x)));
}
static int binaryStep(double x) {
return (x >= 0) ? 1 : 0;
}
static double linear(double x) {
return x;
}
static double rectifiedLinearUnit(double x) {
return Math.max(x, 0);
}
static double tanh(double x) {
return Math.tanh(x);
}

// activation method derivatives:

static double dsigmoid(double x) {
return sigmoid(x) * (1 / sigmoid(x));
}

// error methods:

static double meanSquaredError(double result, double answer) {
return Math.pow(answer-result, 2);
}
static double error(double result, double answer) {
return answer - result;
}

// goal methods:

static double averageOf(ArrayList<Double> inputs) { // average
double sum = 0;
for (double i : inputs) {
sum += i;
}
return sum/inputs.size();
}
static double timesTwo(double x) {
return 2 * x;
}

double learningRate = 0.1;

Neuron[] output;
Neuron[] input;

double error;
double desiredValue;
double predictedValue;

Neuron[][] neurons;

public Network(Neuron[][] pneurons) {
neurons = pneurons;
}

public void step(int iterationcount) { // actual step function
for (int count = 0; count < iterationcount; ++count) {
// forward pass
for (int i = 1; i < neurons.length; ++i) {
for (int j = 0; j < neurons[i].length; ++j) { // sums
Neuron neuron = neurons[i][j];
double sum = 0;
for (int l : neuron.connections) {
Neuron connectedneuron = neurons[i-1][l];
sum += connectedneuron.value * connectedneuron.weight;
}
neuron.value = sigmoid(sum + neuron.bias);
}
}

input = neurons[0]; // input neurons
output = neurons[neurons.length-1]; // output neurons

desiredValue = input[0].value + input[1].value; // change these accordingly
predictedValue = output[0].value;

error = meanSquaredError(predictedValue, desiredValue);

// backwards pass
for (int i = neurons.length - 1; i > -1; --i) {
for (int j = neurons[i].length - 1; j > -1; --j) {
Neuron neuron = neurons[i][j];
for (int l : neuron.connections) {
Neuron connectedneuron = neurons[i-1][l];
double z = connectedneuron.weight * neuron.value; // + bias here
double cost = Math.pow(connectedneuron.value - desiredValue, 2);
double asdf = neuron.value * dsigmoid(z) * 2 * cost;
connectedneuron.weight -= asdf;
}
}
}

String placeholder = "input(s): ";
for (Neuron i : input) {
placeholder += i.value + ", ";
}
System.out.println(placeholder);

placeholder = "output(s): ";
for (Neuron i : output) {
placeholder += i.value + ", ";
}
System.out.println(placeholder);

System.out.println("correct answer: " + desiredValue);
System.out.println("error: " + error);
System.out.println(neurons[0][0].weight + ", " + neurons[0][1].weight);
System.out.println("------------------------------------------\n");

// resetting the values...
for (int i = 0; i < neurons.length; ++i) {
for (int j = 0; j < neurons[i].length; ++j) {
neurons[i][j].value = 0;
}
}

// we randomize (or give training data to) the inputs here.
for (int i = 0; i < neurons[0].length; ++i) {
neurons[0][i].value = Math.random(); // training data here
}
}
}
}

public class NeuralNetwork {
public static void main(String[] args) {
Network.Neuron input1 = new Network.Neuron(3, Math.random(), new int[] {}, 0, 0, 0);
Network.Neuron input2 = new Network.Neuron(2, Math.random(), new int[] {}, 0, 1, 0);

Network.Neuron neuron1 = new Network.Neuron(0, Math.random(), new int[] {0, 1}, 1, 0, 0);
Network.Neuron neuron2 = new Network.Neuron(0, Math.random(), new int[] {0, 1}, 1, 1, 0);

Network.Neuron neuron3 = new Network.Neuron(0, Math.random(), new int[] {0, 1}, 2, 0, 0);
Network.Neuron neuron4 = new Network.Neuron(0, Math.random(), new int[] {0, 1}, 2, 1, 0);

Network.Neuron output1 = new Network.Neuron(0, 0, new int[] {0, 1}, 3, 0, 0); // connections don't matter with output neurons.

Network neuralNetwork1 = new Network(new Network.Neuron[][] {
new Network.Neuron[] {input1, input2},
new Network.Neuron[] {neuron1, neuron2},
new Network.Neuron[] {neuron3, neuron4},
new Network.Neuron[] {output1}
});

neuralNetwork1.step(50);
}
}

He is talking about this

https://en.wikipedia.org/wiki/Backpropagation

I've written that kind of code before. It roughly looks right but the devil is in the details. My guess though there is a mistake in your formula, sometimes the symptom is that it doesn't learn at all but the most insidious thing is when it learns but learns slowly and not very well. I would also look at that learningRate. It's important that it not be too high or too low.