During my university days, I was specialized in artificial intelligence but at the point of time, AI is still very green. As such, I dropped my interests and move toward other bread & butter.

Moving forward to 2017, AI seem to be trending and so perhaps its time to revisit my interests.

So what is neural network?

The reason why the hidden layer is required is to smooth the function from a step-function to a sigmoid function.

Questions

1. Assuming its a convex gradient, what is the error reduction formulae (gradient descent optimization)?
2. How to optimize for non-convex gradient?