Mrthfx

String reversal in Python

Do I really need to have a scientific explanation for my premise?

Algorithm to convert a fixed-length string to the smallest possible collision-free representation?

Placing subfig vertically

Good for you! in Russian

How much stiffer are 23c tires over 28c?

Make a transparent 448*448 image

Do items de-spawn in Diablo?

Offered promotion but I'm leaving. Should I tell?

Aliens englobed the Solar System: will we notice?

Built-In Shelves/Bookcases - IKEA vs Built

Could a cubesat be propelled to the moon?

What are the best books to study Neural Networks from a purely mathematical perspective?

Good allowance savings plan?

Should I take out a loan for a friend to invest on my behalf?

Accountant/ lawyer will not return my call

What Happens when Passenger Refuses to Fly Boeing 737 Max?

Unreachable code, but reachable with exception

MTG: Can I kill an opponent in response to lethal activated abilities, and not take the damage?

Replacing Windows 7 security updates with anti-virus?

How much attack damage does the AC boost from a shield prevent on average?

Is it true that real estate prices mainly go up?

"One can do his homework in the library"

How do I locate a classical quotation?

What are the best books to study Neural Networks from a purely mathematical perspective?

Modern Mathematical Theory for Neural Networks, Cellular Automata, NeuroscienceTwo-layer Perceptron for XORdelay time from mathematical perspectiveReference Request: Neural Networks as Mathematical ModelsSources to learn and understand advanced probability in ML modelsHow can logicians benefit from development of artificial neural networks?Deriving diagonal approximation of Hessian in a neural networkNeural Networks and the Chain RuleWhat are some current best practices for function approximation using neural networks?What does oplus symbol ⊕ do for 2 images in Convolution Neural Networks

5

3

$begingroup$

I am looking for a book that goes through the mathematical aspects of neural networks, from simple forward passage of multilayer perceptron in matrix form or differentiation of activation functions, to back propagation in CNN or RNN (to mention some of the topics).

Do you know any book that goes in depth into this theory? I've had a look at a couple (such as Pattern Recognition and Machine Learning by Bishop) but still have not found a rigorous one (with exercises would be a plus). Do you have any suggestions?

matrices book-recommendation mathematical-modeling neural-networks

share|cite|improve this question

asked 2 hours ago

EliEli

341

New contributor

Eli is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

add a comment | 

5

3

$begingroup$

I am looking for a book that goes through the mathematical aspects of neural networks, from simple forward passage of multilayer perceptron in matrix form or differentiation of activation functions, to back propagation in CNN or RNN (to mention some of the topics).

Do you know any book that goes in depth into this theory? I've had a look at a couple (such as Pattern Recognition and Machine Learning by Bishop) but still have not found a rigorous one (with exercises would be a plus). Do you have any suggestions?

matrices book-recommendation mathematical-modeling neural-networks

share|cite|improve this question

asked 2 hours ago

EliEli

341

New contributor

Eli is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

add a comment | 

$begingroup$

I am looking for a book that goes through the mathematical aspects of neural networks, from simple forward passage of multilayer perceptron in matrix form or differentiation of activation functions, to back propagation in CNN or RNN (to mention some of the topics).

Do you know any book that goes in depth into this theory? I've had a look at a couple (such as Pattern Recognition and Machine Learning by Bishop) but still have not found a rigorous one (with exercises would be a plus). Do you have any suggestions?

matrices book-recommendation mathematical-modeling neural-networks

share|cite|improve this question

asked 2 hours ago

EliEli

341

New contributor

Eli is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

share|cite|improve this question

asked 2 hours ago

EliEli

341

New contributor

Eli is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

share|cite|improve this question

asked 2 hours ago

EliEli

341

New contributor

Eli is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

asked 2 hours ago

EliEli

341

New contributor

Eli is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

asked 2 hours ago

EliEli

341

2 Answers
2

active

oldest

votes

3

$begingroup$

For MLPs, there is a rigorous derivation in the optimization textbook by Edwin Chong and Zak. Although it is notation heavy as all things related to neural networks must be.

This book is for some reason freely available online. See page 219 of https://eng.uok.ac.ir/mfathi/Courses/Advanced%20Eng%20Math/An%20Introduction%20to%20Optimization-%20E.%20Chong,%20S.%20Zak.pdf

I think there is essentially no good mathematical textbook on convolutional neural networks or RNN in existence. People essentially just base their intuition off of MLPs. But it is not hard to create a mathematically rigorous derivation of forward and backward propagation of CNN or RNN.

share|cite|improve this answer

edited 1 hour ago

answered 1 hour ago

Shamisen ExpertShamisen Expert

2,81821945

$endgroup$

add a comment | 

2

$begingroup$

I'd recommend Deep Learning by Goodfellow, Bengio and Courville. I don't know if I'd call it "purely mathematical", but it covers a good amount of math background in the first few chapters. No exercises, though.

share|cite|improve this answer

answered 2 hours ago

Jair TaylorJair Taylor

9,05432144

$endgroup$

• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  Thank you - I've actually had a look at that one too, but while it is good in introducing the main mathematical tools needed for NN, I found it a bit lacking when it came to properly develop the model mathematically.
  $endgroup$
  – Eli
  2 hours ago
  

  
  
  
  

add a comment | 

Your Answer

StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "69"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});

Eli is a new contributor. Be nice, and check out our Code of Conduct.

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3146016%2fwhat-are-the-best-books-to-study-neural-networks-from-a-purely-mathematical-pers%23new-answer', 'question_page');
}
);

Post as a guest

Name

Email

Required, but never shown

3

$begingroup$

For MLPs, there is a rigorous derivation in the optimization textbook by Edwin Chong and Zak. Although it is notation heavy as all things related to neural networks must be.

This book is for some reason freely available online. See page 219 of https://eng.uok.ac.ir/mfathi/Courses/Advanced%20Eng%20Math/An%20Introduction%20to%20Optimization-%20E.%20Chong,%20S.%20Zak.pdf

I think there is essentially no good mathematical textbook on convolutional neural networks or RNN in existence. People essentially just base their intuition off of MLPs. But it is not hard to create a mathematically rigorous derivation of forward and backward propagation of CNN or RNN.

share|cite|improve this answer

edited 1 hour ago

answered 1 hour ago

Shamisen ExpertShamisen Expert

2,81821945

$endgroup$

add a comment | 

3

$begingroup$

For MLPs, there is a rigorous derivation in the optimization textbook by Edwin Chong and Zak. Although it is notation heavy as all things related to neural networks must be.

This book is for some reason freely available online. See page 219 of https://eng.uok.ac.ir/mfathi/Courses/Advanced%20Eng%20Math/An%20Introduction%20to%20Optimization-%20E.%20Chong,%20S.%20Zak.pdf

I think there is essentially no good mathematical textbook on convolutional neural networks or RNN in existence. People essentially just base their intuition off of MLPs. But it is not hard to create a mathematically rigorous derivation of forward and backward propagation of CNN or RNN.

share|cite|improve this answer

edited 1 hour ago

answered 1 hour ago

Shamisen ExpertShamisen Expert

2,81821945

$endgroup$

add a comment | 

$begingroup$

For MLPs, there is a rigorous derivation in the optimization textbook by Edwin Chong and Zak. Although it is notation heavy as all things related to neural networks must be.

This book is for some reason freely available online. See page 219 of https://eng.uok.ac.ir/mfathi/Courses/Advanced%20Eng%20Math/An%20Introduction%20to%20Optimization-%20E.%20Chong,%20S.%20Zak.pdf

I think there is essentially no good mathematical textbook on convolutional neural networks or RNN in existence. People essentially just base their intuition off of MLPs. But it is not hard to create a mathematically rigorous derivation of forward and backward propagation of CNN or RNN.

share|cite|improve this answer

edited 1 hour ago

answered 1 hour ago

Shamisen ExpertShamisen Expert

2,81821945

$endgroup$

share|cite|improve this answer

edited 1 hour ago

answered 1 hour ago

Shamisen ExpertShamisen Expert

2,81821945

answered 1 hour ago

Shamisen ExpertShamisen Expert

2,81821945

answered 1 hour ago

Shamisen ExpertShamisen Expert

2,81821945

2

$begingroup$

I'd recommend Deep Learning by Goodfellow, Bengio and Courville. I don't know if I'd call it "purely mathematical", but it covers a good amount of math background in the first few chapters. No exercises, though.

share|cite|improve this answer

answered 2 hours ago

Jair TaylorJair Taylor

9,05432144

$endgroup$

• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  Thank you - I've actually had a look at that one too, but while it is good in introducing the main mathematical tools needed for NN, I found it a bit lacking when it came to properly develop the model mathematically.
  $endgroup$
  – Eli
  2 hours ago
  

  
  
  
  

add a comment | 

2

$begingroup$

I'd recommend Deep Learning by Goodfellow, Bengio and Courville. I don't know if I'd call it "purely mathematical", but it covers a good amount of math background in the first few chapters. No exercises, though.

share|cite|improve this answer

answered 2 hours ago

Jair TaylorJair Taylor

9,05432144

$endgroup$

• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  Thank you - I've actually had a look at that one too, but while it is good in introducing the main mathematical tools needed for NN, I found it a bit lacking when it came to properly develop the model mathematically.
  $endgroup$
  – Eli
  2 hours ago
  

  
  
  
  

add a comment | 

$begingroup$

I'd recommend Deep Learning by Goodfellow, Bengio and Courville. I don't know if I'd call it "purely mathematical", but it covers a good amount of math background in the first few chapters. No exercises, though.

share|cite|improve this answer

answered 2 hours ago

Jair TaylorJair Taylor

9,05432144

$endgroup$

share|cite|improve this answer

answered 2 hours ago

Jair TaylorJair Taylor

9,05432144

answered 2 hours ago

Jair TaylorJair Taylor

9,05432144

answered 2 hours ago

Jair TaylorJair Taylor

9,05432144