A review of Matrix Methods in Data Analysis
Thomas Dickson
3 minute read
Given the COVID outbreak and associated work from home period I thought now might be a good opportunity to relearn some of the mathematics which underpin statistical learning. I identified the MIT 18.065 course as having potential for a course which covers the fundamentals in detail. I found that the course does a good job of ensuring you learn about Neural Networks from the fundamentals up. The textbook has plenty of references and questions on the key topics, but loses its focus in deeper areas of probability and alternative statistical methods. This post records my experience of following the lecture series and completing the example problems over the previous thirty or so days.
I spent between $1.5$ to $2$ hours watching the lecture and solving questions every day. Early on in the course I found that the lectures were good but didn’t quite dumb it down enough for me - so I purchased the textbook Linear Algebra and Learning from Data. Each lecture has example problems which map to different sections of the textbook. Some of these questions differ between the course and the physical copy, but examine the same concepts that you expect. I was able to answer the questions with a bit of work reviewing the textbook and reading the notes, I found the internet to be very useful for the harder problems.
The course material is grouped into $4$ areas: 1. Linear algebra, 2. Probability, 3. Optimisation, and 4. Deep Learning. It is clear that Gil knows how to teach Linear Algebra. My first exposure to this chap came from watching a keynote at a Julia conference so it was a real pleasure to see his expository skill first hand. He confidently held my hand through the basics of linear algebra, numerical linear algebra (actually solving the real problems) and then into the beginnings of exploring the necessary knowledge of probability. We didn’t spend too long in this part of the map, and I thought that the textbook could have done a better job of including more questions on statistics. The lack of a large section on statistics (and example questions) is a minor point, because this course/textbook is about how to understand neural networks through the use of linear algebra.
I enjoyed the sections looking at Optimisation and Deep Learning (Neural Networks) as they broke down these areas into manageable chunks. I thought the problem sets did a good job of forcing me to review the previous section and understand the concepts. There were plenty of references (predominantly open access) and ideas for projects. The focus of the course is on producing a project investigating an aspect of the course.
If I were to remove the apple from my eye and describe an area of improvement it would be to include more coverage of other methods than neural networks. NN are an essential tool, but it seemed like the section on k-means was almost included as an afterthought. However, there are plenty of other excellent resources out there for other statistical methods (e.g. Elements of Statistical Learning) and I think the course would lose its focus if it spent any more time on this subject.
I would recommend this course to anyone with a spare few hours in their day. The textbook is generally clear, contains good problems and, importantly, plenty of references to begin exploring this area of applied mathematics. The video lectures are great for learning linear algebra, optimisation and neural networks.