Welcome to the beginning of my machine learning journey! This article marks the first post of my progress log, and I plan on continually releasing new updates every Sunday. For more context about myself and this website, check out the about page.
Before even starting my journey, I grew strongly aware about the fundamental importance of math within machine learning.
I knew before even touching a single line of code, I needed to have a firm grasp of linear algebra, calculus, and statistics. I was extremely apprehensive of this idea at first, as a freshman with only some background in single-variable calculus. Combined with my eagerness to get on with the flashy topics like NLP, I felt conflicted on how I should proceed.
That is, until I realized something quite obvious. If I don't know the math, why don't I just learn it? Afterall, I wouldn't want to treat AI as a blackbox of random python library calls, would I?
And so I made it my mission to achieve this within 2 months. I was ready to put in every ounce of the effort required. It was the first time I would get to set aside coding and truly appreciate the beauty of mathematics, while learning from world-class professors.
Once again, welcome to CLOV.
Weeks in Review
The past three weeks were particularly intense, given that I started during finals week. But I enjoyed every minute of it. I've never been this interested in math before, and learning it with a big picture in mind has been much more enjoyable than having grades and exam scores as a goal!
The majority of the past three weeks involved completing the MIT OCW Linear Algebra course by Professor Strang. All the way from Ax = b to eigenvalues and SVD, Strang decomposed every topic into an extremely beginner-friendly approach. His lectures were a joy to watch, pausing to answer his rhetorical questions that had me actively engaged in the material.
And can we take a moment to acknowledge that beautiful ending with how he pulled together the four fundamental subspaces, orthogonality, and eigenvectors simultaneously with SVD? I was in pure awe.
Recently this week, I completed the Khan Academy multi-variable curriculum, covering the basics of gradient descent and other crucial optimization topics. I found multi-variable calculus mostly to be an intuitive expansion of my prior knowledge. Loved seeing overlaps with Linear Algebra with concepts like quadratic approximation (local linearity) and Jacobian determinants.
Additionally, I completed 9 lectures of Harvard's Probability course and worked through 3 of the problemsets. This definitely had me switching into a different mode of thinking than what I was used to. Blitzstein is a fantastic professor. He often delves back into history and explains how to solve famous problems that people once used to consult Newton for. Probability has been extremely eye-opening so far and it's only uphill from here.
Progress Timeline
The Beginning
We started tracking our progress about a week later into our journey, so here is a quick rundown of my progress leading up to December 6th:
- Familarized myself with a general map of data science topics and its distinctions (e.g. AI vs ML vs DL)
- Created a data science roadmap with extensive research and resource compilation.
- Watched 3b1b linear algebra playlist to get a brief conceptual and geometric overview.
- Completed all of Unit 1 Linear Algebra, and watched the lectures for Unit 2.
- Subscribed to and began reading several artificial intelligence newsletters to keep myself updated on latest trends.
December 06, 2023
- Read through all lecture summaries for Unit 2 and took notes.
- I still feel rather uneasy about working with differential equations, mainly because I lack the background.
- Hoping to revisit difficult topics later on for iteration.
December 07, 2023
- Finished the problem sets for the first three sub-units of Unit 2.
- Found interest in neural networks from scratch from the SentDex playlist (and book). Definitely a future project I'm looking forward to.
December 08, 2023
- Completed the rest of the problem sets for Unit 2.
- Ideated on future project ideas: potentially some sort of analysis on chess.com dataset for insightful predictions.
December 09, 2023
- Watched lectures 1-4 of Unit 3.
- Watched a brief overview of calculus 3.
- Understood the relationship of descriptive statistics vs probability and reordered curriculum accordingly.
December 10, 2023
- Finished the final 4 lectures of the entire course and practiced Unit 2 with exam review and exam during the meetup.
- Just setup the blog and got it hosted. Looking forward to starting calculus and probability soon this week.
- Planning on spending a few more days practicing all of Unit 1-3.
December 11, 2023
- Read halfway through the matrix calculus guide.
- Watched lecture 1 of stat 110 and already eager to learn more.
- Starting to take more diligent and organized notes.
December 12, 2023
- Finished reading the rest of matrix calculus. Some of the concepts (e.g. total differentiation) feel a little hazy to me, so a lot more iteration will be required.
- Made some changes to the curriculum introducing a subset of the KA multi-var calc to be better equipped.
- Watched lectures 2 and 3 of STAT 110 and took notes. Having a small background in combinatorics (from discrete) is really helping, and it's awesome to see how well the inclusion/exclusion principle scales.
Overall, I still plan on putting more of an emphasis on Linear Algebra this week by iterating on Unit 3 and doing an overall course-wide review. Glad to have all of winter break to myself to spend grinding everything out. This will be a fruitful journey.
December 13, 2023
- Watched ALL of the required content on Khan Academy Multi-Variable Calculus today and understood it thoroughly. I'm genuinely proud of the progress I've made today. But we have a long way to go! And I'm all the more excited. Of course, I plan on reading all the article recaps and doing many practice problems this week.
- Besides multi, I've reviewed 6/8 lectures for Linear Algebra and I plan on finishing up the remaining Unit 3 review tomorrow. I'm starting to see the shore now 🫡.
December 14, 2023
- Iterated on all calculus content and did practice problems to absorb content.
- Read through the remaining 2 lecture summaries and did all 8 practice problem sets in one shot.
Can't wait to continue with probability and statistics! Made some minor improvements to the website as well.
December 15, 2023
- Completed strategic problems and homework #1 for STAT 110. Watched 3 lectures of STAT 110. Amazed by how calculating probability can tell us absolutely mind-boggling insights about seemingly intuitive yet misunderstood problems (e.g. coin tosses, birthday problem, monty hall).
- It's been a little challenging to wrap my head around it at first, but doing my best to diligently work through problems and reasoning about theorems.
December 16, 2023
- Watched three lectures of Stat 110 learning about different probability distributions and expected values. I can already start to see it's immediate value within data science and it's only fueling me to learn more!
- We had a CLOV session today in which we went over exams for linear algebra review. Tomorrow is gonna be a lot more practice with probability.
Changelog
- We found that it's significantly better to dedicate our weekly meetups for content review, rather than working through the exams together. These problems take significant mental effort and are better suited for individual study. Conceptual reviews are more helpful, as an application of the Feynman Method.
- Moving Matrix Calculus guide into Stage 3 to serve as more of a refresher.
Goals for Next Week
- Work through STAT 110 up until the midterm (Lecture 15).
- Start exploring data science tools with Python, and learn how to apply the mathematical knowledge we learned into code (looking at you numpy!).