Week In Review
2024 is just around the corner, and we've been pushing through the finish line! Although it's only been 5 weeks, it's seriously refreshing to look back and realize how far we've come.
TLDR:
- Wrapped up machine learning specialization and performed spaced review.
- Finished courses 1 - 3 of Andrew Ng's deep learning specialization
- Finished the first 5 lectures of fast.ai
- Read around a 100 pages of the descriptive statistics book and dropped it after finding it too basic. Better focus on probability
- Got to around 90% of the data analysis book and found a good stopping point to move on.
- Practiced EDA and classification with spotify dataset
This week has been full-force on grokking neural networks, by following both courses. Having reached a checkpoint in the DL specialization, it's time for me to iterate heavily on kaggle projects, as well as implementing a neural network myself.
Unfortunately, I did somewhat lose my pace with STAT110, being much more pre-occupied with the ML side of things. Definitely do plan on getting back on it now, as things have settled a bit.
Progress Timeline
December 24, 2023
- Read through chapter 7 of the data analysis. textbook. I'm going to seriously need to experiment with pandas myself on kaggle datasets after it for this stuff to stick well.
- Finished course 3 of the Andrew Ng ML Specialization, marking the end of it. Sweet checkpoint! Iteration plan is to review all notes, labs, and practice throughout next week.
- Watched lecture 19 of STAT 110 on joint distributions.
- Read through 36 pages of "Practical Statistics for Data Scientists"
- Published weekly clov log entry.
Looking to end off 2023 with a BANG - Deep Learning, here I come!
December 25, 2023
- Read through chapter 8 of the data analysis textbook, mainly about joining and concatenation of data frames.
- Watched lecture 20 on the multinomial distribution.
- Read from pages 36 - 77 of the practical statistics book. Just brushing up on confidence intervals, bootstrap, and most importantly sample distributions.
- Reviewed jupyter notebooks for all of course 1, and the first 3 weeks of course 2. Planning on finishing up my full review tommorow! I'm a bit unsatisfied with the quality of the exercises, so I'm definitely looking forward to the kaggle datasets.
- I would love to unpack the adaptive moment optimizer a bit more, but I realized with now is not the best time.
- Thought about some exciting projects to do soon:
- Collaborative and content filtering on the douban books dataset.
- Neural network-based log inspection of distributed systems, especially for consensus, to detect anomalies (definitely a bit advanced, but good to note down).
December 26, 2023
- Watched Lecture 21 of STAT 110, continuing the pace.
- Read through another 35 pages of the statistics book discussing experiments and p-values.
- After covering most of the important material for the stats and data analysis books, I decided to stop continuing to read them, as I felt going any further wouldn't be optimal use of time.
- Went through all of the course 2 and 3 assignments and labs, up until the reinforcement learning one.
- Finally began a mini ML project! Downloaded a spotify dataset and used PCA and XGBoost to try to predict genres given audio features. Model accuracy was only a mere 35% 😅
December 27, 2023
- Learned the basics of the Naive Bayes classifier and SVMs. The "kernel trick" went over my head so I'm definitely going to have to revisit these topics again later.
- Started Andrew Ng's Deep Learning specialization, and completed weeks 1 and 2 of the first course. Most of this is already just review from the ML specialization. Looking forward to the more juicy bits of CNNs and RNNs!
- It was really interesting to see that the logistic regression cost function was really just MLE from statistics.
- Started the fast.ai practical deep learning course and finished the 1st of the 9 lectures. While it doesn't highlight much of the theory or math behind neural networks, the fastai libraries and pytorch are extremely practical tools to be equipped with for real world projects. Therefore, I'm going through both courses to enjoy a bit of both worlds.
December 28, 2023
- Watched lecture 2 of fast.ai and learned about huggingface deployments. Seems like a fun spot to explore around.
- Completed weeks 3 and 4 of course 1 of the deep learning specialization, and almost finishing up course 3 about ML development best practices. Planning on finishing up course 3 and coming back to course 2 tommorow.
- Traced out vectorized forward and back propagation and finally got an intuitive grasp of the matrix dimensions when trained on the entire dataset at once.
December 29, 2023
- Watched lectures 3 and 4 of fast.ai and finetuned an NLP model using huggingface transformers. It was refreshing to finally get a taste of NLP, as it's one of the topics I'm looking forward to the most!
- Finished up course 3 and watched lectures for week 1 of course 2.
- Found some great additional resources from Andrej Karpathy on LLMs. Plan on adding to roadmap soon.
December 30, 2023
- Although today was filled up with errands, I still finished the remaining parts of course 2 of the deep learning specialization. I consider this as a checkpoint, having finished courses 1-3 covering the fundamentals of neural networks. I'd like to spend a few days really iterating on this knowledge thoroughly first, before moving on to CNNs and RNNs.
- Reviewed ML Specialization topics like PCA, Anomaly, K-Means and more in preparation for tommorow's review session.
- Prepared an agenda for tommorow's CLOV session. Looking forward to discussing timelines, reviewing classical ML, and finding some projects!
Goals for next week
As the new year unfolds, we're excited to kick it off by diving deeper into this rabbit hole! Here are a few concrete goals I set for next week:
- Complete 1 fast.ai lecture per day, therefore finishing part 1 and reaching lecture 12.
- Go through all of the lecture notes and labs for courses 1 - 3 to iterate.
- Implement the MNIST classification myself using PyTorch
- Create a neural network from scratch supporting most of the following features:
- Split train test
- Feed-forward architecture with dense layers
- Linear, relu, leaky relu, tanh, sigmoid, softmax
- Gaussian normalization
- L1, L2 norm
- Mini-batch gradient
- Dropout
- Gradient checking
- Moment, rmsprop, adam
- Batch normalization
- Perf analysis with cost function plot, F1 score, MSE