1 hour to overview all slides
Just get rough cards going
1.1.DL.IntroductionToCourseE4040.20210910.pdf
Overview
1.2.DL.IntroductionToDeepLearning.20210120.pdf
Ann can represent any function
Universal approx theorm
Deep learning history
Representation learning etc
1.3.DL.IntroductionToDLcomputingResources.20210113.pdf
Tools:
Python / tensorflow etc
2.1.DL.MachineLearningBasics.20210917.pdf
ML Key Concepts
Learning algorithm
Cost function
Model
Dataset
Optimization Algorithm
Fitting the training data
Finding patterns that generalize to new data
Hyperparameter tuning. Ex learning rate
Supervised vs Unsupervised learning
Linear Regression
Overview Regression / Classification
ML Tasks
Transcription ( Speach audio to words)
Translation (language to language)
Structured Output (iimage segmentation / image captioning)
Anomaly Detection (fraud detection)
Synthesis and sampling (create artistic images in a style)
Performance Measure
Accuracy
Error Rate
Experience
Unsupervised or supervised
Supervised has a label and target variable
Central challenge in ML is generalization
Train / test / validation set
Dividing into batches
Cross validation
Regularization
Mean squared error
Point Estimator
Bias
Bias Variance Tradeoff
Maximum likelihood estimate MLE Maxiumum A-Posteriori Estimation MAP
both Bayesian Estimators
MAP assumes a prior
Prior reduces uncertainty
2.2.DL.MachineLearning.Algorithms.20210917.pdf
(Unsupervised) Data Representation
Try and find a low-dimensional representation
Sparse in th efeature space
PCA
Representation learning
Dimensionality reduction
t_SNE
Create appealing 2d maps from high dimensional data
Curse of dimensionality
Often many more features than training numbers
In high dimensional space everything is empty
Logistic Regression
Maximum Margin Classifiers Support Vector Machine (SVM)
Softmax Multiclass Classification
3.1.DL.DeepFeedforwardNetworks.20210126.pdf
NN History
Perception 1980s
Feed Forward Network
Able to approx higher order function by transforming the input data such that a linear decision boundary can be found
3.2.DL.BackPropagation.20210203.pdf
Think I mostly know it
4.1.DL.Optimization.20200210.pdf
I’ve looked at it
5.1.DL.CNN.20200326.pdf
Make sure I can write the 2d form 5.2.DL.convnets.applications.NVIDIA.V2.169.pdf
5.3.DL.CNN.Architectures.20211104.pdf
Backprop for conv layer
5.3.DL.CNN.Examples.20201124.pdf 6.1.DL.Regularization.20211022.pdf 7.1.DL.PracticalMethodology.20210118b.pdf 8.1.DL.RNN.20201105.pdf 8.1.DL.RNN.20211105.pdf 8.2.DL.RNN.applications.20201111.pdf 12.1.DL.Applications.20211112.pdf