白一帆的博客 | Ethan Blog

Structuring Machine Learning Projects 第二周笔记

ML Strategy(2)

ML Strategy(2) Error analysis Incorrectly labeled DL算法对随机错误有很强的鲁棒性，所以一些随机误差不会影响训练的结果而系统性的错误对模型影响较大 Build your first system quickly then iterate Set up dev/test set and metric Build init...

Posted by baiyf on November 21, 2017

Structuring Machine Learning Projects 第一周笔记

ML Strategy(1)

ML Strategy(1) Orthogonalization 正交化是指让多个超参数的调整互相不影响 ML的基本要求：模型在训练集上拟合情况良好（bigger network）模型在验证集上拟合情况良好(Regularzation,bigger training set) 模型在测试集上拟合情况良好(bigger dev set) 模型在真实环境下表现良好(...

Posted by baiyf on November 20, 2017

Hyperparameter tuning, Regularization and Optimization 第三周笔记

Hyperparameter tuning&Batch Normalization&Programming Frameworks

Hyperparameter tuning&Batch Normalization&Programming Frameworks Hyperparameters 在深度神经网络中有很多的超参数,例如$\alpha,\beta,\beta_1,\beta_2,\epsilon,layers\_l,hidden\_units,learning\_rate\_decay$等...

Posted by baiyf on November 19, 2017

Hyperparameter tuning, Regularization and Optimization 第二周笔记

Optimization algorithms

Optimization algorithms Mini-batch 把整个训练集batch分为多个子训练集mini-batch,mini-batch训练速度远高于batch a epoch 代表遍历了一遍整个训练集 if mini-batch size = m : batch gradient descent $(X^{\{1\}},Y^{\{1\}}) = (X,Y)$ #训...

Posted by baiyf on November 18, 2017

Hyperparameter tuning, Regularization and Optimization 第一周笔记

Practical aspects of Deep Learning

Setting up your Machine Learning Application Bias/Variance trade-off Bias对应训练集误差,Variance对应验证集误差,举例如下 high variance high bias high bias&high variance ...

Posted by baiyf on November 17, 2017

Neural Networks and Deep Learning 第四周笔记

Deep Neural Networks

Deep Neural Network from logistic regression to neural network 深度神经网络的参数维度 \[W^{[l]},dW^{[l]} : (n^{[l]} , n^{[l-1]})\] \[b^{[l]},db^{[l]}: (n^{[l]} , 1)\] \[Z^{[l]} ,dZ^{[l]},A^{[l]},dA^{[l...

Posted by baiyf on November 16, 2017

Neural Networks and Deep Learning 第三周笔记

Shallow Neural Networks

Shallow Neural Network 前向传播 \[Z^{[1]} = w^{[1]}X + b^{[1]}\] \[A^{[1]} = g^{[1]}(Z^{[1]})\] \[Z^{[2]} = W^{[2]}A^{[1]} + b^{[2]}\] \[A^{[2]} = g^{[2]}(Z^{[2]})\] 反向传播 \[dA^{[2]} = -\frac{Y}{...

Posted by baiyf on November 16, 2017

Neural Networks and Deep Learning 第二周笔记

Neural Network Basics

Logistic Regression as a Neural Network sigmoid 在logistic regression后加一个非线性函数,例如sigmoid()可以使输出满足概率特性 \[0 \leq P_i \leq 1\] \[\sum{P_i} = 1\] 损失Loss函数$L$ 损失函数常用预测值和标签的交叉熵定义: \[L(\hat{y},y) = -...

Posted by baiyf on November 14, 2017

Numpy基础

Numpy foundation for deep learning

Numpy 是Python中进行科学计算的库函数,可以处理高性能的多维矩阵运算. Numpy 1.数组 import numpy as np a = np.array([1,2,3]) #定义一维数组 b = np.array([[1,2,3],[4,5,6]]) #定义二维数组 print(a.shape) ...

Posted by baiyf on November 10, 2017

Ethan Blog