Batch Normalization


BatchNorm addresses the internal covariate shift problems by normalizing layer inputs, which makes using large learning rate to accelerate network training feasible.

Comments

Post a Comment

Popular posts from this blog

Python Notes

zip: returns a list of tuples, where the i-th tuple contains the i-th element from each of the argument sequences or iterables. a = [1, 2, 3] b = ['a', 'b', 'c'] for i, j in zip(a, b):      print(i,j) =>1 a =>2 b =>3 c Python __repr__() function returns the object representation. It could be any valid python expression such as tuple, dictionary, string etc.  id(object): return the "identity" of an object. This is an integer which is guaranteed to be unique and constant for this object during its lifetime. Two objects with non-overlapping lifetimes may have the same id() value. hasattr() The hasattr() method returns true if an object has the given named attribute and false if it does not. hasattr(object, name) PIL. Image.resize(size, resample = 0) returns a resized copy of this image Parameters:  -size: the requested size in pixels, as a 2-tuple -resample: an optional resampling filter. This can be one of PIL.I...
Read more

Clustering vs Dimensionality Reduction

Clustering and Dimensionality Reduction are applied to the problem of unsupervised learning. Clustering identifies unknown structure in the data;  Dimensionality Reduction uses s tructural characteristics to simplify data. However, there is a problem called Curse of Dimensionality . In practice, too many features leads to worse performance . Therefore, we need Dimensionality Reduction . It's possible to represent data with fewer dimensions, which requires us to discover intrinsic dimensionality of data. One way to do dimensionality reduction is to perform lower dimensional projections. In the way, we could transform dataset to have less features, in the new feature space, some original features are combined via linear or nonlinear functions . PCA: Principal Component Analysis Find sequence of linear combinations of the features that have maximal variance and are uncorrelated PCA:1st PC 1st PC of X is unit vector that maximizes the s...
Read more

Java Learning

Java 注解： Java 语言中的类、方法、变量、参数和包都可以被标注。 Java标注可以通过反射获取标注内容。 内置的注解： Java定义了一套注解，共有7个，3个在java.lang中，剩下的4个在java.lang.annotation中。 @override--检查该方法是否是重写方法。如果发现其父类，或者引用的接口并没有该方法时，会报编译警告。 @Deprecated--标记过时方法。如果使用该方法，会报编译警告。 @SupperessWarning--指示编译器去忽略注解中声明的警告。 @Retention--标识这个注解应该怎么保存，是只在代码中，还是编入class文件中，或者是在运行时间可以通过反射访问。 @Documented--标记这些注解是否包含在用户文档中。 @Target--标记这个注解应该是哪种Java成员。 @Inherited--标记这个注解是继承于哪个注解类。
Read more