Batch Normalization


BatchNorm addresses the internal covariate shift problems by normalizing layer inputs, which makes using large learning rate to accelerate network training feasible.

Comments

Post a Comment

Popular posts from this blog

Image Similarity using Deep Ranking

Image
Image similarity: the measure of how similar two images are. In other words, it quantifies the degree of similarity between intensity patterns in two images. Triplet: a triplet contains a query image, a positive image and a negative image. How to measure the similarity of two images? L1-norm: Manhattan distance L2-norm: Euclidean distance Loss function: The whole deep ranking architecture can be thought of as a function that would map the image to a point in the Euclidean space. The goal is to learn an embedding function that assigns smaller distance to more similar images. D(f(pi), f(pi+)) < D(f(pi), f(pi-)) <pi, pi+, pi-> such that r(pi, pi+) > r(pi,pi-) f is the embedding function that would map the image to a vector. pi is the query image, pi+ is the positive image, pi- is the negative image and r is the similarity distance between two images. The hinge loss for the triplet is defined as: l(pi, pi+, pi-) = max{0, g+D(f(pi), f(pi+)) - D(f(pi),f(pi-...
Read more

Daily-coding-problem #101

# This problem was asked by Alibaba. # Given an even number (greater than 2), return two prime numbers whose sum will be equal to the given number. def is_prime(n, primes):     for p in primes:         if n == p:             return True         if not n%p == 0:             return False     return True def get_primes(num):     limit = num//2     primes = list()     candidates = list()     for i in range(2, limit+1):         if is_prime(i, primes):             primes.append(i)             candidates.append((i, num-i))     new_candidates = list()     for p1, p2 in candidates[::-1]:         if is_prime(p2, primes):             primes.append(p2)     ...
Read more