python - Initialization of weights

 The main difference between Gaussian variable (numpy.random.randn()) and uniform random variable is the distribution of the generated random numbers:

• numpy.random.rand() produces numbers in a uniform distribution.
• and numpy.random.randn() produces numbers in a normal distribution.

When used for weight initialization, randn() helps most the weights to Avoid being close to the extremes, allocating most of them in the center of the range.

An intuitive way to see it is, for example, if you take the sigmoid() activation function.

You’ll remember that the slope near 0 or near 1 is extremely small, so the weights near those extremes will converge much more slowly to the solution, and having most of them near the center will speed the convergence.

python code to plot cost

import matplotlib.pyplot as plt

%matplotlib inline 

def plot_costs(costs, learning_rate=0.0075):

    plt.plot(np.squeeze(costs))

    plt.ylabel('cost')

    plt.xlabel('iterations (per hundreds)')

    plt.title("Learning rate =" + str(learning_rate))

    plt.show()

#Assuming "costs" is a list of costs obtained during training iterations per hundred

#calling the method with some learning rate

plot_costs(costs, learning_rate)

output:

Generator

Using generator one can create a custom iterator method.

Code to handle file not found

import sys

try:

    with open('journal.txt', 'a') as f:

        f.write('input')

except IOError as exc:    # Python 2. For Python 3 use OSError

    tb = sys.exc_info()[-1]

    lineno = tb.tb_lineno

    filename = tb.tb_frame.f_code.co_filename

    print('{} at {} line {}.'.format(exc.strerror, filename, lineno))

    sys.exit(exc.errno)