It's a major challenge to keep up with the continuous changes to the Keras/TensorFlow neural code library (and the PyTorch library too). I recently upgraded my Keras installation to version 2.6 and so I'm going through all my standard examples to bring them up to date with the inevitable changes.

I was using a new desktop machine and so I had to install TensorFlow 2.6 (which contains Keras 2.6). I ran into unexpected problems when the "wrapt" sub-library refused to build correctly (the installation process builds a .whl file for wrapt instead of using a pre-built .whl file). I found a hack online that suggested issuing the command SET WRAPT_INSTALL_EXTENSIONS=false, before the command pip install tensorflow, and that magically worked. Somehow.

One of the standard examples is the MNIST image dataset. There are 70,000 simple images (60,000 training images and 10,000 test images). Each image has 28x28 pixels and is a handwritten digit from '0' to '9'. Each pixel value is between 0 and 255.

I used the built-in MNIST dataset from Keras but I could have loaded the raw MNIST data using np.loadtxt() or a similar function.

I used the Model() approach by defining separate layers and then passing the first and last layer to the Model() constructor. An alternative design is to use the Sequential() approach. I have no strong preference between Model() and Sequential() -- it's just syntax.

To save time, I only used 2 training epochs with a batch size of 100. In a non-demo scenario I'd use more epochs, but then have to watch for over-fitting.

After the model was trained, I set up a fake 28x28 image with one vertical bar, one horizontal bar, and one diagonal bar. The trained model predicted the fake image is a '5' with pseudo-probability = 1.0000. Many machine learning systems aren't very good at distinguishing between authentic and fake items. This has generated interest in ML systems that output a prediction plus a confidence score of some type.

Good fun!


Left: A clever pseudo Chanel bag made from a paper grocery bag and a chain from a hardware store. Center: A whimsical pseudo Louis Vuitton bag, complete with misspelling. Right: A serious attempt at a Coach bag, but I don't think it will fool many people.

Code below.

  # mnist_tfk.py  # MNIST using CNN   # Keras 2.6.0 in TensorFlow 2.6.0 ("_tfk")  # Anaconda3-2020.02  Python 3.7.6  Windows 10    import os  os.environ['TF_CPP_MIN_LOG_LEVEL']='2'  # suppress warn    import numpy as np  import tensorflow as tf  from tensorflow import keras as K  import matplotlib.pyplot as plt    def main():    # 0. get started    print("\nBegin MNIST using Keras %s " % K.__version__)    np.random.seed(1)    tf.random.set_seed(1)      # 1. load data    print("\nLoading train and test data ")    (train_x, train_y), \    (test_x, test_y) = K.datasets.mnist.load_data()    train_x = train_x.reshape(60_000, 28, 28, 1)    test_x = test_x.reshape(10_000, 28, 28, 1)    train_x = train_x.astype(np.float32)    test_x = test_x.astype(np.float32)    train_x /= 255    test_x /= 255    train_y = K.utils.to_categorical(train_y, 10)    test_y = K.utils.to_categorical(test_y, 10)      # 2. define model    print("\nCreating network with two Convolution, \  two Dropout, two Dense layers ")    g_init = K.initializers.glorot_uniform(seed=1)    opt = K.optimizers.Adam(learning_rate=0.01)      x = K.layers.Input(shape=(28,28,1))    con1 = K.layers.Conv2D(filters=32,      kernel_size=(3,3), kernel_initializer=g_init,      activation='relu', padding='valid')(x)    con2 = K.layers.Conv2D(filters=64,      kernel_size=(3,3), kernel_initializer=g_init,      activation='relu', padding='valid')(con1)    mp1 = K.layers.MaxPooling2D(pool_size=(2,2))(con2)    do1 = K.layers.Dropout(0.25)(mp1)    z = K.layers.Flatten()(do1)    fc1 = K.layers.Dense(units=128,      kernel_initializer=g_init, activation='relu')(z)    do2 = K.layers.Dropout(0.5)(fc1)    fc2 = K.layers.Dense(units=10,      kernel_initializer=g_init, activation='softmax')(do2)      model = K.models.Model(x, fc2)      model.compile(loss='categorical_crossentropy',      optimizer=opt, metrics=['accuracy'])        # 3. train model    bat_size= 100    max_epochs = 2    print("\nStarting training with batch size = %d " % bat_size)    model.fit(train_x, train_y, batch_size=bat_size,      epochs=max_epochs, verbose=1)    print("Training finished ")      # 4. evaluate model    eval = model.evaluate(test_x, test_y, verbose=0)    loss = eval[0]    acc = eval[1] * 100    print("\nTest data: loss = %0.4f \   accuracy = %0.2f%%" % (loss, acc))      # 5. save model    print("\nSaving MNIST model to disk ")    # mp = ".\\Models\\mnist_model.h5"    # model.save(mp)      # 6. use model    print("\nMaking prediction for fake image: ")    # np.set_printoptions(precision=4, suppress=True)    np.set_printoptions(formatter={'float': '{: 0.4f}'.format})      x = np.zeros(shape=(28,28), dtype=np.float32)    for row in range(5,23):      x[row][9] = 180  # vertical line    for rc in range(9,19):      x[rc][rc] = 250  # diagonal    for col in range(5,15):        x[14][col] = 200  # horizontal      plt.imshow(x, cmap=plt.get_cmap('gray_r'))    plt.show()      x = x.reshape(1, 28, 28, 1)    pred_probs = model.predict(x)    print("\nPrediction probabilities: ")    print(pred_probs)        pred_digit = np.argmax(pred_probs)    print("\nPredicted digit: ")    print(pred_digit)      print("\nEnd MNIST demo ")    if __name__ == "__main__":    main()