Running Deep Neural Networks on Bitcoin
Implement a handwritten digit classification in the chain
This post was first published on Medium.
We have implemented a deep neural network for the classification of handwritten digits. The already trained model runs fully on the chain. It is trained offline using the MNIST dataset of handwritten digits. The model takes an image of 28×28 grayscale pixels and outputs a number from 0 to 9.
Introduction to deep neural networks
An artificial neural network is a construction inspired by biological neural networks. The network learns by being exposed to a large number of labeled examples of data. This process is also referred to as guided learning.
The network consists of several components: neurons/nodes, connections, biases and activation functions. These components are grouped together consecutively in layers. The first layer is called “input layer,” where data is sent into the network, and the last “output layer,” whereby the network returns output. A very simple neural network contains only these two layers. To improve performance, we may add one or more “hidden layers” between the two. Networks with hidden layers are called “deep neural networks” (DNN).
An illustration of a deep neural network
Each connection between neurons in the network is weighted with a specific value. Each neural also has a value called “bias” which is added to the sum of the inputs. Learning is the process of finding a set of these weights and biases such that the network will return a meaningful output given some input.
To get a good intuitive sense of how deep neural networks work under the hood, we recommend watching a short video series on the subject.
The network architecture
The DNN for MNIST handwritten digits consists of an input layer of 784 (28 x 28) nodes, a hidden layer of 64 nodes, and an output layer of 10 nodes (number of possible classes/digit). The layers are all interconnected, making the network contain a total of 501760 (784 * 64 * 10) connections.
Nodes in the hidden layer use the ReLU activation function. Argmax is applied to the output nodes to get the correct value, i.e. digit in the classification.
Train the model
The DNN is trained using Keras. With our outlined architecture of the network and using the RMSprop optimizer for training, the model is able to achieve 98% classification accuracy after 50 epochs.
Once the model is trained, the weights and biases need to be exported in a format that we can use in a sCrypt smart contract. For performance reasons, we hardcode these values bytesnot matrices.
Implementation
We have implemented the DNN above, similar to the single-layer neural network (aka, a perceptron) we have implemented previously. The entire code can be found on GitHub.
The function predict() takes in the initial values of the input layer. In our case, it’s the serialized values of a handwritten image. It returns an integer representing the classification result, i.e. the number in the image.
Because sCrypt does not natively support floating-point numbers, we use fixed-point representations by simply scaling values by 10⁸. For example, 0.86758491 becomes an integer 86758491. When we multiply two values, we rescale the result, i.e. divide it by 10⁸.
Potential use cases
DNNs like this can be used in many ways in a smart contract. For example, you can train a model to recognize whether an image contains a hot dog with a certain degree of accuracy. Users are encouraged to scrape the Internet for such images and automatically get paid in bitcoin micropayments for sending them. These images can be aggregated to train the model and improve accuracy.
Watch: sCrypt’s Xiaohui Liu Presentation at BSV Global Blockchain Convention, Smart Contracts and Computation on BSV
width=”562″ height=”315″ frameborder=”0″ allowfullscreen=”allowfullscreen”>
New to Bitcoin? Check out CoinGeeks Bitcoin for beginners section, the ultimate resource guide for learning more about Bitcoin – as originally envisioned by Satoshi Nakamoto – and blockchain.
