I'm just going to show a code snippet the shows the main forward step of a single layer Long Short Term Memory (LSTM) recurrent neural network (RNN). I'm not going to go through how it all works, as there are a ton of great resources online for RNNs. Check out this, by Andrej Karpathy, this tutorial on implementing RNNs in NumPy and Theano, or this fantastic explanation of LSTMs. Note that I've gotten this to work with the MNIST dataset, resulting in some crazy low error rates. If you're interested in seeing that code, let me know!