Scientific Background and Context: Deep learning is part of a broader family of machine learning methods based on learning data representations, as opposed to task-specific algorithms. Learning can be supervised, semi-supervised or unsupervised. Deep learning architectures such as deep neural networks, deep belief networks and recurrent neural networks have been applied to fields including computer vision, speech recognition, natural language processing, audio recognition, social network filtering, machine translation, bioinformatics, drug design and board game programs, where they have produced results comparable to and in some cases superior to human experts. 
Convolutional Neural Networks: A Convolutional Neural Network (CNN) is comprised of one or more convolutional layers (often with a subsampling step), followed by one or more fully connected layers. The architecture of a CNN is designed to take advantage of the 2D structure of an input image (or other 2D input such as a speech signal). This is achieved with local connections and tied weights followed by some form of pooling which results in translation invariant features.
Convolutional Neural Nets in Chemistry: In 2017, Nathan Baker, et al , applied Convolutional Neural Nets to the field of chemistry. The architecture used was a state-of-the-art neural net design that had been applied to Computer Vision tasks. The molecules were input from a database of SMILES strings and decoded into their corresponding 2-D structures using open source software such as RDkit and OpenBabel. The resulting coordinates were then mapped onto an 80 x 80 grid. The Deep Neural Net was then trained on the dataset of resulting and allowed to learn the correlations on its own. One of the most important features about Chemception was that it was not provided with any kind of chemistry knowledge beforehand. All its predictions/classifications were based only on the features it learnt during its training. For 2 of the 3 properties, Chemception outperformed its deep neural network counterparts, and achieved a validation/test AUC of 0.744/0.752 for HIV activity prediction and a validation/test RMSE of 1.51/1.74 kcal/mol for free energy of solvation prediction, which is close to the accuracy of physics-based simulation methods (RMSE ~1.5 kcal/mol).
Capsule Nets: Convolutional Neural Nets perform an excellent job in classification of images and image recognition. However. the major problem with CNNs is that the max pooling step just indicates the presence of certain features, e.g. two eyes or a nose, but does not establish the relationship between the positioning of these features, e.g. two eyes are always above the nose. Due to this problem, a new kind of neural net architecture was developed called Capsule Nets , that can establish relationships between the features just as the human brain does.
A capsule is a nested set of neural layers. In a regular neural network, you keep on adding more layers, while in a Capsule Net you add more layers inside a single layer, or in other words, nest a neural layer inside another. The state of the neurons inside a capsule captures the above properties of one entity inside an image. A capsule outputs a vector to represent the existence of the entity. The orientation of the vector represents the properties of the entity. Capsule nets are one of the first neural net architectures that are not just able to detect the presence of different objects in an image, but also able to study the relationships between those objects.
Broad Goal: We will investigate whether Capsule Nets can be used to circumvent the alignment problem in 3D-QSAR. We also plan to compare the performance of capsule nets against the current state-of-the-art, Convolutional Neural Nets.
- To compare the effectiveness of different forms of molecular input data - string, graph or connectivity-based, and 3D or coordinate-based - in structure-activity relationship modeling with Capsule Nets;
- To benchmark the performance of Capsule Nets against Convolutional Neural Nets and against traditional machine learning methods on one or more cheminformatics datasets;
- To assess whether Capsule Nets can extract meaningful pharmacophore features of molecules from a cheminformatics dataset, and whether Capsule Nets can circumvent the alignment problem in 3D-QSAR.