Memory Augmented Network 

 

Technology 

 

 Sequential data is any kind of data where the order matters. There are many kind of data that has such characteristics. Time-relevant data can be a typical example. However, there are other sequential data that is time irrelevant such as text data and genetic data. These data contain important structural relationships among items and that’s the reason why the order is important.

 

 When analyzing sequential data, sequence modeling can helps predicting what word/letter comes next with those data. There are many sequence model in deep learning such as Recurrent Neural Network with Long-Short Term Memory networks. They use internal memory to remember the characteristics of previous data. However, their memory is typically too small, and is not compartmentalized enough to accurately remember facts from the past.  This cause the model not able to keep track of long-term dependencies Memory augmented networks use external memory to solve the problem. They store the previous data in their external memory and learn how to do it efficiently. They learn how to store, focus and forget information during training. These are benefits what we can earn using memory augmented networks.

 

1. Memory network can keep track of long-term dependencies             

2. Memory network uses attention mechanism which helps focusing on important part of sequence.

3. Memory networks can connect the information and catch the structural relationships within memory.

​

Application Field

​

Bio – drug discovery

 The goal of an automated drug discovery model is to find chemically feasible SMILES strings. We consider legitimate SMILES strings as sentences composed of character used in SMILES notations. The objective of a model then is to learn hidden rules of forming sequences of letters that correspond to legitimate SMILES strings.

 

 To generate a valid SMILES, in addition to correct valence for all atoms, one must count ring opening and closure, as well as bracket sequences with several bracket types. However, regular RNNs are unable to solve the sequence generation problems because their inability of understanding algorithmic rules that generate sequences.

 

 For such a reason, memory-augmented neural networks has been used as a solution. The method assumes that there exists an arbitrary language which encodes the molecule structure. The language possesses a grammar, so a model that misunderstands the grammar generates infeasible molecule designs.  Indeed, memory-augmented networks generates 9% more feasible molecule designs than networks without memory.

 

 This line of thought is an extended work of Turing Machine. The molecule structure is ’programmable’, hence it is important to find a ’computer’ that can ’compile’ molecule structures under legitimate rules. Our team had extensive works on designing such computational brain last year.  We develop ’a computational drug discovery process’ using related knowledge.