High-Capacity Recursive Neural Coding


Background

Recursive Auto-Associative Memory or RAAM ( Pollack 1990 ) is a method for solving tree structures in a feed-forward network. The RAAM architecture is very similar to that of encoder networks (Ackley et al., 1985; Cottrell et al., 1987), consisting of a compressor network and a reconstructor network which are simultaneously trained. The principal difference is that in a RAAM the compressor and reconstructor are used recursively to encode and decode, respectivley.

The figure above shows how a RAAM encodes the tree (E(F G)). First we feed (F G) into the compressor network, resulting in output a. Then we feed in (E a), giving b. To decode, we feed b into the reconstructor network, giving (E a). At that point we need some kind of "terminal test" to tell us that E, F, and G are terminals (requiring no further decoding), while a is non-terminal that must be fed again into the reconstructor, giving (F G).

<- Previous Next ->