Brain-Inspired Learning

Improving upon gradient descent optimization.

Supervised Backpropagation

Gradient descent optimization is currently the foundation of state-of-the-art artificial intelligence (AI), but supervised learning with backpropagation has limits.
Deep neural networks are computationally expensive to train with backpropagation, and the most capable AIs require large GPU clusters which consume kilowatts of power.
Backpropagation requires continuous activation functions and the existence of large gradients through all network layers. This makes networks brittle and susceptible to adversarial attacks.
Supervised backpropagation is greedy and vulnerable to training data bias. Enormous training datasets are required to mitigate this vulnerability.
Disappearing gradients limit the depth of networks that can be trained effectively.

Alternative Learning: Hybrid Supervised-Unsupervised Learning

Training images are simple shaded renderings of targets in all orientations

The fully-convolutional network has binary activation.
Training is a combination of Hebbian clustering and local supervised learning.
The fourth layer is trained on class, rotation, and position ground truth.
The last two segmentation layers are trained to regenerate activation in previous feature layers.

Brain-Inspired Local Learning

The network learned orientation-selective edge features in the second convolutional layer. Active orientation channels are displayed as colors in the middle and right feature maps.

The fourth layer correctly classifies, locates, and determines the orientation of the two aircraft.

The last generative layer segments the two aircraft found in the input image (right).

Robust

The network was tolerant of cloudy images and partially occulded targets.

Whole Satellite Image

The network classified and segmented all but four aircraft correctly in a satellite images of Beijing international airport