Learning without Forgetting

The paper addresses the continual expansion of a convolutional neural network when the training data for its existing tasks is no longer available. The proposed Learning without Forgetting method uses only the data for the new task to train the network, while preserving the network's responses on the original tasks, avoiding the need to store or retrain on old data as the number of tasks grows.

In experiments, Learning without Forgetting performs favorably against the commonly used feature extraction and fine-tuning adaptation techniques, and performs similarly to multitask learning that assumes access to the original task data. A more surprising observation is that it may be able to replace fine-tuning when old and new task datasets are similar, yielding improved new-task performance and offering a practical way to grow vision systems incrementally.