Model Training and Generalization

Machine learning aims at replacing hand-coded functions with models automatically trained from data. Such automation is a reasonable and certainly inevitatable trend in computer science. The main ‘trick’ however remains in training models that generalize well to unseen data. Training a model involves a potentially large set of data dubbed the training set. It is somewhat easy for the machine, and especially for deep neural networks to fully memorize these samples.