Transfer Learning and Visualization of Neural Networks for Artistic Images,
Nicolas Gonthier, Yann Gousseau, and Saïd Ladjal, Télécom Paris
You can see this video (enhanced version) in fullscreen here
Abstract
Transfer learning from large-scale natural image datasets, particularly ImageNet, fine-tuning standard deep convolutional neural network models and using the corresponding pre-trained network have become the de facto method for art analysis applications. Nevertheless, there are large differences in dataset sizes, image style and task specifications between natural image classification and the target artistic images, and there is little understanding of the effects of transfer learning. In this work, we explore some properties of transfer learning for artistic images. We compared different ways to obtain an image classifier: fine-tuning, or not, of pre-trained models and training models from scratch. We also use feature visualization techniques in order to understand more precisely what the network learned on those specific artistic datasets. Those visualization of deep neural networks internal representations can help to highlight how neural networks build up their “understanding” of images. We observed that the network could specify some pre-trained filters in order to adapt them to the new modality of images. On the other hand, the network can also learn new, highly structured filters specific to artistic images when the lower-level layers of the initial model are “frozen”. In particular, it is possible to obtain classifiers with equivalent classification performances but with different hidden representations, that can be specific to artistic images or not.
Biographies
Nicolas Gonthier received a Data Science M.Eng. degree in ISAE-Supaéro and an M.Sc. degree in applied mathematics in the University of Toulouse, both in 2017. Currently he is a Ph.D. student at Télécom Paris, Institut polytechnique de Paris. He is funded by an interdisciplinary grant (IDI IDEX) from the University Paris-Saclay. His research interests include deep learning, image processing and machine learning for cultural heritage.
Yann Gousseau received the engineering degree from the École Centrale de Paris, France, in 1995, and the Ph.D. degree in applied mathematics from the University of Paris-Dauphine in 2000. He is currently a professor at Télécom Paris. His research interests include the mathematical modeling of natural images and textures, stochastic geometry, computational photography, computer vision, image and video processing.
Saïd Ladjal is a former student of École Normale Supérieure, France. He received a master degree from École Polytechnique and a Ph.D. degree from École Normale Supérieure de Cachan in 2005 in mathematics. He is currently an associate Professor at Télécom Paris. His research interests include image quality, computational photography and image understanding.
URI/Permalien:
