深度神经网络内部迁移的信息几何度量分析
Information Geometric Measurement of Internal Transfer of Deep Neural Network
投稿时间:2017-08-20  修订日期:2017-10-15
DOI:
中文关键词:  深度学习  迁移学习  信息几何
英文关键词:deep learning  transfer learning  information geometry
基金项目:国家自然科学基金项目(面上项目,重点项目,重大项目)
作者单位E-mail
费洪晓 中南大学 hxfei@csu.edu.cn 
陈力 中南大学  
李海峰 中南大学 lihaifeng@csu.edu.cn 
何嘉宝 中南大学  
谭风云 中南大学  
摘要点击次数: 380
全文下载次数: 0
中文摘要:
      使用深度神经网络处理计算机视觉问题时,在新任务数据量较少情况下,往往会采用已在大数据集上训练好的模型权值作为新任务的初始权值进行训练,这种训练方式最终得到的模型泛化能力更好。对此现象,传统解释大多只是基于直觉分析而缺少合理的数学推导。本文将深度神经网络这种网络结构不变下层间的学习转为深度神经网络内部的迁移能力,并将学习过程变化形式化到数学表达式。考虑数据集对训练过程带来的影响,利用信息几何分析方法,确定不同数据集流形之上的度量和联络,实现不同数据集之间的嵌入映射,同时将参数空间的变化也放入流形空间,探究其对学习过程的共同影响,最终实现对这种内部迁移现象的数学解释。经过分析和实验验证可得内部迁移过程其实是一种能使网络可以在更广空间进行最优搜索的变化,有利于模型可以在学习过程中获得相对的更优解。
英文摘要:
      When deep learning is used to deal with computer vision tasks, under little number of new task data, the pre-trained model weight based on a very large data deep neural network is trained as an initial weight to get better generalization ability. At this point, former explanations are based on intuitive analysis and lack of reasonable mathematical methods. In this paper, deep neural network, which trained on internal layers with fixed structure, has been changed into internal transfer ability in deep neural network. And changes of the learning process are formalized into a mathematical expression. Considering the influence of the data set on the training process, the information geometric analysis method is used to determine the metrics and connections over manifolds of different data sets, which can realize the embedding mapping between different data sets. At the same time, the change of parameter space is also put into a manifold space to explore its common influence on learning process. Finally, a mathematical explanation is provided for the internal transfer phenomenon. Also, after the analysis and experiments, the process of internal transfer is a change which can make the network search for optimal search in a wider space, so that the model can obtain a relative better solution in learning process.
  查看/发表评论  下载PDF阅读器
关闭