+高级检索
深度神经网络内部迁移的信息几何度量分析
DOI:
作者:
作者单位:

作者简介:

通讯作者:

基金项目:


Analysis on Information Geometric Measurement of Internal Transfer of Deep Neural Network
Author:
Affiliation:

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
    摘要:

    使用深度神经网络处理计算机视觉问题时,在新任务数据量较少情况下,往往会采用已在大数据集上训练好的模型权值作为新任务的初始权值进行训练,这种训练方式最终得到的模型泛化能力更好.对此现象,传统解释大多只是基于直觉分析而缺少合理的数学推导.本文将深度神经网络这种网络结构不变下层间的学习转为深度神经网络内部的迁移能力,并将学习过程变化形式化到数学表达式.考虑数据集对训练过程带来的影响,利用信息几何分析方法,确定不同数据集流形之上的度量和联络,实现不同数据集之间的嵌入映射,同时将参数空间的变化也放入流形空间,探究其对学习过程的共同影响,最终实现对这种内部迁移现象的数学解释.经过分析和实验验证可得内部迁移过程其实是一种能使网络可以在更广空间进行最优搜索的变化,有利于模型可以在学习过程中获得相对的更优解.

    Abstract:

    When deep learning is used to deal with the computer vision tasks, under little number of new task data, the pre-trained model weight based on a very large data is trained as an initial weight to get better generalization ability. At this point, former explanations are based on the intuitive analysis and lack of reasonable mathematical methods. In this paper, deep neural network, which trains on internal layers with fixed structure,changed into internal transfer ability in deep neural network. The changes of learning process are formalized into a mathematical expression. Considering the influence of the data set on the training process, the information geometric analysis method is used to determine the metrics and connections over manifolds of different data sets, which can realize the embedding mapping between different data sets. At the same time, the change of parameter space is also put into a manifold space to explore its common influence on learning process. Finally, a mathematical explanation is provided for the internal transfer phenomenon. Meanwhile, after the analysis and experiments, the process of internal transfer is identified as a change which can make the network search for optimal search in a wider space. Therefore, the model can obtain a relative better solution in learning process.

    参考文献
    相似文献
    引证文献
文章指标
  • PDF下载次数:
  • HTML阅读次数:
  • 摘要点击次数:
  • 引用次数:
引用本文

陈力,费洪晓,李海峰?覮,何嘉宝,谭风云.深度神经网络内部迁移的信息几何度量分析[J].湖南大学学报:自然科学版,2019,46(2):97~104

复制
历史
  • 收稿日期:
  • 最后修改日期:
  • 录用日期:
  • 在线发布日期: 2019-03-01
  • 出版日期:
作者稿件一经被我刊录用,如无特别声明,即视作同意授予我刊论文整体的全部复制传播的权利,包括但不限于复制权、发行权、信息网络传播权、广播权、表演权、翻译权、汇编权、改编权等著作使用权转让给我刊,我刊有权根据工作需要,允许合作的数据库、新媒体平台及其他数字平台进行数字传播和国际传播等。特此声明。
关闭