杜圣杰, 贾晓芬, 黄友锐, 郭永存, 赵佰亭. 面向CNN模型图像分类任务的高效激活函数设计[J]. 红外与激光工程, 2022, 51(3): 20210253. DOI: 10.3788/IRLA20210253
引用本文: 杜圣杰, 贾晓芬, 黄友锐, 郭永存, 赵佰亭. 面向CNN模型图像分类任务的高效激活函数设计[J]. 红外与激光工程, 2022, 51(3): 20210253. DOI: 10.3788/IRLA20210253
Du Shengjie, Jia Xiaofen, Huang Yourui, Guo Yongcun, Zhao Baiting. High efficient activation function design for CNN model image classification task[J]. Infrared and Laser Engineering, 2022, 51(3): 20210253. DOI: 10.3788/IRLA20210253
Citation: Du Shengjie, Jia Xiaofen, Huang Yourui, Guo Yongcun, Zhao Baiting. High efficient activation function design for CNN model image classification task[J]. Infrared and Laser Engineering, 2022, 51(3): 20210253. DOI: 10.3788/IRLA20210253

面向CNN模型图像分类任务的高效激活函数设计

High efficient activation function design for CNN model image classification task

  • 摘要: 激活函数(Activation Functions,AF)对于卷积神经网络学习、拟合复杂函数模型来说具有十分重要的作用,为了使神经网络能更好更快的完成各类学习任务,设计了一种新型高效激活函数EReLU。EReLU通过引入自然对数函数有效缓解了神经元“坏死”和梯度弥散问题,通过分析激活函数及其导函数在前馈和反馈过程中的作用对EReLU函数的数学模型探索设计,经测试确定EReLU函数的具体设计方案,最终实现了提升精度和加速训练的效果;随后在不同网络和数据集上对EReLU进行测试,结果显示EReLU相较于ReLU及其改进函数精度提升0.12%~6.61%,训练效率提升1.02%~6.52%,这有力地证明了EReLU函数在加速训练和提升精度方面的优越性。

     

    Abstract: Activation Functions (AF) play a very important role in learning and fitting complex function models of convolutional neural networks. In order to enable neural networks to complete various learning tasks better and faster, a new efficient activation function EReLU was designed in this paper. By introducing the natural logarithm function, EReLU effectively alleviated the problems of neuronal "necrosis" and gradient dispersion. Through the analysis of the activation function and its derivative function in the feedforward and feedback process of the mathematical model of the EReLU function exploration and design, the specific design of the EReLU function was determined through test, and finally the effect of improving the accuracy and accelerating training was achieved; Subsequently, EReLU was tested on different networks and data sets, and the results show that compared with ReLU and its improved function, the accuracy of EReLU is improved by 0.12%-6.61%, and the training efficiency is improved by 1.02%-6.52%, which strongly proved the superiority of EReLU function in accelerating training and improving accuracy.

     

/

返回文章
返回