Nguyen, V.-T. and Luong, T.-K. and Popovici, E. and Trinh, Q.-K. and Zhang, R. and Nakashima, Y. (2021) An Accurate and Compact Hyperbolic Tangent and Sigmoid Computation Based Stochastic Logic. In: 2021 IEEE International Midwest Symposium on Circuits and Systems, MWSCAS 2021, 9 August 2021 through 11 August 2021.
MWSCAS-FINAL.pdf
Download (423kB) | Preview
Abstract
In this paper, a proof-of-concept implementation of hyperbolic tanh(ax) and sigmoid(2ax) functions for high-precision as well as compact computational hardware based on stochastic logic is presented. Nonlinear activation introducing the non-linearity in the learning process is one of the critical building blocks of artificial neural networks. Hyperbolic tangent and sigmoid are the most commonly used nonlinear activation functions in machine-learning system such as neural networks. This work demonstrates the stochastic computation of tanh(ax) and sigmoid(2ax) functions-based Bernstein polynomial using a bipolar format. The format conversion from bipolar to unipolar format is involved in our implementation. One achievement is that our proposed implementation is more accurate than the state-of-the-arts including FSM based method, JK-FF and general unipolar division. On average, 90% of improvement of this work in terms of mean square error (MAE) has been achieved while the hardware cost and power consumption are comparable to the previous approaches. © 2021 IEEE.
Item Type: | Conference or Workshop Item (Paper) |
---|---|
Divisions: | Faculties > Faculty of Radio-Electronic Engineering |
Identification Number: | 10.1109/MWSCAS47672.2021.9531838 |
Uncontrolled Keywords: | Chemical activation; Computation theory; Computer circuits; Learning systems; Mean square error; Neural networks; Stochastic systems; Bernstein polynomial; Bipolar format; High-precision; Hyperbolic tangent; Non-linear activation; Proof of concept; Sigmoid function; Sigmoids; Stochastic logic; Unipolar format; Hyperbolic functions |
Additional Information: | Conference code: 171744. Language of original document: English. |
URI: | http://eprints.lqdtu.edu.vn/id/eprint/8595 |