Abstract. Uses automated search to discover novel activation functions and identifies Swish (x·sigmoid(x)) as a particularly effective alternative to ReLU, matching or exceeding it on a range of image and language tasks.
Tags:neural-networksactivationsswish
This site is currently in Beta. Contact: Chris Paton