资源分类
》 资源 》算法》pytorch_highway_networks

pytorch_highway_networks

2019-09-18 | |  119 |   0 |   0

PyTorch Highway Networks

Highway networks implemented in PyTorch.

Highway Equation

Just the MNIST example from PyTorch hacked to work with Highway layers.

Todo

  • ~~Make the Highway nn.Module reuseable and configurable.~~

  • Why does softmax work better than sigmoid? This shouldn't be the case...

  • Make training graphs on the MNIST dataset.

  • Add convolutional highway networks.

  • Add recurrent highway networks.

  • Experiment with convolutional highway networks for character embeddings.

Notes

  • ELU doesn't work better than RELU for the layer activation.

  • Softmax seems to work better than sigmoid for the gate function?!


上一篇：LOLA_DiCE

下一篇：DAGGER

用户评价
全部评价

热门资源

  • TensorFlow-Course

    This repository aims to provide simple and read...

  • Klukshu-Sockeye-...

    KLUKSHU SOCKEYE PROJECTS 2016 This repositor...

  • DeepFaceLab_Linux

    DeepFaceLab的Linux Ubuntu 版本

  • Tensorflow-TCN

    Tensorflow TCN The explanation and graph in ...

  • DuReader_QANet_BiDAF

    Machine Reading Comprehension on DuReader Usin...