RBM layer ? (+DBN)

[cbalint13]

• Anyone thinking or planning to implement $subj ? I think it would be quite usefull to have:

RBM, GRBM with CD-k, PCD-k

[shelhamer]

Contributing an RBM layer and CD-k solver would be a fine start. I know there is interest in probabilistic models and RBMs in Caffe from conversations at conferences and the like so it's a worthy project for the interested researcher / developer.

#274 is an early but cancelled attempt at an RBM layer. At the time I suggested a hybrid solution in Caffe + Theano, at least for the purposes of prototyping, but have since changed my mind and agree that having it all in Caffe would be helpful and interesting. One might train models with discriminative and generative components if it's all in the same framework.

[elador]

If there is no RBM layer yet in Caffe, what is this commit then if I may ask? I am a bit confused.

7f43120

[kloudkl]

Don't think it works at all. Either write one from scratch, or use Pylearn2.

[cbalint13]

@shelhamer ,

I am willing to contribute with RBM layer. My first step would be:

• Use this very clear example: https://github.com/cudamat/cudamat/blob/master/examples/rbm_cudamat.py
• Implement in caffe's way of API, add very simple MNIST example with some visualisation.

If all fine in a very future next stage would do:

• Use deepnet as reference: https://github.com/nitishsrivastava/deepnet
• Add l1, l2 constraints
• Add sparsity constraints
• Add gaussian-rbm (linear input)
• Add CD-k, and PCD
• More examples with CIFAR-10, ReLU within GRBM samples and many more.
• More multimodal examples

You can close this Issue, will post a PR (asap) instead.

[kashefy]

@cbalint13
Did you end up creating a PR for this? still planning to? dropped it?

[krikru]

+1 on @kashefy's comment. Why was this issue closed?

[cbalint13]

@krikru ,

• An effort is going on: Add a Restricted Boltzmann Machine layer #3568 . My time is limited but this on my top todo list and will try enhance with more features.

[krikru]

@cbalint13 Sweet. I was just curious :)