Hi, first of all thank you for sharing caffe, it really looks great! I run into an issue training the imagenet model. The training error is not decreasing even after 40k iterations. I am using the imagenet configuration provided in the examples folder. The model is trained with caffe from the dev branch (since master has not been merged with boost-eigen yet).
The train loss starts out around 7.1, then decreases close to 6.9 in the first 150 iterations, and then remains above that value for 40k iterations and likely beyond. I run the same configuration on the boost-eigen branch and had not trouble with the training. The error started decreasing after about 1k iterations and everything seems to be working fine.
Am I missing something? Can you reproduce this behavior? I am running caffe on Ubuntu 12.04 with Cuda 5.5 (GTX 770).
Hi, first of all thank you for sharing caffe, it really looks great! I run into an issue training the imagenet model. The training error is not decreasing even after 40k iterations. I am using the imagenet configuration provided in the examples folder. The model is trained with caffe from the dev branch (since master has not been merged with boost-eigen yet).
The train loss starts out around 7.1, then decreases close to 6.9 in the first 150 iterations, and then remains above that value for 40k iterations and likely beyond. I run the same configuration on the boost-eigen branch and had not trouble with the training. The error started decreasing after about 1k iterations and everything seems to be working fine.
Am I missing something? Can you reproduce this behavior? I am running caffe on Ubuntu 12.04 with Cuda 5.5 (GTX 770).