Continuous Integration / Build Monitoring

[shelhamer]

Configure a buildbot to check master and dev on push and report test status and the like.

[kloudkl]

To deal with the problem arised in #47, it was suggested that we should try Chromium build infrastructure which is based on buildbot as proposed here to test under various development environments. Obviously, it was not mature enough to take it seriously because the contributions at that time were not as active as they are today and the workload pressure of peer reviewing was not a problem yet.

Maybe trying to test all the OS and GCC/CUDA compiler combinations is an overkill for a project that's not as huge as Chromium. But at least we should cover the most popular versions of Ubuntu, Red Hat/CentOS and iOS combining the default compilers released with them.

[shelhamer]

Our current intention is to cover our main research platform, Ubuntu 12.04 LTS, in CPU and GPU modes on Tesla class hardware with GCC 4.6 and CUDA 5.5 as demands on our infrastructure allow.

Of course diversifying is better, so we would be open to an (automatic) means of reporting other build results too, once the first step is in place.

We'll look into the Chromium build infrastructure as an alternative to vanilla buildbot. Thanks for the suggestion.

[kloudkl]

It is perfect to start with such a popular platform. I am using the same OS and compilers too.

[shelhamer]

In addition, a bot will respond to a post-commit hook on master to re-generate and publish the docs automatically.

Further down the line, we can have hooks to build + test pull requests.

[shelhamer]

The required equipment will arrive next week, and we hope to have the build bot configured soon after.

[kloudkl]

@shelhamer, the good news is that there is no need to setup and maintain the continous integration infrastructure. Travis-CI is perfect for open source projects hosted on GitHub. It is very easy to get started.

"Travis CI is a hosted continuous integration service. It is integrated with GitHub and offers first class support for:

C
C++
Clojure
Erlang
Go
Groovy
Haskell
Java
JavaScript (with Node.js)
Objective-C
Perl
PHP
Python
Ruby
Scala
Travis CI's build environment provides different runtimes for different languages, for instance multiple versions of Ruby, PHP, Node.js. It also comes preinstalled with a variety of data stores and common tools like message brokers."

[shelhamer]

@kloudkl thanks for the suggestion, but we still need to set up our own
buildbot for the GPU tests and are particular library dependencies.

Sorry for the hold-up on this everyone. I'll take another stab at it after
ICML.

On Wed, Jun 18, 2014 at 9:08 AM, kloudkl notifications@github.com wrote:

@shelhamer https://github.com/shelhamer, the good news is that there is
no need to setup and maintain the continous integration infrastructure.
Travis-CI http://docs.travis-ci.com/user/getting-started/ is perfect
for open source projects hosted on GitHub. It is very easy to get started.

"Travis CI is a hosted continuous integration service. It is integrated
with GitHub and offers first class support for:

C
C++
Clojure
Erlang
Go
Groovy
Haskell
Java
JavaScript (with Node.js)
Objective-C
Perl
PHP
Python
Ruby
Scala
Travis CI's build environment provides different runtimes for different
languages, for instance multiple versions of Ruby, PHP, Node.js. It also
comes preinstalled with a variety of data stores and common tools like
message brokers."

—
Reply to this email directly or view it on GitHub
#150 (comment).

[kloudkl]

You're right. Maybe there are other CI tools that are easier to set up and manage especially considering that virtual machines would be necessary as the tests have to be run on multiple operating systems.

[bhack]

Could we close this ticket? Or take it open until we find also a GPU CI solution?

[shelhamer]

Closing since Travis-CI is ready thanks to @huyng and @jeffdonahue and the CPU-only build #561 will further improve continuous integration time.

GPU build monitoring, perhaps best done nightly instead of continuously for resources, can follow as a refinement of our newly-established monitoring.