OpenDataPlane is open source project hosted on github. It supports various architectures, build environments and toolchain levels. This talk is about the evolution of the CI system to support ODP development. It describes which variants we used for testing and what is our current solution, why we use Travis and Shippable frameworks for automated testing of GitHub projects, and why we wrote our own homegrown tools. We also explore how Docker can overcome these problems by permitting the test environment to be more tightly defined and controlled. This talk discusses the motivation, benefits, and practical issues surrounding CI for development of open source applications.