#summary This page documents how to setup a buildbot for Melange.
#labels Contents-Review,Importance-Useful,Phase-Deploy

= Introduction =

Buildbot is an open source framework for automating software build, test run, and release. At Melange, we're using buildbot for these automations. The instructions are for a mint clean debian based machine, however due to various choices made in this page, the instructions shouldn't be a lot different for other distros. 

= Instructions =

{{{
  $ BASEDIR='/mnt/teraref'
  $ wget https://raw.github.com/pypa/pip/master/contrib/get-pip.py
  $ sudo python get-pip.py‎
  
  # Add the following line to your /etc/apt/sources.list if you're on a debian based system to install firefox.
  $ deb http://packages.linuxmint.com debian import

  $ sudo pip install virtualenv
  $ sudo apt-get update
  $ sudo apt-get install libcurl4-gnutls-dev libexpat1-dev gettext libz-dev libssl-dev python-lxml python-dev firefox
  $ sudo apt-get install libxml2-dev libxslt1-dev xvfb openjdk-7-jre make
  $ cd $BASEDIR; mkdir buildbot; cd buildbot;
  $ virtualenv venv; cd venv;  . bin/activate
  $ pip install buildbot buildbot-slave
  $ buildbot create-master master
  $ mv master/master.cfg.sample master/master.cfg

  # Let's see if the everything is in order as of now.
  $ buildbot start master; tail -f master/twistd.log
  # If the logs appear okay, we're good to go!
 
  # Let's create a slave, and see if it's running okay.
  $ cd $BASEDIR/buildbot/venv; buildslave create-slave slave localhost:9989 example-slave pass
  $ buildslave start slave; tail -f slave/twistd.log
  # Again, If the logs appear okay, we're good to go!
}}}

The next step involves copying the config files from `/configuration` directory in melange-project-root, to the buildbot machine's master folder - `$BASEDIR/buildbot/venv/master/`
The file private.py has some fields, whose description is documented in the file itself. The fields are pretty straightforward, and the description about them in the same file should be sufficient. 

Now, let's restart the buildbot - 

{{{
  $ cd $BASEDIR/buildbot/venv/master
  $ buildbot restart
}}}

At this point, it might make sense to run these commands, just to check everything is in order - 

{{{
  $ cd $BASEDIR/buildbot/venv
  $ buildbot master start; tail -f master/twistd.log
  $ buildslave start slave; tail -f slave/twistd.log
}}}

Next, we point our browser at the IP at which buildbot is hosted, and point our browser to port 8020. The waterfall page should appear. Try doing a force-build and see if the buildbot gives expected results.

==How to enable buildbot to send emails?==

Now this is a bit tricky, since postfix can be hard to configure. But if you follow the below mentioned steps, things should go smoothly.

{{{
  $ sudo apt-get install postfix mailutils libsasl2-2 ca-certificates libsasl2-modules

  # Open the postfix config file.
  $ sudo vim /etc/postfix/main.cf

  # Add the following lines in there.
  relayhost = [smtp.gmail.com]:587
  smtp_sasl_auth_enable = yes
  smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
  smtp_sasl_security_options = noanonymous
  smtp_tls_CAfile = /etc/postfix/cacert.pem
  smtp_use_tls = yes

  # Next Open postfix sasl_passwd file.
  $ vim /etc/postfix/sasl_passwd

  # Add the following line to it.
  [smtp.gmail.com]:587    USERNAME@gmail.com:PASSWORD
  # USERNAME being the buildbot's email id, and PASSWORD being well, password.

  # Next, run the following commands in your shell.
  $ sudo chmod 400 /etc/postfix/sasl_passwd
  $ sudo postmap /etc/postfix/sasl_passwd
  $ cat /etc/ssl/certs/Thawte_Premium_Server_CA.pem | sudo tee -a /etc/postfix/cacert.pem
  $ sudo service postfix reload

  # After that try to send an email from the instance using a script similar to http://www.tutorialspoint.com/python/python_sending_email.htm
  

  # The email sending should ideally fail, but then go to the logs -
  $ tail -50 /var/log/mail.log

  # You'll find that Gmail has sent a URL to authenticate.
  # Point your browser to this URL, and authenticate.
  # Next, try sending email again, and you should be good to go.
  # Hooray!
}}}

Also, you might need to look at master.cfg of buildbot in order to decide for the value of a few parameters.

NOTE: Builds might fail sometimes because of /tmp getting full, since buildout downloads stuff in there, and doesn't clear it - A cron job is setup using tmpreaper, and it clears everything in /tmp that is older than 1 day.