Installing ReviewBoard on Amazon EC2

Recently at HuzuTech I updated our installation of ReviewBoard, which is a great code review tool. Rather than just update packages though, I decided to rebuild the server for shits and giggles. Herein lies my method, just in case you want to set one up yourself.

Pre-requisites

This guide assumes that you have an AWS EC2 account set up, and that you are comfortable with creating instances and SSH’ing into them. There are plenty of other guides out there to take you through those basics.

What You’ll End Up With

  • 1 (one) EC2 server micro instance running with a 32-bit Amazon Linux AMI
  • 2 (two) VCS installed on the server: subversion and git
  • 1 (one) Data Volume containing:
    • 1 (one) ReviewBoard site
    • 1 (one) PostgreSQL database
  • 1 (one) Glowing feeling of a Job Well Done

The precise security configuration and so forth for the instance is likely to vary depending on what your preferred policies are.

Creating The Instance

Sign in to the EC2 web console and select “Launch Instance” for the region you prefer. I’m using US East because it’s cheap.

The following few screens show the Quick Launch instance wizard thing which seems to have just sprung up on the site, but you should be able to work out what to do in the normal/classic wizard.

The important points are:

  • Use a Basic 32-bit Amazon Linux AMI
  • Use a micro instance

Give your instance a sane name and create and download a new key pair for it. You can use an existing pair, but I tend to use a new key pair for each image for finer grained access control. Make sure it’s the 32-bit Amazon Linux AMI which you choose.

In the next screen, you may find it useful to add termination protection and check that it’s selected a micro instance.

Next, set up an new security group for it. You can’t change the security group later (although you can add and remove ports in the group), so make sure you create a new one unless you are certain you want to share an existing one. Make sure SSH and HTTP ports are open and accessible from your current public IP address. If in doubt, you can make the ports accessible from any address for the moment by specifying a source of 0.0.0.0/0.

That’s the basic instance set up. Click “Save details” and then you should see the launching screen:

Close it. Now create the volume to use for data. Expand the “Elastic Block Store” menu at the left side if it isn’t already and select Volumes. Click the “Create Volume” button and make it about 1 GiB or so, depending on the size of the repositories you’ll store on it:

Now all that remains to do is to attach it to the instance you just launched. Select the new volume and click “Attach Volume” and attach it to your fresh instance:

That’s the basic setup done. Now on to installing bytes.

Setting Up The Instance and Data Volume

SSH into your server:

$ ssh -i ~/.ec2/tools-reviewboard.pem ec2-user@ec2-107-22-136-39.compute-1.amazonaws.com

       __|  __|_  )
       _|  (     /   Amazon Linux AMI
      ___|\___|___|

See /usr/share/doc/system-release/ for latest release notes.
There are 14 security update(s) out of 18 total update(s) available
[ec2-user@ip-10-116-186-53 ~]$

The first thing you’re likely to notice is that the instance requires some updates already, so we will install those.

NOTE: for the rest of the article, I’ll just show the commands to run without showing the output, except where the output is important or interactive. Also, the $ at the start of a line indicates a command to type - you don’t type the dollar, but type the rest…

$ sudo yum --assumeyes update

That should update the basic image to the latest versions of the pre-installed packages.

Now, before we go any further, lets set up the data volume. You should have already created a new volume and attached it to the ReviewBoard server instance. If this is the first time you’ve used separate data volumes (something you should really get into the habit of using), the thing to remember is that it’s sitting as a raw, unformatted block device initially. In order to use it, it needs to be partitioned, formatted, and then mounted.

First, format it. I’m assuming that your device is /dev/xvdf. If it isn’t, adjust the instructions as appropriate. Run fdisk on the device. It’s interactive so the following few steps include the console output for clarity.

$ sudo fdisk /dev/xvdf

Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0xb6d71f5e.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').

Command (m for help):

Enter m in order to show a menu.

Command action
   a   toggle a bootable flag
   b   edit bsd disklabel
   c   toggle the dos compatibility flag
   d   delete a partition
   l   list known partition types
   m   print this menu
   n   add a new partition
   o   create a new empty DOS partition table
   p   print the partition table
   q   quit without saving changes
   s   create a new empty Sun disklabel
   t   change a partition's system id
   u   change display/entry units
   v   verify the partition table
   w   write table to disk and exit
   x   extra functionality (experts only)

Command (m for help):

Enter n to create a new partition.

Command action
   e   extended
   p   primary partition (1-4)

Enter p to create a primary partition.

Partition number (1-4):

Enter 1.

First cylinder (1-261, default 1):

Press enter to select the default.

Last cylinder, +cylinders or +size{K,M,G} (1-261, default 261):

Press enter to select the default. The actual value for the default will depend on how large a volume you created with the Amazon web console. This is a 2 GiB volume.

Command (m for help):

Enter w to write the changes to the disk. The partition will be created and you should see:

The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

The next step is formatting the partition. Use mkfs to create a new ext4 partition:

$ sudo mkfs -t ext4 /dev/xvdf

Now create an empty folder in /mnt for the partition, and edit /etc/fstab to mount it on boot.

$ sudo mkdir /mnt/rb.data
$ sudo nano /etc/fstab

The entry to add to fstab should look like this:

/dev/xvdf     /mnt/rb.data   ext4    defaults    0   0

If you chose a different folder, filesystem or have it available on a different device, remember to use your values here.

The final task is simply to mount the volume, and then run df to check that it has mounted:

$ sudo mount /mnt/rb.data
$ df -h

Filesystem            Size  Used Avail Use% Mounted on
/dev/xvda1            7.9G  1.1G  6.8G  14% /
tmpfs                 302M     0  302M   0% /dev/shm
/dev/xvdf             2.0G   67M  1.9G   4% /mnt/rb.data

If you don’t see your volume in the Mounted on column, check all the previous steps worked.

Installing Lots Of Stuff

Now to install all we need for ReviewBoard Cut and paste this full block into the command line:

$ sudo yum --assumeyes install \
    ant \
    ant-apache-regexp \
    autoconf \
    gcc \
    gcc-c++ \
    git \
    httpd \
    icu \
    java-1.6.0-openjdk-devel \
    libicu \
    libicu-devel \
    libjpeg \
    libjpeg-devel \
    libpng \
    libpng-devel \
    lucene \
    make \
    memcached \
    mod_wsgi \
    patch \
    postgresql \
    postgresql-devel \
    postgresql-server \
    python-devel \
    svn \
    telnet

Now to install the python prerequisite packages:

$ sudo easy_install \
    python-memcached \
    pycrypto\>=2.4.1 \
    psycopg2 \
    django-storages

Pyconfig will probably spew out lots of warnings, but it should be OK to ignore those.

Next, install ReviewBoard:

$ sudo easy_install ReviewBoard

Now you have a choice – install pylucene so that you can have full text search in ReviewBoard, or skip it and just move on to setting up the database.

Installing Pylucene (optional)

Although you don’t need Pylucene installed, it’s nice to have it because of the search features it adds to ReviewBoard. However, it can be a bit of a bugger to install, so although these instructions work for now, if they’re broken by the time you read this, you may want to skip it.

Brave? Then on-wards…

Create a folder to build and install Pylucene from, download and extract the archive:

$ mkdir pylucene
$ cd pylucene
$ curl apache.mirror.rbftpnetworks.com/lucene/pylucene/pylucene-3.4.0-1-src.tar.gz | tar -xvz

tar may take a while to extract the large archive being fed to it, but once it’s finished, compile jcc:

$ cd pylucene-3.4.0-1/jcc
$ sudo env JCC_JDK=/usr/lib/jvm/java-1.6.0-openjdk NO_SHARED=true python setup.py build
$ sudo env JCC_JDK=/usr/lib/jvm/java-1.6.0-openjdk NO_SHARED=true python setup.py install
$ cd ..

Now to build Pylucene. First you have to edit some values in the Makefile, so open that:

$ nano Makefile

And then paste these lines at the top of the very top of file, before the block comment:

JAVA_HOME=/usr/lib/jvm/java-1.6.0-openjdk
PREFIX_PYTHON=/usr
ANT=ant
PYTHON=$(PREFIX_PYTHON)/bin/python
JCC=$(PYTHON) -m -m jcc.__main__
NUM_FILES=3

Now you should be ready to build it:

$ make

This can take a long time (30 minutes or so), so go get some tea. After it’s complete you can now install it:

$ sudo make install

That’s Pylucene dealt with. On to the database.

Creating the ReviewBoard Database

The database will also reside on the data volume. This setup assumes that you only want one PostgreSQL cluster. I’ve no idea why you’d want more than that, but if you do, you’re advanced enough to figure out how to make the next few steps fit into that configuration yourself.

First, create a folder for the database, and give it to the postgres user:

$ sudo mkdir -p /mnt/rb.data/db/pg
$ sudo chown postgres:postgres /mnt/rb.data/db/pg

And now set that data location in the postgres conf file:

$ echo "PGDATA=/mnt/rb.data/db/pg" | sudo tee /etc/sysconfig/pgsql/postgresql

With that sorted, we can now tell PostgreSQL to create the cluster:

$ sudo service postgresql initdb

Next, we want to change the database authentication method for connections from 127.0.0.1 in order to allow ReviewBoard to connect. To do this, edit /mnt/rb.data/db/pg/pg_hba.conf and change the method used for 127.0.0.1/32 from ident to md5:

$ sudo nano /mnt/rb.data/db/pg/pg_hba.conf

Near the bottom of the file, change the line host all all 127.0.0.1/32 ident to host all all 127.0.0.1/32 md5.

All that’s left is to start it and then create the empty ReviewBoard database:

$ sudo service postgresql start
$ sudo su postgres -c "createdb reviewboard"

Now create a database user for reviewboard. I’m going to be lazy here and create them as a superuser because I can’t be arsed setting up their correct priviledges. Don’t be as stupid as I am:

$ sudo su postgres -c "createuser -sP reviewboarduser"

Be sure to give them a decently strong password.

Now, in case you have to reboot your server, you’ll want to make sure things start on boot. To do this, you can use ntsysv:

$ sudo ntsysv

Use the cursor keys to move, space to select, and tab to move between areas. Make sure you enable httpd, memcached and postgresql.

Just to make sure it’s all running, reboot the instance.

$ sudo shutdown -r now

This will kick you out of your session, but just give it a minute or two and then log back in. The address of your server should still be the same at this point, but if it isn’t, you can find it again from the AWS web console.

After you’ve logged back in, check that the webserver is running to make sure the ntsysv command above worked. The easiest way to do this is by using telnet to request the default page from the local webserver:

$ telnet 127.0.0.1 80
GET /

You should see a screed of HTML returned. If you see a connection refused or timed out, go back to the ntsysv section above and try again. Note that ntsysv doesn’t start the services for you, it just marks them to start on boot, so you have to perform that shutdown step to test it.

Creating A ReviewBoard Site

That’s pretty much the server setup now. All that remains is to create a review board site. There’s a handy interactive script to do this. Create a directory for the site and then run the interactive script:

$ sudo mkdir -p /mnt/rb.data/www
$ sudo rb-site install /mnt/rb.data/www/reviewboard.huzutech.com

The script is fairly informative, but if you need more information, see the ReviewBoard documentation for creating sites:

Basically, the only thing to remember is that you are using a postgresql database, memcached, apache and wsgi.

Give ownership of the web directory to Apache, and add the generated site config to the apache conf folder, then restart apache:

$ sudo chown -R apache:apache /mnt/rb.data/www
$ sudo ln -s /mnt/rb.data/www/reviewboard.huzutech.com/conf/apache-wsgi.conf /etc/httpd/conf.d/XX_reviewboard.huzutech.com.conf
$ sudo service httpd restart

Note that the link name starts with XX_. This is important so that it’s loaded in the correct order.

Finished!

That’s it! You should be able to see your server if you visit the public address in a web browser. There are several post setup things that you probably want to do, such as setting up a proper vhost file, locking user registration and setting up projects and repositories.

If you installed lucene, don’t forget to add the lucene index updating cron job to cron, otherwise you’ll get errors when you try to search and it can’t find the search index files.

Check out the guides on the main ReviewBoard website for more information about all that. Then sit back and learn from all the glorious code reviews you see rolling in.

Comments