Ubuntu

I usually develop and test my webpages on various VirtualBox virtual machines and since I am doing a lot of console work I use a framebuffer which enables using larger resolutions than the default console. Until now I was using Ubuntu 8.04 LTS virtual machines, but recently I upgraded to 9.10. Ubuntu 9.10 (Karmic Koala) is using grub2 so the simple adding of vga=791 or similar value to boot options won't work. Here's how to enable the framebuffer on virtual consoles:

1. Enable kernel modules

$ sudo nano /etc/initramfs-tools/modules

Add two lines at the end of the file enabling two kernel modules:

vesafb
fbcon

Prevent blacklisting the vesafb module:

$ sudo nano /etc/modprobe.d/blacklist-framebuffer.conf

Look for the line blacklist vesafb and comment it out:

#blacklist vesafb

2. Modify grub configuration

$ sudo nano /etc/default/grub

Remove the comment from the GRUB_GFXMODE line and set the desired resolution:

GRUB_GFXMODE=1024x768

There's no need to set bit depth, just the screen resolution. Then open the very first config file in /etc/grub.d

$ sudo nano /etc/grub.d/00_header

Look for the code set gfxmode=${GRUB_GFXMODE} and insert one line after it so the code looks like this:

if loadfont `make_system_path_relative_to_its_root $(){GRUB_FONT_PATH}` ; then
  set gfxmode=${GRUB_GFXMODE}
  set gfxpayload=keep
  insmod gfxterm
  insmod ${GRUB_VIDEO_BACKEND}

3. Update your boot configuration

$ sudo update-grub2

$ sudo update-initramfs -u

That's it! Now reboot and watch your enhanced virtual console with the framebuffer.

Meet Lighttpd: it is a fast, small footprint and higly configurable webserver with advanced features. It's often a common alternative to Apache. At the time of writing the latest versions from the 1.4.x branch is considered stable while Lighttpd 1.5.0 is an ongoing work with new features.

One new feature is the mod_deflate server module which is a modified version of mod_compress shipped with 1.4.x versions. mod_deflate can compress any output from lighttpd static or dynamic - that is, lighttpd can compress the output of dynamically generated pages with this.

Websites with lots of dynamically generated content receive significant bandwidth saving with dynamic output compression, and visitors of those websites benefit faster page loading times.

There are patches available which makes building a version of Lighttpd 1.4.x from source which includes mod_deflate.

1. Building and packaging Lighttpd on Ubuntu

Before building on an Ubuntu hardy system, some prerequisites need to be installed. After getting lighttpd source, it needs to be patched to support mod_deflate. Standard debian packages are to be built which can be easily installed on any target system (assuming the same OS architecture)

Install tools for building and get the Lighttpd source from the package repository:

$ sudo apt-get install build-essential fakeroot
$ sudo apt-get install debhelper cdbs libssl-dev libbz2-dev libattr1-dev libpcre3-dev libmysqlclient15-dev libgamin-dev libldap2-dev libfcgi-dev libgdbm-dev libmemcache-dev liblua5.1-0-dev dpatch patchutils pkg-config uuid-dev libsqlite3-dev libxml2-dev libkrb5-dev
$ mkdir ~/lighttpd
$ cd ~/lighttpd
$ apt-get source lighttpd

Download the patch and apply it:

$ wget http://redmine.lighttpd.net/attachments/download/632/lighttpd-1.4.19.mod_deflate.patch
$ cd lighttpd-1.4.19
$ patch -p1 < ../lighttpd-1.4.19.mod_deflate.patch

Build and install the package:

$ fakeroot dpkg-buildpackage

After building completes you will see the freshly built deb packages in the parent directory:

$ cd ..
$ ls -lgG
total 1336
drwxr-xr-x 8   4096 Aug 16 08:05 lighttpd-1.4.19
-rw-r--r-- 1  66387 Oct  8  2008 lighttpd-1.4.19.mod_deflate.patch
-rw-r--r-- 1  70422 Aug 16 08:07 lighttpd-doc_1.4.19-0ubuntu3_all.deb
-rw-r--r-- 1  11536 Aug 16 08:08 lighttpd-mod-cml_1.4.19-0ubuntu3_i386.deb
-rw-r--r-- 1  10958 Aug 16 08:08 lighttpd-mod-magnet_1.4.19-0ubuntu3_i386.deb
-rw-r--r-- 1   6918 Aug 16 08:08 lighttpd-mod-mysql-vhost_1.4.19-0ubuntu3_i386.deb
-rw-r--r-- 1   8492 Aug 16 08:08 lighttpd-mod-trigger-b4-dl_1.4.19-0ubuntu3_i386.deb
-rw-r--r-- 1  19002 Aug 16 08:08 lighttpd-mod-webdav_1.4.19-0ubuntu3_i386.deb
-rw-r--r-- 1  38501 Aug 16 08:03 lighttpd_1.4.19-0ubuntu3.diff.gz
-rw-r--r-- 1   1106 Aug 16 08:03 lighttpd_1.4.19-0ubuntu3.dsc
-rw-r--r-- 1   2433 Aug 16 08:08 lighttpd_1.4.19-0ubuntu3_i386.changes
-rw-r--r-- 1 267770 Aug 16 08:08 lighttpd_1.4.19-0ubuntu3_i386.deb
-rw-r--r-- 1 815568 Mar 12  2008 lighttpd_1.4.19.orig.tar.gz

Lighttpd depends on libterm-readline-perl-perl so install this as well:

$ sudo apt-get install libterm-readline-perl-perl

If apt-get is whining about libterm-readkey-perl as a dependency try:

$ sudo apt-get -f install libterm-readkey-perl libterm-readline-perl-perl

Then you can install lighttpd from the freshly built deb package:

$ sudo dpkg -i lighttpd_1.4.19-0ubuntu3_i386.deb

If you need additional packages, install them as well:

$ sudo dpkg -i lighttpd-mod-magnet_1.4.19-0ubuntu3_i386.deb
$ sudo dpkg -i lighttpd-mod-webdav_1.4.19-0ubuntu3_i386.deb

Copy mod_deflate module to its place:

$ sudo cp -d ~/lighttpd/lighttpd-1.4.19/debian/tmp/usr/lib/lighttpd/mod_deflate.so* /usr/lib/lighttpd/

2. Configuring mod_deflate

Load the module in /etc/lighttpd/lighttpd.confand comment out mod_compress:

    server.modules = (
                "mod_access",
                "mod_alias",
                "mod_accesslog",
    #           "mod_compress",
                "mod_deflate",
    #           "mod_rewrite",
    #           "mod_redirect",
    

Also comment out mod_compress configuration directives:

    #### compress module
    #compress.cache-dir          = "/var/cache/lighttpd/compress/"
    #compress.filetype           = ("text/plain", "text/html", "application/x-javascript", "text/css")

Configure the mod_deflate module:

    # mod_deflate settings
    deflate.enabled = "enable"
    deflate.compression-level = 9
    deflate.mem-level = 9
    deflate.window-size = 15
    deflate.bzip2 = "enable"
    deflate.min-compress-size = 200
    #deflate.sync-flush = "enable"
    deflate.output-buffer-size = 4096
    deflate.work-block-size = 512
    deflate.mimetypes = ("text/html", "text/plain", "text/css", "text/javascript", "text/xml")
    deflate.debug = "enable"

Detailed information about configuration options is available on the Lighttpd mod_deflate documentation page. The last line is only to check if mod_deflate is working.

Fire up Lighttpd:

$ sudo /etc/init.d/lighttpd start

Visit a static page: http://your.server/index.lighttpd.html and check the lighttpd error log at /var/log/lighttpd/error.log:

2009-08-16 09:27:22: (mod_deflate.c.1232) Content-Type: text/html
2009-08-16 09:27:22: (mod_deflate.c.1239) mime-type: text/html
2009-08-16 09:27:22: (mod_deflate.c.1267) enable compression for  /index.lighttpd.html
2009-08-16 09:27:22: (mod_deflate.c.1283) add Vary: Accept-Encoding for  /index.lighttpd.html
2009-08-16 09:27:22: (mod_deflate.c.305) output-buffer-size: 4096
2009-08-16 09:27:22: (mod_deflate.c.307) compression-level: 9
2009-08-16 09:27:22: (mod_deflate.c.309) mem-level: 9
2009-08-16 09:27:22: (mod_deflate.c.311) window-size: -15
2009-08-16 09:27:22: (mod_deflate.c.313) min-compress-size: 200
2009-08-16 09:27:22: (mod_deflate.c.315) work-block-size: 512
2009-08-16 09:27:22: (mod_deflate.c.1350) Compress all content and use Content-Length header: uncompress len= 3574
2009-08-16 09:27:22: (mod_deflate.c.948) compress: in_queue len= 3574
2009-08-16 09:27:22: (mod_deflate.c.881) compress file chunk: offset= 0 , toSend= 3574
2009-08-16 09:27:22: (mod_deflate.c.351) gzip_header len= 10
2009-08-16 09:27:22: (mod_deflate.c.386) compress: in= 3574 , out= 0
2009-08-16 09:27:22: (mod_deflate.c.1003) compressed bytes: 3574
2009-08-16 09:27:22: (mod_deflate.c.451) flush: in= 0 , out= 1555
2009-08-16 09:27:22: (mod_deflate.c.489) gzip_footer len= 8
2009-08-16 09:27:22: (mod_deflate.c.919)  in: 3574  out: 1563
2009-08-16 09:27:22: (mod_deflate.c.1020) finished uri: /index.lighttpd.html , query:

3. Install and configure php for Lighttpd

Install the CGI version of PHP

$ sudo apt-get install php5-cgi
$ sudo lighty-enable-mod fastcgi

Place a php file named info.php file in your server root directory:

<?php phpinfo(); ?>

Now visit http://your.server/info.php and check the log:

2009-08-16 09:34:56: (mod_deflate.c.1232) Content-Type: text/html
2009-08-16 09:34:56: (mod_deflate.c.1239) mime-type: text/html
2009-08-16 09:34:56: (mod_deflate.c.1267) enable compression for  /info.php
... snip ...
2009-08-16 09:34:56: (mod_deflate.c.919)  in: 42054  out: 7057
2009-08-16 09:34:56: (mod_deflate.c.1020) finished uri: /info.php , query:

That's it. If it works, be sure to comment out deflate.debug in lighttpd.conf as it's generating a fairly large amount of entries in your server log.

Caveats

mod_deflate doesn't support caching compressed files in a directory like mod_compress does. This means that every http request matching a configured mime-type will trigger mod_deflate compression. This usually produces higher cpu utilization in favor of lower bandwidth usage. If you experience high cpu load, try lowering the deflate.compression-level.

Bazaar is a nice version control system for me, it allows to keep my project directory versioned without much fiddling with the vcs-savvy stuff.

1. Get updated version of Bazaar

On Ubuntu 8.04 LTS (hardy) the bazaar packages are kind of outdated, but there is a simple solution. The PPA for Bazaar Developers is a Launchpad project but it contains package repositories for all supported Ubuntu systems.

To get the packages jut edit /etc/apt/sources.list.d/sources.list, add the following lines:

deb http://ppa.launchpad.net/bzr/ppa/ubuntu hardy main
deb-src http://ppa.launchpad.net/bzr/ppa/ubuntu hardy main

You can subsitute hardy with your ubuntu version if it's supported. Add the OpenPGP key id for the repository:

$ sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 8C6C1EFD 

Be sure to check the proper key-id of the repository. At the time of writing this is 8C6C1EFD. Note: apt-key needs the gnupg package to do this.

Then update and install the new versions:

$ sudo apt-get update
$ sudo apt-get install bzr bzrtools

2. Get the bzr upload plugin

The bzr-upload plugin incrementally uploads changes to a dumb server. Web sites are often hosted on servers where bzr can't be installed. In other cases, the web site must not give access to its corresponding branch (for security reasons for example). Finally, web hosting providers often provides only ftp access to upload sites. This plugin uploads only the relevant changes since the last upload using ftp or sftp protocols.

The bzr-upload is not available as a package for Hardy. Luckily, installing this is really easy, just type:

$ mkdir -p ~/.bazaar/plugins
$ bzr co lp:bzr-upload ~/.bazaar/plugins/upload

The plugin is instantly available for use after checking out. Now you can use the plugin via:

$ bzr upload sftp://username@server:22/~/sites/my.coolsite.com/
$ bzr upload --auto

See bzr help upload for detailed usage information.