Lighttpd and FCGI for Rails -- To Static or Not?

I’ve noticed a little instability with my setup recently, at present I have Lighty configured to manage the FastCGI processes itself.

The relevant part of my config looks as follows:

fastcgi.server= (".fcgi" =>("oobaloo.co.uk" =>  (    "socket" => "/tmp/oobaloo-lighttpd-fcgi.socket",    "bin-path" =>  "/var/www/servers/www.oobaloo.co.uk/current/public/dispatch.fcgi",    "bin-environment" => ("RAILS_ENV" => "production" ),    "max-load-per-proc" => 4,    "min-procs" => 1,    "max-procs" => 3,    "idle-timeout" => 90  ))

However, I’ve read people suggest that it’s better (when running more than one rails application) to instead use static processes, and there’s a page on the Rails wiki describing this setup.

So, I changed my Lighty config to the following:

fastcgi.server = (".fcgi" =>  ("localhost-7000" =>    ("host" => "127.0.0.1", "port" => 7000,  "bin-environment" => ("RAILS_ENV" => "production"))),  ("localhost-7002" =>    ("host" => "127.0.0.1", "port" => 7002,  "bin-environment" => ("RAILS_ENV" => "production"))),  ("localhost-7001" =>    ("host" => "127.0.0.1", "port" => 7001,  "bin-environment" => ("RAILS_ENV" => "production"))))

and instead started the processes manually using the `spawner` with the following script:

!/bin/sh

/var/www/servers/\
www.oobaloo.co.uk/current/script/process/spinner \
-c ’/var/www/servers/\
www.oobaloo.co.uk/current/script/process/spawner -p 7000 -i 3’ \
-d

However, I noticed that over time new processes were being created to handle `dispatch.fcgi` for the blog. This got to the point with about 6 or 7 processes that would choke the VPS and cause it to become relatively unusable.

Is this normal for other processes to be created? Under what conditions would additional processes be spawned?

Anyway, for the time being I’m back on Lighty managed processes—at least the number of processes remain static.

Excitement with some big news!

Things have been pretty quiet here ony my blog of late, but there has been a reason!

Approximately 3 weeks ago my position at Perfect Information was made redundant and I focused my time at looking for future employment, as well as finishing up a few small side projects I’d been working on.

I’m absolutely overjoyed to say that last Tuesday I was officially offered (and accepted) a developer position with ThoughtWorks UK, somewhere I’ve always had an urge to apply to, but had always been quite scared by.

An old colleague of mine joined and had always had great things to say about the place, so I thought I’d give it a go. Whilst out on holiday I had an email asking me to get in touch to arrange a phone interview. Since then I’ve been through the phone interview, coding test, 2 developer interviews, some more intelligence/aptitude testing, personality tests and a further interview with a senior manager. But I made it. I’m to become part of Roy’s Social Experiment.

I’m thoroughly looking forward to getting started. My first day will be on the 2nd of May, and I’ll hopefully also get the chance to post a little about starting as an experienced hire, and what it’s like to be a ThoughtWorker.

In the meantime I’ll be catching up on some reading, writing, and of course coding. In particular, something really good fun I’m putting together that I hope to have released before then!

Until then, time to open something fizzy and alcoholic!

Upgrading Typo and Rails

Firstly, I needed to get Ruby 1.8.4 compiled to upgrade my 1.8.3 release, as well as compile in support for Readline (since I posted about a problem I’d had with Rails’ console earlier without this):

cd ~/wget ftp://ftp.gnu.org/gnu/readline/readline-5.1.tar.gzwget ftp://ftp.ruby-lang.org/pub/ruby/ruby-1.8.4.tar.gztar zxvf ruby-1.8.4.tar.gztar zxvf readline-5.1.tar.gz

cd readline-5.1./configure --prefix=/usr/localmakesudo make install

cd ../ruby-1.8.4.tar.gz./configure --prefix=/usr/local --enable-pthread\  --with-readline-dir=/usr/localmakesudo make install

That should update your version of Ruby, you can check all went well by running `ruby -v` and checking you see: `ruby 1.8.4 (2005-12-24) [i686-linux]`.

The next step was to get my server up and running with the latest code from Typo’s trunk (and place Rails 1.1 in the vendor directory—I wanted to leave my other applications on 1.0 for the time being).

To do that, I created a separate directory at the same level as my current Typo revision. Within the directory I also have a symlink named `current` that points to the revision I’m currently running from.

So, I ran the following:

cd /var/www/servers/www.oobaloo.co.ukmkdir trunkcd trunksvn co svn://typosphere.org/typo/trunk .

That should pull down the latest revision, I also then renamed the directory to indicate the revision that it corresponds to:

cd ..mv trunk trunk-983

Next, we need to place the Rails 1.1 release inside the Typo application’s `vendor` directory. This is apparently what is likely to be bundled into future Rails releases: everytime you create a new application via `rails myapp` it’ll copy the Rails source over for you.

So, I do the following (inside the `trunk-983` directory):

cd vendormkdir railscd railssvn co http://dev.rubyonrails.org/svn/rails/tags/rel_1-1-0/ .

Again, you’ll see a flash of info from Subversion letting you know it’s all been downloaded successfully.

Finally, I wanted to do the necessary database migrations so that my database schema is up-to-date. To make sure I didn’t lose anything I used `mysqldump` to backup the database beforehand (into a `db` folder I use for all my backups).

cd ../../dbmysqldump typo_production -u typo -p > typo-pre-983.sql

Check that the file was created with all the content, and then you can make the final step (running the Rails migration):

cd trunk-983rake migrate RAILS_ENV=production

You should hopefully see the migration stuff succeed, finally, you can switch the `current` symlink over:

rm currentln -s trunk-983/ current

And you’re off and running!

1..2..3.. you're back in the room!

After a few woes over the past few days all seems well again.

First, my email died—re-sending old emails, which, even after deleting boatloads just seemed to keep filling up my inbox. In an attempt to keep better track of things and provide a far better service I’ve switched to a new (and separate) provider. I’m going to see how it goes for a month and go from there. I ended up choosing one in the UK to provide a snappier IMAP experience.

If that wasn’t enough, the hostserver my VPS is on hit a ‘kernel bug’ earlier this morning and wiped me out for an hour or so, just as well I don’t host my own email!

So, if you’ve been trying to download code, read posts or contact me, that’s why I’d ‘gone dark’, but the LEDs are back on and blinking now!

Mint Follow-up

Firstly, one of my concerns was that I no longer had a way of viewing Pages per Visit, this was a great way of determining whether people navigated around my site having got here. For me this is important as if other people link to my posts, or are referred here through my comments to other posts/pages, they’re interested in what I have covered.

Mint has something similar to this (Total vs. Unique in it’s Pages panel), but the excellent Fresh View pepper let’s me visualise it. I can tell from the spikes (directly correlated to me making a posting (and receiving hits from referrers such as Planet Ruby on Rails and Developer Fusion) that there are more views per visit for that day, than a non-posting day. Leading me to believe that when other people find their way here (by way of Google, for instance) that they stick with the article they found. Perhaps making it easier to find related posts may encourage people to stay?

The brilliant Outclicks pepper also lets me see where people go afterwards, and the page they were on beforehand. It doesn’t really help me target posts better, but it does let me know that people are finding things useful, if not the things I directly write.

lets me know whether I'm doing relatively better, or worse, between two time periods. For me, this is weekly. I know that because I posted more times this week, I've served markedly more requests. A great encouragement to keep posting. However, what's more important is that I can see which posts are retaining popularity through being referenced elsewhere, or found by Google. If I can keep on finding interesting things to post about, perhaps people will stick around.

Overall, it’s feeling much better. I still miss a way of seeing the amount of time a user spends on a particular page. Google Analytics had a great page that could show you a breakdown of pages, based on pages people spent most time at, the average time a user spent on any given page etc. Very useful stuff. Perhaps I should look at the Pepper API and see whether I could do it. Unless someone’s seen something else to do it?

Have a good weekend!

Great Hosted Email Needed

My email seems to be going a little screwy right now (currently hosted by my old ASP.NET provider – CrystalTech). Their webmail has never been that great to use, and anti-spam/virus has also been a little flaky—I ended up just rejecting any emails with any attachments.

So, I’m now looking to find somewhere else to host my emails. I could move it onto my VPS, but I’d rather not have something as essential as email at the will of my tinkering! It’d also be nice to have really, really solid anti-virus and anti-spam protection that I just don’t have time to administer.

Essentially, I’m looking for something along the lines of MessageLabs, but usable (and affordable) for an individual. I also need IMAP (rather than POP3—I access emails all over the place), and a good webmail app would be really nice too. Finally, it has to just be a hosted email service, I don’t really need any webhosting as part of the package.

So, does anyone have any good recommendations?

Ruby and Readline

D’oh. Looks like I can’t use `script/console` just at the moment on my VPS’s install of Ruby, which I seem to remember just installing via my host’s `apt` repository for Red Hat Enterprise 4.

Anyway, here’s the output of the command::

Loading development environment.  /usr/local/lib/ruby/1.8/irb/completion.rb:10:in `require':  no such file to load -- readline (LoadError)    from /usr/local/lib/ruby/1.8/irb/completion.rb:10    from /usr/local/lib/ruby/1.8/irb/init.rb:252:in `load_modules'    from /usr/local/lib/ruby/1.8/irb/init.rb:250:in `load_modules'    from /usr/local/lib/ruby/1.8/irb/init.rb:21:in `setup'    from /usr/local/lib/ruby/1.8/irb.rb:54:in `start'    from /usr/local/bin/irb:13

Do I have to rebuild Ruby for `readline` to be included? Or is anyone aware of any clever hackery to avoid it? I’d just rather not be tinkering with Ruby on my server which would surely kill a few things whilst I do.

Incidentally I need this because I’m trying to get a clever way of migrating data from MySQL to Postgresql and rather than hacking away at SQL statements I thought it’d be much nicer to bend Chad Fowler’s suggestion in Rails Recipes to extract data to YAML from MySQL and then back from YAML to Postgresql and let Ruby and ActiveRecord do the legwork.

New Typo Theme

If you’re viewing this through your web-browser then you will (hopefully) notice the new theme that I’m trialling. I was previously using Hemingway a brilliant theme produced for the original Typo theme contest at TypoGarden.

Although I loved its two column layout, it could become a little difficult to read the home-page, and for the usual sidebar stuff (links to archives, searching and the like) you had to scroll to the bottom of the page, not that great a usability thing.

Then today I noticed Brighthouse on the TypoGarden site. It’s a much lighter, brighter looking theme, and hopefully just as clear and perhaps even a little more usable?

So, to all those who have seen the site both pre and post Hemingway, how do you feel? Think it’s worth the swap? Or am I to expect a great swathe of unapologetic apathy :)

ReSharper 2 is shaping up!

I’ve been using ReSharper with Visual Studio 2003 for probably 15 months or so now, and have found it to be one of the most invaluable tools I use (along with Reflector, CruiseControl.net, NUnit, TestDriven.net, Clover.NET, the list goes on :). But ReSharper just makes the development process nicer.

I can add references and not worry about having to add the using statement myself, ReSharper pops up the red for me automatically. It makes writing code test-first easier, I can write a statement then have ReSharper write the new class/property/method when it recognises it doesn’t already exist. And now, with ReSharper 2 (via. their EAP program) I’ve been using their built-in unit testing runner.

Previous EAP builds have been a little buggy, and the unit test runner has been pretty darn slow, but this seems to have improved with the latest build. At least to the point where I can use it and tolerate the occasional unhandled exception message appear. I just can’t wait for it to be released, which I believe is very soon!

In fact, I find their unit test runner experience to be superior to TDD.net’s, and NUnit GUI’s.

1. It sits in a window giving me a constant green/red bar. With TDD.net it kind of hides away after I run the tests and go back to the code. It’s a good constant reminder of progress. A positive psychological thing.
2. When I then update the code and rebuild, it will colour the tests that existed and ran previously a slightly different colour. Obviously these should all be green, but, its nice to see which tests I’d just run and just added etc.
3. It has a very nice assertion message viewer that is formatted. It’s surprising just how much nicer this is to use. Both NUnit’s GUI and TDD.net just display regular text. It can be difficult to separate messages from the stack trace, and messages can even be hard to read.
4. For `Assert.AreEqual(string, string)` assertions, it’s posssible to get a diff view of the comparison. This would’ve been extremely useful where we’ve previously done stuff in the past to compare generated HTML and templates for mail merges, where its often just an `Environment.NewLine` out, or a space misplaced in a sea of markup.

In fact, it makes me wish that Ruby’s Test::Unit unit test framework had a way of colouring the messages it displayed so my Rails’ `rake test_units` and `test_functional` commands would be easier to see at a glance in the terminal window.

Don’t worry, despite my happiness with a ReSharper + Visual Studio 2005, I still have hankerings to get back to the Ruby on Rails development later tonight and tomorrow! Let the fun begin! I’ve got some great projects lined up, including one or two multiplayer online games! More on those soon!

Oh, and apologies for the lack of regular updates, it won’t happen again. I promise.

How to improve your photos in Photoshop

One of the quickest and simplest ways is to use Photoshop’s Levels through an adjustment layer, allowing you to leave the original image intact. To accomplish this, click the half-black and half-white circle icon in your layer panel (shown on the right here). Then, in the pop-up menu, select “Levels…”. This will pop up an editor which will look something like the shot below.

This window displays the distribution of information from dark to light across the image. Using the sliders across the bottom it’s possible to adjust the dark point, mid point, and white point through the sliders.

So, to improve the contrast and stop the image feeling a little flat, we’ll move the dark slider across (telling Photoshop to consider that the new position is the dark point—making lighter areas appear darker). We’ll also slide the mid-tone slider across to the right to make the whole image appear darker, and move the left slider in slightly to lighten the top-end. The new positions look as in the image to the right.

They’re relatively small changes, but they make quite a large improvement as you’ll see from the adjusted image to the left. Now, although this image looks far more vibrant, and less flat, it also looks markedly red. So, time to adjust things a little more to remove that redness so it looks a little more natural.

To do that, click that black/white semicircle thing again, but this time we'll insert a "Color Balance" adjustment layer in. Once that pops up, we just move the top slider a little to the left until the image looks a little better. We just want to tone the over-bright redness a little.

This leaves us with a final image as follows:

And for comparison to the original:

As you can see, the image is bright and vibrant, the blues, reds, greens and blacks are solid, and it looks much more lively. Overall, much more pleasing. The Levels adjuster is one of the quickest ways I’ve found to help a photo jump out at you, although there are more advanced ways to adjust more precisely—using the Curves adjuster for instance, maybe I’ll try and find an example to show that in the future. My next job is to remove some of the dust and other bits ‘n pieces in the mesh that distract the eye in the background.

Incidentally, I picked up this (and my other retouching ‘skillz’) through Katrin Eismann’s excellent book “Photoshop Retouching & Restoration”. It has some great walkthroughs, including some amazing work with restoring old destroyed photos. I’d thoroughly, thoroughly recommend it to anyone looking to improve their photos with Photoshop!