Each year, thousands of people around Ireland join Concern Worldwide in the Concern Fast. During the Fast, people give up food for 24 hours and raise as much money as they can for some of the world’s poorest people. One aspect of this years campaign is an online element, where registered fasters can get their own online profile page. Through this, they are able to take sponsorships and return any money that they have raised offline directly through the site.

Kirkdesigns built the Concern Fast website with Drupal 6, CiviCRM, a rather large handful of community contributed modules and of course, a selection of custom modules to facilitate the specific needs of the campaign.

The main site

Concern Fast main navigationThe homepage is built using a custom Panels 3 layout with a couple of dedicated content types that are used to build nodes to drop into the various panel areas. The menu on the right is built from a standard Drupal menu, with customised templates that allow each menu item to be styled with an icon as well as allowing the description for each menu item to be displayed under the name.

Landing pages

Most of the other landing pages are built from standard Panels layouts. The Menu Node Views module was used to enable Drupal Menu items to be selected and used within Views. This made it possible build a view out of a Drupal menu that could be output as a grid and dropped it into a panel. This makes maintenance of the landing pages fairly simple, as the content is primarily driven from the Drupal menu system.

The blog

The blog is a public Organic Group of which several of the Concern content team are members. This allows them to log in and post new blog articles as they please. Having the blog built as an Organic Group meant that user permissions could be fine grained within the context of the blog.

Support site

Organic groups was also used for the campaign's support team. This gives them a central place where they can access documentation (built using a Drupal Book) as well as providing a user search interface (built with an exposed views widget), and a CiviCRM contact search widget. It also contains a number of quick links to administration items that they frequently use (again, another standard Drupal menu). The group is a closed, private group, meaning that membership is handed out by the group administrator only, and that the content of the group is inaccessible to the general public.

Other pages

Other general 'content' pages are simple static nodes with a CCK ImageField and an ImageCache preset to give the images rounded corners and a consistent look. The user search facility was built with Views and an exposed filter widget and of course, some custom theming.

User profiles

Concern Fast donation barometerUser profiles are a major part of the Concern Fast website and they have been customised quite considerably. Again, Panels was used to replace the standard user profile pages and provide an advanced means of laying them out as desired. Several custom Panels plugins were developed to hook into various parts of CiviCRM and make its information available on user profile pages. For example, the donation barometer uses the CiviCRM public API to get details of sponsorships for each user. Likewise, the Sponsor Comments section is another custom Panels plugin that gets its data from CiviCRM.

The registration process

The user registration process has also been customised somewhat to provide a smooth end user experience and better integration with CiviCRM than you get out of the box.
Logintoboggan allows users to log in using their email address as well as user name, as well as allowing users to select their password upon registration, whilst still requiring them to validate their email address before gaining full access to the site. Since users were to be able to login with their email address, it was decided that usernames would be automatically generated upon registration based on the user's name. This was done in a custom module using hook_nodeapi.

The activation process

A 'returning user activation' element was also built for this campaign, allowing Concern to send out personalised emails and letters providing previous year's fasters at the start of the campaign, containing a unique activation URL from which they could reactivate their account from previous years. Essentially, this meant importing several thousand existing contact records into CiviCRM, generating and reserving a username for each one, and building a multistep account activation form where users could confirm (and alter if necessary) their personal details and set up a password for their new account.

Profile setup wizard

The Profile Setup module is used to provide a 'profile setup wizard' to newly registered users. On confirming their email address they are taken directly to a multi-step form, guiding them through the process of customising their user profile. This includes setting their own personal greeting, their donation target and profile picture. Most of this information is stored in a Content Profile node, including the user profile picture which is a CCK ImageField with a CCK ImageField Crop widget to allow users to crop their profile picture whilst uploading. A custom snippet adapted from here copies the image from the CCK field to Drupal's native user profile picture, allowing it to be used where and when a standard profile picture can be. Unique Avatar ensure that profile pictures are not still cached once they have been changed by the user and ImageCache Profiles allows the profile pictures to be displayed using ImageCache profiles - rounding the corners and presenting them in a consistent fashion across user profiles.

Data synchronisation

All of the above was actually small fry in comparison to the work that went into the backend. Concern are a very well established company, with a complicated infrastructure and well defined internal processes and needs. The website would be little use to them as an entity by itself and integration was needed with other internal systems.

Since CiviCRM was being used to capture user data as well as handling donations and sponsorships, this data needed to be moved from there to other internal systems. CiviCRM provides a fairly decent hook specification, making it possible to hook in at appropriate points, grab the data and send it via a SOAP web service to the required internal systems. This is done during the registration process, whenever a user updates their contact details, and whenever a donation or sponsorship is made through the site.

Not only is the CiviCRM data synchronised to another system, but the synchronisation process is actually a two way process, allowing the data to be modified on the external system (which their telemarketing team use as their interface to the data) and have any changes used to update the data directly in CiviCRM. This means that user data can be edited at either end and still be kept consistent. Services were also built allowing the external system to trigger certain events on the website such as the sending of a password reset email, or resending a welcome email.

Modules Used

Almost 150 modules were used in the construction of the site. The most important of these are listed below:

  • CCK - for building the custom content types.
  • ImageField, FileField, Image API and ImageCache - for adding images, documents and other resources to the site.
  • Views - used to build numerous content listings and layouts throughout the site.
  • Panels 3 - used to build a number of non-standard layouts, including the front page and other landing pages.
  • Organic Groups - to build a blog which multiple people could contribute to, as well as a private area for members of the campaign's support team.
  • CiviCRM - to store customer data and process donations and sponsorships.
  • SOAP Client - to synchronise data with other internal systems.
  • Webform - for building custom order and contact forms.

Other modules include Pathauto, WYSIWYG, AddThis, Better Formats, Google Analytics, XML sitemap, Secure Pages, URL Alter, Menu Node Views, Unique Avatar, Taxonomy Image, Page Route, Nodewords, Custom Breadcrumbs, Tagadelic, reCAPTCHA, Rules, Remember Me...

Bugs and patches

As with any Drupal project, building the actual site was only half the story. Most of the time was spent researching, bug fixing, and developing custom code. For reference, here is a list of Drupal bugs specific to this project (My full bug tracker on drupal.org is now at around 30 pages – that is about 750 individual bugs!) that I either reported, patched or helped get patched:

And, the same goes for CiviCRM – which also has a very active and responsive developer community. Big thanks to Donalad Lobo (dlobo on #civicrm) who always seemed to have the time to answer my questions:

Performance

The website runs on a dedicated server with 8GB ram and is powered by NginX and fastcgi – much better than Apache/mod_php. The database also runs on a dedicated server and both machines are connected with a fast Ethernet connection.

The standard Drupal database cache was replace with memcached using the Memcache API and Integration module. This provides a substantial performance boost to the site for anonymous users and greatly reduces the load on the Database. APC was also used as a PHP op-code cache.

Whilst working on the performance of the site, I used ab on yet another machine, connected directly to the web server via fast Ethernet to perform benchmarks for each step of the process. A summary of the results, which can be seen below, clearly show the improvements gained for each step of the process.

no caching
Concurrency: 2
Throughput: 1.4 req/sec
95% response time: 1845ms

Drupal Page caching (database)
Concurrency: 5
Throughput: 22 req/sec
95% response time: 308ms

Drupal Page caching + memcache
Concurrency: 10
Throughput: 29 req/sec
95% response time: 475ms

Drupal Page caching + memcache + APC
Concurrency: 50
Throughput: 95 req/sec
95% response time: 418ms

By using a combination of Memcache and APC, the overall responsiveness of the site was dramatically improved, as was its capacity to perform under a heavy load.

Drupal's database logging was also replaced with logging via the Syslog which also helped to greatly reduce the number of database queries per page, whist still providing much needed logging. PhpLogCon is used to parse, display, search and filter the syslog logs, which is far superior to Drupal's internal watchdog anyway. And, a single CSS sprite was used for most of the icons on the site, greatly reducing the number of http requests per page. CSS aggregation, JavaScript aggregation and JavaScript minification (using Javascript Aggregator) were also employed to help improve load times.

Tip: If you want to improve the performance of your sites, join the High Performance group on drupal.org. There is wealth of useful information, and plenty of experts ready to answer your questions.

Summary

Another client site added to the Kirkdesigns portfolio, a happy client, and a pat on the back for Drupal, which continues to prove itself as a world leading CMS solution. It's extensibility, flexibility, huge user base and active community make it suitable for almost every imaginable web based application . As always, I learned a huge amount on the way, most of which can of course be applied to future projects.

About Kirkdesigns

Kirkdesigns is the company name for Tom Kirkpatrick, a freelance web developer from Oxford, UK – currently living in Dublin, Ireland. I hope you enjoyed reading this case study.

Comments

Very nice design, prominent easy flowing interface, very impressive.

That's a very impressive project, Tom and thanks for sharing the write up.

Quick question, if you don't mind...What were the main factors on selecting memcache over the drupal db cache?

cheers

dub

Hi Dublin Drupaller,
Drupal's database caching can put quite a considerable load on the databas since there are a huge number of calls to cache_set and cache_get for every single page request. Using the devel module clearly highlighted that it was these queries that were taking the time. Results from lookups to the cache table can contain a fair amount of data (fully built html pages) and with the setup for this site, the database server is hosted on a separate machine, even with a 100mbps network link between the two machines network latency begins to build up and become an issue. Using Memcache as the cache backend massively reduces the load on the database and improves the overall performance of the application in return.

If you take another look at the article, I have updated the performance section to also include a summary of the benchmarks that I performed on the site whist working on the performance. I think the results speak for themselves.

--
Tom
www.systemseed.com - drupal development. drupal training. drupal support.

thanks for the detailed answer Tom. very informative and very impressive benchmarks.

cheers

dub

nice to read all what you have been upto :) wow, does seem really complex

its great to have folks like you in the community who are willing to step up and addresss some of the issues that software has. CiviCRM has gotten significantly better with your input :)

lobo

Yes, this is a great case study. It clearly outlines the complex solution put in place for this site. with thanks.

Very impressive. Which theme do you use for this project?

Hi Igal,
I built the theme for this site, using the excellent zen theme as a starting point - as I do for all the custom themes that I build. The zen theme framework is well thought out, well tested, and provides an excellent 'base' theme that can be easily customised.

--
Tom
www.systemseed.com - drupal development. drupal training. drupal support.

2-way synchronization with CiviCRM! Holy sh*t! That's a lot of work. Especially for a one-off project. With CiviCRM's pre hooks it's doable on this side, but only if you have a similar pre hook on the other side. And the performance impact is significant.

________________________
dave hansen-lange
Technical Lead
Advomatic.com
Great White North office
Canada

You mention customizing the profiles but the profiles visible to anonymous users are just a name and photo. Do you have a screenshot of the more customized ones? Also, I thought Profile Setup only worked with core profile? Did you get it working with Content Profile?

Michelle

Hi Michelle, I should have included a screenshot of a profile page in the article, and unfortunately now that it has been promoted to the drupal.org homepage I can not edit it again. However, point your browser to http://www.concernfast.org/niallomurchu for an example of a faster profile page.

In answer to your question, with a little bit of fiddling, I did indeed manage to get Profile Setup working with content profile. I think all the patches have been committed now so it should be working with the latest release of profile_setup.

--
Tom
www.systemseed.com - drupal development. drupal training. drupal support.

yep, looks like it does: http://drupal.org/cvs?commit=274070

Nice work on the site-- and nice contribution!

_
Don't be a Help Vampire - read and abide the forum guidelines.
If you find my assistance useful, please pay it forward to your fellow drupalers.

Ah, yeah, that one looks quite a bit different.

Great that you've gotten the patches committed. I haven't looked at PS for quite a while. Will have to give it another look, now!

Michelle

There is a great tips to all the viewers that if u doing any work fast u read that case and step forward to that work.

Kirk

This is impressive. Thanks for sharing all the lessons.

Did you write a custom template for the menu?

Best regards

Sridhar

Acer Aspire 5745
[i5 430M, 3GB, 320GB]
Ubuntu 12.04 (Precise Pangolin)
Drupal 6.15, 7.x
Aware Hosting, DigitalOcean, Go Daddy, LimeDomains, Rackspace, Tata Instacompute