Howto - Circumvent/Workaround Anonymous user limitation and, tracking first visits in Drupal

This long and detailed post on a lazy Sunday afternoon is something I've wanted to do for a while. It supplies workaround for a number of problems or "security" features in Drupal. When I had to work through the problems outlined here, I was in the "drupal learning curve" so I had to hunt down the info and the sparse documentation on the subject did not help. I hope people find solutions the problems addressed here. If anybody does this in a different manner, let the community know too.

Problem

How to get around issues of anonymous users

There are lots of reasons why Drupal does not store session info for anonymous users. (sources needed)

But there may be reason why you may want to store variables, such as anonymous use of a shopping cart, tracking the user as he navigates your website. Using the techniques outlined in the tutorial you will be able to successfully track anonymous users on your website.

This tutorial also outlines how to get around the problem of storing info based on session, according to Drupal, an anonymous users has very little customization. Even if they are customized you can carry over anonymous setting to a user, when they create an account or log in. There is a technique later on in this tutorial explaining how to work around this problem too.

Here are the problems in drupal.
1. Drupal does not establish persistence sessions for anonymous users. Meaning, you can store $_SESSION data, but it will get reset when the user logs in, see problem #3.
2. A cookie is not set until the second visit.
3. Drupal resets the session id, every time a user logs in.

Solutions

The first issue we can take care of by saving session data in a table based on the session id, or saving in the $_SESSION variable.

The second and third issues issues is a bit more complicated.

The first thing you need to do is create a new module. I call mine "freshuser" module, because of the system that I'm working on, but you can name is whatever you want.

You will need to create 3 files. freshuser.info, freshuser.module, freshuser.install

freshuser.info

name = Fresh User
description = This module provides a user management and user integration with drupal
core = 6.x
package = FreshSuite
version = "1.0"

freshuser.install

You need to create two tables in the freshuser.install file

function freshuser_schema(){
	$schema['freshuser_firstvisit'] = array(
	'description' => t('Drupal doesn keep track of the users first visit. This table assists in the workaround.'),
    'fields' => array(
      'id' => array(
        'description' => t('The unique identifier for the first visit tracking of the user.'),
        'type' => 'serial',
        'unsigned' => TRUE,
        'not null' => TRUE),
      'timestamp' => array(
        'description' => t('The timestamp when the user made his first visit.'),
        'type' => 'int',
       	'size' => 'normal',
        'not null' => FALSE),
      'sid' => array(
        'description' => t('Session id we gave on the first visit.'),
        'type' => 'varchar',
		'length' => 64,
        'not null' => FALSE),
      'http_referrer' => array(
        'description' => t('The HTTP Referrer of the first visit.'),
        'type' => 'text',
        'size' => 'normal',
        'not null' => TRUE),
	  'landing_page' => array(
        'description' => t('The landing page that the user first came to the site with.'),
        'type' => 'text',
        'size' => 'normal',
        'not null' => TRUE),
      ),
    'primary key' => array('id'),
    );
    $schema['freshuser_map_sid_to_uid'] = array(
	'description' => t('This table maps session IDs to Drupal User IDs.'),
    'fields' => array(
      'uid' => array(
        'description' => t('The drupal user ID assigned to the user.'),
        'type' => 'int',
       	'size' => 'normal',
        'not null' => FALSE),
      'sid' => array(
        'description' => t('Session id given to the user.'),
        'type' => 'varchar',
		'length' => 64),
      ),
    );
   return $schema;
}

function freshuser_install(){
	// Create my tables.
	drupal_install_schema('freshuser');
}

function freshuser_uninstall(){
	drupal_uninstall_schema('freshuser');
}

This schema created two tables
1. freshuser_firstvisit
2. freshsuite_map_sid_to_uid

The purpose of the first table "freshuser_firstvisit" is to track if the anonymous is visiting the site for the first or second time.

The second table "freshuser_map_sid_to_uid" maps the session_id's to users so you can match the data once an anonymous user creates an account. This is not really necessary but illustrates how the workaround works. Read on for more explanation later.

Tracking anonymous users in Drupal

A prerequisite.

If you want to track anonymous users in Drupal, you have to prepare for it. This is whether you want to create shopping carts, let the the user tryout the functionality of you website, and you want persistent data. Let me explain, you shouldn't use $_SESSION to store cart information or any other information about a user. It will be destroyed once the user creates a new account or logs in. I recommend you create your own table that stores the information based on the session id of the user. The problem with this is #1 and #3 listed above, in that drupal resets the session id every time the user logs in or out.

To prepare, I do the following: In my user based tables I have two fields to track the user, "user_type" and "user_value". User type will specify 1=anonymous 2= logged in. then I will know what type of data value is in "user_value",

in pseudo code

if (user_type==1[anonymous]) 
then 
    user_value=session_id, // 32 byte session id
elseif  (user_type==2[loggedin]) 
then 
    user_value=user_id // will be integer

Thus I know if the the record in the database refers to an anonymous user or a logged-in user. (based on the value in "user_type")

freshuser.module

In the freshuser.module file I create the following function

function bootstrap(){

}

I want to run this function as soon a possible, I know there is a hook_init or something but for now I just put it like this.

function bootstrap(){

}
bootstrap()

so the function gets called when the file is parsed.

The first order of business is detecting if the user is logged in.

function bootstrap(){
	global $user; // get the global $user variable
	if($user->uid){
		// user is logged in, there is no need for any cirumventing
		if(!isset($_SESSION['tracked'])){
			$_SESSION['tracked'] = true; // set it for the sake of consistency
		}
		return true;
	}
}
bootstrap();

If the user is logged in ($user->uid !=0) we set a custom variable in the session to tag him as tracked. "Tracked" means they've been through the first two visits to the website. So the reason for a $_SESSION['tracked'] is that we want to know who has been tracked and who wasn't. If the user is logged in then we don't need to workaround anything.

function bootstrap(){
	global $user; // get the global $user variable
	if($user->uid){
		// user is logged in, there is no need for any cirumventing
		if(!isset($_SESSION['tracked'])){
			$_SESSION['tracked'] = true; // set it for the sake of consistentcy
		}
		return true;
	}
	if(isset($_SESSION['tracked']) && $_SESSION['tracked']){
		// user has been successfully tracked and we don't really have to do anything
		return true;
	} 
}
bootstrap();

Any user who has been successfully tracked through the first two visit (which we will discuss momentarily) or is logged in does not need anything done to circumvent problem #2 listed above.

If the user is not logged in or tracked, we need to determine if this is the first or second visit. The cookie is not set until the second visit. Since there maybe some functionality you may want to do in on the user's first visit. Such as, track ads, or email yourself if GoogleBot is indexing you site. We do the following:

function bootstrap(){
	global $user;
	if($user->uid){
		// user is logged in, there is no need for any cirumventing
		if(!isset($_SESSION['tracked'])){
			$_SESSION['tracked'] = true; // set it for the sake of 

consistentcy
		}
		return true;
	}
	if(isset($_SESSION['tracked']) && $_SESSION['tracked']){
		// user has been successfully tracked and we don't really have to do anything
		return true;
	} else {
		// this can be the first or second visit.
		// lets determine which one

		$sid = session_id(); // get the current session id
		if(!db_fetch_array(db_query("SELECT * FROM {freshuser_firstvisit} WHERE sid = '%s'", $sid))){
                        // this is the 1st visit - no record
			db_query("INSERT INTO {freshuser_firstvisit} SET sid='%s', http_referrer='%s', timestamp=%d, landing_page='%s'", $sid, $_SERVER["HTTP_REFERER"], time(), $_SERVER["HTTP_HOST"] . $_SERVER["REQUEST_URI"] . $_SERVER["QUERY_STRING"]); // add record to db track user landing page etc.
		} else {
			// this is the 2nd visit since a record already exists
			$_SESSION['tracked'] = true;
                        $_SESSION['o_sid'] = $sid;
		}
	}
	return true;
}
bootstrap();

Breakdown:

$sid = session_id(); // get the current session id
if(!db_fetch_array(db_query("SELECT * FROM {freshuser_firstvisit} WHERE sid = '%s'", $sid))){

the "if" statement will tell us if there is a record in the database for the session_id. If there is no record, that means the user is on his 1st visit and we will then add a record to the "freshuser_firstvisit" table.

db_query("INSERT INTO {freshuser_firstvisit} SET sid='%s', http_referrer='%s', timestamp=%d, landing_page='%s'", $sid, $_SERVER["HTTP_REFERER"], time(), $_SERVER["HTTP_HOST"] . $_SERVER["REQUEST_URI"] . $_SERVER["QUERY_STRING"]); // add record to db track user landing page etc.

I have added some information, basically, when the first visit occurred, the landing page, and the HTTP_REFERER. this can be valuable for adwords tracking etc. In this space you can do what every you want, and it will only be executed on th first visit of an anonymous user. (this does not take into account authenticated users coming from another site. They will need to be taken care of separately.)

You must create a record for the session id. you are free to add any other field. I'm just using the above info as an example of what you can do, that you wouldn't be able to do otherwise.

We need to create the record because we can't store any values in the $_SESSION variable on the first visit. We need another way of determining if the visit is the first or another. If there is a record on the database we know that the user has already been to the website.

} else {
// this is the 2nd visit since a record already exists
$_SESSION['tracked'] = true;
$_SESSION['o_sid'] = $sid;
}

After the user has visited the site at least once we can then set the variables.

$_SESSION['tracked'] = true; is set so we don't have to keep querying the database on every visit. The database only needs to be queried on the first and second visits.

We also set the $_SESSION['o_sid'] value to the current session_id. The reason will become clear soon. Mainly to solve problem #3 listed above.

Solving Problem #3

Now we have a problem of associating the session data to a user when he creates an account or logs in. Say, for example, you have a shopping cart, and it's in a table will all the data for an anonymous users. The problem is that it is based on a session id. Once the user logs in, the session id is reset. So your tracking data is worthless, because the user has a new session id.

For this we create the following function is actually implemetn hook_user. It also creates a custom hook.

function freshuser_user($op, &$edit, &$account, $category = NULL){
	global $user;
	$sid = session_id();
	
	if($op == "login"){
		if(isset($_SESSION["o_sid"]) && $_SESSION["o_sid"]!=""){
			// user just logged in so we have to account for the change in session id and notify all module that depend on session id's
			$sid = session_id();
			$o_sid = $_SESSION["o_sid"];
			module_invoke_all("user_session_id_to_user_id", $o_sid, $user->uid, $sid); // invokes the custom hook
			$_SESSION["o_sid"] = $sid;
		}
	}
}

What this hook does is anytime a user logs in it checks for a previous session id and then calls it new hook called hook_user_session_id_to_user_id with three variables, the old session id, the user id, and the new session id. This allows any module that implements this hook to track changes in the session ids.

Sample hook implementation

For the last function I have an implementation of the "hook_user_session_id_to_user_id", yeah I know its a long name, but you can change it to whatever you want.

function freshuser_user_session_id_to_user_id($o_sid, $uid, $sid){
	// we need to log the change in session id to keep record for future use.
        // code to create record that matches the $o_sid to the $uid
        // code to create record the matches the $sid to the $uid
        
}

Now just enable the module in the administration panel.

IMPLEMENTATION

To take advantage of all the hard work you just did. We should go back to what I said earlier regarding preparing you code and tables for anonymous user data.

From here on I will outline briefly how to create a shopping cart that can use the module we created above.

When I created my shopping cart I store the cart infomation in table and not in the session. For a number of reasons.
1. Doesn't clog up the database sessions table.
2. Allows for faster load times. I only access the cart when I need to, and doesn't get loaded on every page.
3. I can perform statistical analysis on the cart on a later date. Like, time intervals between adding an item to when they check out. Or, what item's are most frequently added to the cart. There are hundreds of variable to extract.
4. Allows for saving of the cart for later.
5. User can be loggedin, on mutliple computers and have his cart.

To take advantage of the freshuser module above a shopping cart should create the following additional fields to a table to allow anonymous shopping.

1. user_type
2. user_variable

when saving the cart to a table, and the user is anonymous, you tag it with the session id. user_type = '1' // for anonymous
and user_value = 'session_id()'

to prevent the loss of the identifier [session id] when a user logs in, (because the session id is reset) you implement the "hook_user_session_id_to_user_id".

So when the user logs in and the session id changes you will implement the "hook_user_session_id_to_user_id" hook and assign the user_id to the card instead of the session_id.

function yourmodule_user_session_id_to_user_id($o_sid, $uid, $sid){
	// find the cart record based on the $o_sid value.
	SQL = WHERE user_type='1' and user_value = $o_sid,

	// and then update the record to the user_id
	SQL = UPDATE .... SET user_type = '2' and user_value = $uid;
}

Cautionary Note

These technique workaround some important security features of Drupal. You are responsible to take care of the security.

Update: Fixed the $schema as noted below