Hello again!
A client has asked for an upgrade that I think will fit nicely into your module.
We're going to be keeping the:
- Ip
- Uid
- Path
- Timestamp
- All the other geolocation data
The plan is to database the data every time the geolocation happens, or if the geolocation is more than a variable amount of hours old. So for instance a fresh record every 24 hours. We'll be keeping that data in the table for a variable amount of type, wiping it out using cron after a variable amount of days. So, say, wiping out after 1 year.
The other piece going along with that would be a statistics table that keeps total hits broken down to the zip code and month level that should be easy to maintain forever.
Want me to write that in for you again?
Comments
Comment #1
lance.gliser commentedI noticed another issue talking about exposing to views. I think this data structure would be pretty handy for that too.
Comment #2
lance.gliser commentedI was thinking about the reporting side of it, so I have more to add to the plan.
I'd like to have two views, one of the historical, one of the recent. Both sitting at admin/reports/smart-ip. I'm figuring on exposing the data of the tables to views, creating a couple default views with exposed filters, and embedding them.
A nice addition, not slated for version one of this new feature would be to use the proximity filter from location module if it's installed to enable searching in a radius from a given zip for X month.
Comment #3
arpeggio commentedHello,
Yes, the suggested features are welcome to Smart IP module. The features you are suggesting are:
Those features sounds good. Thanks.
Comment #4
lance.gliser commentedThought you'd like an update on progress. I've been a bit detained, but making decent progress. The schema, and data is being stored on my local copy. Now I need to rig up the views exposure and default views to use for reporting.
Comment #5
arpeggio commentedThank you for the update. Reminder: please make sure that the additional features are modular, so that the user have option to enable/disable them. Thanks!
Comment #6
lance.gliser commentedThe heavy lifting of the statistics options is already optional, and in a separate include file. I did add one additional setting to the base options though. In order to have meaningful history, I had to accommodate for sessions that might last upwards of weeks, which would look like there was a lack of activity. To counter this, I've introduced an option to force a fresh geocode on a visitor, if the geocode data in their session is more than X hours old.
Comment #7
arpeggio commentedThanks lance.gliser. You're making a good progress.
Comment #8
lance.gliser commentedAlright, round 1 of the feature ready for testing. I added a dependency on views for the reporting end. Check it out!
Comment #9
lance.gliser commentedAn additional piece should be added, but I'm a bit short on time at the moment. There's a framework in place for whoever would have time to do it. We could use the Location module's proximity filter.
Bah. I think I also forgot to remove a drupal_set_message() that warns about each geolocation log. Might check that while you're reviewing the rest.
Comment #10
arpeggio commentedThat's great! Please give me time, I'm catching a deadline. Thank you.
Comment #11
arpeggio commentedI made a quick review of your patch. Here are my few comments:
- Why do we need to remove the variable_set('smart_ip_extract_zip_done', FALSE); in smart_ip.install?
- Please avoid escaping single quotes:
Can I ask for the source code? I'm getting error using my patch application.
Comment #12
lance.gliser commentedThat's odd. That line you mentioned "variable_set('smart_ip_extract_zip_done', FALSE);" certainly does not need removed. I must have deleted it by accident at some point.
I'll try to keep the translation strings in mind, just have a habit of single quoting strings, thank you.
I'm attaching a zip for you of the entire project with alterations, including commenting out the statistics drupal_set_message() I warned about, and wrapping the view's labels in t().
Comment #13
arpeggio commentedI have used your demo application, its cool! However, it seems it captures the history of one user only, the admin (maybe because it's still in development stage). I noticed that this will be implemented as sub module, that's great! we can easily ported this feature to Drupal 7. Looking forward for the completion of this application. Thanks.
Comment #14
lance.gliser commentedNow you've slightly confused me.
The package I attached should have the statistics included right into the main module. It could be taken into a sub module, but I didn't develop it that way.
I'm also not having issues tracking other users. I've got hits from anonymous and other users as well. It might be that you're not seeing others, because it's not giving you a fresh geoencode until after the expiration timestamp. If you log in as admin, and it clocks you, it'll be X (24 by default) hours, until it geocodes you again to force a statistic update.
Perhaps an action should be taken at login to force a fresh geocode? It's not a huge hit on statistical accuracy. Whichever you feel more meaningful.
Comment #15
arpeggio commentedI'm sorry to confuse you. Yes, I understand that statistics is directly integrated into the main module, what I mean is that I saw a folder under modules directory of Smart IP with template files for creating new module which I assume that the statistics application will be created as sub module of Smart IP (which is the right move).
In my point of view, when our users start to test the statistics application they will expect immediately logs from their dummy visitors and some will think its not working with other users. If you designed that way because you think forcing a fresh geocode at login will affect the site performance, in my opinion its better to leave it that way or take no action at login. It will be unnoticeable in the long run.
Comment #16
lance.gliser commentedSo in terms of actions you need to commit it to the module the only remaining item is:
Required:
Create statistics as a sub module that can be optionally enabled, and brings dependancy on views.
Documentation on how statistical hits are created.
Anything else required?
Comment #17
arpeggio commentedI suggest adding in the documentation the simple installation and usage steps also. After this statistics application is completed future task for this:
- Port to Drupal 7
- Add IP lookup capabilities to the watchdog module (as in Smart IP issue: http://drupal.org/node/1047824). I think it is more appropriate to integrate this to your statistics application, what do you think?
Comment #18
lance.gliser commentedIt shouldn't be hard to add that ip lookup to the detail statistic view. It's a kind of "field" that views could produce fairly easily.
It will take me at least a few days to get these things handled.
I do not currently have D7 site. Will you be doing the port, or should I plan on doing that as well?
Comment #19
arpeggio commentedIf you’re available, why not? =) This is an easy task for the one who author the application and I can help in code review/testing… However, I can also do the porting but I will need your assistance because this application is huge compared to your last contribution.
Comment #20
lance.gliser commentedAlright, here's round two for you.
A few changes are required in the main module still. I need the ability for people set the decay time on geolocations coming from your module so we're sure it's set how they like. Also moved the geolocation firing into a single function to fire, a bit simpler to call. I made sure every login and out also fires a fresh geolocation to clear up the behavior you saw.
Otherwise the functionality has been moved into a sub module, that brings in the views dependency. I whipped up the custom field handler you needed to do whois lookups in the detailed view was well.
This will not play nicely with the previous version of the code if you have it still installed. You'll need to removed the two statistical database tables from the database manually, as they have changed which schema owns them during transition.
It's still lacking some documentation beyond the field descriptions, I commented the help section out until we decide on some clean language.
I don't think I can afford time for the D7 port right now. I'm already doing this one off client hours because it's become somewhat of my pet project.
Have a go at it!
Comment #21
arpeggio commentedCool! I have tested the latest patch, it now display the latest log without waiting for the timestamp to expire. Now, I am currently reviewing the code...
I have question, why do we need to swap:
to:
Comment #22
lance.gliser commentedIt's a bug I found in the code while doing all the revisions. It seems they are backwards in your latest dev version. You would only want to set the user data, if the account being passed in is the same as the user being acted against. I thought about it a bit, and I'm not entirely sure how another user would even get passed to the function, but I wanted to cover the possibility.
Comment #23
arpeggio commentedI see, I think that would be very rare or may not be possible to get other user's location stored to another user's data.
I noticed that you simplified the:
to:
Using the simplified version would introduce bug when using the debug mode. Do we have concern if we still use the former?
At http://localhost/admin/reports/smart_ip_statistics/detailed
the filter field labeled with "And:" is confusing to the user... How does that field work?
I made some modification on statistics applcation. These modifications are just corrections, to comply with Drupal coding convetion. Like spaces in control statements, placement of commas. Please review the diff file I created.
* Just a reminder of the documentation.
Comment #24
lance.gliser commentedI didn't recall making that change to the set_session_data function, so I downloaded a fresh copy of smart IP just now. V6.x-1.2-alpha1, the current stable release I patched against, includes the bugged function lines you mention. I suppose it would be wise to get that updated while we patch in the rest.
The detailed view's exposed filter you mentioned "And:" is a default way of handling searching date ranges for views. I didn't create that one, just exposed it. We probably could stand to update the default view to say "Time Between:" so the "And:" makes more sense. The label that would need to be altered is in: /modules/smart_ip_statistics/views/smart_ip_statistics.default_views.inc on line 391:
'label' => t('Time'), would become 'label' => t('Time Between'),
Comment #25
arpeggio commentedActually the attachment at #23 is already the latest version. I’ll apply the suggested label for your statistics views.
Comment #26
lance.gliser commentedI'm a bit lost arpeggio. Were there more changes, or is this patch considered committed and issue fixed?
Comment #27
arpeggio commentedlance.gliser, I've sent you message in your contact form, did you received it?
Comment #28
lance.gliser commentedI did not. It seems I had an incorrect email in my communications tab, even though it let me confirm it... Very odd.
Comment #29
arpeggio commentedIs it ok if you share your email?
Comment #30
lance.gliser commentedMy contact information should be correct. I updated it. But I sent along my information to you just in case.
Comment #31
arpeggio commentedOk thanks. Already sent you a message.
Comment #32
arpeggio commentedThe attachment is the latest version of Smart IP with statistics application patched. Please have a quick review then commit it to CVS.If you don’t have CVS account please apply now. You may want to use this thread for your CVS application.
Comment #33
lance.gliser commentedI spent a bit of time going through the results of the Coder module's review. I'll do a bit more cleanup, and should bet it applied sometime this week.
Comment #34
arpeggio commentedOk. Are you using the source code at comment #32 for your cleanup? For me that source code is ready for your CVS commit and if there are corrections found then it is strongly recommended be applied to that source code.
Comment #35
lance.gliser commentedActually I fell ill last week. I applied for a csv account, and am still waiting to hear back.
Comment #36
arpeggio commentedI hope you're fine now. I checked your CVS application and its status is "postponed (maintainer needs more info)". My suggestion is to follow up and mark your application as active (It seems you've already stated the needed info).
Comment #37
lance.gliser commentedThank you for the tip off of the required status change, I've updated my request.
Comment #38
birdsnbees commentedSubscribing.
Comment #39
lance.gliser commentedFor those curious, still waiting on approvals. Went through the process and haven't heard back.
Comment #40
socialnicheguru commentedhas this been incorporated into the module?
Comment #41
arpeggio commentedNot yet, lance.gliser is responsible for this task.
Comment #42
lance.gliser commentedarpeggio, they haven't gotten back to me. I welcome you to commit this one yourself. My time has become a bit more invested lately anyway. Please take what you have, and enjoy. It's been a pleasure working with you.
Comment #43
arpeggio commentedlance.gliser, I see maybe a follow up can get their attention to review your pending request. Same here. Thanks =)
Comment #44
socialnicheguru commentedis this in d7?
Comment #45
arpeggio commentedNo, this is in D6 only. For D7, you may want consider IP Geolocation Views and maps.
Comment #46
lance.gliser commentedComment #47
heddnDrupal 6 is no longer supported