Implement face-detection

For reference:

Face detection in pure PHP (without OpenCV)
http://svay.com/blog/index/post/2009/06/19/Face-detection-in-pure-PHP-(without-OpenCV)

I think this method only works on faces you can see fully, not half or 3/4 profiles.

Would it be possible to add face detection as an optional component? Then people who can set up their servers properly would be able to take advantage of it.

See the recently released smartcrop module! Looks promising, and doesn't require 3rd party libraries. Sorry, posted this in the wrong place.

#2 sounds like a good idea!

I agree that face detection would ba an awesome addition ... and I dont think that the need for a php extension should prevent you from moving forward... even if only 1/3 of sites using image cache have the technical ability to add a php extension, that means that's 47,000+ sites could make use of this.

That's a lot...

Well, even I gave my opinion in #4, it didn't look easy to install this PHP extension. I tried to install OpenCV, then install the PHP extension but I didn't succeed :( Google didn't help. I don't know if there is anyone who recently uses OpenCV in PHP. It is however a bit off topic.

I agree with #2, very useful feature for the "haves" out there. I support implementation and perhaps disabling the option when support isnt available.

just ran across this:
http://svay.com/blog/index/post/2009/06/19/Face-detection-in-pure-PHP-(without-OpenCV)

seems promising

love to sub

face tagging
as i don't know more about php or js, but i can explain others, how they can make a module for this kind of feature.
Since there are many modules for serving same kind of functionality. but neither one for this purpose, i know many of people are looking for this feature. so if any one who knows programming, i can tell how he could make this.

A. http://code.google.com/p/jquery-image-annotate/

demo : http://www.flipbit.co.uk/jquery-image-annotation.html

B. http://www.fotonotes.net/

with this two tools you can tag faces on image but for user references functionality you must have to use user references field or Tag User module.

AKA : write a module by using one of tool, that i mentioned above & then integrate it with user references field or Tag User module. ]

Thanx.

Sub

Using an external service can elevate server configuration (OpenCV with PHP is something not easy to install)
http://developers.face.com/docs/api/faces-detect/

This is awesome, didn't know this service existed!

subscribing

Subscribing

Closest thing I could find right now (not really facial detection):

imagefield_focus
smartcrop

Hehe :)
Subscribing.

subscribe

Subscribing.

Subscribing

I found something that looks promising, though all the demos were static and didn't allow for image UL to test, but it appears to work... Anybody want to check this out? It is jQuery based and looks easy to implement.

http://papermashup.com/jquery-face-detection/

Looks legit, would love to get this implemented.

would like it

Subscribing

+1

this would be awesome and I would be willing to send some cash your way grendzy if this got added.

Also very interested.

grendzy, what would it cost to sponsor this? I have a website that is cropping photos to chest and crotch shots, not the desired effect I'm going for.

i would be willing to contribute to get this done, both financially and codeing if need be.

Just sent a message to grendzy the maintainer. If he doesn't respond we should consider grouping our resources together and making a proposal in the developer forums.

we should totally start a pool to get this going.

What's the best way to do the pooling of cash? Chipin is an option, but I'm open to suggestions. We would also need to define in more detail what we are looking to be accomplished. We also need a way that both us and the developer can test the module (ie pictures we expect to work with the face detection).

Chipin would be my suggestion. I wouldn't expect this to be to expensive.

First I think we need to define the project, then get some quotes so we know how much to make the chipin for.

Should this be apart of the smart crop module or a separate project? I only ask because it needs to be addressed. I would say an optional component, either as separate module or as its own imagecache action bundled with the smart crop module.

Everything out there I see uses the OpenCV php extension, I would expect that requirement would force this to be a standalone module. I would prefer if we found a way to do it without the use of OpenCV as OpenCV, as a requirement, would greatly reduce the use of the module.

I'm on rackspace's managed hosting, so I can not install php extensions. This link from #21 offers up a jquery solution.

Website: http://papermashup.com/jquery-face-detection/
Author's Website: http://facedetection.jaysalvat.com/
GitHub: https://github.com/jaysalvat/jquery.facedetection

...also, this is the link I posted in #8 http://svay.com/blog/index/post/2009/06/19/Face-detection-in-pure-PHP-(without-OpenCV)

So I sugest we have this project create a new module under the smartcrop project that uses no additional php module. Possibly the methode from #35 or a methode based on #35.

grendzy has a good idea. We should have a test bed of images, and I see no reason not to use flickr. Although we should use a more specific tag #drupalsmartcropface.

Do we have anyone in mind that is willing/able to do the work?

Why not only use pictures that can be committed to drupal's git, flickr is big enough

I wouldn't recommend committing more than what is needed for the configuration page. Or do you mean to just use pictures that are gpl?

Yes, lets only use GPL images.

I can give it a whirl as time allows. I'm finishing up bug fixes for a social network / knowledge repository for a client of Acquia and might have some time. I'm starting to look at Image Annotate (http://drupal.org/project/image_annotate) for image tagging. I'd suggest that what we need to do first is to demonstrate of the integrity and understanding of the base library with GPL images. Probably use forward facing pictures with medium/high levels of color contrast for initial test cases. I've made a lot of progress with making custom modules from scratch and will help to integrate the library referenced above into a drupal module.

I didn't read all of the above comments, but what would you see the reason / benefit of this? To associate with users? I could see something cool like this: Identify faces and automatically give a tag to those images of something generic, like 'face' or 'untagged person' and have the ability for a person to give those faces an Image Assist / User Reference tag associated with the coordinates we could get from the above library.

What could also be cool is if you could take the associated users of the faces and if there are untagged people have a way, maybe a queue, where those users could get User Points to help you identify the other faces.

Additionally it would be cool to see if you could do some sort of comparison of face regions for percentage of similarity... but first things first.

I think the money incentive would only be necessary to get some folks to pitch in some time. Perhaps as feature rewards. First person to submit demonstrable feature sets gets $X amount and bragging rights. ;)

If agreed upon, we should set the first phase for facial detection given via image upload and using a photo with faces on a white background. Seems like that would be a baseline. What would this be worth? $50 buck? Next phase get it happening with Dupal. How much is this worth? $50 to $100? Any more phases / feature sets that we could set ante for?

Steven

Here is an example where smart crop utterly fails. The image is being cropped to the persons chest because of the high contrasting white text on a black shirt. I also have another user who's image was being cropped to their crotch, but I'll spare them the further embarrassment.

Full Image:
http://ivaw.org/sites/default/files/public/styles/350w/public/pictures/p...

Cropped Image:
http://ivaw.org/sites/default/files/public/styles/full-teaser-listing2/p...

Taking time to read the above comments and checking out Smart Crop, here's what I hypothesize: creating another Image action for the above mentioned facial detection, but not under Smart Crop, though I can see the awesomeness of implementation along with it. For ImageCache, do whatever other actions you want then Smart Crop it, then use the Face Detection on that. I bet the entropy concept is similar but if the detection exists in a library already I think it would be more beneficial to have it separate from Smart Crop.

A note: ImageCache and SmartCrop seem to be about the use of image manipulation using different image libraries. Does facial recognition stand separate because it is more of a detection algorithm as opposed to a manipulation? Seems like there is pixel change when doing ImageCache operations. Perhaps the test is if you need access to the ImageCache functions / API.

Thoughts?

Not sure if this is possible, but if you could set a threshold for face detection certainty, that would be great. Because then we could have a fallback where if no faces are detected with a high enough certainty, something like smart crop could be used instead.

stevenmhouse: you seems to be having some confusion. While facial recognition would be very cool. What the intent was [at least my intent was] to crop centered on the face of the subject. This way in the examples animelion posted the center or focus of the crop would not be the chest, but the face.

Because of this it is facial detection, and image manipulation based on this. Some others should chime in here if I am wrong.

Right on. I came to this thread via google search and didn't mean to hijack a thread. With what you are looking seems to be what I'm describing above, or at least similar to it. What I could see beneficial to the community would be to have one project that would have the face detection capability (useful as stated above without only being in imagecache/smartcrop. This could have an API that could interface with imagecache. Looking at value for Drupal it seems like if you could do face detection there are other use cases it would be benefical for, and if this generated the area to crop to it would fit great in imagecache. For your scenario, what are the variables? As an action, would you take all faces and crop them to separate instantiations, or would you only want one face in a picture?

From an initial test I got bad results. See attached images. There is no PHP so it should just run... looking into it now.

Using only the files provided in the download, here's the code generating the error:

function detect() {
try {
var coords = ccv.detect_objects(grayscale( $$.get(0)), cascade, 5, 1);
} catch(e) {
options.error($$, 2, 'This image is not valid');
return [];
}

so... ccv.detect_objects() isn't working. 15 min into it.

DL'd another face w white background and tested w no luck... going to have to step through the JS. Any jQuery gurus want to help out? Gonna set breakpoints and step through the code, but don't konw what I'll find.

Here's the JS code from index.html that is firing the error, generated in jquery.facedetection.js and passed back the JS in the head tag:

$(function() {
$('#try').click(function() {
var $this = $(this);

var coords = $('img').faceDetection({
complete:function() {
$this.text('Done!');
},
error:function(img, code, message) {
$this.text('error!');
alert('Error: '+message);
}
});

So, definatly, this is the line in the try/catch that is failing:
var coords = ccv.detect_objects(grayscale( $$.get(0)), cascade, 5, 1);

Anybody else willing to try?

wow this is awesome.... ++++++

We shouldn't be doing it in js. It should be done in php as a part of the imagecache pipeline.

The js would be a cool addition to the focus crop imagecache plugin though.

My own usecase is with images that are coming in via an api. I am talking about thousands of images. This makes anything that has to happen in the browser rather cumbersome.

subscribe

+1

I played around with the face.com api today... it seems to be fairly good at detecting faces, and their API can be used freely (up to 5000 requests per hour).

I created a simple API module and a cropping action which uses a detected face as the center for cropping... with the few images I tested, this seems to work well. You can find the code in this sandbox: https://drupal.org/sandbox/bforchhammer/1337254

Be warned that this is just a proof-of-concept; a few problems are listed on the sandbox page, and there are probably many more; if you're feeling adventurous, feel free to have a look, test it out, comment... It would be great to know whether people think this approach is worth putting more effort into (or not)... ;-)

I think anything to help get this off the ground would be a good idea. I will have to try this out over the weekend to see how well it works.

One potential problem, don't imagecache actions accure at page load. Putting an api between inside of the page load could cause some spastic page load time.

One potential problem, don't imagecache actions accure at page load. Putting an api between inside of the page load could cause some spastic page load time.

As far as I know images are only created once and then cached for subsequent requests...

right, however, take an image intensive site with 40 or so images that need to go through this process. Even if the api is really quick, say a second per request, then you are still looking at adding 40 seconds to a page request.

@frob yes, but only for the first time this page is viewed. After that, no problem.
And 2 quick ways to get around this:

1. You can use vbo to generate all imagecache images at once
2. if you have boost installed you can set it up to crawl all pages, so all images should be ready before users visit the site

Four use-cases that I can see are:

1) Face Detection ImageCache action to center on a face and crop with some variables to adjust pixel padding buffers and scale of the face to make it include the neck or not, and smaller heads vs. larger face-only crops.

2) Face Detection Photo Tagging Assistant - use the functionality to find faces in uploaded photos and set some required action for the user to type in the name of the person that would also bring up their User Relationships list(s), this could also use the ImageCache action for presentation in some ways showing only the face OR it could simply put a box around the face and a text input field for the person's name. I haven't messed much with Facebook Connect integration, but it seems like it'd be a good idea to base the code on something FB uses so it can pass it along on shares.
This could then be combined with Tag User and/or Image Annotate:
http://drupal.org/project/taguser
http://drupal.org/project/image_annotate

3) Facial Recognition Comparison - have the site comb over all photos and be able to pick out who might be in a photo based on other photos available, like iPhoto on Mac OS does, then making it easy to go through all the questionable tagging and choose Yes, it's _Bob_Whoever_ or No here's the name (text input linked-into User Relationships AJAX dropdown) or an X to not tag it at all. I haven't thought this through much but not sure how this would be set up as a queue, but seems possible, yet a big, big toll on the server on large sites with lots of user uploads.

4) User Relationship Recommendation - based on either completed tagging on photos of other friends (if you're in a photo with Jill on Jill's photo, it would recommend Jill as a friend), or Facial Recognition to make assumptions without more than a profile image as a basis and recommend that way.

Facial Detection and Facial Recognition are two different things. First seems more easily attainable, second sounds like it might need to have some major $$$ backing to actually work well enough to be called useful and likely bulk up a database.

I just ran across this, and though I haven't tested it, it's the first I've seen of face detection in a Drupal module.

http://drupal.org/project/face_detection

So, now if we can just get everything from ImageCache, Annotate, Image_annotate, User Relationships, and taguser to all play nice together, and have someone develop an add-on to a pluploader for multi-image uploading that would find all faces and make it easy to tag them all at the same time it would be a pretty decent system.

Well, the module description page says that the module uses the jQuery facedetection library, i.e. a client-side library (javascript). That may help solve use cases such as the "photo tagging assistant", but for things like ImageCache integration we'll need a server-side solution (i.e. php)...

#63 ... perhaps node.js could bridge that gap here

This has gotten way sidetracked.

All this needs to be is a server-side php script that creates a smart crop based on the detection of a face in the image.

If you want facial recognition then I would say that is another task all together.
If you want bulk image upload, there are other modules for that that have nothing to do with this --that is a separate issue.

As for node.js; there is already a php face detection extension. The idea for this was to implement a way to do this with no other server side configuration.

The idea for this was to implement a way to do this with no other server side configuration.

I agree... and that's actually what I experimented with in two sandboxes; Both come with a cropping mechanism.

• Facedetection + Crop uses mauricesvay's php-facedetection script
• Face.com API uses the face.com webservice for face detection

I will have to checkout those sandboxes, might be what we are looking for with this.

@frob, what if you want face detection without cropping to that box that is found? Face detection is a function that could be reusable in various ways while detection+crop is just one need. Reread my #61 ideas. Yes, detection and recognition are two different functions. The bulk upload mention was to include a function when bulk uploading to immediately, while the remaining items are uploading, to begin tagging people in photos as it finds faces (detection).

@bforchhammer, I tried both and got errors upon attempting to add them to an image effect set, but they did show up as an option in the dropdown list. A for effort, getting excited.

:)+)

There could be other effects that could be added to an image like centering an overlay over detected faces (heh, turn all profile photos into smiley faces or something, or just some faint border or gun sight, to add flavor), so many possibilities especially if image annotation could base on the detected area and then return maybe a contextual menu for adding as a friend or last few activity items/actions on the site.

@deanflory,

It sounds like you want to take this out of smartcrop and turn it into its own module.

@bforchhammer, I tried both and got errors upon attempting to add them to an image effect set, but they did show up as an option in the dropdown list. A for effort, getting excited.

Sorry about the errors; it was just simple experiments I did a few months ago, and I didn't test things extensively... If anyone wants to work on it, I'd be happy to grant access to the sandboxes. Let me know.

I installed the module and was able to get past the errors. https://github.com/bforchhammer/php-facedetection needs to be downloaded in /sites/all/libraries/php-facedetection.

With my "article_thumb" image style, which uses scale and facedetect crop, images with no faces detected show up fine. Images with faces cause the site to hang for over a minute. After that, the site just gives up and loads with an empty image wrapper. Calling the image directly via URL hangs and then shows a white screen.

@bforchhammer... what do you suggest?

(disregard this message)

I've been successful in getting the module to run. The issue I was experiencing earlier was caused by the fact that the one image I had with a face in it was also massive (2448x3264), causing the system to hang. I shrank the image in Photoshop and re-uploaded it. Here's my feedback:

1) The image conversion is painfully slow. This site is the only thing I'm running on my test server (P4, 1GB RAM) and a page with only 4 "facedetect" images on it took 90 seconds to load. Other image styles load almost instantly.

2) My test photo was of a person standing upright with his face toward the top of the photo. The module tries to center the face in the newly rendered image. This created "black" space at the top of the image. The module should "crop in" when a face is too close to the edge to prevent the creation of black space.

Great work so far!

1) Yes, large images are difficult. I'm no expert with face detection algorithms; maybe there's a better one out there which is quicker... otherwise, speed could probably be improved by automatically scaling the image down for the face detection process.

2) That's an edge case which actually shouldn't be too hard to fix; I guess we just have to check the cropping values before applying them, and move them respectively if they go outside the original image dimensions.

I can't wait to try this out.

The face detection could probably be sped up with any number of preprocessing: shrink the image, convert to 8-bit color, multiple passes, etc.

@frob, it may work better as it's own module, then have others hook into that functionality, that way all modules wanting the function don't have to duplicate code. I can see face detection working like focus crop and smart crop currently do, maybe with an order of focus, then face, then smart crop, that way one could set a focus area that is not just the face centered, or if a user doesn't set anything the face detection works it's magic and if there's no face then smart crop centers on the most complex area. With those 3 items I can't see the need for anything else!

Trying bforchhammer's new goodies again with the php-facedection library in place!

Bad news for face.com: Facebook will soon close the API after having bought this company.

Not sure if you've found this one yet but it looks pretty solid, opencv algorythm in pure php:
https://github.com/felixkoch/PHP-FaceDetector/blob/master/FaceDetector.php

@JurriaanRoelofs cool, thanks for the pointer. I hadn't seen that one... Code looks good, but similar to mauricevay's library it would need to be adjusted to work with "image resources" instead of "image filenames". New sandbox anyone?

A similar (and better updated) library was pointed out in #35:
https://github.com/mauricesvay/php-facedetection

While I was active in this issue (5 comments, the last one was also mine), I've just created a new module implementing face detection. It is quite different from smartcrop: from the concept (focal point instead of crop) to the implementation (it works independantly from the Image toolkit).

http://drupal.org/project/image_focus

@jcisio : cool, but do you plan to implement it for Drupal 7 too? I didn't try your module, because I no longer use D6, and I think in January, 2013, it would be better to experiment with modules written for D7 or D8, not D6... :) Anyway, thanks for the efforts!

#82 as I wrote this module for my own need, it started with a 6.x version. Any try to port it to D7 is welcome, and I don't think it would take more than 30 minutes to port. BTW, this issue is not an appropriate place to discuss about it.

@jcisio : it's OK :)

"Any try to port it to D7 is welcome, and I don't think it would take more than 30 minutes to port"

Especially for the module's developer, who knows its code! ;)
Unfortunately currently I don't have time to deal with it, hopefully someone will do it.

For me the ideal cropping module would decide the focal point of the image in three steps:
1. Manually let user select the focal point(s). This should be optional, I suspect I want to bother users with this in 15% of my use-cases.
2. Algorithmically try to recognize faces. If that fails:
3. Algorithmically determine entropy, a solid baseline in case there are no (recognizable) faces.

Just came across this and wondering if anyone has had success with this module or another in tandem to recognize faces to determine focal points and help "smart crop" images.

I have this module installed, but still see some room for improvement, such as on this page:
https://www.kobejet.com/en/song/go-distance

You can see several of the images awkwardly crop the faces.