Drupal 7 port done

Thanks for this D7 port! I've been looking forward to testing it because the existing 7.x-dev file doesn't work at all. So I finally got to it tonight, but unfortunately I can't get it to import my GEDCOM file properly (about 450 individuals). I get this error the first pass:

PDOException: in drupal_write_record() (line 6868 of drupal7/includes/common.inc).

Then, subsequent imports produce this error:

messagePDOException: SQLSTATE[42S01]: Base table or view already exists: 1050 Table 'family_relations_temp' already exists: CREATE TABLE {family_relations_temp} (`nid` VARCHAR( 128 ) NOT NULL , `famc_xref` VARCHAR( 128 ) NULL , `fams_xref` VARCHAR( 128 ) NULL) ENGINE = MYISAM; Array ( ) in family_import_submit() (line 100 of drupal7/sites/all/modules/family/import.inc).

The module creates nodes but there are no relationships to other nodes (no ascendants or descendants), so it seems to fail when it finds something in a GED file it doesn't like.

In addition, the version on the modules page is listed as 6.x because of the info file. Uninstalling the module requires a manual drop of the family tables because D7 can't see the disabled module to uninstall it in the GUI.

* * *

For what it's worth, I was able to get the D6 version to do quite a lot but there are some pretty serious problems that need fixing that are off-topic to this thread. But because GEDCOM files have a lot of different types of data in them, my working methodology for D6 was to create a tree in Ancestry.com, export the GEDCOM file, edit and streamline the file with some genealogy software (currently using Family Tree Maker), re-import the file into Ancestry, then export again for the Family module. That last step was necessary for some reason but I can't explain why.

It's too bad the support for this module has died because it could be really great with a few tweaks.

Yes adding nodes works, that's great.

But supporting GEDCOM files is essential. People who work with genealogy records often maintain thousands, or tens of thousands of names, and sometimes hundreds of records for each name, so the module's import function is important. Plus, different genealogy programs have varying advantages, so using the GEDCOM format to import/export is common. Obviously, manual entries become impractical at some point, especially considering the many potential bits of data involved for each individual.

If your firm were to support this module, I have many suggestions to offer you based on my experience with the D6 version, which was adequate but critically flawed in a few important ways. Autocomplete fields vs long and nonsensical dropdown lists, for example, are a huge sticking point when adding or modifying records.

There are lots of genealogists waiting in the wings for a genealogy module to come forward that is robust and flexible. A popular feature request is to allow individual names to become users that could create their own branches, thereby adding to the overall genealogy record count, but for which an authenticated user would have relative control to add/delete, or make changes. Shared records is risky but unavoidable. One of the problems with familysearch.org is that people can overwrite records and ruin years of hard work for other users. Another pain point is the lack of revisioning, which Drupal could alleviate.

Finally, I could also see the Family module being adapted for other things beyond genealogy per se, like plotting company owners, officers, partnerships or clients, or even project management, and money flows.

Cheers, and thanks again.

I apologize for the delays in responding to this...

Zarevac: awesome job on the port. The project still has some issues to work out (as do the other versions), but still -- many thanks.

Committing to HEAD/7.x.

I have attempted to use this version of the module and the D6 version, but have the same problems with both. The gedcom file I want to import has 3500+ individuals and 900+ families (approx 76,000 lines). I always get a php execution timeout (60 sec) after about 1050 or so individual nodes are created - even if I enter a limit of 1000 records. Seems the limit is ignored. The only way to make another attempt after this is to first manually delete everything the module has created in the database or you get errors like in #1 above.

My limited understanding leads me to believe the problem is the overhead associated with creating and populating nodes. Because I noticed that an uninstall left all the created nodes untouched, I experimented copying the code from the import.inc file that deletes nodes when 'replace' is selected, into the uninstall function. Result was repeated timeouts after deleting about 350-400 nodes.

I have to wonder if Drupal is cut out for this sort of work. It would be nice to be able to have custom tables in the database that Views and Panels ,etc. could recognise. Other stand-alone scripts that manage their own database can import my gedcom file in under 30 sec., problem-free.

The performance problem you're having is not likely to be with Drupal, which can certainly handle this task, but with your server and your Apache settings, including and especially your php.ini file.

And regardless of whether you delete all the family nodes upon import or you build on what you have (something that doesn't work properly in the Family module anyway), the import performance is not about how many preexisting nodes you have, but about much time you can give to the php job, along with the general performance of your server.

So if you change the timeout and the Family module happens to like your particular GEDCOM file, you can import a large number of individuals. I've successfully imported over 30,000 names, but that took about 8-10 minutes in my case with my server at the time. If you can change your settings to support longer import times you should see all your nodes represented in Drupal. How they may look is another issue.

If you're not using the Backup and Migrate module, you should. It's especially important for developing with the Family module. Restoring your site rather than deleting nodes can save you a lot of time. Otherwise, you're creating new node numbers for the same names on each import because Drupal is very careful about not creating node conflicts. (I also highly recommend using Drush.)

Lastly, there are unfortunately many serious problems with the Family module in parsing GEDCOM files, and it makes developing with this module a very frustrating experience. Especially difficult problems are:

• how it deals with incomplete data,
• how it handles alternative names,
• how it builds Parents 1 and 2 in the right order, and
• how easy it is to edit data.

The last point is one of the most frustrating problems because the Family module makes it nearly impossible to fix things after import. It's beyond me why Family builds the list of names for parents in a drop down list when editing family groups. Most bizarre is why the parent dropdown lists are not ordered alphabetically. The module is just not built for long lists of individuals. So to deal with this you have to scroll down and hunt for your name amid lots of people with the same names out of order. (You can also try to speed things up by using the tab keys back and forth, but you're just going in order through those long lists.) A HUGE improvement for the Family module would therefore to be to make these fields work with Drupal's built-in autocomplete function.

So old_dog, if you have any luck working with this module, let me know! I've had to put my Family-related project on the shelf until some of the serious problems with this module are fixed.

Regarding timeouts & settings:
I am running my tests on a XAMMP installation on a local Windows XP system. The timeout setting is 60 sec. which is double what you can expect on a shared hosting environment. From perusing the code of other stand-alone scripts, the import function reads the php time-out setting and then checks the running time regularly in the loop. As running time approaches the timeout, details of progress are saved and the loop is broken. I am not sufficiently knowledgeable to know how far up the chain you have to go in order to reset the clock, but I suspect you need to regenerate the submit form. The import should then automatically resume where it left off and repeat until done. Perhaps in Drupal you can use the Batch API to achieve the same.

I see a strategic problem with the module:
The way I work is to maintain all my research in a proprietary system (Legacy) off-line and periodically generate a gedcom to upload to my site. The problem I see with the Family module is that it makes no sense to import a revised gedcom unless 'replace' is selected. Otherwise you end up with thousands of orphan nodes. However, if nodes have acquired body text and/or comments between updates, it all gets deleted with the nodes. So, perhaps the import should update existing nodes rather than replace them. But how can you do that when all the gedcom ID numbers from the initial import have been dumped?

We??? Who and how many are you? (just curious)

I don't have any way of producing an XML file, so I guess the module will become even more useless.

From Wikipedia: On 23 January 2002, a draft (beta) version of GEDCOM 6.0 was released for developer study only, as it was not a complete specification, and developers were recommended to not begin implementation in their software. For example, descriptions of the meaning and expected contents of tags were not included.

While I applaud the forward progress to XML it seems prudent not to drop the import of GEDCOM 5.2 files. I haven't had the time to work with the family module lately. Does anyone use the import patch I sent in?

I was idly thinking today about creating a geneology program in Drupal and ,lo, here it is already. Well done Zarevac for the conversion to 7. I'll be interested to see how this develops.

I've been hacking away at the import code and learning php and drupal at the same time...
Stuff I've noticed:
- the form fields are ignored -see http://drupal.org/node/1353036
- the first individual and group nodes created are empty, because they are created before any details are read.
Subsequent data is out of phase, although appearing to be correct. This results in the last individual and
group records to be lost. see http://drupal.org/node/405170#comment-1531520
- the use of the "family relations temp" xref table is a waste of effort and a source of error. The logic assumes
that the first 'fams' record returned is the husband. The wife's record could very possibly occur earlier in
the gedcom file and so be returned as husband. Other possibilities are husband unknown or, in the case of
multiple marriages, getting the partners mixed up since only one 'fams' is recorded for an individual.

I'm over-hauling the code to eliminate the xref table and replace the cumbersome switch/case logic with a 'drill down' strategy using if/else-if structure. Results are promising. But even that structure will become difficult to follow when processing for all the gedcom tags is added. I think it would be better to have separate parsing functions for the various types.

As part of eliminating the xref table, I have added the standard gedcom id fields to the schema and use them to create the node references for parents, etc. I believe these gedcom id's could be used in the various queries instead of having to create direct node-id links. And if you ever want to import an updated gedcom, these fields would be essential for any sort of update process on the database.

....carrying on...

See comment #8 at http://drupal.org/node/405170

I have virtually no experience with the Family module, and I was unaware that a D7 port existed before today. However I'm an avid genealogy buff, and would be interested in helping out with future work.

Some time last year, I needed a project to learn the entity and field api stuff in D7. Towards that end, I wrote some code that would import a GEDCOM file from an ancestry.com export, create nodes from that import, link everything up via node references, and present parts of my tree as a descendency chart. This was primarily a learning exercise for me, but I did get quite a bit further along in the code than I originally expected to. I had actually forgotten about this code in the many months since I wrote it. But today I came across this thread, and it reminded me of the work that I had done.

As this was not an attempt to port any existing module, I do not know if my code has any relevance to this project or this discussion. If not, please pardon the interruption. If, however, there is anything in my code that would be of benefit to your efforts to start a new D7 family module from scratch, I'm happy to share it. While far from complete, I have had great success importing standard GEDCOM files with as many as 1600 persons from Ancestry.com, Family Tree Maker, and TNG.

This code is in a public git repository at git://github.com/billbostick/unkle.git, If you have the time and inclination to take a look, I'd be happy to answer any questions that you might have.

@ Bill Bostick
I would also be interested in looking over your efforts, but am also unsuccessful in following your link.
Perhaps you could zip it up and post it here as an attachment?

I have been using this module and my efforts to solve the problems that I perceive with it as a learning exercise in Drupal, PHP, HTML, CSS. So far, I've modified the import script to handle time-outs, not lose or mix up data, and eliminate the need to create the cross reference table.

I've since abandoned trying to improve this module and am working on a similar but different approach. Instead of nodes, I use some custom entities. Using this approach, I can assign the id's from the gedcom import directly to the primary keys of individual and family entity tables, and the cross-reference data in the gedcom (famc, etc.) can be copied directly into the tables. Also, when I want to replace the existing database with a new gedcom import, I can simply truncate the tables (happens in milliseconds). An additional benefit is that the tables are automatically removed on uninstall. My experience with the current module is that it takes about 3 times as long to delete a node as to create it, requiring additional code for handling time-outs on import replacement and uninstall (prime reasons for my change of approach).

I've managed to create a pedigree chart using only views, panels, and page manager plus some CSS.
I'm particularly interested to see how you've done the descendancy chart.

I wonder if jerdiggity has anything to say.

Try the repo page at https://github.com/billbostick/unkle. I sent an email to Zarevac via his contact form with the same link... pretty sure it'll get you to the code. If I post a zip file here, it might be out-of-date pretty quickly, so I'd rather point to the repo.

It sounds like you've taken a lot of the same steps I took, old_dog, especially with regards to the use of entities. As I look at this code for the first time in 6 months, I realize that there's a lot more unfinished parts than there are finished parts. I've got the individual and family content types working to the extent that I could parse a GEDCOM file and build a relationship tree using node references. Using this, I added the ability to create simple descendency charts. As I was learning the entity stuff, I created a framework for notes, repositories, sources, and events, but I have not yet tied these entities to individual or family records. I'm using nodes for families and individuals, but these should probably be entities as well.

A few simple examples on a live site (which could go away at any time):

http://www.sprode.net/node/6131
http://www.sprode.net/node/6131/tree
http://www.sprode.net/node/6131/tree/vert

The GEDCOM import code might be the most useful thing here. I was able to get it to work reliably with GEDCOM files from multiple sources. As it exists, it's a single massive operation that consumes a bunch of php memory. I attempted at one time to use the drupal batch api to handle this, but I have not yet got this code working.

Regardless of the usefulness (or lack thereof) of this code, I'd be interested in helping out with this module in the future.

Thanks for the quick response.
It will take me some time to digest it all.

Hi guys,
very interested in this module for a project in the near future. I just want to point out, referring to Zarevac comment that for D7 will be much better to handle location data with the Geofield/Addressfield combo and Openlayers to display them.

From Wikipedia: In February of 2012 FamilySearch outlined a major new project around genealogical standards called GEDCOM X.
The file format is described here:https://github.com/FamilySearch/gedcomx/wiki/File-Format

I wish the maintainer would resume his work on this module. It has a lot of promise. There are many people working with genealogy who are looking for useful ways to aggregate and display the complex ways that human create lives together.

Regarding GEDCOM X files, @mato thanks for the link. This looks like an important evolution, and it's good that it's on git. FamilySearch is an important contributor to this field, obviously, so it's good to pay attention to what they're working on.

I for one am inspired by lots of possibilities with this module. I was able to get quite far with D6 but there were/are serious problems. But with some more work, I actually think it can be adapted for other types of information than just human genealogy.

What about thinking outside of the box for a moment and consider unusual things like forensics. Or maybe deep research into corporations and their partnerships and employees, for example. Consider the utility of tracking complex relationships among networks of professionals with this kind of tool, as an alternative to the over-popularized social networks like Facebook or LinkedIn. Sometimes you just need a template you know without the requirement to enter a boat load of personal data, or to be forced into public forums that involve other people.

Now with semantic data in D7 and a variety of interesting ways to set up nodes, fields, taxonomies, and all sorts of informational relationships, Drupal can do more to manage the complexity of the human relationships that endlessly confound us, as well as things like business interactions, animal science -- even astronomy and beyond.

Why not? A relationship builder like this could be extremely useful if it were more stable and flexible.

Wonderful! Looking forward to the new release! Keep at it, good job Zarevac (and everyone else who has contributed)!

Is the purported maintainer of this project involved in any way?
Does he have any interest whatsoever?

You might be better off putting your new version in your sandbox so that issues and patches can be handled in a more normal fashion -- depending, of course, on your willingness to act as a real maintainer should.

Looking forward to testing it out, thanks for your work on this.

I found some minor bugs:
- in the family group section - data in the [place of union] field is not saved correctly. I looked thru the code already but found no mistake yet ... i'm going to dig a bit deeper
- in the permission logic (or maybe just in the actual given permissons?) on the sandbox seems to be a mistake. I can add and edit individuals and groups but even those are shown to me as private. This is in no way a desirable behaviour.

questions and suggestions:
date fields: As far as i understand the code the module stores dates directly with the php/mySQL date functions and is not using the date API. Right? In the current state there is no way to change the date format (or widget) for the date field. Furthermore estimated dates are not possible. If i recall correctly this was already a feature in an older version (or was it in your relation based attempt?)
family location: What exactly is this contenttype good for? Wouldn't it be more elegant and flexible to use something like location and/or geofield for that purpose?

Hello all,

I have recently spent some time on version 7 of this module. As a basis I took the code of #37. There were many improvements in presentation of the ascendant and descendant trees, which now now look much better, as well as the presentation of the data for individuals (e.g. added age, improved and cleaned up translatable string, html and css).

I would like to share my code, but I am not sure how to go about. I could just post a zip, but maybe a more official version would be nicer. Who can advice me.

I guess posting a .zip file would be appropriate for this thread...
Your other options would be:

1. apply to take over maintainership of this module.
2. open your own sandbox and place your version there.

Well then, here is my version of "family" based on the code of #37. I made some more changes.
- Code (php, html, css) has been changed notably in many places, especially the tree views, and in the family individual and family group presentations.
- I did nothing with the location functionality. It is probably better to use the location and gmap modules to connect those with individuals and groups. At least, that is what I do on my site.
- After installation I cloned the views, and then worked with those to adapt them to my taste. The provided views are a good starting point for that.
- You can add more fields (like image) to the content types after installation to extend the possibilities.
- I have not paid particular attention to security. Use at your own risk.
- Also I see more ways of improving the code. Maybe I'll set up a sandbox later.

Let me know what you think

NB. I got this message when uploading the file:

For security reasons, your upload has been renamed to family-20140819.tar_.gz.

So you'll have to rename it back to family-20140819.tar.gz (remove the underscore).

As a followup to my previous post, I post a new file with my version of the "family" module. I fixed a bug in the autocompletes for individuals and groups. This time the file is also .zip, which probably suits more people.

Again, have fun with it, and let me know your comments.

I started a sandbox (called Pedigree) where I work on my D7 version of Family Tree. You can find it here:
https://www.drupal.org/sandbox/jos_s/2325179

It is my first sandbox, so let me know if you have suggestions.

Various problems importing GEDCOM with #46:
Notice: Undefined property: stdClass::$ancestor_group infamily_individual_insert() (line 162 of .../family/individual.inc)
Notice: Undefined variable: month in family_changeDateFormat() (line 83of .../family/common.inc)
PDOException: SQLSTATE[22007]: Invalid datetime format: 1292 Incorrect datetime value: 'Apr--Probate' for column 'deathdate' at row 1: INSERT INTO {family_individual} ...

Then after import during browsing:
Fatal error: Call to undefined function family_calculate_age() in .../family/view.inc on line 121
Fatal error: Call to undefined function family_check_short_date() in .../family/tree.inc on line 173

Pedigree does look better than the old one.

@bradp : I have not used the GEDCOM import feature, so I haven't looked at the code for that part of the module. To judge from the error messages you posted, it looks like maybe the dates are not recognized and imported. When I have more time I may test importing GEDCOM files.

As to the errors "Fatal error: Call to undefined function": are you sure you copied all the new files for the module? The functions are defined in "common.inc". Did you clear the cache?

One of these days I'll update git with a newer version.

@bradp : I had a quick look at the import code. It looked like the variable for $month was not defined properly. I corrected that, and committed the code to git. Can you please check if this works now?

There are also some other changes in the code, mainly concerned with layout.

Notice: Undefined property: stdClass::$ancestor_group in family_individual_insert() (line 162 of .../family/individual.inc)

Each individual node that is created by the import routine results in a call to family_individual_insert() in order to populate the family_individual table in the database.

Alterations to that function associated with auto-complete have broken it as far as import is concerned.

Note: It's probably more appropriate to discuss Jos' work in the issue queue at his sandbox.