automatic import

I would also be interested.

I second this. Automated import would make my life so much easier.

Same here. I'm working on trying to accomplish this very same task.

I've sponsored a development of this, and the module is coming along great. E-Mail me at drew@imoria.com to inquire. If you are interested in helping me sponsor this development (the module is already usable, but will be getting some spiffy upgrades) - I will be happy to give you a patch that will make node_import drupal-5 and cron seamless as soon as today. I will also update you with patches as soon as they come in.

awesome news, Drew! Will this support automatically updating nodes as well (or could it tie in with the work here)?

Yep, auto update based on customizable criteria. (ie: if (this = that) {update or do not update node}) Haven't taken a look at the work there yet, I'll get back to you after I do.

To be specific, this patch can/will be able to:

1. Trigger via cron
2. Set Max Execution Time
3. Set Number of Nodes per cron
4. Cancel jobs
5. Ability to upload straight from file in a folder already on server (default "files") or upload new file
6. Ability to skip "preview" step
7. Import Based on Criteria (if(this=that){import}) where "this" is a column header in the csv file and "that" is a text field. For my project it will be if the updated date is earlier (so it will have =, !=, >=, and <=)

Like I said in my previous post, about 3/4 of this is finished and ready go. Anyone who can help me fund the last 1/4 of it can have the patch as it is now (perfect for everything I listed except # 7). The faster I get the developer the money, the faster we can have this reviewed by the maintainer and get a patch, or updated module up there. Anything is appreciated, let me know if you're willing to help.

Sounds great! Please do you know roughly how fast the automatic import performs e.g. how many nodes per minute for a given spec machine?

The reason I ask, is I want to know if some performance improvements were also implemented. Please see comments on:
http://drupal.org/node/270584

Thanks

120 nodes per minute is about average. There were some tweaks made, and CCK is to blame for some of the lag, but when you get up 75,000 + it slows down to about 10 / minute.

EDIT: at first, it flies... maybe 700 - 800 / minute.

Sounds useful! are there currently any plans for a 6.x port? :) Scott

Ok, thanks. I can help out with testing, if necessary. Please what's the current status of the patch for:

"5. Ability to upload straight from file in a folder already on server (default "files") ..."

Subscribing (And very exicted about this!)

Subscribing

Subscribing

sub

Now that everyone's here...

What/How much would it take to get this completed?

Any update on this? It sounds very promising!

I would very much want to see such an improvement to an already great module. Is anyone doing anything or is this thread dead?

subscribing

any updates on this...?

Subscribing ... Any updated on this?

Drew, Let me help to accomplish automatic import, I need it too, very ugrent :)

subscribing

Hi,

If i correctly understand problem you need to add new records in tvs/cvs file and then import it in automatically by node Import module via cron. If so, I had the same problem.
For solving that I have created new module Node import via cron, it requires Node import module.
User or bot should append content to the existent files for tasks automatically. And then it is imported via cron.
I hope it will be useful for you. Please test it.

Now I'm working on other module which can convert data from XML to CVS and even images. This is done is for only my case, but in future I'm going to implement that in general way.

Please write here your ideas.

Setting as duplicate of http://drupal.org/node/684512.

dealancer I need drupal 7 module for