hi all.

My problems are :
1. when I import from CSV data(approx. below 500), it works with no problem.
But over 1,000 data, it's not imported over 750 no more
and then importing percentage is shown up and down.
The columns of data have GUID field with unique target and some custom columns.

2. If I delete GUID field, when I import data(over 10,000), it's never stopped like infinite loop.

Basic configurations are all set.
And GUID field is located last field after all custom fields.
I input the integer unique values into GUID fields.

So how can I fix these problems?
If you guys need to know more about my damn problems, just say it. I'll tell you in detail.

Thanks in advanced.
Best Regards.

Kevin Lee.

P.S just one more thing, when I stopped the importing process, this message is appeared below.
But I don't know whether to be related :
"Notice: unserialize(): Error at offset 35 of 38 bytes in FeedsSource->load()
(line 522 of /var/www/dpoi/drupal-7.10/sites/all/modules/feeds/includes/FeedsSource.inc)."

Comments

twistor’s picture

twistor commented

Can you try this patch, http://drupal.org/node/712304#comment-5076258?

procarrie’s picture

procarrie commented
Category: task » bug
Priority: Critical » Major

hi again.
I knew the reason of my problem.
I'm using PostgreSQL. But if I use MySQL, it's not gonna happen the problem.

Well.. what is difference between PostgreSQL and MySQL in this problem?

Basically, I want to use PostgreSQL. But I don't know how to set or do something for PostgreSQL.

Please tell me how to solve.

Thanks in advanced.
Best Regards.

Kevin Lee.

twistor’s picture

twistor commented
Assigned: procarrie » Unassigned
Priority: Major » Normal

Have you verified that it works with MySQL?
Did you try the patch I pointed to?

procarrie’s picture

procarrie commented

// twistor
Actually I don't know how to apply patch file.
Could you tell me simply way?
Thank you for helping me.

dman’s picture

dman commented

(0_<)

How to apply a patch to a contributed module

That page was three lines long when it started, but it's grown an extra line every time someone asks the same question...

procarrie’s picture

procarrie commented
Assigned: Unassigned » procarrie

// twistor
I patched your file.

I used the commend : #patch -p1 --dry-run feeds-text-to-blob-712304.patch

After that, I got the message below:
patching file feeds.install
Hunk #1 FAILED at 70.
Hunk #2 succeeded at 533 with fuzz 1 (offset -88 lines).
1 out of 2 hunks FAILED -- saving rejects to file feeds.install.rej

Now.. what should I do more?

dman’s picture

dman commented

This is a question that can be answered!
It usually means that the path is out of date compared to the version of the code you are applying it to.
Part of it was still close enough to be applied, but one chunk was trying to patch code that had already been changed, or was too different to be merged..

It seems the patch linked is supposed to apply to a 6.x-1.x-dev version, yet this issue is flagged as 7.x, so there is likely to be an issue here.
Right now, It could be that this solution will not apply to this question.

procarrie’s picture

procarrie commented

//dman

So you mean that patch is old.

Then, do you get the solution?

My problem is still occurred. When I use MySQL, it works well.

But I use PostgreSQL(using unique target), it's not working.

The percentage is back and forth from some part.

colan’s picture

colan
he/him
Primary language English
Location Toronto 🇨🇦
commented
Status: Active » Closed (duplicate)

Looks like a duplicate to me. Further fixing should be done over there.

seattlehimay’s picture

seattlehimay commented
Title: the problem about importing large data (CSV) » PostgreSQL and infinite looping on import
Assigned: procarrie » Unassigned
Status: Closed (duplicate) » Active

Reopening this one as I'm having similar problems.

I'm using Postgres. Tried to import a tab delineated file of approx 125 records. Records are not excessive in size--doubtful this is size related so changing the title of issue as it may be misleading.

After clicking import, the status bar quickly goes up to 37%, then stays there. Seems to enter some loop where tens of thousands of nodes get inserted into the database (I have versioning turned on, if it matters). Had to kill the import. Just like procarrie, I see a BUNCH of these errors:
Notice: unserialize() [function.unserialize]: Error at offset 35 of 38 bytes in FeedsSource->load() (line 522 of /mypath/modules/feeds/includes/FeedsSource.inc).

twistor’s picture

twistor commented

Can you try the patch in this issue? #712304: Batch import does not continue where it left off, instead starts from the beginning.

seattlehimay’s picture

seattlehimay commented
Status: Active » Fixed

Jeez. My apologies for wasting your time. I had seen the patch, but thought it had already been committed to dev. Indeed, this fixed the problem. (Now onto the UTF 8 errors....)

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

JB_ineris’s picture

JB_ineris commented
Status: Closed (fixed) » Active

Reopening the issue as i applied the patch and still in trouble.
I might be missing something.

I use a PostgreSQL/PostGIS database.
I have applied the patch and checked the feed.install file. OK
I have reloaded the Feed Module.
I use node reference to reference my nodes A to already imported nodes B.
I have set up php.ini :
- memory_limit = 1024M
- max_input_time = 120
- max_execution_time = 300
- upload_max_filesize = 10M

I am trying to load a file with 30000 nodes of light size (4 short columns).
The issue occurs after only a few seconds.
I have exactly the same issue as described :
- using GUID, the importing percentage goes to 0% after initializing and never moves up. The result is 55 nodes imported and never more.
- without the GUID, infinite loop with several time the same node.

I have tried a lighter file with 300 nodes. Behaviour is about the same. I just have up and down percentage bar without the GUID, but it goes into an infinite loop.

Any idea ?
Thanks for any help.

JB_ineris’s picture

JB_ineris commented
Status: Active » Closed (fixed)

Found the trouble.
Didn't see my database configuration needed an update, maybe once i installed a module or a patch.

So the patch definitely fixes the issue.

JB_ineris’s picture

JB_ineris commented
Issue summary: View changes

nothing