Indexing slow with large index

I am using Solr in an application that I am writing that will (in the first instance) hold about 50,000 records with the text of a pdf in the Solr index.

I found that during the import the index stalled at about 1,000 records. After tracing it down, it was because optimize was being called after every document. After about 1,000 records the indexing got so slow and only 1-2 documents were being added per cron run.

To fix the problem in the short term, I commented out line $solr->optimize(FALSE, FALSE);