The 2026 Drupal Business Survey is open. Your response will help shape the future of the Drupal business ecosystem.The ParserCSV.inc library we're using for CSV parsing does not support batching at the moment. A CSV file can only be parsed in one piece.
As ParserCSV is the only candidate for batching on the parser level in the Feeds suite, this issue is prerequisite for #744660: Expand batch support to fetchers and parsers.
| Comment | File | Size | Author |
|---|---|---|---|
| #4 | 850298-4_parser_csv_batching.patch | 10.3 KB | alex_b |
| #3 | 850298-3_parser_csv_batching.patch | 9.85 KB | alex_b |
| #2 | 850298-2_parser_csv_batching.patch | 4.18 KB | alex_b |
| #1 | 850298-1_parser_csv_batching.patch | 2.96 KB | alex_b |
Comments
Comment #1
alex_b commentedHere we go:
- Time based batching (this is what existing infrastructure in Parser CSV allows us to do).
- Make Iterator report file position.
- Make Iterator accept a file position on where to start parsing.
Needs tests. Time based batching will likely need to go in favor of a more testable number-of-lines based batching.
Comment #2
alex_b commented- Introduces line based batching.
Comment #3
alex_b commented- Added tests
- Series of bug fixes
- All tests passing
Read test in patch to understand the new functionality.
Comment #4
alex_b commented- Clean up naming: lastLinePos() instead of lineLimitReached(), setLineLimit() instead of setNumberOfLines().
- Deprecated timeoutReached() in favor of lastLinePos().
Comment #5
alex_b commentedCommitted.