Add importing and export for CSV and XML

tdellaringa - July 11, 2007 - 17:33
Project:Path redirect
Version:6.x-1.x-dev
Component:User interface
Category:feature request
Priority:normal
Assigned:Dave Reid
Status:active
Description

Lovely module, sorely needed too! However, adding one redirect at a time is VERY time consuming. I have something like 75 to do, some way to do multiple would be a great boon, even if it is just a page with 10 lines of fields to submit at once.

Thanks

Tom

#1

jjeff - July 11, 2007 - 19:13

How do you suppose an interface for this would work? I'm open to the idea, but there are several things that need to be entered for each redirect, so it's not quite as simple as a textarea with 1 URL per line.

#2

tdellaringa - July 11, 2007 - 20:54

glad you asked! I'm a usability engineer by trade so I appreciate the concern. My initial thought would be that you would have say 10 rows on the page each with the 5 fields, where the redirect type would change do a dropdown. So you'd have:

alias-to-redirect | to | ? | # | type

Now, I don't know how you process this, but I'm assuming you could run each row through the process the same way you are doing 1 - you just do up to 10.

Granted horizontal space is an issue, you'd get less room for the fields.

However, possibly a better idea is to allow some kind of CSV file to be submitted and run the process on that. I'm just thinking out loud here, I'm not sure what the best way would be. If I could submit a text file that is like

originalURL/bad, newURL/good, ?foo, #link, 301
originalURL/bad, newURL/good, ?foo, #link, 301
originalURL/bad, newURL/good, ?foo, #link, 301
originalURL/bad, newURL/good, ?foo, #link, 301

Then have it processed, that would be the easiest for the user, I think. I just work offline and create my text file, upload the file and you do your magic on it, running each row and adding the redirects.

Again, I have no idea how much work that is on your end, or if it is even possible. Where do the redirects end up - how does it work?

#3

jjeff - July 11, 2007 - 21:37

A usability engineer you say!!?? Well there are a great many things that we could use your help on!!

I like the comma separated stuff because, as you say, you can create these as a separate text file and bring 'em in all at once. And this format could also work for export, so we're basically building an import/export system for this module. And the import should be really easy to parse because commas aren't valid URL characters (I think). So I can simply do a split by line break, and then by commas, and run those into the path_redirect_save() function. (I make it sound so easy!)

I'll take a look at this and see what I can come up with.

(Seems like there should perhaps be a similar thing for path aliases as well.)

#4

tdellaringa - July 13, 2007 - 15:05

Exactly my thoughts! Glad it makes sense. I think it's probably easier that way than trying to come up with a form that works with all the fields. I'll keep an eye on this thread looking for an update. If you want to e-mail me when it's ready, its pixelmech at gee mail dot com.

Thanks for your hard work!

#5

HorsePunchKid - October 13, 2007 - 01:19

I'm afraid commas are valid in URLs, or at least you're likely to encounter them. Some of the crustier CMSes that adopted the "users don't care about URLs" philosophy use them curiously often. Witness:

http://web.worldbank.org/WBSITE/EXTERNAL/0,,contentMDK:20130468~pagePK:5...

*ack* Now that said, I'm pretty sure that spaces are not valid in URLs, nor should they appear in any of the other parts of the input. Or perhaps tabs would work, since tab-delimited files are fairly well supported. CSV could still work, but unless there's a PHP parser library you can count on being installed, it's probably not worth it.

#6

HorsePunchKid - December 25, 2007 - 00:45
Title:Need multiple add feature» Import redirects from a delimited file
Priority:critical» normal

More accurate title given where this discussion went. A "quick add" feature might be nice, still...

#7

HorsePunchKid - December 27, 2007 - 18:32

Here is a proposal:

  • An "import" tab on the admin page.
  • A form therein comprising a file upload widget and a redirect type selector (default: 301).
  • The uploaded file shall be tab-delimited; records separated by /[\n\r]+/.
  • The first two columns list the "path" (from) and "redirect" (to) in that order.
  • An optional third column contains a redirect type to override the default selected.
  • The query string and fragment are automatically extracted.
  • Errors (e.g. from already-existing redirects) will be non-fatal and reported when the import is complete.

Any thoughts? Would it be better to prepare a query and bypass path_redirect_save()?

#8

gemini - April 7, 2008 - 04:10

I would be interested in this module to redirect over 4K URLs. I used it to redirect less than 100 URLs and it worked beautifully, but is it gonna handle over 4K just as good?

#9

HorsePunchKid - April 14, 2008 - 03:11

I don't think the query for looking up the redirect will be noticeably slower for 4,000 entries compared to 100 entries. Paging through 4,000 entries in the admin interface might be rather burdensome. If necessary, we could perhaps add a quick filter box on the admin page.

#10

gemini - April 16, 2008 - 22:36

What I actually did - created a CCK old index reference field and imported all page indexes from the old site. Then I uses custom_errors module and added some code to extract ids from old ulrs and look up in the current db. So far it works fine.

#11

geodaniel - October 28, 2008 - 22:49

Subscribing to this.. it'd be great to see.

#12

Dave Reid - March 15, 2009 - 07:25
Title:Import redirects from a delimited file» Add importing and export for CSV and XML
Version:5.x-1.x-dev» 6.x-1.x-dev
Assigned to:Anonymous» Dave Reid

Basic export functionality has just been committed to the 6.x-1.x-dev version. Sorry, new features are not going to be ported back to Drupal 5, so upgrade! I'm also working on getting the import working.

#13

silurius - April 17, 2009 - 23:24

Dave, are there plans to include import functionality as well? Currently trying to decide how I want to handle replicating the ~100 redirects I created in a dev site today to a QA site.

#14

Dave Reid - April 18, 2009 - 18:48

There are, but I'm looking into how to best implement it. Probably what I'm going to go with is to implement integration with the Node import module so that I don't have to recreate and maintain import functionality.

#15

jeffschuler - September 13, 2009 - 02:41

Subscribing!

The ability to bulk-import would be useful on it's own, and would also allow for some other cool functionality, (within Path Redirect or in other/new modules.)

For instance: Fairly frequently, I wish I could auto-generate appropriate redirects when an automated alias path is changed.
ex: Say I want to change all of my Plant content type nodes from the default content/[title-raw] to plant/[title-raw]. A glue module might auto-create redirects for all old=>new paths...

#16

epersonae2 - September 15, 2009 - 23:53

I have over 100 static pages to redirect for a current project. I have a spreadsheet listing the old pages and the new paths/node IDs. In lieu of built-in module functionality, do you have any suggestions for importing the redirects directly into the database? (Otherwise, I'm probably going to see if I can do the .htaccess using a CSV as the starting point.)

#17

marvil07 - November 26, 2009 - 07:17

subscribing

 
 

Drupal is a registered trademark of Dries Buytaert.