Download & Extend
Create from Web » Issues

How to use this modul? - A litle doku please

Posted by xxm on March 23, 2010 at 8:46am
Project:Create from Web
Version:6.x-1.3
Component:Documentation
Category:support request
Priority:normal
Assigned:Unassigned
Status:active

Issue Summary

Is someone out there who could write a smal doku, screecast or somethink simular for that interresting module? I've no idea how to use and to configure ist for the basics steps.

Login or register to post comments

Comments

#1

Posted by B747 on March 24, 2010 at 12:49pm

Enable the module, go to /createfromweb and trying entering a Wikipedia url whilst selecting wikipedia as the operator....and it should work....

#2

Posted by xxm on March 25, 2010 at 9:19am

Please tell the very basic steps, what to do do get operator_wikipedia.inc to work.

1. enable the module
2. domain/createfromweb - to create an node and to make some inputs
--> I get "could not retrieve wikidata" after putting "Jeremy Rifkin", a node is created, only with "Jeremy Rifkin" as the title. What is missing in the config

3. /admin/settings/createfromweb - to configure mapping fileds, if needed
--> BUT HOW? ANY EXAMPLE? NEEDED FOR BASIC FUNCTIONS?
4. ?
???

In the /createfromweb page is nothing to select.
Where to change the wikipedia url to an other language?
Any changes that have to be done in operator_wikipedia.inc or somewere else to get the module to work?

#3

Posted by brevity on March 25, 2010 at 9:10am

maybe a permission issue but i assume you already tried as admin (for other users allow createfromweb via /admin/user/permissions ) -- are there any drupal error messages showing up / in the watchdog? cheers

#4

Posted by brevity on March 25, 2010 at 9:16am

does the other service, 'books from isbndb' work? select this as 'service operator' just enter 'catcher in the rye' or something and a list of ten books should show up ... attached image as it could look like

AttachmentSize
createfromweb_example_isbndb.png 61.74 KB

#5

Posted by xxm on March 25, 2010 at 9:23am

I only get "title: Jeremy Rifkin" as Result.
What could be wrong?

#6

Posted by brevity on March 25, 2010 at 9:34am

Jeremy Rifkin does not seem to be related anyhow with an isbndb query of 'catcher in the rye' .. wow

on which OS are you running drupal -- i guess i never tried the module on Windows, i hope there are no linefeed/encoding issues

#7

Posted by xxm on March 25, 2010 at 10:37am

I've used the wikipedia oparator. And wikipedia has an article about Rifkin, but the wikipedia oparator does'nt retrieve anything.
For the isbndb oparator I get the 10 results, but how to make an node of it.
Debian is the os.

#8

Posted by brevity on March 25, 2010 at 10:37am

sounds good! so why not click 'create 1' to create a node of it (after expanding a result entry)? you can adjust some values in the text preview and then later in the node preview ...

#9

Posted by xxm on March 25, 2010 at 12:19pm

As I told, for wikipedia, what are my interessed, the results is only "title: Jeremy Rifkin". So no possibility to "create xy" but it shows a created node with the searched term "jeremy rifkin" as the node title.

What could be missing?

#10

Posted by brevity on March 25, 2010 at 12:45pm

ok, i see, thanks --

sorry have overlooked that issue: it seems that wikipedia currently does not like to file_get_contents their articles in raw, as in:

warning: file_get_contents(http://en.wikipedia.org/w/index.php?title=The_Catcher_in_the_Rye&action=...) [function.file-get-contents]: failed to open stream: HTTP request failed! HTTP/1.0 403 Forbidden in /home/david/www/february/sites/all/modules/createfromweb/operator_wikipedia.inc on line 72.

getting the url with the browser or wget no prob, so i suggest to replace:

<?php
    #$raw = (file_get_contents($rawurl)) or drupal_set_message("could not retrieve wikidata", ERROR);
    $raw = `wget -qO- '$rawurl'`;
<
?>

generally the idea was to retrieve the wikipedia raw text and extract some data from the infoboxes thereof

the title is constructed earlier out of the wiki url without the need for this content (so that's why u only got title in the first place)
also, if u do loads of queries google (which is used to find the corresp. wiki article) may want u to set an api key

#11

Posted by xxm on March 25, 2010 at 1:37pm

great, it's cool. Thx.
wget makes life more easy :-)

I got these data:

region: Western world
era: contemporary
color: lightsteelblue
image_name: JeremyRifkin RV.jpg
image_size: 200px
name: Jeremy Rifkin
birth_date: 1945
main_interests: economy, political science, scientific and technological change
notable_ideas: End of the working society, Empathic Civilization
title: Jeremy Rifkin
body: Jeremy Rifkin (born 1945, Denver, Colorado), founder and president of the http://www.foet.organd creator of the http://www.foet.org/tir.html, is an American economist, writer, public speaker and activist who seeks to shape public policy in the United States, the European Union, and around the world. He is the bestselling author of numerous books on the impact of scientific and technological changes on the economy, the workforce, society, and the environment.

Now, how do I get the data mapped into cck fields like
image_name:--> cck_imgfield
or
birth_date:--> cck_birthdate

?

And how do I get the image result in an other language like in english?
image is part of the body
This is from de.wikipedia.org

title: Jeremy Rifkin
body: Datei:JeremyRifkin RV.jpg Jeremy Rifkin (* 26. Januar 1943 in Denver, Colorado) ist ein US-amerikanischer Soziologe, Ökonom, Publizist sowie Gründer und Vorsitzender der Foundation on Economic Trends (FOET; Sitz in Washington D.C., USA). Er unterrichtet unter anderem an der Wharton School der Universität von PennsylvaniaWharton Alumni Magazine: http://www.wharton.upenn.edu/alum_mag/issues/winter2008/wharton_leaders..... Winter 2008 und ist Berater diverser Regierungen und auch der EU-Kommission. Er gilt als Theoretiker der Zugangsgesellschaft.der Freitag: http://www.freitag.de/2001/27/01271101.php. 29. Juni 2001

#12

Posted by brevity on March 25, 2010 at 1:40pm

choose similar names for your cck fields as found in the result (createfromweb tries to find the best match between fields) or try to create a mapping in the createfromweb settings

haven't tried much of special cck fields except text, so don't know how to map into cck_imgfield -- but u might look at the cck_link discussion http://drupal.org/node/713442

for further adaptation to your needs one could think of making a copy of that operator_wikipedia.inc eg to replace the image_name with an absolute url to the image etc

#13

Posted by xxm on March 26, 2010 at 9:50am

The english wikipedia results points to my imgfield

field_image_name: JeremyRifkin RV.jpg

but it could'nt be loaded to the upload imgfield form.
1. How could that be done?

The german results does'nt hit it. The results from the german wikipedia has less infos and shows the image as the following result in the body text.

title: Jeremy Rifkin
body: Datei:JeremyRifkin RV.jpg Jeremy Rifkin (* 26. Januar 1943 in Denver, Colorado) ist ein US-amerikanischer Soziologe, Ökonom, Publizist sowie Gründer und Vorsitzender der Foundation on Economic Trends (FOET; Sitz in Washington D.C., USA). Er unterrichtet unter anderem an der Wharton School der Universität von PennsylvaniaWharton Alumni Magazine: http://www.wharton.upenn.edu/alum_mag/issues/winter2008/wharton_leaders..... Winter 2008 und ist Berater diverser Regierungen und auch der EU-Kommission. Er gilt als Theoretiker der Zugangsgesellschaft.der Freitag: http://www.freitag.de/2001/27/01271101.php. 29. Juni 2001

2. How do I get the imagename mapped to the field_image_name as in the english result?

#14

Posted by brevity on March 26, 2010 at 10:22am

1) one would need to see how cck imgfield stores images and hack that similar as done in the refered cck_link thread

2)
wikipedia articles vary a lot in information representation; the english page here uses an infobox of template 'philosopher' which the createfromweb wikipedia operator tries to parse, but the german page does not (yet) use an infobox

sorry i cannot be of more detailed help now

#15

Posted by asb on June 3, 2010 at 9:36pm

The fix from #10 (commenting out #$raw = (file_get_contents($rawurl)) or drupal_set_message("could not retrieve wikidata", ERROR); and replacing it with     $raw = `wget -qO- '$rawurl'`;) results in:

warning: file_get_contents(http://en.wikipedia.org/w/index.php?title=Johannes_Brahms&action=raw&sec...) [function.file-get-contents]: failed to open stream: HTTP request failed! HTTP/1.0 403 Forbidden in /var/www/drupal/sites/all/modules/createfromweb/operator_wikipedia.inc on line 72.

Using $raw = (file_get_contents($rawurl)) or drupal_set_message("could not retrieve wikidata", ERROR); in line 72 of operator_wikipedia.inc results in:

warning: file_get_contents(http://en.wikipedia.org/w/index.php?title=Johannes_Brahms&action=raw&sec...) [function.file-get-contents]: failed to open stream: HTTP request failed! HTTP/1.0 403 Forbidden in /var/www/drupal/sites/all/modules/createfromweb/operator_wikipedia.inc on line 72.

So where's the difference? Either of it does fill out any CCK fields, except the node title.

Very strange... -asb

#16

Posted by brevity on June 3, 2010 at 9:50pm

strange indeed ;) the first warning one should not talk about 'file_get_contents' when it's "commented out" -- just remove the line for a try?!

#17

Posted by asb on June 3, 2010 at 11:21pm

Oops, #$_SESSION['createfromweb']['operator']['result2'] = $raw; (next line) is no comment, and # obviously doesn't "comment out" the line. Too bad if one doesn't speak PHP and has to fumble with code... // works better.

Now I'm getting a "body:" result and no error message. This looks promising ;) ... but still no mapping. This can't be so hard?

From http://de.wikipedia.org/w/index.php?title=Johannes_Brahms&action=edit:

{{Personendaten
|NAME=Brahms, Johannes
|ALTERNATIVNAMEN=Marks, G. W. (Pseudonym); Würth, Karl (Pseudonym)
|KURZBESCHREIBUNG=Pianist und Komponist der Romantik
|GEBURTSDATUM=7. Mai 1833
|GEBURTSORT=[[Hamburg]]
|STERBEDATUM=3. April 1897
|STERBEORT=[[Wien]]
}}
</code

My CCK fields:

<ul>
<li>field_geburtsort, Label. "Geburtsort"
<li>field_sterbeort, Label: "Sterbeort"
<li>field_beschreibung, Label: "Beschreibung"
<li>...
</ul>

Mapping (per <code>./admin/settings/createfromweb<code>):

<code>
geburtsort=geburtsort;
sterbeort=sterbeort;
beschreibung=kurzbeschreibung;

Result: no mapping.

...
kurzbeschreibung=beschreibung;
...

Result: no mapping.

...
KURZBESCHREIBUNG=beschreibung;
...

Result: no mapping.

...
KURZBESCHREIBUNG=field_beschreibung;
...

Result: no mapping.

...
beschreibung=KURZBESCHREIBUNG;
...

Result: no mapping.

As it seems today will be the day I loose my mind...

#18

Posted by e-anima on October 20, 2010 at 11:49am

i tried it too and get as far as you "asb".
There is a lot of data parsed and alot of unnecessary stuff like "not filtered out" like

(automatically put to starring cck field)
starring:

{{Unbulleted list|Anna Torv|Joshua Jackson|Lance Reddick|Kirk Acevedo|Blair Brown|Jasika Nicole|John Noble}}

When i will need that for my next project i will take a deep look into the module. I think it needs much work to be done. I like the module idea.

#19

Posted by brevity on October 20, 2010 at 11:57am

thanks for your consideration!

yes, it could need a fresh-up ... regarding drupal 7 it would be a chance to intregrate with token module and others