By dallas63 on

Hi guys,

Just a quick question :-)

I would like to scrape some of the data (not everything) from the table on this page: http://varde-sportsfiskerforening.dk/list_reports.php and display it in an other site - also in a table. How do i achieve this?

I don't have a clue about how to get started, and it would really mean a lot to get some help :-) Since the new site I'm making is for a nonprofit organization.

Thanks a lot.

⋅ Categories: Drupal 7.x

Comments

pbarnett’s picture

pbarnett commented

This might be helpful - http://davidvielmetter.com/tricks/howto-convert-an-html-table-to-csv-usi...

Alternatively, you could view the page source and copy/paste the HTML table into a WYSIWYG editor and edit it there...

dallas63’s picture

dallas63 commented

Thank you for quick reply.

I don't think the solutions are updating continuously - and that's what I need. Not a snapshot of the table right now, but a table that updates its content from the other table.

WorldFallz’s picture

WorldFallz commented

You might ask if they provide a feed of some type-- xml, rss, whatever. If they do, you can probably use the feeds module to import it. Even if not, you might be able to create a custom parser for feeds.

Otherwise, maybe you can use drupal_http_request.

koorenne’s picture

koorenne commented

I don't know if there is a better drupal solution, but you might want to check out http://simplehtmldom.sourceforge.net/ for the scraping part. I've used this a long time ago to scrape a table of baseball scores (this code is far from complete). You probably want to write your own module and also take care of some caching.

// Load the API
include('simple_html_dom.php');

// Create and populate the DOM object
$html = new simple_html_dom();
$html = file_get_html('http://www.google.com/');

// Parse every row and look for TH and TD
$table = $html->find("div[class=ranking] table tr");
$i=0;
foreach($table as $row) {
  foreach($row->find('th') as $cell) {
      $header[] = $cell->innertext;
  }
  foreach($row->find('td') as $cell) {
      $data[$i][] = $cell->innertext;
  }
  $i++;
}

// insert drupal theme function to theme a table with $header and $data
koorenne’s picture

koorenne commented

Here's the code for a small module called 'scraper' implementing simplehtmldom.

// Create menu
function scraper_menu() {
  $items['demo'] = array(
    'page callback' => 'scraper_page', 
    'access arguments' => array('access content'), 
  );
  return $items;
}

function scraper_page() {
  // Load the PHP Simple HTML DOM Parser
  $path = $_SERVER['DOCUMENT_ROOT'] . base_path() . drupal_get_path('module', 'scraper') . '/simple_html_dom.php';
  require_once($path);

  // Create a DOM object from a URL
  $html = new simple_html_dom();
  $html->load_file('http://varde-sportsfiskerforening.dk/list_reports.php');

  // Find the second table
  $rows = $html->find('table', 1)->find('tr');

  // Loop through all rows and save content
  $i=0;
  foreach($rows as $row) {
    foreach($row->find('th') as $cell) {
        $header[] = $cell->innertext;
    }
    foreach($row->find('td') as $cell) {
        $data[$i][] = $cell->innertext;
    }
    $i++;
  }
  
  // Return the themed table HTML
  return theme('table', array('header' => $header, 'rows'=> $data));
}