Scraping a messy html website with PHP

I am in the following situation. I am trying to convert a messy scraped html code to a nice and neat xml structure.

A partial HTML code of the scraped website:

<p><span class='one'>week number</span></p>

<p><span class='two'>day of the week</span></p>
<table class='spreadsheet'>
table data
</table>

<p><span class='two'>another day of the week</span></p>
<table class='spreadsheet'>
table data
</table>

<p><span class='one'>another week number</span></p>

ETC

Now I want to create the following xml structure with php:

<week number='week number'>
 <day name='day of the week'>
  <data id='table data'>table data</data>
 </day>

 <day name='another day of the week'>
  <data id='table data'>table data</data>
 </day>
</week>
<week number='another week number'>
 ETC
</week>

Have been trying the simple html dom method, but have no idea how to get the next sibling and check wether it is a new day of the week, a new table data or a new week etc..

I am, of course, also open to other solutions.

Thanks.

Cheers, Dandoen

Answer

There is no silver bullet. A typical way to handle this, would be to first filter the html through htmltidy, to get a somewhat predictable tag soup, and then feed it to a parser (Such as DomDocument). Then use DomXPath to select the nodes that you need and assemble an intermediate structure of associative arrays and finally transform this into an output xml document.

Hint: Use firebug’s “Copy XPath” feature to grab the xpath expression for a node.