How can I extract data from Wikipedia's list pages?

Question

Essentially, what I am trying to achieve is extract the name and country of birth of every composer from this Wikipedia list. This is my first attempt ever at using SPARQL, but here's what I've managed so far:

SELECT ?name ?birthplace WHERE {
?person foaf:name ?name .
?person dbpedia2:placeOfBirth ?birthplace .
?person <http://purl.org/dc/terms/subject> <http://dbpedia.org/resource/Category:Women_classical_composers> .
?country rdf:type dbpedia-owl:Country .
FILTER (str(?birthplace) = str(?country))
}
ORDER BY ?name

I'm running this through dbpedia's SNORQL and while I get an output in the format I want, I'm only using results from the "Women classical composers" Wikipedia category which only covers a portion of what is displayed in the "List of female composers by name" list. I've tried finding categories common to all the composers in the list which would allow me to select them with relative ease, but couldn't find any which were truly all-encompassing.

I do realise my question is rather similar to this one, but I unfortunately could not figure out my problem after reading this.

Accepted Answer

This one can't be done against the public endpoint.

There is a wikipedia data set that contains facts of the form "Page X links to Page Y", so if you had this data loaded into your SPARQL engine you could replace

?person <http://purl.org/dc/terms/subject>
    <http://dbpedia.org/resource/Category:Women_classical_composers> .

with

<http://dbpedia.org/resource/Category:Women_classical_composers> 
    <http://dbpedia.org/ontology/wikiPageWikiLink> ?person .

but the "wikipedia pagelinks" are not loaded into the public endpoint. If you loaded DBpedia into your own triple store, you could do this query.

Note to get your list of Woman classical composers you can (i) load just the pagelinks in your own triple store and do the rest against the DBpedia endpoint or (ii) use tools like awk to suck just a few pagelinks out of the N-Triples file (much faster than loading it all into a triple store) or (iii) extract links from the Wikipedia page.

How can I extract data from Wikipedia's list pages?

Follow this question

Upcoming Events

Related questions