Querying for literals in a more efficient manner

Question

I'm building a small keyword search on a locally installed Virtuoso server containing the DBpedia dump. For example, if the user is searching for Japan, the keyword search will basically implement this query:

SELECT * WHERE {
 ?subject  <http://www.w3.org/2000/01/rdf-schema#label> ?Literal
 filter(REGEX(STR(?Literal) ,'.*Japan.*','i'))
}

The problem is that this query takes too much time to run. I wonder if there's another alternative or best practice to overcome this.

I thought of adding LIMIT and OFFSET to the query and preview to the user results 10 by 10, but i think it will take more time to match to the query OFFSET. What do you think ?

Accepted Answer

It's important to understand how such queries are run. Your query will scan through all values of rdfs:label in the data (which is a lot) looking for the substring "Japan". This is never going to be efficient: REGEX is not an alternative for full-text search.

Thankfully, many SPARQL engines support full-text search indirectly, Virtuoso included:

SELECT * WHERE {
  ?subject  <http://www.w3.org/2000/01/rdf-schema#label> ?Literal .
     FILTER bif:contains(?Literal, "Japan")
}

Note again that the bif:contains function is not defined in SPARQL, but is rather a custom function implemented by Virtuoso. This version of the full-text query will run as a lookup on a inverted index (not scanning lots of data) and should be much more efficient.

EDIT Actually, the syntax might be:

SELECT * WHERE {
  ?subject  <http://www.w3.org/2000/01/rdf-schema#label> ?Literal .
  ?Literal bif:contains "Japan" .
}

(I don't have the necessary foo to find the official Virtuoso documentation, but various examples quote the above syntax.)

Querying for literals in a more efficient manner

Follow this question

Upcoming Events

Related questions