Do you use full text search with SPARQL? If so how and why?

We use the IndexingSail from uSeekM as documented here: https://dev.opensahara.com/projects/useekm/wiki/IndexingSail, and here http://opensahara.com/blog/geospatial-search-rdf-data. It provides a layer on top of triplestores that use the Sesame API, thus can be used with a large number of triplestores (Sesame Native, Bigdata, 4Store, Owlim, Mulgara, ...). The reason for using the uSeekM is that we don't have vendor lock-in. The same search syntax can be used for all these stores. The open source version of uSeekM uses a Postgres database with GIN inverted indexes for building the text-index. SPARQL and SerQL queries are rewritten by uSeekM and federated between the text-index and the triplestore.

Example query:

PREFIX search: <http://rdf.opensahara.com/search#>
SELECT ?record
WHERE {
  ?record a <http://purl.org/ontology/mo/Record>
  ?record ?predicate ?match .
  FILTER(search:text(?match, "Florence & Machine"))
} ORDER BY ?record

The syntax for search:text filters is described here: https://dev.opensahara.com/projects/useekm/wiki/IndexingSail#Full-Text-Search

Disclaimer: I work for TalkingTrends, and uSeekM is developed by us for this purpose. It also does geospatial indexing of geometries for efficient GeoSPARQL querying.

I haven't used it personally, but Ontotext's BigOWLIM also lists support for integrated full-text search as one of its features.

Then there is the Nepomuk LuceneSail, which I have used in the past. Example query: Search for any resource an RDFS-label value that contains the string "person":

PREFIX search:   <http://www.openrdf.org/contrib/lucenesail#>
SELECT ?x ?score ?snippet WHERE {
?x search:matches ?match.
?match search:query "person";
  search:property rdfs:label;
  search:score ?score;
  search:snippet ?snippet. 
}

The main use case for any such solution typically has to do with document management. For example, we have built an RDF-based system for experimental researchers to document their research, where lots of semi-structured data is added (samples x, y, z, experiments a, b and c, and so on), but where users can also add (Office, PDF, ...) documents (for example as input literature, or as a publication resulting from a particular bit of research).

We have used a full-text search solution to facilitate doing keyword searches that consider both the structured RDF (tags, entity types, all other sorts of annotations) as well as the full-text of the documents. A non-indexed triplestore will be dead-slow when evaluating string searches on literals that basically contain the entire text of a multipage word file, so we use an indexer.

Allegrograph also has the ability to do free text searches. In SPARQL you use the fti:match predicate.

Overview: http://www.franz.com/agraph/support/documentation/v4/text-index.html

Example SPARQL: http://www.franz.com/agraph/support/learning/SPARQL-fti-match.lhtml

I worked developing a project where user assets were tracked by a resource id, which had label, comments, and other properties. Text searching on any literal was a way to provide a loose matching mechanism for user assets, that allowed them to quickly narrow results. Since it searched all literals, the semantic meaning of the text became unimportant, but as a filter mechanism it allowed one to quickly narrow results by its target text.

Also, FWIW, the use cases that always drives me to using free text search is:

• Because a regex doesn't do the trick. Either regex on a literal would involve scanning too many literals and bring the query to a crawl, or the search expression isn't something I can do in a regex easily, e.g. search 5 keywords order independent, or search for near neighbors, etc.
• Or, if the search is something that you present to a user and inject into your SPARQL. Far more users are more comfortable with Google like search expressions than with a regex pattern, so the choice to use free text is driven by your expectation of user need.

Anzo supports it in a way similar to, but not interoperable with, the systems you mention. There's some incomplete documentation of it at http://www.openanzo.org/projects/openanzo/wiki/SPARQLExtensions (search for "textmatch").

We use it for the "obvious" reason -- it gives users an easy way to start narrowing down large sets of data to the records they're interested in.

So far, I think the other answers miss the point that text that is relevant to a node can be dispersed to other nodes. For instance, suppose you're looking for a company and you do a search for "Larry Page". Well, :Larry_Page may exist as a :Person, but you want :Google to show up. Somehow the system needs to follow the link from :Larry_Page to :Google.

Back in 2006 I was involved in something called the Global Performing Arts Database. GloPAD was based on PostgreSQL, but the data model was similar to an RDF system in that we had a very detailed ontology that distributed text into many nodes. I built a system that queried our data dictionary and could construct a graph of traversal paths across tables and rows. The system was capable of static analysis of this graph as well as dynamic traversal in real time.

A key thing we developed was traversal rules that would determine which links would yield relevant text and which wouldn't. The most important rule in the box was that the system would not traverse links that went to an object of the same type as the starting object. In the example above, for instance, there might be a

:Microsoft :isCompetitorof :Google .

but the system would not follow it because :Microsoft is also a :Company. In GloPAD this was easy because the types were mutually exclusive.

In GloPAD we did the traversal whenever a new record was added or was changed, creating a set of synthetic documents that were indexed in a conventional full-text system. This had the advantage of predictable query time. Performance was fine and our customers were happy with the search results.

It would also be possible to find matching node with matching text and traverse the graph at query time to determine what other nodes are related and how to rank them. Here it would be easy to tweak the ranking function to depend on relationship types and relationship distance, but I'd be concerned that queries could be slow if you hit a highly connected part of the graph.

The main use case of "full text search" in SPARQL is to delude customers by giving them a buzzword. With "full text search" managers get a checkbox to judge a SPARQL-based product or service without having to understand and think about which tools to use for solving which problems. Instead you should distinguish between:

• string searching as simple detection of substrings in literals
• literal pattern matching which is not supported by SPARQL.
• text retrieval based on ranking algorithms (basically vector space model), such as implemented in Lucene

People must likely think about the third but they only get the first.

The Virtuoso Faceted Browser Service is a powerful demonstration of its Full Text Indexing in action for querying over large scale Linked Data.

Try the following steps for a live demonstration of its use:

1. Goto: http://lod.openlinksw.com -- remember, you have 21 Billion+ triples (I state this number with what follows in mind)
2. Enter pattern: School
3. When you get the first results page, click on the "Type" link in the Navigation section
4. You'll see: http://lod.openlinksw.com/fct/facet.vsp?cmd=load&fsq_id=36
5. Few Retry Button Clicks later (depending on state of working set cache) you see: education.data.gov.uk in the list
6. Then you'll have: http://lod.openlinksw.com/fct/facet.vsp?cmd=load&fsq_id=37
7. Optionally, click on "Places" link from the Navigation section and you get: http://lod.openlinksw.com/fct/facet.vsp?cmd=load&fsq_id=38
8. Click on "Entity1" link at page top and you get a list
9. Use "Save" for permalink and sharing
10. etc..