[foaf-protocols] P2P FOAF search

Dan Brickley danbri at danbri.org
Thu Jun 4 18:17:51 CEST 2009


On 4/6/09 17:24, Luke Maurits wrote:
> Greetings list,
>
> I'd like to gauge the community's interest in developing an "official"
> protocol (perhaps built on top of XMPP) for peer to peer searching of
> FOAFspace.

+1 on XMPP :) I made some experiments with it here - 
http://svn.foaf-project.org/foaftown/jqbus/intro.html

> The broad idea is this:  Each node in the network would typically "know
> about" (i.e. know the URI of and have an efficiently searchable
> representation of) a relatively small number of FOAF profiles, on the
> order of magnitude of 10.  I envisage that the typical client
> application for the network would also provide some other services,
> like activity stream aggregation or the like, so the obvious candidate
> list of "known about" FOAF profiles would be precisely those that the
> user is "watching".  Anyway, each node is also aware of a relatively
> small number of other nodes on the network, and a user can search for a
> FOAF profile by name, mbox, mbox_sha1sum, whatever, by passing the
> search criteria and its contact details (IP, port) to each of the few
> nodes it knows about - and they either respond if they know of any
> profiles matching the criteria or pass it on to *their* known nodes,
> and so on, propagating the query throughout the network until somebody
> answers.  Basically this is exactly the same idea behind how P2P file
> sharing applications like Limewire locate files, just applied to FOAF.
>
> The motivation is essentially just that a network like this might let
> us get closer to complete and up-to-date searchability of FOAFspace
> faster and cheaper than waiting on people to build large data centres
> and index everything by extensive scuttering.

I certainly think this kind of search is worth exploring, but I'd 
hesitate re using it just for things described directly in the FOAF 
vocabulary. One of the core things re FOAF is that it's built on top of 
the general purpose RDF system for describing things. The FOAF spec just 
provides a handy dictionary for saying things about people, but it was 
always intended to be used with data about other things too - in 
particular the artificts of friendship and collaboration, including 
photos, videos, music, books, papers, software etc.

So I encourage you to explore this but try to have a design which from 
day 1 doesn't exclude these inter-related types of data. For example 
searching in this way for information about opensource software could 
use DOAP, an RDF vocab which links FOAF to the world of opensource 
collaboration. Or SIOC, which covers in more detail the description of 
content and discussion in online fora.

Did you take a look at SPARQL yet?


> By the community agreeing upon an exact protocol for doing this, we'd
> be letting anybody write clients to join the network, in whatever
> language they like.  It also opens the door to people coming up with
> alternative ideas for the network, for instance some people may like
> the idea of, rather than having a node know about 10 or 20 profiles
> that the user is interestd in, having a node which just fields queries
> out of a database of say 1000 or so profiles fetched by scuttering.
> That sort of node would significantly improve the quality of searches
> and is quite within the resource limitations of a dedicated hobbyist.
> Possibly there are yet other kinds of node someone might decide to
> implement.

If the base tech can be built from existing pieces (SPARQL, XMPP; maybe 
Lucene-style full text search too) ... it should be possible to tweak 
and experiment with various indexing and harvesting strategies...

> I know of at least one person other than myself who has expressed some
> interest in developing software for a network like this, and there are
> quite possibly others who will become interested, so hopefully if we
> can put some work into developing a protocol then work on clients may
> begin to slowly materialise.

http://aksw.org/Projects/xOperator is also related, see also 
http://blog.dbtune.org/post/2008/02/25/Playing-with-SPARQL-and-XMPP and 
I think Benjamin Nowack has made some similar experiments, re the layer 
for going between user-supplied queries and sparql: see 
http://semsol.org/semcamp/sparqlbot


> Anyway, if anyone wants to ask questions about P2P searching of
> FOAFspace, point out existing work along the same lines, express
> interest in working on the protocol or anything like that, feel free to
> reply.

Definitely interested! I think the first point to think about is what 
requirements these scenarios might have which diverge from straight "out 
of the box" SPARQL querying. One thing to keep an eye on is the new 
SPARQL group at W3C, who for example are looking at adding constructs 
for full-text query (see 
http://www.w3.org/2009/sparql/wiki/Feature:FullText) and are also I 
think talking about simple distributed query as well. The work on 
service description there is also relevant - see 
http://www.w3.org/2009/sparql/wiki/Feature:ServiceDescriptions

Re the XMPP side, be warned that the binding of SPARQL to XMPP I worked 
on (the jqbus stuff) has a serious flaw: it puts the entire response 
from the SPARQL db into a single XMPP IQ message. For large results this 
is problematic; it won't pass between all server-to-server links in the 
public Jabber network...

Do keep us posted as you explore this,

cheers,

Dan


More information about the foaf-protocols mailing list