[foaf-protocols] P2P FOAF search
danbri at danbri.org
Thu Jun 4 18:17:51 CEST 2009
On 4/6/09 17:24, Luke Maurits wrote:
> Greetings list,
> I'd like to gauge the community's interest in developing an "official"
> protocol (perhaps built on top of XMPP) for peer to peer searching of
+1 on XMPP :) I made some experiments with it here -
> The broad idea is this: Each node in the network would typically "know
> about" (i.e. know the URI of and have an efficiently searchable
> representation of) a relatively small number of FOAF profiles, on the
> order of magnitude of 10. I envisage that the typical client
> application for the network would also provide some other services,
> like activity stream aggregation or the like, so the obvious candidate
> list of "known about" FOAF profiles would be precisely those that the
> user is "watching". Anyway, each node is also aware of a relatively
> small number of other nodes on the network, and a user can search for a
> FOAF profile by name, mbox, mbox_sha1sum, whatever, by passing the
> search criteria and its contact details (IP, port) to each of the few
> nodes it knows about - and they either respond if they know of any
> profiles matching the criteria or pass it on to *their* known nodes,
> and so on, propagating the query throughout the network until somebody
> answers. Basically this is exactly the same idea behind how P2P file
> sharing applications like Limewire locate files, just applied to FOAF.
> The motivation is essentially just that a network like this might let
> us get closer to complete and up-to-date searchability of FOAFspace
> faster and cheaper than waiting on people to build large data centres
> and index everything by extensive scuttering.
I certainly think this kind of search is worth exploring, but I'd
hesitate re using it just for things described directly in the FOAF
vocabulary. One of the core things re FOAF is that it's built on top of
the general purpose RDF system for describing things. The FOAF spec just
provides a handy dictionary for saying things about people, but it was
always intended to be used with data about other things too - in
particular the artificts of friendship and collaboration, including
photos, videos, music, books, papers, software etc.
So I encourage you to explore this but try to have a design which from
day 1 doesn't exclude these inter-related types of data. For example
searching in this way for information about opensource software could
use DOAP, an RDF vocab which links FOAF to the world of opensource
collaboration. Or SIOC, which covers in more detail the description of
content and discussion in online fora.
Did you take a look at SPARQL yet?
> By the community agreeing upon an exact protocol for doing this, we'd
> be letting anybody write clients to join the network, in whatever
> language they like. It also opens the door to people coming up with
> alternative ideas for the network, for instance some people may like
> the idea of, rather than having a node know about 10 or 20 profiles
> that the user is interestd in, having a node which just fields queries
> out of a database of say 1000 or so profiles fetched by scuttering.
> That sort of node would significantly improve the quality of searches
> and is quite within the resource limitations of a dedicated hobbyist.
> Possibly there are yet other kinds of node someone might decide to
If the base tech can be built from existing pieces (SPARQL, XMPP; maybe
Lucene-style full text search too) ... it should be possible to tweak
and experiment with various indexing and harvesting strategies...
> I know of at least one person other than myself who has expressed some
> interest in developing software for a network like this, and there are
> quite possibly others who will become interested, so hopefully if we
> can put some work into developing a protocol then work on clients may
> begin to slowly materialise.
http://aksw.org/Projects/xOperator is also related, see also
I think Benjamin Nowack has made some similar experiments, re the layer
for going between user-supplied queries and sparql: see
> Anyway, if anyone wants to ask questions about P2P searching of
> FOAFspace, point out existing work along the same lines, express
> interest in working on the protocol or anything like that, feel free to
Definitely interested! I think the first point to think about is what
requirements these scenarios might have which diverge from straight "out
of the box" SPARQL querying. One thing to keep an eye on is the new
SPARQL group at W3C, who for example are looking at adding constructs
for full-text query (see
http://www.w3.org/2009/sparql/wiki/Feature:FullText) and are also I
think talking about simple distributed query as well. The work on
service description there is also relevant - see
Re the XMPP side, be warned that the binding of SPARQL to XMPP I worked
on (the jqbus stuff) has a serious flaw: it puts the entire response
from the SPARQL db into a single XMPP IQ message. For large results this
is problematic; it won't pass between all server-to-server links in the
public Jabber network...
Do keep us posted as you explore this,
More information about the foaf-protocols