[Fwd: Re: [rdfweb-dev] Slug: A Simple Semantic Web Scutter]

[Leigh Dodds]

Leigh Dodds wrote:
> 
Apologies, missed this question whilst migrating subscriptions
between email addresses:

> -------- Original Message --------
> Subject: Re: [rdfweb-dev] Slug: A Simple Semantic Web Scutter
> Date: Fri, 10 Dec 2004 08:48:54 -0600 (CST)
> From: elijah wright <elw at stderr.org>
> To: Leigh Dodds <ldodds at ingenta.com>
> CC: rdfweb <rdfweb-dev at vapours.rdfweb.org>
> References: <41B98E8C.3090707 at ingenta.com>
> 
> 
>> http://www.ldodds.com/blog/archives/000167.html
>>
>> First version of a simple Java scutter. Does nothing especially 
>> clever, just accumulates a local cache of RDF for subsequent 
>> processing/manipulation with other tools. Implements the ScutterVocab 
>> to keep track of its travels.
> 
> have you done any scalability testing yet?  it would be most useful to
> know what the bounds of a "workable" data set for your scutter are.

I'm afraid I haven't. As noted in the 'blog posting it was mainly a
development exercise to try writing a Scutter, I've not spent time
on performance testing, tuning or clever scuttering strategies.

I think the Scutter Vocabularly support is a pretty useful feature in
its own right though. The data it collects is ripe for further 
processing to extract metrics about crawling speeds, errors encountered,
rates of change of FOAF documents, etc.

Cheers,

L.