Journal 2025-06-04

So I might have fixed the path error I'd made for myself. This is a test post to find out. I'd better check image paths too. Here's Claudio having a walk -

Ok, a definite improvement. It has revealed another little issue. Escaping, that other bugbear. My pipelines need some escaping because representations of this content - markdown & HTML - are going through SPARQL queries. Looks like I've got a bit of it in the wrong place. While the image of Claudio resolves correctly in the index.html recent posts, that's not the case for the single-page renderings. The image link there looks like :

<img src=\"/images/2025-06/claudio-walk.jpeg\" alt=\"walking dog\">

Escaped quotes. Ops.

Eye Reasoner

That was pleasing. I've been wanting to do something with a reasoner in #:transmissions for a while now but never really had practical justification. I just did it for fun. Took about 30 mins.

The Eye Reasoner is Euler Path-based, works with N3. The JS lib couldn't be easier in use. Here's the Processor (pipeline node) which contains it :

(source)

Here's the transmission (pipeline) :

(source)

• and it in action :

Journal 2025-06-03

The Tensegrity Suite

It occurred to me that if anyone shows interest in the stuff I've been playing with then there needs to be a good place to discuss things. The domain (AI) is pretty hot right now, but my particular corner is niche. Still, things need to be public, must be easy to maintain. It occurred to me sticking things in Git would be useful. Then I discovered GitHub supports Discussions on repos. That'll do. So I've set up a repo called tensegrity and on it enabled Discussions. It's new to me, but I think it should work. If you are reading this then you should probably go say hello over there.

Bloody Paths!

What I have going on with this stuff right now is that a #:Transmissions pipeline reads content from markdown files in a particular dir tree, posts this off (lightly annotated with metadata) to a SPARQL store. This initial use case I call #:Postcraft, pulls this back out again and renders it as a static Web site. Like what you are looking at. This is massively over-engineered for a static site builder, but it's what seemed like a good sanity check for getting such data in and out of the store.

The thing is I've also got #:Semem looking at the contents of that store. This is intended as an LLM-oriented knowledgebase. I'd already got some bits in place, like generating embeddings from the content and use of them for semantic search in the vector store similarity sense.

I had a big push on this in the past week or so, sorting out the #:Ragno ontology for describing knowledgebases and maybe implementing a bunch of knowledge augmentation and retrieval algorithms. I say maybe because a large part of the coding was done by AI assistants, and they're not exactly great at checking their own homework. But this report on interactions gives me cautious optimism.

One thing I know for sure, this stuff is currently very fragile. And confusing. I don't know why I bothered asking ChatGPT for an illustration of this post because I could simply have taken a pic of my office.

So anyway, the bloody paths thing. To kick the tyres of #:Semem I needed a smaller set of docs than this blog stuff. I've hardly put any content in the #:Postcraft part of #:Semem itself yet. It will eventually look after the docs, but as of now, sod all there.

So I ran the #:Postcraft transmissions over it. Bugger! Totally berserk paths. The primary motivation behind #:Transmissions is to make complex processing easier to manage. However, the decomposition comes at a cost. Here are the current commands I have to run to render this site:

cd ~/hyperdata/transmissions # my local path
./del-dan.sh # clear SPARQL graphs
./trans postcraft-statics ~/sites/danny.ayers.name/postcraft #
./trans md-to-sparqlstore ~/sites/danny.ayers.name/postcraft
./trans sparqlstore-to-html ~/sites/danny.ayers.name/postcraft
./trans sparqlstore-to-site-indexes ~/sites/danny.ayers.name/postcraft

Here's the transmissions.ttl for sparqlstore-to-html :

@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix : <http://purl.org/stuff/transmissions/> .

:sparqlstore-to-html a :Transmission ;
   :pipe (:p10 :p20 :p30  :p40 :p50  :p55  :p60 :p70
          :p80  :p90 :p100) .

:p10 a :SPARQLSelect ;
     :settings :selectArticles .

:p20 a :ForEach ;
    :settings :resultIterator .

:p30 a :Restructure ;
    :settings :prepArticle .

:p40 a :MarkdownToHTML ;
    :settings :mdHTML .

:p50 a :Templater ;
    :settings :articleContentTemplate .

# store the HTML snippet
:p55 a :SPARQLUpdate ;
     :settings :updateArticleHTML .

# make the page
:p60 a :Templater ;
    :settings :articlePageTemplate .

:p70 a :PathOps ;
  :settings :filenameConstructor .

:p80 a :PathOps ;
  :settings :filenameExtension .

:p90 a :FileWriter ;
  :settings :htmlField .

:p100 a :SPARQLUpdate ;
     :settings :recordRendering .

Each of the :settings objects is a little block of statements. Each of the processors in the pipeline is a JS module. But each part is constructed simply, and I do have tolerable debugging facilities in place. It's just a bloody drag to trace through the pipeline nodes to find where I've done something stupid in path resolution. Hey ho. Claudio tells me it's walk time again. When we get back I may just have play with some of the other stuff that's cluttering up the office right now.

Journal 2025-06-01

cd /etc/nginx/sites-available
nano tensegrity.it.conf

server {
        server_name tensegrity.it;
        listen 80;

        # Hide nginx version
        server_tokens off;

        location / {
                root /home/www/tensegrity.it;
                index index.html index.htm index.ttl;
                autoindex on;
        }
}

ln -s /etc/nginx/sites-available/tensegrity.it.conf /etc/nginx/sites-enabled/tensegrity.it.conf
nginx -t
systemctl restart nginx
certbot

ln -s /home/hyperdata/semem/docs/postcraft/public /home/www/tensegrity.it/semem
ln -s /home/hyperdata/hyperdata/docs/postcraft/public /home/www/tensegrity.it/hyperdata
ln -s /home/hyperdata/transmissions/docs/postcraft/public /home/www/tensegrity.it/transmissions

/home/danny/hyperdata/hyperdata/workspaces/postcrafts/hyperdata-docs/content/raw/entries/2025-05-04_the-stack.md

Journal 2025-05-31

This morning I was woken by Federico's tractor about 15 mins before my alarm. But that was fine, last night I prepared a flask of coffee. I did about 2 hours worth of dev with Claude before getting out of bed. Then went shopping, I've just fixed the Claude artifacts and I'm very happy. A very productive morning!

JSDoc RDF Plugin

It does what JSDoc does, with an output in RDF Turtle : jsdoc-rdf on GitHub. Something I've wanted for years but never got around to. It became more pressing now I've started integrating LLMs etc into my knowledgebase setup - in Semem (which still needs a bit of tweaking for my artificial midsummer break deadline).

OpenTelemetry in Transmissions

This seems a must-have for my pipeliney thing. Ok, every node in a transmission (= pipeline) is an implementation of a Processor. That interface is descended from ProcessorImpl which does the work of running the processors. This latter now has hooks into Open Telemetry. There is space for hooks in the code responsibility for the construction of pipelines, as well as in the API bits. But being lower demand, they are lower priority.

I haven't a clue yet how best to make use of this, I believe I'll need Prometheus and/or Grafana somewhere.

Filesystem watching for transmissions

Not unrelated to the telemetry thing. Right now, the #:Postcraft doc pipelines to render this blog process every single source markdown file. So I just added this fs watcher.

The idea is that this will be a long-lived background process. When a file in a given dir tree changes, this thing wakes up and does a HTTP POST to a given endpoint. For the #:postcraft use case, the intention is that there will be a #:transmissions mini-service that will receive the POST and run the markdown-to-sparqlstore, sparqlstore-to-html pipelines.

As yet untested, but the ability to trigger transmissions via HTTP is something I've had in mind all along. There is something approximating implementation, but I'm only now getting to the stage where I can start trying it out in practice.

Tip of the Day

I've currently got Windsurf hammering away at Semem. It was having diversions where it lost its thread somewhat, then repeated earlier approaches that failed... attention deficit, dude! But this seems to have helped :

I suggest you create a file WINDSURF.md and in it place note on strategies which work and those which don't areas of the codebase that likely need attention etc.

It's also a good read for me (same deficit). Also :

remember to update WINDSURF.md as you go along. Leave yourself a note in it to remind yourself about it.