Journal 2025-06-14

Remind me why I'm doing this...

I had one of those moments just now. I've intermittently put a lot of time into Semem, my Semantic Memory thing. The sessions have got increasingly intense, it went something like this :

  1. Rough scaffolding implementation - a setup that included embeddings for semantic search plus using an LLM for concept extraction. It also had a bit of support for a temporal dimension, short-term & long-term memory (I lifted what I could from the design of Memoripy, given I'm using Node JS). I got a SPARQL store client in there, but wasn't really doing anything with it at this stage, data was mostly JSON.
  2. Ragno ontology design - I needed a model of a knowledgebase to build from. I went through several back-of-envelope cycles with it, the usual malarky of trying to pin things down to keep some semblence of control while still leaving things open enough to allow serendipity to play. Then I stumbled on the NodeRAG paper. Aha! This is what I want. Except they had things quite tightly pinned down, and the notion of non-retrievable resources on the Semantic Web? Nah. So I kinda merged the floppy thing I already had with this more rigid thing, implemented the basic structures. A huge bonus with the paper was that it outlines a bunch of different algorithms for working with knowledgebases.
  3. Ragno implementation - with considerable help from Claude I set about implementing them in the Semem environment. Lots of cycles. Claude is such a fibber about when things are complete & working. After a long journey through the gaslighting, I believe most, if perhaps not all, work.
  4. ZPT ontology design - that's Zoom, Pan, Tilt, not as Claude kept insisting in the docs Zero-point traversal. I looked that up, very much expecting it to be a thing, it makes sense for a curve, though traversal hints at this other form of graphs. Apparently not. Claude's imagination. If it becomes a thing in quantum in the next year or two, I'm joining a monastery. Zoom, Pan, Tilt is for knowledgebase navigation, using those cinematic analogies. I'd had it sketched out for a while. Finding the NodeRAG paper was also serendipitous in helping me think about this separately from the core knowledgebase description. (foraging for the win!) Maybe not orthogonally, but at least loosely-coupled. Being still fresh from the Ragno implementation I rolled with the momentum, got this bashed out. Again, I believe it mostly works - all the demo scripts (under examples/) have worked at least once. Heh, except those under pending, haven't a clue where I'm up to with those.
  5. MCP Implementation - I'd already got approximate HTTP API endpoints in place (along with very sketchy UI for trying things out), and even a first-pass, untested MCP. But it became increasingly obvious this was something I needed to do a bit more properly. Dual function : selectively expose the stuff under the hood for sanity-checking, plus it might actually be useful.

Which brings me back to my starting point. Useful for what..?!

I spend so much time making tools to make tools (ditto in real life) I forgot the original motivation.

Benchmarks! The primary goal of all modern software!

Ok, that was it, Personal Knowledge Management. Which may be project specific, relatively narrow domain. Not one size fits all. I play around with music and associated technology, have all kinds of other little hobby (and house-related) projects on the go. I've thousands of bookmarked links relating to this stuff. I've got loads of my own notes scattered all over the place, backups of intermittent blog posting going back a couple of decades. This is all knowledge that could potentially help me with my activities, if only I could access it in a useful way.

Yeah, so. How do I get all the material into my soon to be knowledgebase? Well, the other project I've been putting the other bulk of my time into is Transmissions, a pipeliney thing into for just such operations. That too is mostly in place/rather fragile in places.

I'm going to be spending the next few weeks (until 2025-07-03) at least making a note of loose strings in my tensegrity structure of projects. Then I'm going to try to avoid this stuff for a month.

Wish me luck.

PS. I have a Use Case : use Semem to help write a paper about...Semem! #:dogfood