Journal 2025-02-27

I broke my earlier #:postcraft site builder setup (that publishes this) a few weeks ago. Rather than spend time fixing that as it was I decided to push on in the direction I want things to go. #:postcraft is an application of #:transmissions, my pipeliney thing.

A key aspect of my plans is using a SPARQL store for data (content) management. I've made quick & dirty #:transmissions #:processors to do SPARQL UPDATE and SELECT. They're using #:nunjucks templating - which involves a bit of code duplication at the moment, I already have a :Templater processor. But I want to get the publishing working end-to-end before tidying things up (there is tonnes of tidying to do).

I've got as far as having near-enough templating (the #:sparql-diamonds strategy) to get the rendered content in #:messages. The target paths for the ultimate HTML pages is in the messages too. But while I've got some basic (JSON) message structuring in the :Restructure processor, it only covers moving things around, I need to mess with the JSON keys & values as strings to get the target file path/names.

I'm aware there are existing libs for JSON transformation kind of thing. I actually had to use one, JMESPath, as a core part of my last quasi-dayjob. In a past life I spent quite a lot of time around XSLT, same general idea. But I've been reluctant to commit to any lib for this, because #:transmissions processors are intended to be small, composable units, and I needed to discover what was needed. Restructuring the shape of the JSON is one aspect. I've got a copy - (currently misnamed rename) and a remove.

StringOps Processor

Ok, another pass at the procedure for making a new #:transmissions #:processor (I want AI to do all this for me asap).

Rough idea of what's needed, with example(s)

Input :

a root target path A is specified as a config setting (eg. /home/danny/sites/strandz.it/postcraft/public)
a relative path B is provided in the message (the SPARQL SELECT results give a message.currentItem.relPath.value, eg. data/input/2025-02-11)
a slug is provided C in the message (message.currentItem.slug.value, eg. input2)
a filename extension is specified D as a config setting (eg. .html)

Process :

A,B,C,D are joined to make E

Output :

Side Effects : none.

Hmm. In this case it isn't a simple string concat, an fs path is needed (with appropriate path.sep as needed), Ok, add another Input :

the nature of the operation - ok, for now all that's needed is concat-like, either simple string or path

Determine (more formal) Processor Signature

Heh, I see the Signature (expressed in a comment) src/processors/example-group/ExampleProcessor.js is lagging what I have in code. The inputs can come from message, manifest.ttl or config.ttl transparently in that order of precedence. ...And even the code isn't updated to where I'm at in my head - Processor.getProperty(predicate term) should be marked deprecated, in favour of getValue/getValues. I'll get back to that...

A major #:todo is to make a Single Source of Truth, terms for (something like) a sig: http://purl.org/stuff/processors ontology. It should be possible to reason about these things.

But this'll do for now. First pass, in a transmissions.ttl I'll have :

:px a :StringOps ;
  :settings :filenameConstructor .

In the corresponding config.ttl :

:filenameConstructor a :ConfigSet ;
  :asPath true ;
  :values (:a :b :c :d) .
  :a :string "/home/danny/sites/strandz.it/postcraft/public" .
  :b :field "currentItem.relPath.value" .
  :c :field "currentItem.slug.value" .
  :d :string ".html" .

The path in :a should come from the manifest.ttl (for site strandz.it), but that'll happen automatically anyway. But it will need a sensible default in config.ttl.

Copied and renamed dir src/applications/example-application to src/applications/test/stringops

Do boilerplate

First question is, will this new processor fit in an existing group? The functionality is cross-JSON-structure string-ish ops. I do have a group processors/text. But the immediate use case is for a path, and there's processors/fs. Hmm... For now I'll put this new processor in processors/json, not a big deal (which suggests it should go in processors/staging). Decide later!

If a new processor group was needed, I'd need to make a copy of (& rename) src/processors/example-group (and add a ref in src/engine/AbstractProcessorFactory.js). But using src/processors/json only means copying src/processors/example-group/ExampleProcessor.js and adding a ref in src/processors/json/JSONProcessorsFactory.js.

Implement!

I made a little util to help with lists for (which I'm certain Bergi will tell me is unneccesary), used in /home/danny/hyperdata/transmissions/src/processors/json/Restructure.js as: GrapoiHelpers.listToArray(config, this.settingsNode, term)

Ew, I think I may have some code duplication around Restructure. A couple of a days ago I added src/utils/JSONUtils.js to address fields in the message, which I needed for MarkdownToHTML. Whatever, there's JSONUtils.find() which can do the getting needed here (hmm, it maybe should be incorporated into processor.getValues() too).

Test!!