Journal 2025-03-03

I'm battling with my tendency to get overwhelmed by stuff. So much to do... Hey ho, keep breathing, get on with it.

The irony is, a major motivation behind what I'm working on at desk is to get the computer to reduce the amount of stuff I have to do. C'est la vie.

Ok. Apart from the stack of real-world admin paperwork staring at me from the big table, my main priority is getting my documentation setup sorted (again). I think I'm overcome some bits that were blocking me, yesterday I had a bit of new code dev go very smoothly.

Postcraft Transmissions

So I've called my doc setup #:postcraft, tagline the most over-engineered static site builder ever! Let me review how far I've got. Two separate #:transmissions right now - pipeliney things expressed in Turtle RDF. The first one walks the local filesystem, reads the markdown format docs, templates them into SPARQL UPDATE queries, posts them off to a store :

# src/applications/test/file-to-sparqlstore/transmissions.ttl

@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix : <http://purl.org/stuff/transmissions/> .

:md-to-sparqlstore a :Transmission ;
     :pipe (:p10 :p20 :p30 :p40) .

:p10 a :DirWalker ;
    :settings :dirWalker .

:p20 a :FileReader ;
     :settings :readerSet .

:p30 a :MakeEntry ;
     :settings :entryDefaults .

:p40 a :SPARQLUpdate ;
     :settings :sparqlUpdate .

The config is mostly declared in another Turtle file, statements referred to by the :settings properties above.

The other will run queries over the SPARQL store, template the results into HTML :

...
:sparqlstore-to-html a :Transmission ;
 :pipe (:p10 :p20 :p30  :p40 :p50 :p70 :p80) .

:p10 a :SPARQLSelect ;
     :settings :sparqlSelect .

:p20 a :ForEach ;
    :settings :resultIterator .

:p30 a :Restructure ;
    :settings :prepArticle .

:p40 a :MarkdownToHTML ;
    :settings :mdHTML .

:p50 a :Templater ;
    :settings :articleContentTemplate .

:p70 a :Templater ;
    :settings :articlePageTemplate .

:p80 a :Restructure ;
    :settings :resolveFilepath .

:p90 a :FileWriter ;
    :settings :saveArticlePage .

Aha! I remember where I was up to. Thank you Rubber Duck Journaling.

The last couple of processes in that pipeline didn't support what was needed for determining the file path/name. But last coding I did was implement a new process for that (src/processors/json/StringOps.js - it might well get renamed/moved to something better later, but is ok for now) along with a integration test app for it (src/applications/test/stringops).

Used in a #:transmission like :

:pFilename a :StringOps ;
  :settings :filenameConstructor .

With statements in an associated config.ttl :

:filenameConstructor a :ConfigSet ;
  :asPath true ;
  :targetField "aFilepath" ;
  :values (:a :b :c :d) .
  :a :string "TEST_PASSED" .
  :b :field "fields.fieldB" .
  :c :field "fields.fieldC" .
  :d :string "TEST_PASSED" .

The :field properties refer to JSON paths in the message flowing through the transmission. The objects indicated get joined into a fs-compatible path. So when invoked with a message at command line :

cd ~/hyperdata/transmissions # my local path
./trans stringops -m '{"fields": {"fieldB" : "TEST","fieldC":"_PASSED"}}'

The resulting message will contain :

{
  "aFilepath": "TEST_PASSED/TEST/_PASSED"
}

Oops. It's missing :d. Grr.

This is puzzling, the config dataset contains :

<http://purl.org/stuff/transmissions/d> <http://purl.org/stuff/transmissions/string> "TEST_PASSED" .
<http://purl.org/stuff/transmissions/d> <http://purl.org/stuff/transmissions/string> ".html" .

Hah, fool danny. I'd got another block in the config for testing the processor when :asPath was false, when the values should be treated as simple strings and concatanated rather than using path.join(). I'd used the same names : :a :b :c :d, thus making pairs of statements for each of those subjects.

The JS code was fine, after renaming the nodes in the Turtle to :a2 :b2 :c2 :d2 it now produces:

"aFilepath": "TEST_PASSED/TEST/_PASSED/TEST_PASSED",

Dogwalk time.