Hello (again) World!

Hello (again) World!

actually Journal : 2024-05-18 but as this will be the first visible post, that title will do

Well, that was incredibly gruelling. But I've now got a working static site generator, #Postcraft, and I'm using it to generate this page. It's not pretty, but it's a start.

#Postcraft is only an application of something I'm calling #Transmissions, a pipeliney thing to help me put together other (more interesting) systems. I've spent ages on this thing, and raced the last mile to get this working end-to-end, leaving loads of dreadful hacky code in my wake. But it puts me in a good place from which to move forward. My immediate next step is just to sort out the TODOs in the codebase, so I can dip in to tidy up and know what I'm doing. Meanwhile, I can start using it for some of that more interesting stuff, and blog about it!

2024-05-18

Journal : 2024-05-16

Journal : 2024-05-16

Yesterday, after lots of dreadful hackery, I finally got to a key milestone with #Postcraft static site builder (an application of #Transmissions). It's working end-to-end. There's still a massive amount of work to do on all levels, but it kind-of corresponds to a Minimum Viable Product (that only I will ever use). Enough is in place that I can actually start using it, it feels like a huge weight off my shoulders.

Today, 5 priorities :

  1. Make the pages bare-minimum pretty to put online
  2. Put online

Stop. Two will do.

The page styling (CSS now, later little bits of JS etc) files should be declared in the manifest, but for now I'll just copy files over manually. Actually, why bother copying, might as well just point to the originals for now.

Oh FFS!!! It seems I changed something last night after it was all working, and now it's not. I've done git reset --hard, nope. (Andwith that wiped some bits of tidying. Hey ho).

So I'll have to go back to the start of the pipeline, trace things step-by-step. Which does actually give me what might be a good idea : the services do have a signature, very like those of function arguments/return value - what they expect to receive, what they pass on. Copilot in this (markdown) typing just suggested a "contract" for a service - I like that. So it should be straightforward to add some flagging, I guess in the common base class (in Service.js), in the first instance just for logging/debugging. But the same thing should lend itself to a natural interface for unit-ish testing.


Carried over from yesterday:

Plan

Postcraft

Remember I need to take down the old server

  • upload to server

  • make everything work

  • Hello World (again) post

  • Transmissions/Postcraft overview post

  • pinned posts

  • Atom feed

FOAF-Retrospective

  • FOAF video cleanup
  • Placeholders + manifest for site
  • find papers

Kaggle-Preferences

  • get set up, run the demo
2024-05-18

Journal : 2024-05-15

Journal : 2024-05-15

Slow progress on #Postcraft.

Yesterday I spent a very long time trying to figure out why, although I had s:Unfork working in isolation, it wasn't working after s:DirWalker. I got it working, but haven't a clue what the problem was. Note to self : write more tests.

I just spent 2 hours trying to figure out why my RDF handling code wasn't working after a switch(x){ case y: ... }. Foooolish Danny. I'd forgotten to put return in the case blocks

I wasn't getting the properties I needed in the context to render the front page. But now I believe I am.

{
  "rootDir": "/home/danny/HKMS/postcraft/danny.ayers.name",

  "siteURL": "https://danny.ayers.name",
  "subdir": "blog",

  "indexPage": {
    "filepath": "public/blog/index.html",
    "templateFilename": "layouts/mediocre/index-page_template.html"
  },

  "filepaths": [
    "2024-05-03_one.md",
    "2024-05-03_two.md"
  ],
}

Ok, I'm no longer concerned with the original markdown files, so before going any further I'll change filepaths to slugs. These will include the local subdirs leading to the slug proper, but no need to worry about that now. Especially since I noticed that s:DirWalker isn't recursing properly.

Front page is rendering - Major milestone reached!!!

TODO include milestone screen shot here

The code is so buggy & hacky, and the current end result so ugly. But it's working end-to-end.

There's a massive amount of code tidying & refactoring to do, but that's not one of the priorities right now. They are:

  • Make the pages pretty enough to put online, and put online
  • Organise the TODOs so that I'm clear what needs doing
  • Write the 'Hello (again) World!' post

I've done enough prep for these that they shouldn't take too long.

Next I need to sort out my plans for #FOAF-Retrospective. I'll be needing #Transmissions & #Postcraft in various places for that, so all this will still be getting attention.

I must avoid yak shaving but as I'm sorting out all the TODOs, I should keep an eye on the project management project bigger picture.

Plan

Postcraft

Remember I need to take down the old server

  • upload to server

  • make everything work

  • Hello World (again) post

  • Transmissions/Postcraft overview post

  • pinned posts

  • Atom feed

FOAF-Retrospective

  • FOAF video
  • Placeholders + a:manifest for FOAF-Archive
  • find papers

Kaggle-Preferences

  • get set up, run the demo
2024-05-18

Journal : 2024-05-14

Journal : 2024-05-14

This morning, post office then shopping with Mari. I did the minimum that seemed necessary on Bergi's 'Unified Landscape' post, but when I pinged him he'd told me he'd already posted it. Not to worry, I'm very encouraged that he's been looking at similar things to me.

Progress on #Postcraft! Last night I got the pipeline as far as rendering the individual entry HTML. I did get rather quick & dirty with it, but that's fine for now. I've already got a huge todo list, mostly refactorings that will be straightforward. Now I just want to keep the momentum up, get the whole thing working end-to-end, make it nice later.

TODO The how-to guide needs tips, s:ShowMessage etc

    trm:pipe (:s1 :s2 :s2 :s4 :s5 :s6 :s7 :log1 :s8 :s9 :s10 :s11 :s12 :s13 :s14 :s15 :s16) .
...
:s7 a :PostcraftPrep .

:log1 a :ShowMessage .

:s8 a :MarkdownToHTML .

Oops!

s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.s2.

Check that pipe.


TODO it'd be nice to give the context etc objects names, but it's something around Prototype,

   context.constructor.name = 'Context'
   ->
   TypeError: Cannot assign to read only property 'name' of function 'function Object() { [native code] }'

not obvious, https://stackoverflow.com/questions/17169335/how-do-i-specify-constructor-name-on-a-custom-object


That was a little annoying. I tried s:Unfork where it needed to go (after s:DirWalker etc) and it didn't work. After a lot of circular fiddling, it now works. I have no idea why it didn't before.

Ok, that Front Page.

How much to do in the pipeline, how much to hardcode in the template? (nunjucks is ok with that)

I could use another s:DirWalker to get the list of raw entry filenames, pass that to s:Template to render the front page. I might need to add a s:Sort service to get the entries in the right order (GitHub #Copilot suggested that, it's a good one).

Or... carry over what I've already got/can get from the earlier s:DirWalker. That's probably the way to go.

Oops. It goes into an infinite loop when I add a subdir to the source dir. s:DirWalker needs a test, fix recursion. Leave that for now. What there is in the context after s:Unfork is:

  "filepaths": [
    "2024-05-03_one.md",
    "2024-05-03_two.md"
  ],

Hmm. It's a bit clunky, I'll have to remap those to the raw content HTML files. But it's not terrible, it can take the info from a single source, the bits of context derived from manifest.ttl.

Plan

Postcraft

Remember I need to take down the old server

This was TODO but now #RELOCATE : paths for entry pages/fix errors in PostCraftPrep - can refactor to simplify? check s:ConfigMap

  • add s:Unfork, template & save the front page
  • (part of above) make a start on figuring out filter/order of posts
  • make individual entry pages presentable
  • make the front page presentable

probably not today

  • upload to server

  • make everything work

  • Hello World (again) post

  • Transmissions/Postcraft overview post

  • pinned posts

  • Atom feed

FOAF-Archive

  • Placeholders + a:manifest for FOAF-Archive
  • FOAF video

Kaggle-Preferences

  • get set up, run the demo

#RELOCATE transmissions topology - multiple lists, series/parallel

support-it@blackview.it

I believe I've got s:Unfork working. I made a s:Fork to test it in isolation with a little test application. Fork/Unfork aren't very good names, something a like Multicast might be better. Leave that for the Great Refactoring, which I'm do some time after #Postcraft is up and running.

It's funny, while I was trying to figure that out, the thought of providing something like a GOTO service crossed my mind. There is a natural progression from pipeline to flowchart-shaped structures, to a little language (with Turtle syntax!). But I have to invoke YAGNI here. It will be interesting to see what other steps in that direction I have to take, but the goal right now is to make it easy to get things done. And get them done.

Oops...I forgot I need to do generate the final entry pages next, before Unforking.

#RELOCATE pc:ContentGroup might be better as a more general trm:Group

to add s:Unfork after the entry content generation phase, then

Yesterday I said : Ok, for TODOs, I'll have a top-level todo/index.md file, plus monthly files like todo/2024-05.md. Then have Plan blocks in this journal for daily bits.

What I've got dangling, I'm not sure how best to manage. It would be good to have a Where I'm At Status somewhere. And a Requirements. And a convention for flagging doc chacteristics in terms of changeability.

Note to #RELOCATE, re. #GTD : I need to be clearer in demarcating my planning activity and the actual work. I'm revising too much when I hit the code. I need clear plans written down before I start, no matter how long they take. Then code faster.

context.entryContentToPage = { sourceDir: sourceDir, targetDir: targetDir, templateFilename: templateFilename


Plan

Bergi 'Unified Landscape'

Phone Return!!!

Postcraft

Fork is more like multicast - rename?

Kaggle-Preferences

  • get set up, run the demo

FOAF-Archive

  • Placeholders + a:manifest for FOAF-Archive
  • FOAF video

Unfork

Ok, the Topology question. Postcraft is quite a good example on which to be thinking about this.

  • Phase One : Markdown -> HTML at raw entry level, then raw entries -> final entry pages
  • Phase Two : raw entries -> front page etc.

Phase One forks into multiple processing paths, one for each markdown file. To avoid uneccessary complications, it makes sense to make sure Phase One has completed before starting Phase Two. So,

I believe how I've got s:DirWalker means that order is maintained subsequently, even though each path runs independently (parallelism for efficiency isn't something I need to think about now).


  • three semi-independent pipelines defined in transmission.ttl : markdown -> HTML rawEntries; rawEntries -> entries; rawEntries -> front page
  • one 'continuous', with an Unfork or some such to bring the control flow down to one path
  • one more blocky, with ContentGroups being set up from manifest.ttl in ConfigMap at the start

All three approaches could be supported, but for now I should just pick the one that seems most suitable for the current task. Considerations :

  • keeping it simple, with sensible defaults
  • keeping concerns separate :
    • transmission.ttl = topology
    • services.ttl = details of individual service configurations together they define the application
    • manifest.ttl = application instance configuration

I need some fresh air.


2024-05-18

Journal : 2024-05-13

Journal : 2024-05-13

The 90:90 rule : the first 90% of the code accounts for the first 90% of the development time. The remaining 10% of the code accounts for the other 90% of the development time. (I was about to type a different version, but GitHub #Copilot suggested that, it's a good one).

I've been rather blocked by #Postcraft, it is a major priority for me to have a blogging[^1] system in place. So I'll go quick & dirty now to get something up and running, refactor later.

In the Postcraft transmission, I've got Phase 1 working, from markdown to raw HTML (the notion of phases is useful, I'll use that).

s:ConfigMap pulls in the configuration from manifest.ttl, it's at the context.sourceDir etc level for Phase 1, but because Phase 2, I've got that going into a sub-object entryContentToPage, so after Phase 1 the context includes:

{
   "entryContentToPage": {
    "sourceDir": "public/post-content-cache",
    "targetDir": "public/blog",
    "templateFilename": "layouts/mediocre/post-page_template.njk"
  },

  "contentBlocks": {
    "link": "public/post-content-cache/2024-05-03_two.html",
    "title": " TEST TWO",
    "created": "2024-05-03",
    "updated": "2024-05-13",
    "content": "<h4>TEST TWO</h4>\n<p>here is the content of the second post</p>\n"
  },

  "filepath": "/home/danny/HKMS/postcraft/danny.ayers.name/public/post-content-cache/2024-05-03_two.html",

  "rootDir": "/home/danny/HKMS/postcraft/danny.ayers.name",
  "filename": "2024-05-03_two.md",

}

The value of contentBlocks.link I guess should be the full URL of the page. But before I sort that out, wherever that is assigned is where I need to get the target filepath for the entry page. Also the Slug (with my own conventions).

s:Template has signature (Good Word!) :

  • Input
  • context.templateFilename or if not present,
  • context.template
  • context.contentBlocks - {title, body...} or whatever
  • Output
  • context.content - the templated content

Ok, quick & dirty, I'll make an s:EntryContentToPagePrep hardcoded service to do the remapping.

Ok, nearly working. The link URL that gets templated isn't quite right : https://danny.ayers.name/blog/2024-05-03_one.html.html.html - oops!

[^1]: I'm using 'blogging' as shorthand for note capture etc etc, a fundamental part of my workflow/personal knowledge management [^2]. [^2]: New to me, (GitHub-flavoured) Markdown footnotes

Plan

  • Bergi 'Unified Landscape'
  • prep phone for return

Postcraft

NEXT - paths for entry pages/fix errors in PostCraftPrep - can refactor to simplify? check s:ConfigMap

Kaggle-Preferences

  • get set up, run the demo

FOAF-Archive

  • Placeholders + a:manifest for FOAF-Archive
  • FOAF video

#RELOCATE transmissions topology - multiple lists, series/parallel

support-it@blackview.it

Progress on Postcraft! I believe I've got s:Unfork working. I made a s:Fork to test it in isolation with a little test application. Fork/Unfork aren't very good names, something a like Multicast might be better. Leave that for the Great Refactoring, which I'm do some time after #Postcraft is up and running.

It's funny, while I was trying to figure that out, the thought of providing something like a GOTO service crossed my mind. There is a natural progression from pipeline to flowchart-shaped structures, to a little language (with Turtle syntax!). But I have to invoke YAGNI here. It will be interesting to see what other steps in that direction I have to take, but the goal right now is to make it easy to get things done. And get them done.

Oops...I forgot I need to do generate the final entry pages next, before Unforking.

#RELOCATE pc:ContentGroup might be better as a more general trm:Group

to add s:Unfork after the entry content generation phase, then

Yesterday I said : Ok, for TODOs, I'll have a top-level todo/index.md file, plus monthly files like todo/2024-05.md. Then have Plan blocks in this journal for daily bits.

What I've got dangling, I'm not sure how best to manage. It would be good to have a Where I'm At Status somewhere. And a Requirements. And a convention for flagging doc chacteristics in terms of changeability.

Note to #RELOCATE, re. #GTD : I need to be clearer in demarcating my planning activity and the actual work. I'm revising too much when I hit the code. I need clear plans written down before I start, no matter how long they take. Then code faster.

context.entryContentToPage = { sourceDir: sourceDir, targetDir: targetDir, templateFilename: templateFilename


Plan

Bergi 'Unified Landscape'

Phone Return!!!

Postcraft

Fork is more like multicast - rename?

Kaggle-Preferences

  • get set up, run the demo

FOAF-Archive

  • Placeholders + a:manifest for FOAF-Archive
  • FOAF video

Unfork

Ok, the Topology question. Postcraft is quite a good example on which to be thinking about this.

  • Phase One : Markdown -> HTML at raw entry level, then raw entries -> final entry pages
  • Phase Two : raw entries -> front page etc.

Phase One forks into multiple processing paths, one for each markdown file. To avoid uneccessary complications, it makes sense to make sure Phase One has completed before starting Phase Two. So,

I believe how I've got s:DirWalker means that order is maintained subsequently, even though each path runs independently (parallelism for efficiency isn't something I need to think about now).


  • three semi-independent pipelines defined in transmission.ttl : markdown -> HTML rawEntries; rawEntries -> entries; rawEntries -> front page
  • one 'continuous', with an Unfork or some such to bring the control flow down to one path
  • one more blocky, with ContentGroups being set up from manifest.ttl in ConfigMap at the start

All three approaches could be supported, but for now I should just pick the one that seems most suitable for the current task. Considerations :

  • keeping it simple, with sensible defaults
  • keeping concerns separate :
    • transmission.ttl = topology
    • services.ttl = details of individual service configurations together they define the application
    • manifest.ttl = application instance configuration

I need some fresh air.

2024-05-18