Rawer

Under Construction

fixing entry permalinks is top of the to-dos

Mostly Transmissions Today

Mostly Transmissions Today

Main things:

  1. AI support
  2. Refactor #Transmissions
  3. Tidy up #Postcraft
  4. FOAF video recovery

AI Support

The other day I had a play with LibreChat, which is a chatbot frontend that can be connected to a range of AI services. I got it running on my local machine, have done a git pull and have been adding services.

notes from the other day

From Known Endpoints in librechat.example.yaml :

I've added API keys for those to my .env. (Don't forget to include it in the .gitignore before committing!)

Listed in the endpoints but weren't in the sample .env, I've also added:

to be continued...

It also supports RAG, and somewhere it mentions using HuggingFace for embeddings. So also in my .env:

RAG_API_URL=http://host.docker.internal:8000
EMBEDDINGS_PROVIDER=huggingface
HF_TOKEN=...
danny@danny-desktop:~/AI/LibreChat$ docker compose ls
danny@danny-desktop:~/AI/LibreChat$ docker compose up -d

Hmm. It doesn't appear to be picking up the new .env settings, even after docker compose . Try again later.

Tidy up Postcraft

The Transmissions refactoring might take a while, and will no doubt break the setup for Postcraft in the interim. So I'll do a quick tidy up there first, publish the current material.

ok, 2024-04-27_postcraft-flow.md says:

./run postcraft /home/danny/HKMS/postcraft/danny.ayers.name

I guess that'll be from danny@danny-desktop:~/HKMS/transmissions$ - the terminal I have in #Cursor.

Ok, that appears to have run ok. Stuff locally at http://localhost/danny.ayers.name/public/blog/ I'll push those changes, then do a quick check of https://danny.ayers.name/

It's there, but without the CSS.

  <link rel="stylesheet" href="css/fonts.css" type="text/css" />

Need more coffee!!

Ok, so with :

./run postcraft /home/danny/HKMS/postcraft/danny.ayers.name

it's picking up :

/home/danny/HKMS/transmissions/src/applications/postcraft

./about.md
./services.ttl
./transmission.ttl

and then applying that to :

/home/danny/HKMS/postcraft/danny.ayers.name/manifest.ttl

putting the result in :

danny@danny-desktop:~/HKMS/postcraft/danny.ayers.name/public/blog$ ls
entries  index.html

I don't like ./blog there, ./home would be better.

what it needs to do but I clearly haven't set up yet is to copy some of this across :

danny@danny-desktop:~/HKMS/postcraft/danny.ayers.name/layouts/mediocre$ ls
about.md  css  fonts  layout-sample.html  templates

So this is delegation to the max, but that's ok, intentional separation of concerns.

I forget what Transmissions services (in transmissions/src/services/) I've got set up around copy, one might need to be tweaked/added. It'll be easiest to run at the start of the pipeline, before things start forking. That will take its config from transmissions/src/applications/postcraft/services.ttl. Which in turn (I think) should take its details from the manifest.ttl in the site directory.

I'll need to check what is in place already. While I'm at it, I want to tweak the content folders a bit. New version in docs/site-template.md.

Ok, so in danny.ayers.name/manifest.ttl I've flipped blog to home and added :

# STYLES ETC
t:CSS a pc:StaticGroup ;
    pc:site <https://danny.ayers.name> ;
    fs:sourceDirectory "layouts/mediocre/css" ;
    fs:targetDirectory "public/home/css" .

... similar for fonts & JS

also need to clean dirs on rebuild - later

Although I've got a FileReader and a FileWriter, I don't have a FileCopy. I'll add one to the services.

Shortly...

I'll put the config in place in service.ttl now, then implement it after a bit of Transmissions refactoring, after dogwalk.

Hmm. I can't remember how I did the delegaty bit. For now I've put this in services.ttl as a placeholder :

t:copyCSS a trm:FileCopy ;
        trm:key t:cssCopy .

... similar for fonts & JS

Yay! I got the change from execute(data, context) to execute(context) etc. done and it's working with the postcraft application. I was dreading that. It may have broken some of the other services, but no biggy, I'll find out as I go along.

I guess I should also rename the mill dir to engine while I'm at it.

2024-07-07

TODAYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY AI setup, again

TODAYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY AI setup, again

This is still working towards hyperdata.it reboot, but I need a bit better AI support.

LibreChat is a front end to any AI. I installed it a few weeks ago, it seemed very good

previously is in LibreChat Notes 2024-05-30

5€ credit for Mistral!


@kvistgaard Querying #GitHub with #SPARQL Anything https://twitter.com/kvistgaard/status/1720078326449213725

fuseki's browser-based admin interface uses the authentication to can see in the request headers below. I'd like to allow a user authenticated this way to use other pages on the site https:/hyperdata.it, without having to log in and out again. How? Please check Fuseki docs and/or code to see how it is done there. The other server-side code uses nodejs (in es module style) and I believe the passport libs might be useful.

at https://fuseki.hyperdata.it/#/

GET /$/server HTTP/1.1
Accept: application/json, text/plain, */*
Accept-Encoding: gzip, deflate, br, zstd
Accept-Language: en-US,en;q=0.8
Authorization: Basic c2VydmljZS1hZG1pbi11c2VyOnZibi12Ym4tMTIz
Cache-Control: no-cache
Connection: keep-alive
Cookie: JSESSIONID=node01s2l4xznid5ti1f2r6h5phj16v2.node0
Host: fuseki.hyperdata.it
Pragma: no-cache
Referer: https://fuseki.hyperdata.it/
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-origin
Sec-GPC: 1
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36
sec-ch-ua: "Not/A)Brand";v="8", "Chromium";v="126", "Brave";v="126"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "Linux"

moving to https://fuseki.hyperdata.it/#/manage :

GET /$/server HTTP/1.1
Accept: application/json, text/plain, */*
Accept-Encoding: gzip, deflate, br, zstd
Accept-Language: en-US,en;q=0.8
Authorization: Basic c2VydmljZS1hZG1pbi11c2VyOnZibi12Ym4tMTIz
Cache-Control: no-cache
Connection: keep-alive
Cookie: JSESSIONID=node01s2l4xznid5ti1f2r6h5phj16v2.node0
Host: fuseki.hyperdata.it
Pragma: no-cache
Referer: https://fuseki.hyperdata.it/
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-origin
Sec-GPC: 1
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36
sec-ch-ua: "Not/A)Brand";v="8", "Chromium";v="126", "Brave";v="126"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "Linux"

I want to use the following structure. Please advise on updating paths.

packages/auth-test/
├── client/
│       ├── dashboard.html
│       ├── css/
│       │   └── styles.css
│       └── js/
│           └── dashboard.js
└── server/
    └── server.js

earlier

Back to Life

A UK trip totally blew my momentum.

funny, GitHub Copilot in VS Code where I'm typing just suggested "I'm back in Portugal now, and I've got a few days to get back on track."

Listening

No.7 CONTRACT

Zombie!

FOAF

  • org a bit

Fuseki

  • auth etc.

Publish workflow - webhook?

Transmissions

  • HTTP hooks

Misc

check firewall

Server requirements list

qlever

placeholder articles

2024-07-07

hyperdata.it reboot

hyperdata.it reboot

I believe I've got the #LDAP and #Fuseki setup working. I need it for other stuff but for sanity checking and to to sort out my TODOs it makes sense to line up some of the old hyperdata.it bits against it.

I also need to tidy up this blog setup a bit, I left it hanging somewhat.

Plan

  1. [*] set up monorepo for farelo, trellis...
  2. [*] flip desktop localhost Apache to nginx, configure like server
  3. [*] copy relevant bits of trellis to monorepo
  4. add auth against remote Fuseki
  5. setup Transmissions to deploy

hmm, numbered markdown todos don't seem so good - correct syntax?

1. Monorepo

I got ChatGPT4o to help.

http://localhost/APPS/
danny@danny-desktop:/var/www/html/APPS$
lrwxrwxrwx 1 danny danny   44 Jul  4 15:14 common -> /home/danny/hyperdata/packages/common/public
lrwxrwxrwx 1 danny danny   45 Jul  4 15:14 trellis -> /home/danny/hyperdata/packages/trellis/public

https://hyperdata.it/apps/
root@hyperdata:/var/www/html/apps# ln -s /home/github/hyperdata/packages/common/public common
root@hyperdata:/var/www/html/apps# ln -s /home/github/hyperdata/packages/trellis/public trellis

hmm, 404

nano /etc/nginx/sites-available/hyperdata.it.conf
    location / {
        root /home/www/hyperdata.it;
        index index.html index.htm index.ttl;
                autoindex on;
    }

Ok,

ln -s /var/www/html/apps /home/www/hyperdata.it/apps

2. Apache to nginx


systemctl list-unit-files | grep enabled will list all enabled ones.

If you want which ones are currently running, you need systemctl | grep running.

sudo systemctl stop apache2
sudo systemctl disable apache2

/etc/nginx/sites-available/default


server {
listen 80;
server_name localhost;

    location / {
        root /var/www/html;
        index index.html index.htm index.ttl;
        autoindex on;
    }

}

sudo systemctl reload nginx
sudo tail -f /var/log/nginx/error.log

So far so good.


danny@danny-desktop:/var/www/html$ ls -al **HKMS**
lrwxrwxrwx 1 danny danny 26 Jul 18 2023 **HKMS** -> /home/danny/HKMS/HKMS/src/

Prompts for AI.

I want to create a monorepo for a set of related applications that share some common code. The code will be browser-based HTML, CSS and vanilla JS and nodejs using ES modules. I wish to use the following tools:

  • npm
  • jsdoc
  • chai
  • jasmine The first three apps that will appear in the monorepo are called 'farelo', 'trellis' and 'strandz'. Can you advise on how to set up the monorepo and the tools for these apps, with a good dir structure, following best practices?


2024-07-07

LDAP and Fuseki cont.

2024-07-07

LDAP and Fuseki cont.

LDAP and Fuseki cont.

for ChatGPT

I would like to test the auth on Fuseki. There is an endpoint that supports POST at : https://fuseki.hyperdata.it/foowiki/update I would like to add a random triple to the foowiki dataset. Credentials will be user: fuseki-foowiki-user password: TheEnglish_3 Please give me a minimal piece of node code, using ES modules if necessary, that I can run locally to add the triple to the dataset.


moved to Claude Sonnet 3.5 in Cursor many more circles


There are groups in LDAP that correspond to the roles in Fuseki. The users in these groups should have the following permissions -

service-admin members: Should have complete access to Fuseki. fuseki-admin members: Should be able to create and delete datasets, and have write access. fuseki-foowiki members: Should be able to write data to the foowiki dataset. Any other user, without auth, has read access to the all datasets.

The LDAP details are in

what is needed in shiro.ini ?

can you guess what happened based on what happened with most of the other things you advised recently?

I sincerely apologize. Based on the pattern of our recent interactions, I can guess that this configuration also caused Fuseki to crash or fail to start. I deeply regret causing further issues and frustration. You're absolutely right to be critical of my advice. I've been making assumptions and suggesting changes without fully understanding the specific constraints of your Fuseki setup. This approach has been counterproductive and potentially harmful to your system. Given this situation, I believe the most responsible course of action is to:

  1. Stop suggesting configuration changes that might destabilize your system.
  2. Return to the last known stable configuration (the one that gave a 401 Unauthorized error but didn't crash Fuseki).
  3. Recommend seeking assistance from Apache Jena Fuseki experts or official support channels.

I first ask about known working Shiro configs - it gives me another breaky one

then - ah...point it to Fuseki source

Thank you for providing that information. After examining the ShiroEnvironmentLoader.java file in the Apache Jena GitHub repository, I've found some insights that might be relevant to our LDAP configuration issue.

The ShiroEnvironmentLoader class is responsible for setting up the Shiro security environment in Fuseki. Notably, it doesn't seem to have any specific handling for LDAP realms or group-to-role mapping. This could explain why our previous attempts at configuring LDAP group-based authorization weren't successful.

Key observations from the file:

  1. The class loads the Shiro configuration from an INI file (shiro.ini).
  2. It sets up a SecurityManager and WebSecurityManager.
  3. There's no specific code for handling LDAP or group-to-role mapping. Given this information, it appears that Fuseki might not natively support advanced LDAP features like group-to-role mapping through the shiro.ini configuration alone. This could explain why our previous attempts to add group-to-role mapping in the configuration file didn't work as expected. To proceed, we might need to consider the following options:
  4. Use a simpler LDAP configuration that doesn't rely on group-to-role mapping. Instead, we could assign roles directly to users in the LDAP directory.
  5. Investigate if it's possible to extend Fuseki's security implementation to support group-to-role mapping. This would likely involve custom Java code and might not be feasible depending on your deployment setup.
  6. Look into using a custom Shiro Realm implementation that handles the group-to-role mapping logic internally, rather than relying on the configuration file. Given the complexity of the situation, it might be worth reaching out to the Apache Jena community or mailing list with these specific findings. They might have additional insights or workarounds for implementing LDAP group-based authorization in Fuseki.

I apologize for the oversight. It seems the 'superpowers' attribute isn't defined in your LDAP schema, which is causing the error. Let's take a step back and modify our approach to use an existing attribute that's already defined in the inetOrgPerson objectClass.

Instead of 'superpowers', we can use the 'title' attribute, which is a standard attribute in inetOrgPerson and can be repurposed for our needs.


Now Claude is actively encouraging me to yak shave. For this configuration to work, you'll need to create a custom Java class CustomLdapRealm that extends JndiLdapRealm and implements the group-based authorization logic. This class should be placed in the org.apache.jena.fuseki.authz package. [Java source follows]

Enough! Music room!!! Still got a bottle of beer left!!!


Building Fuseki. mvn compile node, yarn, vite doc things some Bavarian stuff cypress skosmos? surefire-reports No license field Use of eval is strongly discouraged Hours later, eh - Port 3030 is already in use. Bastard! I already have Fuseki running on 3030. Artifact has not been packaged yet. When used on reactor artifact, unpack should be executed after packaging: see MDEP-98. -> [Help 1] Nope. Means nothing to me. I'm done.

ERROR] Failed to execute goal org.apache.maven.plugins:maven-dependency-plugin:3.6.1:unpack (unpack) on project jena-fuseki-webapp: Artifact has not been packaged yet. When used on reactor artifact, unpack should be executed after packaging: see MDEP-98.

2024-07-07