nullify

I have now open sourced AMQL - a set of .NET AI model engineering capabilities: https://github.com/simonsoanes/AMQL 

This tool has the ability to create models, fine tune/train them, merge them, create MoE clusters, prune them, import and export different formats (safetensors and gguf), quantise a model and introspect inference operations to find exactly what pathway triggered a particular token response.  It can also export any changes to models as low-rank adapters (LoRA) to be loaded.

It also has its own format compatible with VIndex3 which is model agnostic, however I had to add support for variable quantisation.

This is a tool I developed for my own model engineering needs when building Ariadne (both the platform and the foundation models that underpin it).  It allows investigation of hallucinations (over-fit for compliance due to post-training) and even fixing specific ones by adjusting the tensors and logits directly.  It also has the ability to convert a model to a classifier or embedding model.

Most of these things are possible in the Python ecosystem already, but this combines them into a single utility that can do so at fairly high performance and both with and without using GPU compute.  Apologies that it only supports CPU and CUDA, I do not have AMD GPU systems available to test any ROCm code on.

This is an R&D level tool; it isn't intended for normal users, however I hope it's of use to someone. It is part of an automated workflow normally.

Its natural partner is TensorSharp, which provides a higher-performance inference server. Alongside Zhongkai Fu’s excellent ongoing work, I’ve contributed tensor parallelism, dynamic LoRA adapters and several other features to it.

I've recently decided to open source some groupware software I built to replace applications that were running on PHP on my home network.  These are brand new apps built to my own standards  and running on .NET 10 with the fewest possible dependencies (for example, the e-mail app doesn't store your password at all and supports client side end to end encryption using OpenPGP) - even the icons in the webmail are custom made.

The CalDav server is designed to support the webmail client and android/apple phones, and supports push notifications for calendar and contact changes, as well as SyncML.

https://github.com/simonsoanes/CalDav - Nullify CalDav, my CalDav and CardDav server supporting calendars, contacts, tasks and notes.

https://github.com/simonsoanes/Webmail - Nullify WebMail

These applications are going to be monitored and maintained by my AI platform, so will get ongoing updates and be kept on a current version of all libraries, however most functionality was custom written from scratch with no dependencies - for example, the PGP implementation is from scratch (which does mean I've not necessarily provided all historic encryption and signing algorithms and it hasn't been audited, but it does pass standard interop tests).

I kept having out of memory situations where I was unable to finish fine-tuning or quantization jobs when I had hundreds of GB's of video memory seemingly free - I kept getting:

Failed to load checkpoint: Some modules are dispatched on the CPU or the disk

It turns out that in a unified memory Grace Blackwell system you need to drop your OS cache or it can consume too much of the unified memory, resulting in paging to disk instead of GPU use.

sudo sh -c 'sync; echo 3 > /proc/sys/vm/drop_caches'

Today I tried to get voice recognition working using hardware acceleration on an ARM64 system with an Nvidia GPU.  I eventually found the easiest way was to make my own Dockerfile based on their own, but using this base image:-

FROM nvidia/cuda:12.3.2-cudnn9-runtime-ubuntu22.04

This comes with all the necessary Cuda support libraries pre-loaded and allowed me to quickly compile, build and publish a Grace Blackwell compatible docker image of faster-whisper.  Once I've tested it a bit and set up an automated pipeline I'll publish it.

The difference between CPU and GPU processing was only a second or so, but it does make the recognition feel snappy and you can use the large model easily instead of the tiny one.

So I was trying to give my current personal AI project access to Gitea, and had to read the code to get it working.  If you want to start it up on docker and use http as the transport, then your docker-compose is:-

version: "3.8"

services:
  gitea-mcp:
    image: gitea/gitea-mcp-server:latest
    container_name: gitea-mcp
    restart: no
    ports:
      - "8080:8080"
    environment:
      - MCP_MODE=http
      - GITEA_URL=https://yourgiteainstance
      - GITEA_TOKEN=yoursecrettoken

The latest version of .NET is now out and available to download along with Visual Studio 2026. The update to .NET 10 adds significant performance improvements including things like automatic use of AVX instructions where available, and a significant improvement in process startup time.  There's also some language additions that make things more succinct.

It's a little early to upgrade production things, but I've already updated this website - I had to use pre-release versions of the Postgres EF naming conventions Nuget package as it's not yet compatible with version 10 yet, it looks like they hardcoded a max version number into the .NET 9 version.  Hopefully I'm just a day early and they'll be out by the official release date.

The new version of Visual Studio 2026 looks really slick, it's quite a good incremental improvement with a lot of Copilot features, first party support for Podman and few good UI and debugging improvements - in particular showing the contents of parameter variables and if condition results at breakpoints without needing to manually dig into them.

I also see that Jetbrains are hoping to have an updated version of their IDE, Jetbrains Rider that's .NET 10 compatible out tomorrow too.

I was fighting with unsloth / Nvidia Cuda / Python versioning today. I eventually gave up on unsloth and managed to use pytorch directly - however as part of the journey I found that Microsoft, Nvidia and Linux have made GPU paravirtualisation so smooth it's invisible.

First of all, it just works in WSL - you can just install the Nvidia drivers (inside WSL) and magically get full GPU support, with no prior configuration of how much of the GPU you are taking, it's shared dynamically and very smoothly.  You can just use the GPU as if it's local once you've installed the drivers.

Which is clearly intended to handle container tech and brings us on to the Nvidia container toolkit which provides the local libraries for the containers, and Podman GPU support.

• Install CUDA and the Nvidia drivers on the host
• Install the CUDA and Nvidia drivers on the WSL instance / VM, or even the Nvidia Container Toolkit.
• Install the Nvidia Container Toolkit on the docker machine/podman machine in WSL ("podman machine ssh" to get into it)
• Use "--gpus all" on podman when running a container to enable the GPU's to be passed through to the container!

Overall I was very surprised, as Nvidia historically put a lot of road blocks on paravirtualisation of their cards.  It's great to see this working.

Regarding the situation with unsloth, I found it was breaking even though the libraries were installed - and think different bits have different dependencies.  It could do with some re-engineering to guarantee it's consistency, even their own containers weren't working and I tried several tags.

Instead, I created my own micro-environment to work in with a Dockerfile based on Nvidia's pytorch package:-

FROM nvcr.io/nvidia/pytorch:25.09-py3
EXPOSE 8888
WORKDIR /app
RUN pip install jupyter
CMD ["jupyter", "notebook", "--ip=0.0.0.0", "--port=8888", "--no-browser", "--allow-root"]

Once this had been built with a quick:-

podman build -t aiplatform .

I could run the container using

podman run --gpus all -p 8888:8888 aiframework

mklink "C:\Program Files\RedHat\Podman\docker.exe" "C:\Program Files\RedHat\Podman\podman.exe"