Fixing NTP failing with Chrony on Kubuntu 26.04

Kubuntu 26.04 — Chrony NTP Sync Failure (Home Network)


Root cause: Ubuntu 26.04 switched from systemd-timesyncd to chrony by default, with NTS-secured pool servers (*.canonical.com) enabled out of the box. Chrony's authselectmode directive defaults to mix, but the effective behavior observed was that the NTS pool entries had require applied automatically (visible as R in EOpts via chronyc selectdata). Since those NTS pool sources were unreachable on the home network, chrony refused to synchronize from any source — including the healthy, agreeing local servers — logging:

Can't synchronise: no required source in selectable sources

Fix: Added to /etc/chrony/chrony.conf:

authselectmode ignore

This disables chrony's automatic require/trust injection for authenticated (NTS) sources, so selection is governed purely by the explicit prefer/trust flags in the sources file. The NTS pool stays configured (useful when off the home network) but no longer blocks sync when unreachable.

Verification:

sudo systemctl restart chrony chronyc sources # local servers show as selected (*/+) chronyc tracking # Leap status: Normal, Reference ID populated

Key diagnostic commands used: chronyc tracking, chronyc sources, chronyc selectdata

Proxmox Backup Server - Recover a ZFS Datastore that is 100% full

This has happened to me a couple times now where my ZFS pool has filled to 100% causing not only backups to fail, but also the ability to run garbage collection. When this occurs you for example might see errors like:

TASK ERROR: atime safety check failed: update atime failed for chunk/file "/mnt/datastore/ZFS-HDD/.chunks/bb9f/bb9f8df61474d25e71fa00722318cd387396ca1736605e1248821cc0de3d3af8" - ENOSPC: No space left on device

The way I have fixed this is to temporarily reduce the reserved ZFS filesystem space, run immediately a garbage collection, then increase the reserved space back to what it was. The steps are:

1. Run a Prune job and flag some backups for deletion, so that the garbage collection routine has something to actually remove.
2. In the shell, run:

echo 10 > /sys/module/zfs/parameters/spa_slop_shift

3. IMMEDIATELY run the garbage collection against the datastore. Expect this to be slow going. Make sure no backups try to run during this time.
4. After the garbage collection has freed up space, make sure you set the ZFS space to what it was before by running

echo 5 > /sys/module/zfs/parameters/spa_slop_shift

5. Run a verify job against your backups to detect any issues from running out of space.


Done! If everything went as planned, your datastore is functional again and backups can now resume.

VM Prep for conversion to PVE from VMWare

To make my life easier I have been working at automating the conversion process from VMWare virtual machines to Proxmox VE 9. The hardest part being of course having to load the VirtIO SCSI driver into Windows and having it recognise that at boot the first time it comes up in PVE.

To accomplish this there are two scripts I run. The first script that I made will:

- Save hardware inventory (CPU, RAM, Disks, IP) to C:\temp\notes.txt
- Uninstall VMware Tools (Registry & Chocolatey)
- Install QEMU Guest Agent & VirtIO Drivers from a specified network share
- Setup the VirtIO SCSI driver as a service within Windows set to autostart at boot

The second script that I downloaded from here attaches a fake drive to the machine while it is running using the VirtIO SCSI controller. This is the magic that makes it fully bootable immediately.

After the first boot check the notes.txt file and update the network adapter with a static IP if required, and if there are additional unmounted drives use that document to ensure you use the right drive letters when attaching them again.

The first script: https://github.com/proudcanadianeh/Powershell-Scripts/blob/master/Windows/VMWareToPVE.ps1
The second script: https://github.com/croit/load-virtio-scsi-on-boot/tree/main

When importing to Proxmox using the Import Guest feature, be sure to unselect the checkbox to Prepare for VirtIO SCSI, change the SCSI controller to VirtIO Single, and the network adapter to VirtIO Paravirtualized.

SentinelOne Log Collector for Fortigate with Windows

I found all the official documentation to be confusing at best, or straight up misleading and wrong at worst for this issue. Below are the steps that worked for me.

Get your API Key:
Open your SentinelOne dashboard, go to Policy and Settings, API Keys (Under Singularity AI SIEM), click the Add Key button and choose to make a Write Key. Save this key for later.

Install the Fortigate Parser:
At the top of your dashboard, click Marketplace. Find the Fortigate Firewall option and add that to your account (Top level, or you can choose an individual site. Make sure your API key is from the same level)

Install The agent:
1. Download and install the .msi from
https://app.scalyr.com/scalyr-repo/stable/latest/ScalyrAgentInstaller-2.2.16.msi
2. As administrator, open C:\Program Files (x86)\Scalyr\config\agent.json
3. Set the api_key value to the API key you made before
4. Within the Monitors portion of the agent file, put in the following:

monitors: [
{
module: "scalyr_agent.builtin_monitors.syslog_monitor",
protocols: "tcp:601, udp:514",
accept_remote_connections: true,
message_log: "fortinet.log",
parser: "marketplace-fortinetfortigate-latest"
}
]
Configure for SentinelOne:
1. As administrator, create a new file
C:\Program Files (x86)\Scalyr\config\agent.d\settings_server.json with the following contents:
{
scalyr_server: "https://xdr.us1.sentinelone.net"
}

3. Configure firewall to send syslog to the ip/hostname of the server the agent is running on.
(On 7.4 this is under Log and Report, Log Settings, then the Syslog Logging option)

Graylog 6.3.1 on Synology DSM 7.2.2

Below is the Docker Compose script to setup Graylog. Copy and paste this then make a new Project with this script. The default password I used was changeme123 but I strongly encourage you to generate your own SHA256 hash and replace the "GRAYLOG_ROOT_PASSWORD_SHA2" field with it. 

 Once it has been build and is running, you must go into the graylog-1 container and open the log. At the top it should have a randomly generated username and password that is needed for a pre-install configuration page available on port 9000. 

 Note, for this to work on Synology you MUST have this environmental variable as included below:

opensearch.bootstrap.system_call_filter: false

 

# For DataNode setup, graylog starts with a preflight UI, this is a change from just using OpenSearch/Elasticsearch.
# Please take a look at the README at the top of this repo or the regular docs for more info.
services:
  # MongoDB: https://hub.docker.com/_/mongo/
  mongodb:
    image: "mongo:7.0.21"  
    restart: "on-failure"
    networks:
      - graylog
    volumes:
      - "mongodb_data:/data/db"
      - "mongodb_config:/data/configdb"  
  # For DataNode setup, graylog starts with a preflight UI, this is a change from just using OpenSearch/Elasticsearch.
  # Please take a look at the README at the top of this repo or the regular docs for more info.
  # Graylog Data Node: https://hub.docker.com/r/graylog/graylog-datanode
  datanode:
    image: "${DATANODE_IMAGE:-graylog/graylog-datanode:6.3.1}"
    hostname: "datanode"
    environment:
      GRAYLOG_DATANODE_NODE_ID_FILE: "/var/lib/graylog-datanode/node-id"
      # GRAYLOG_DATANODE_PASSWORD_SECRET and GRAYLOG_PASSWORD_SECRET MUST be the same value
      GRAYLOG_DATANODE_PASSWORD_SECRET: "27566328401b278558782c47432b2fc788045d8b87dbd7563b48b3e4d36fbd87"
      GRAYLOG_DATANODE_MONGODB_URI: "mongodb://mongodb:27017/graylog"
      opensearch.bootstrap.system_call_filter: false
    ulimits:
      memlock:
        hard: -1
        soft: -1
      nofile:
        soft: 65536
        hard: 65536
    ports:
      - "8999:8999/tcp"   # DataNode API
      - "9200:9200/tcp"
      - "9300:9300/tcp"
    networks:
      - graylog  
    volumes:
      - "graylog-datanode:/var/lib/graylog-datanode"
    restart: "on-failure"
  # Graylog: https://hub.docker.com/r/graylog/graylog-enterprise
  graylog:
    hostname: "server"
    image: "${GRAYLOG_IMAGE:-graylog/graylog:6.3.1}"
    depends_on:
      mongodb:
        condition: "service_started"
      datanode:
        condition: "service_started"
    entrypoint: "/usr/bin/tini --  /docker-entrypoint.sh"
    environment:
      GRAYLOG_NODE_ID_FILE: "/usr/share/graylog/data/data/node-id"
      # GRAYLOG_DATANODE_PASSWORD_SECRET and GRAYLOG_PASSWORD_SECRET MUST be the same value
      GRAYLOG_PASSWORD_SECRET: "27566328401b278558782c47432b2fc788045d8b87dbd7563b48b3e4d36fbd87"
      GRAYLOG_ROOT_PASSWORD_SHA2: "494a715f7e9b4071aca61bac42ca858a309524e5864f0920030862a4ae7589be"
      GRAYLOG_HTTP_BIND_ADDRESS: "0.0.0.0:9000"
      GRAYLOG_HTTP_EXTERNAL_URI: "http://localhost:9000/"
      GRAYLOG_MONGODB_URI: "mongodb://mongodb:27017/graylog"
    ports:
    - "5044:5044/tcp"   # Beats
    - "5140:5140/udp"   # Syslog
    - "5140:5140/tcp"   # Syslog
    - "5555:5555/tcp"   # RAW TCP
    - "5555:5555/udp"   # RAW UDP
    - "9000:9000/tcp"   # Server API
    - "12201:12201/tcp" # GELF TCP
    - "12201:12201/udp" # GELF UDP
    #- "10000:10000/tcp" # Custom TCP port
    #- "10000:10000/udp" # Custom UDP port
    - "13301:13301/tcp" # Forwarder data
    - "13302:13302/tcp" # Forwarder config
    networks:
      - graylog
    volumes:
      - "graylog_data:/usr/share/graylog/data/data"
    restart: "on-failure"
networks:
  graylog:
    driver: "bridge"
volumes:
  mongodb_data:
  mongodb_config:
  graylog-datanode:
  graylog_data:
 
 
 

VCenter Remediation fails with HPE Oneview SPP image

 If a VMWare host is failing in remediation after adding the HPE Firmware and Drivers Addon most likely the Smart Update Tools isn't configured correctly. 

On the impacted host enable SSH and connect, then run the command:

sut -set mode=AutoDeploy

 

This might take a minute to run and appear to not be doing anything, just wait. Once complete it should immediately be ready to go. 

 If SUT isn't installed, make sure the HPE Customization for HPE Servers is installed in the image. 

You can verify the current mode by running:

sut -exportconfig

 

Deploy Chocolatey packages with Microsoft Intune

 Deploying Chocolatey packages using Intune

Let me start by saying this is currently working with Chocolatey version 2.2.2 in August of 2023.

To deploy chocolatey packages using Intune three scripts are required. Install.ps1, uninstall.ps1, and detection.ps1. 

These three scripts along with a pre-compiled .intunewin file are available at this url:

ChocoPackages/Intune Deployments (github.com)

To setup the package follow these steps:

1. Add a new Windows App (Win32)

2. Upload the .intunewin file

3. Change the Name, Description, and add the Publisher. I like to do information on this page, but that is optional. 

4. On the Program tab, enter this for the Install Command:

powershell.exe -ExecutionPolicy Bypass -file ./install.ps1 {packagename} 

 Then enter this for the Uninstall Command:

powershell.exe -ExecutionPolicy Bypass -file ./uninstall.ps1 {packagename}

Your page should look like this:

5. For the Requirements page, I usually just choose 64 Bit and an old build of windows. 

6. For the Detection Rules, you need to modify the detection.ps1 script with the name of the package and then upload it here under the "Use a custom detection script" option.

7. For dependencies ideally you would have Chocolatey as a package and add it here, but that is optional. 

8. Under assignments choose who will get this package and then you are done!

Enjoy having one click self updating deployments using Intune and Chocolatey!