Immich hints and tips

A dump of my immich experience

Getting lists of filenames from an album.

Create an API key from your Immich instance.

NOTE: You will need album.read and asset.read

Then get an ID from an album to get images from.
Open een album in your browser and copy the ID from the URL

Code to get a filelist using Curl

url -s -H "x-api-key: 2Nk4sO4eEm001Cm1Dsnl32UVDEvxxxxxxxxxxxxxxxxx" "https://myphotos.example.com/api/albums/f6a300c2-5027-4c38-a367-xxxxxxxxxxxxxx" | jq -r '.
assets[].originalFileName'

Fixing WhatsApp

When ingesting WhatsApp media, the dates in the database will contain the ingest date. This is because the GPS/Date and other exif information are removed from the Media in WhatsApp.

NOTES:

  • Always import your camera media first, these will contain all exif info, if you upload WhatsApp media containing the same image it can be skipped. (Look for deduplication tip below)
  • WhatsApp autouploaded using the App on your phone rarely needs adjusting. (Taking a photo and uploading it the same day will fix the wrong day issue)

Luckily the WhatsApp media contains the date in the filename.

git clone https://github.com/FlorianKrauseResearch/Immich-Metadata-Update.git
(somewhere on your desktop system/laptop)

Look at installation and usage here: https://github.com/FlorianKrauseResearch/Immich-Metadata-Update
Create a new API key with enough rights!

This software will connect to your immich instance, searches for ingestdates and whatsapp filenames discrepancies.
And wil fix these in the immich database.

I’ve got a directory containing above code for every user, with their own .env file, and custom filters

I’ve edited immich_metadata_update/filters.py

BUILTIN_PATTERNS: dict[str, DatePattern] = {
    "whatsapp": DatePattern(
        name="WhatsApp",
        regex=r"^IMG-(\d{8})-WA\d{4}\.\w+$",
        date_format="%Y%m%d",
    ),
    "whatsappvid": DatePattern(
        name="WhatsApp",
        regex=r"^VID-(\d{8})-WA\d{4}\.\w+$",
        date_format="%Y%m%d",
    ),
    "screenshot_basic": DatePattern(
        name="Screenshot (basic)",
        regex=r"^Screenshot_(\d{8})-\d{6}\.\w+$",
        date_format="%Y%m%d",
    ),
    "screenshot_full": DatePattern(
        name="Screenshot (with app name)",
        regex=r"^Screenshot_(\d{4}-\d{2}-\d{2}).*$",
        date_format="%Y-%m-%d",
    ),
    "signal": DatePattern(
        name="Signal",
        regex=r"^signal-(\d{4}-\d{2}-\d{2})-\d{2}-\d{2}-\d{2}-.*$",
        date_format="%Y-%m-%d",
    ),
}
python3 run.py --preset whatsappvid
python3 run.py --preset whatsappvid --apply corrections.json

Incorrect MAP location (0,0 problem, AKA Null Island)

Sometimes media has a incorrect GPS location, or it is missing, or as above set as 0:0

You CAN change the location of Images using the MAP in Immich.
(Select MAP > Day or image > Menu: Change location)
(Also under Utilities)
Immich WILL NOT change your image!, It will write a sidecar file with updated location info.

How I like to fix this:
Download the images for which you want to remove the GPS information.
Delete from Immich.
Run below script over those images to remove Exif information and reupload.

exiftool -gps:all= FILENAME

Loads of the same images

Deduplicate? : Use Utilities > Review duplicates

Select camera instead of WhatsApp image to keep.
(Most of the time bigger and has all exif information!)

Burst photos or simular photos? Use Stacking. This will show only ONE thumbnail in albums/timeline.

Another solution is moving them to Archive!

Uploading using immich-go

https://github.com/simulot/immich-go

./immich-go upload from-folder --server http://192.168.1.2:2283 --api-key GdMq6lZU8Szw6Lc2TXXXXXXXXXXXXXXXXXXXXXX  --folder-as-album=FOLDER ~/Pictures/Screenshots/

NOTE: Subdirs become new albums.

Immich Power Tools

https://github.com/immich-power-tools/immich-power-tools

  • Manage people data in bulk : Options to update people data in bulk, and with advance filters
  • People Merge Suggestion : Option to bulk merge people with suggested faces based on similarity.
  • Update Missing Locations : Find assets in your library those are without location and update them with the location of the asset.
  • Potential Albums : Find albums that are potential to be created based on the assets and people in your library.
  • Analytics : Get analytics on your library like assets over time, exif data, etc.
  • Smart Search : Search your library with natural language, supports queries like “show me all my photos from 2024 of “
  • Bulk Date Offset : Offset the date of selected assets by a given amount of time. Majorly used to fix the date of assets that are out of sync with the actual date.

PYTHON script to download an album (with a filename filter)

NOTE: At the bottom you can remove the # comments to also REMOVE from immich

import requests
import os

IMMICH_URL = "http://192.168.1.2:2283/api"
API_KEY = "2Nk4sO4eEm001Cm1Dsnl3XXXXXXXXXXXXXXX"

ALBUM_ID = "c4ce0661-0c4c-4c49-b6c1-XXXXXXXXXXXXXXXXXXXXX"
FILENAME_PREFIX = "VID_"  # filename filter

HEADERS = {
    "x-api-key": API_KEY
}

DOWNLOAD_DIR = "./downloaded"
os.makedirs(DOWNLOAD_DIR, exist_ok=True)


def get_album_assets(album_id):
    r = requests.get(
        f"{IMMICH_URL}/albums/{album_id}",
        headers=HEADERS
    )
    r.raise_for_status()
    return r.json()["assets"]


def filter_assets(assets):
    # simulate SQL LIKE 'IMG_2023%'
    return [
        a for a in assets
        if a["originalFileName"].startswith(FILENAME_PREFIX)
    ]


def download_asset(asset):
    asset_id = asset["id"]
    filename = asset["originalFileName"]

    url = f"{IMMICH_URL}/assets/{asset_id}/original"

    r = requests.get(url, headers=HEADERS, stream=True)
    r.raise_for_status()

    path = os.path.join(DOWNLOAD_DIR, filename)

    with open(path, "wb") as f:
        for chunk in r.iter_content(8192):
            f.write(chunk)

    return path


def delete_assets(asset_ids):
    r = requests.delete(
        f"{IMMICH_URL}/assets",
        headers=HEADERS,
        json={"ids": asset_ids}
    )
    r.raise_for_status()


def main():
    print("Fetching album assets...")
    assets = get_album_assets(ALBUM_ID)

    print(f"Total assets in album: {len(assets)}")

    print("Filtering by filename...")
    filtered = filter_assets(assets)

    print(f"Matched assets: {len(filtered)}")

    downloaded = []

    print("Downloading...")
    for asset in filtered:
        try:
            path = download_asset(asset)
            downloaded.append((asset["id"], path))
        except Exception as e:
            print(f"Download failed: {asset['id']} - {e}")

    # VERIFY
    print("Verifying...")
    if len(downloaded) != len(filtered):
        print("Download mismatch. Abort delete.")
        return

    for _, path in downloaded:
        if not os.path.exists(path) or os.path.getsize(path) == 0:
            print(f"Invalid file: {path}")
            return

    print("Verification OK")

    # DELETE
    ids_to_delete = [asset_id for asset_id, _ in downloaded]

    #print("Deleting assets...")
    #delete_assets(ids_to_delete)

    print("Done!")


if __name__ == "__main__":
    main()

Immich and (not) Google Timeline

Google killed timeline.

I’ve been experimenting in the past with GPS (gpx) mappers and alternatives.

Now I’ve installed Dawarich, which can use the photos in my immich library.

https://dawarich.app

Spin up a docker instance, create an API key in immich, and GO.

I’ve imported google’s timeline.json from my phone.

Auto post locations to your instance API using:

This week, a lot of networking and Home Automation fixing

(Sorry except for a mikrotik script, not much details)

  • Installed a new Home Assistant instance for testing on my Proxmox
  • Moved all dhcp/dns to my main Mikrotik, second backup in progress
  • Decommissioned an old trunked switch (lot of work)
  • Decommissioned an old router/firewall server (10+ years old)
  • Redraw my network in DrawIO
  • Installed Homelable to scan and draw my network (see below)

Meanwhile I am looking for a new welder.
Also doing some woodwork. (bird feeder stand)

Bird feeder pole (has a brownish tint now)

This is a cool project (Homelable). Below a still incomplete network.

Some code below to generate a CSV for DNS/DHCP entries from a Mikrotik.
(NOTE: seems spaces in comments/name/entries break stuff)

ssh admin@10.x.x.x "/ip/dns/static;export" | grep 10.1.0 | awk '{
    addr=""; comment="" ; name=""
    for(i=1;i<=NF;i++) {
        if($i ~ /^address=/) addr=$i
        if($i ~ /^comment=/) comment=$i
        if($i ~ /^name=/) name=$i
    }
    if(addr && name) print addr, name, comment
}' > dnsstatic.out

ssh admin@10.x.x.x "/ip/dhcp-server/lease;export terse" | awk '{
    addr=""; comment=""
    for(i=1;i<=NF;i++) {
        if($i ~ /^address=/) addr=$i
        if($i ~ /^comment=/) comment=$i
    }
    if(addr && comment) print addr, comment
}' > dhcpstatic.out

for f in $(seq 1 254) ; do lease=$(grep "10.1.0.$f " dhcpstatic.out | cut -f3 -d=) ; dns=$(grep "10.1.0.$f " dnsstatic.out | cut -f2,3 -d" " |sed s/name=//g | s
ed s/comment=//g | sed s/\ /,/g | head -1) ; echo -n 10.1.0.$f, ; echo -n $lease ; echo -n "," ; echo $dns  ;done > ips.csv

Meanwhile filling my own hosted “Spotify” clone, but better. (Navidrome)
(My immich server is also ingesting while we speak.)

Navidrome

Mailbox resorting attachments

(I often forgot to download important attachments, like orders and important paperwork.

I am using paperless-ngx to make pdfs searchable.

  • export (takeout) mail from for example gmail
  • extract all attachments using below script
  • sort into media types
  • use dangerzone.rocks installation to sanitize (remove malware/virus)
cat All\ mail\ Including\ Spam\ and\ Trash-002.mbox | formail -des munpack

Cleanup script

find -type f -iname '*.desc' -exec rm  {} \;


for f in =X*; do
    new=$(echo "$f" | sed -E 's/^=X//; s/X=(\.[0-9]+)?$/\1/')
    mv -- "$f" "$new"
done

for f in *.[0-9]*; do
    base=${f%.[0-9]*}

    if [ -f "$base" ] && cmp -s -- "$base" "$f"; then
        echo "Removing duplicate: $f"
        rm -- "$f"
    fi
done

for f in *.[0-9]*; do
    n=${f##*.}          # 1, 2, 3 ...
    base=${f%.*}        # winmail.dat
    ext=${base##*.}     # dat
    name=${base%.*}     # winmail

    mv -- "$f" "${name}-${n}.${ext}"
done

for f in -*; do
    new="${f#-}"
    mv -- "$f" "$new"
done

for f in -*; do
    new="${f#-}"
    mv -- "$f" "$new"
done

for f in *X; do
    mv -- "$f" "${f%X}"
done


mkdir -p pdf images audio text movies bww zip midi html vcf xml
mv *PDF pdf 
mv *pdf pdf
mv *gif images
mv *GIF images
mv *jpg images
mv *bmp images
mv *BMP images
mv *jpeg images
mv *JPG images
mv *Jpg images
mv *png images
mv *tif images
mv *eps images
mv *PNG images
mv *Png images
mv *svg images
mv *psd images
mv *mp3 audio
mv *MP3 audio
mv *wma audio
mv *wav audio
mv *m4a audio
mv *txt text
mv *wri text
mv *doc text
mv *docx text
mv *xls text
mv *XLS text
mv *ppt text
mv *pptx text
mv *xlsx text
mv *mp4 movies
mv *MP4 movies
mv *avi movies
mv *mov movies
mv *MOV movies
mv *mpg movies
mv *MPG movies
mv *bww bww
mv *abc bww
mv *pio bww
mv *zip zip
mv *ZIP zip
mv *tgz zip
mv *tar zip
mv *rar zip
mv *mid midi
mv *html html
mv *htm html
mv *vcf vcf
mv *xml xml

Next thing to do .. sanitize PDF’s

paperless-ngx to ingest

Smoke detector (ShellySmoke) notification Home Assistant with friendly-names

Quick ‘n simple post, so I won’t forget.

Multiple triggers in one Automation, use friendly_name to distinguish entities that triggered the notification.

Message send example “Smoke detected ShellySmokeKitchen”

alias: Smoke detected
description: ""
triggers:
  - type: smoke
    device_id: bbd0c2ebb949b38a28c6ecca67fb4ebe
    entity_id: 888d20ea2075873284708125491695b5
    domain: binary_sensor
    trigger: device
  - type: smoke
    device_id: 31bda70d60fc815681fe951469e11a0a
    entity_id: 3a9538b025f7d263b5e145d6f4dda859
    domain: binary_sensor
    trigger: device
  - type: smoke
    device_id: 1a33c981b1dfca13202b5efe0237fea1
    entity_id: 6bf59d8de54a7bdef0bab7daf483221f
    domain: binary_sensor
    trigger: device
  - type: smoke
    device_id: 0496d720c844a5648d8b3789b8f4d093
    entity_id: e635593a29dc9af3977224b54bf19140
    domain: binary_sensor
    trigger: device
conditions: []
actions:
  - action: notify.pushover
    metadata: {}
    data:
      message: Smoke detected  "{{ state_attr(trigger.entity_id, 'friendly_name') }}"
      data:
        priority: 2
        sound: siren
        expire: 300
        retry: 30
mode: single

Software i use(d)

My much used set of tools (old draft I edited)

File managers

  • Midnight Commander
  • Thunar
  • Nautilus
  • nnn
  • Dolphin

Generic tools

  • rsync
  • dcfldd – an enhanced version of dd
  • screen/tmux
  • fdupes
  • ghex / xxd

Connecting/networking tools

  • mosh – roaming, faster, using ssh
  • sshfs – mount remote filesystems over ssh (see ssh tricks)
  • wavemon – wifi info
  • GNS3 – Graphical Network Simulator-3
  • Wireshark
  • PHPIpam
  • Homelable

Encryption

  • Luks
  • ecryptfs

Graphical/Photo

  • Gimp
  • Eog
  • rawtherapee
  • Inkscape
  • Darktable
  • PureRef
  • digiKam
  • Krita
  • Hugin
  • ImageMagick

Documents/Notekeeping

  • Vim
  • Joplin
  • Paperless-ngx
  • DrawIO – For network drawings
  • LibreOffice/OnlyOffice -> EuroOffice
  • Scribus

3D Print/Lasercutting

  • Cura
  • Orcaslicer
  • Bambuddy
  • LightBurn
  • Model editors: Blender and OpenScad, Meshroom, Sketchup

Coding

  • Mostly CLI
  • PlatformIO
  • Arduino IDE
  • VSCodium (stripped Visual Studio)

Electronics

  • Fritzing
  • Kicad
  • Logic

Ebook / Comic readers

  • Calibre
  • FBReader

GFX/Video

  • Blender
  • kdenlive
  • OBS Studio
  • Shotwell
  • Handbrake
  • ffmpeg
  • VLC
  • Kodi/Libreelec

Music

  • CLI abc tools
  • Musescore
  • Bagpipe Music Player
  • Audacity (after removing all bad things be-ing added in 2021)
  • LMMS
  • Ardour

Virtualisation/Emulation

  • proxmox
  • libvirt
  • ovirt
  • guestfish
  • dosbox
  • pcem
  • Martypc

Password management

  • Keepass / KeepassXC

API

  • Postman
  • Flask (Python) see ledserver

Beamer

  • Mapmap
  • LPMT
  • Qprompt

Brewing

  • Brouwhulp
  • Brewfather (web)

Web/App alternatives for bad companies (Mostly own hosted)

  • Gmail – Hosted elsewhere – Thunderbird + web
  • Google Drive – Nextcloud
  • Spotify – Navidrome
  • Google Photos (I never used this, i used Gallery2/3/Wipigo ) – Immich
  • Youtube – Jellyfin for own movies
  • Whatsapp/Google Chat – My own mattermost server, signal and IRC
  • Teams – Ownhosted jitsi
  • Teamviewer – Rustdesk
  • Google Timeline – Dawarich

Server generic

  • Databases : mariadb/mongodb/influxdb/sqlite
  • Grafana
  • NodeRed
  • gitea
  • HomeAssistant
  • Bookstack
  • Librenms
  • Check_mk (i’ve started with Netsaint (1999), Nagios,Icinga,
  • Mosquitto

Unsorted stuff

  • vimperator (old)
  • links/curl/dsniff/urlsnarf
  • Databases (mariadb/mongodb/influxdb/sqlite)
  • Metabase
  • Adminder
  • Scrot (snapshot tool)
  • Luminace-hdr
  • GDlib
  • Dia (old)
  • Blackmagic Fusion
  • gcalcli
  • Gcalcron
  • Domoticz
  • MQTT-Explorer
  • Android studio
  • Nextion IDE
  • Irssi
  • Mutt
  • Mailcow
  • Tinymediamanager
  • Cacti (old)
  • Winbox (Mikrotik)
  • iptraf
  • ntopng
  • Twiki/Foswiki (old)
  • Digikam
  • My own photo manager
  • Cewe Photobooks
  • qdlsrdashboard
  • gphoto2
  • Ktechlab
  • Ardour5
  • Puredata
  • Cadence
  • Natron
  • yt-downloader
  • 4k downloader
  • Taggers
  • Mp3tag
  • Tinymediamanager
  • MusicBrainz Picard
  • Beets
  • Music players
  • MOC
  • SoftSqueeze
  • Clementine
  • My Badly Designed Sound Machine
  • Server stuff
  • Namazu2
  • Netdata
  • Ntopng
  • Snort/Snortsam
  • Virtualisation

Window managers (See other post)

  • Xmonad (current)
  • Gnome (current)
  • Enlightenment (old)
  • Compiz (old)
  • Fluxbox (old)
  • Ratpoison (tried)
  • Twm (tried)
  • Xfce (old)
  • Window Maker(old)
  • Sawfish (old)
  • Kde (old)
  • i3 (old)
  • IceWM (old)
  • Motif (old)
  • Flwm (old)
  • Fvwm (old)
  • Ximian desktop (bought) (old)

  • Home Drawing
    • Sweethome3D
    • Blender
    • Sketchup
    • Drawio
  • Old but cool
    • Mainactor
    • Appleshake
    • Zbruch
    • Povray
    • Lightzone

Other tools:

Git, tig, xdotool, nmon, ntop, iotop, etc etc (lijstje genereren)

Using Reolink IPcams for motion detection automations in Home Assistant

I’ve been testing automations using an IPCam instead of a PIR.

If dark, person detected on Reolink camera -> turn on light.

CODE

alias: Motion detect garden
description: ""
triggers:
  - type: motion
    device_id: 3459783a628a0065fd5540af988f0b60
    entity_id: 4f4193f01ceb5c5ec21a2e5bf3f6c9e3
    domain: binary_sensor
    trigger: device
    for:
      hours: 0
      minutes: 0
      seconds: 2
conditions:
  - condition: sun
    before: sunrise
    after: sunset
  - condition: device
    type: is_off
    device_id: 2189b12c73679451fa1af406b0b492d1
    entity_id: 4bb88f1b2f7bd1d6701cc924ecbb3cb1
    domain: switch
    for:
      hours: 0
      minutes: 0
      seconds: 5
actions:
  - action: switch.turn_on
    metadata: {}
    data: {}
    target:
      device_id: 2189b12c73679451fa1af406b0b492d1
  - type: toggle
    device_id: 348056095d7d3217280b235cf0969dc6
    entity_id: 7c02a433ce73b03df62d2187a97ad9b0
    domain: light
  - action: light.toggle
    metadata: {}
    target:
      entity_id: light.shellyrgbw2_2c4013_channel_4
    data: {}
  - delay:
      hours: 0
      minutes: 0
      seconds: 5
      milliseconds: 0
  - action: light.toggle
    metadata: {}
    target:
      entity_id: light.shellyrgbw2_2c4013_channel_4
    data: {}
  - delay:
      hours: 0
      minutes: 5
      seconds: 0
      milliseconds: 0
  - action: switch.turn_off
    metadata: {}
    target:
      device_id: 2189b12c73679451fa1af406b0b492d1
    data: {}
  - type: toggle
    device_id: 348056095d7d3217280b235cf0969dc6
    entity_id: 7c02a433ce73b03df62d2187a97ad9b0
    domain: light
mode: single

Testing software for a paperless office.

I always wanted to have everything on my fileserver. But indexing or searching for the right document is a pain in the *ss.

I wrote several tools for this but wanted to try something new.

Maybe its a perfect solution for all my datasheets I use in my electronics as well.

So I installed Paperless-ngx using docker.

Paperless-ngx is a community-supported open-source document management system that transforms your physical documents into a searchable online archive so you can keep, well, less paper.

Features

  • Organize and index your scanned documents with tags, correspondents, types, and more.
  • Your data is stored locally on your server and is never transmitted or shared in any way.
  • Performs OCR on your documents, adding searchable and selectable text, even to documents scanned with only images.
  • Utilizes the open-source Tesseract engine to recognize more than 100 languages.
  • Documents are saved as PDF/A format which is designed for long term storage, alongside the unaltered originals.
  • Uses machine-learning to automatically add tags, correspondents and document types to your documents.
  • Supports PDF documents, images, plain text files, Office documents (Word, Excel, PowerPoint, and LibreOffice equivalents) and more.
  • Paperless stores your documents plain on disk. Filenames and folders are managed by paperless and their format can be configured freely with different configurations assigned to different documents.
  • Beautiful, modern web application that features:
    • Customizable dashboard with statistics.
    • Filtering by tags, correspondents, types, and more.
    • Bulk editing of tags, correspondents, types and more.
    • Drag-and-drop uploading of documents throughout the app.
    • Customizable views can be saved and displayed on the dashboard and / or sidebar.
    • Support for custom fields of various data types.
    • Shareable public links with optional expiration.
  • Full text search helps you find what you need:
    • Auto completion suggests relevant words from your documents.
    • Results are sorted by relevance to your search query.
    • Highlighting shows you which parts of the document matched the query.
    • Searching for similar documents (“More like this”)
  • Email processing: import documents from your email accounts:
    • Configure multiple accounts and rules for each account.
    • After processing, paperless can perform actions on the messages such as marking as read, deleting and more.
  • A built-in robust multi-user permissions system that supports ‘global’ permissions as well as per document or object.
  • A powerful workflow system that gives you even more control.
  • Optimized for multi core systems: Paperless-ngx consumes multiple documents in parallel.
  • The integrated sanity checker makes sure that your document archive is in good health.

I’ll keep adding to this page at a later time

Upgraded lasercutter

I bought a lasercutter a few years ago. Time for a upgrade.

I moved from a 5.5Watt to a 20Watt version.
Also included an Air-assist! (Not in picture) Which helps a lot.

First serious thing .. a sign to put on my office pc for my girlfriend.
She turns my computer off, when it looks like its not doing anything.

Often Machine Learning things, bash scripts are running which can’t be paused or stopped.

This will take care of interruptions.

Example with/without air assist.

(Didn´t cut in a single pass, and brown edges!)