Category Archives: Computer

Scraping podcast which uses a javascript to obfuscate mp3 links

wget-ting the page only gave me flat html, but no readable links.

We need the rendered version, phantomjs wil help

wget https://bitbucket.org/ariya/phantomjs/downloads/phantomjs-2.1.1-linux-x86_64.tar.bz2

printsource.js

var system = require('system');
var page   = require('webpage').create();
var url    = system.args[1];
page.open(url, function () {
  console.log(page.content);
  phantom.exit();
});

Run phantomjs

phantomjs-2.1.1-linux-x86_64/bin/phantomjs printsource.js  https://xxxxxxxx/show/xxxx > out

So now i got the rendered page, get mp3’s and titles, for this example

cat out | sed 'N;s/\n/,/' | cut -f2,7 -d\" | while read line ; do
mp3=$( echo $line | cut -f1 -d\")
title=$( echo $line | cut -f3 -d\> | tr -d '/<>[]]\!,;' | tr -d "'" | sed s/CDATA//g | sed s#title##g | sed s/:/-/g )
echo "$mp3 $title"
wget $mp3 -O "$title.mp3"
done

bash downloadscript
done

Portable Logitech Media Server again

See post: https://www.henriaanstoot.nl/2014/04/10/portable-squeeze-server/

In the past i’ve used a home build Logitech Squeezebox server (as it was called then), Picore player and tried volumio.
Picore player has been sitting in my livingroom for ages, but was converted to a Node-Red Dashboard and recently Home Assistant Dashboard. (Has been a dasticz daskboard also)

Today i build another version, smaller and with a screen.
Why? .. because of being ‘offline’ or ‘offgrid’ on our holidays.
The car we are driving only has a Aux input.

Most of the installation is as mentioned on:
https://docs.picoreplayer.org/projects/add-a-display/

I edited  /opt/bootsync.sh
to get /dev/sda1 mounted persistent
use pcp br after editing.

Default user/pass : tc piCore

Controlling the thing is via touch or a app on my phone using wifi hotspot.

  • Audio cable 3.5mm
  • Raspberry 3
  • Large usb thumbdrive
  • 3.5inch RPi Display – 480×320 Pixel – XPT2046 Touch Controller
  • car cigarette lighter adapter for power

PiCore uses below alliases

ceChange directory to /mnt/mmcblk0p2/tce
ceoChange directory to /mnt/mmcblk0p2/tce/optional
m1Mount the boot partition /mnt/mmcblk0p1
m2Mount the second partition /mnt/mmcblk0p2
c1Change directory to /mnt/mmcblk0p1
c2Change directory to /mnt/mmcblk0p2
vicfgEdit configuration file config.txt using vi
vicmdEdit boot file cmdline.txt using vi
u1Unmount the boot partition /mnt/mmcblk0p1
u2Unmount the second partition /mnt/mmcblk0p2

Shutting down piCore is done by cutting the power, due to everything being mounted readonly. EXEPT
When you are using LMS server installation, which uses a database.
But there is a tweak for a shutdown button.

I’m using GPIO 16 because i’ve got a screen connected.
Active LOW, means you have to connect a pushbutton/switch between GND and GPIO pin. (nearest Vcc OR Gnd)

Table lamp hack

Added: ino file 20220525

Bought a cheap table lamp a few weeks ago.
Runs on batteries and when you flip it over, it turns on or off.

I thought, when i strip this thing of its internals. I can make a wifi/mqtt enabled one.


Opening it up today, i saw a minimalistic print and a battery holder. There was a tilt switch like

Which i wanted to replace by a mercury one i bought in a bunch of sensors a few years ago.

So why go though all the trouble stripping and replacing .. so i didnt

GND and 5v to the batteries, and D4 to the tilt switch. (Measure which side you have to take!) .. I used a pull down of 3k3 ohms

Esp was flashed in the past with Easy ESP .. well lets keep that one for now.

INO version

#include <ESP8266WiFi.h>
#include <PubSubClient.h>
#include <Wire.h>
#include <Ethernet.h>

const char* ssid = "SSID";
const char* password = "PASSWORD";
const char* mqtt_server = "mqttserver";
const char* mqtt_username = "";
const char* mqtt_password = "";
const char* clientID = "wankel";

const int tiltPin = 4;
int tiltState = 0;    
int previousState = 0;    
WiFiClient espClient;

PubSubClient client(espClient);

void reconnect() {
  while (!client.connected()) {
    if (client.connect(clientID, mqtt_username, mqtt_password)) {
    } else {
      delay(2000);
    }
  }
}

void setup()
{
  {
    client.setServer(mqtt_server, 1883);
    pinMode(tiltPin, INPUT);
  }
}
void loop() {
  tiltState = digitalRead(tiltPin);
  if (tiltState != previousState) {
    if (tiltState == HIGH) {
      client.publish("onoff-wankel/wankel/State", "0"); //
    } else {
      client.publish("onoff-wankel/wankel/State", "1"); //
    }
    delay(100);
  }
  previousState = tiltState;

  {
    if (!client.connected()) {
      reconnect();
    }
    client.loop();
  }
}
Node red + led server

Example is using my ledserver, see other post, but i intent to made a easy to configure node red panel where the to be controlled devices are preconfigured.

[
    {
        "id": "9ec21acaec91aecc",
        "type": "mqtt in",
        "z": "54f3b5b461471f2c",
        "name": "",
        "topic": "onoff-wankel/wankel/State",
        "qos": "2",
        "datatype": "auto",
        "broker": "8c74c5f6.9a7a48",
        "nl": false,
        "rap": true,
        "rh": 0,
        "inputs": 0,
        "x": 400,
        "y": 260,
        "wires": [
            [
                "0fe77b535517f818"
            ]
        ]
    },
    {
        "id": "159f65f444a0d7c2",
        "type": "http request",
        "z": "54f3b5b461471f2c",
        "name": "1 - 30 red",
        "method": "GET",
        "ret": "txt",
        "paytoqs": "ignore",
        "url": "http://ledserver:8080/range/01/30/ff0000",
        "tls": "",
        "persist": false,
        "proxy": "",
        "authType": "",
        "senderr": false,
        "credentials": {},
        "x": 900,
        "y": 280,
        "wires": [
            []
        ]
    },
    {
        "id": "5806fbfd0e99daab",
        "type": "http request",
        "z": "54f3b5b461471f2c",
        "name": "1 - 30 black",
        "method": "GET",
        "ret": "txt",
        "paytoqs": "ignore",
        "url": "http://ledserver:8080/range/01/30/000000",
        "tls": "",
        "persist": false,
        "proxy": "",
        "authType": "",
        "senderr": false,
        "credentials": {
            "user": "",
            "password": ""
        },
        "x": 910,
        "y": 220,
        "wires": [
            []
        ]
    },
    {
        "id": "0fe77b535517f818",
        "type": "switch",
        "z": "54f3b5b461471f2c",
        "name": "",
        "property": "payload",
        "propertyType": "msg",
        "rules": [
            {
                "t": "eq",
                "v": "0",
                "vt": "str"
            },
            {
                "t": "eq",
                "v": "1",
                "vt": "str"
            }
        ],
        "checkall": "true",
        "repair": false,
        "outputs": 2,
        "x": 660,
        "y": 260,
        "wires": [
            [
                "5806fbfd0e99daab"
            ],
            [
                "159f65f444a0d7c2"
            ]
        ]
    },
    {
        "id": "8c74c5f6.9a7a48",
        "type": "mqtt-broker",
        "name": "mqttserver",
        "broker": "mqttserver",
        "port": "1883",
        "clientid": "",
        "usetls": false,
        "compatmode": true,
        "keepalive": "15",
        "cleansession": true,
        "birthTopic": "",
        "birthQos": "0",
        "birthPayload": "",
        "closeTopic": "",
        "closePayload": "",
        "willTopic": "",
        "willQos": "0",
        "willPayload": ""
    }
]

ZFS replace disk

I’m using ZFS for my main fileserver, this pool was created over 10 years ago.
Meanwhile i’ve: Swapped broken disks, switched disks for bigger ones and effectively resized my storage 2 or 3 times. Never had any corruption.

Yesterday i say a warning that one of the disks in the pool was OFFLINE.
Today i replaced it using below command’s

  • Put the disk in OFFLINE mode (if needed, mine was already offline)
    • zpool offline tank sdb
  • Remove disk from system
    • echo 1 | sudo tee /sys/block/sdb/device/delete
  • Remove the disk physically
  • Insert the replacement disk. And copy headers/structure from another disk
    • sgdisk –replicate=/dev/sdb /dev/sda
    • sgdisk –randomize-guids /dev/sdb
  • Run the zpool replace command. 
    • zpool replace tank /dev/sdb
  • Use online command to activate disk (no needed in my case, it already did that)
Tips:

# My labels with serials fell off :(
dd if=/dev/sdb of=/dev/null -> blinky led .. 

# What is the serial?
sudo hdparm -i /dev/sdb | grep Serial

Output

root@latex:~# sgdisk --replicate=/dev/sdb /dev/sda
The operation has completed successfully.
root@latex:~# sgdisk --randomize-guids /dev/sdb
The operation has completed successfully.
root@latex:~# zpool replace tank /dev/sdb
root@latex:~# zpool status
  pool: tank
 state: DEGRADED
status: One or more devices is currently being resilvered.  The pool will
        continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
  scan: resilver in progress since Wed May 18 11:31:21 2022
    5.64T scanned out of 14.4T at 331M/s, 7h42m to go
    1.88T resilvered, 39.16% done
config:

        NAME             STATE     READ WRITE CKSUM
        tank             DEGRADED     0     0     0
          raidz1-0       DEGRADED     0     0     0
            sda          ONLINE       0     0     0
            replacing-1  REMOVED      0     0     0
              old        REMOVED      0     0     0
              sdb        ONLINE       0     0     0  (resilvering)
            sdc          ONLINE       0     0     0

errors: No known data errors

Gotek stuff

Recently back to old skool retro!

As posted https://www.henriaanstoot.nl/2022/05/03/c64-and-sd2iec/

Wellll, i bought some goodies from gotek.nl

A new SUB-D(23) to scart video cable for amiga.

A 9pindin mouse to PS/2 convertor

My tank mouse died .. beyond repair. Blue 9pins to ps2 works great!

A switchless (using key combination) diskdrive switcher df0<>df1

I used to make a floppy switch myself in the past, i used a wirewrap socket, and a cross switch. Made a hole in the back, with the little switch.
This one does not need a hole in your case for the switch.

Own floppy switch

Gotek Floppydrive Emulator with Rotary Encoder

This drive supports a lot of disk formats and systems, i will have to look into that ..
https://github.com/keirf/FlashFloppy/wiki/Host-Platforms

Commodore day

Update: 20220514 – Vic Graf cartridge and more

One Vic-20 working ( switched some keyboards and chips around )
Something i made in 1984? .. then the fuse in my vic-20 power blew (250v 160mA)
Another Vic-20 – with a Bad U31 (Oscillator)? or Vic video chip?
Vic Graf Cartridge, graph a function with annoying sound

Manual : https://archive.org/details/VIC_Graf_1982_Commodore/page/n11/mode/2up

I’ve got a load of cartridges, some of them i tested:

  • Vic-20 – Super expander plus 3K ram ( also some draw and sound functionality in this one )
  • Vic-20 – 32K Ram expander (switchable)
  • Vic-20 – 3K expander
  • C64 – KCS power cardridge
  • C64 – Final Cartridge III
  • C64 – Data Manager 2 – Data Base, hard to find information on this one, will post later on this one.

Mqtt Bash Nodered Notify

When running scripts which take a long time, i don’t want to wait for things to finish before i can start the next one.

For example, using my dedup script or compiling stuff. I wanna know when it is finished.

So i made some scripts

Maybe you can hear the spoken text in the background playing downstairs

I’ve put a function in .bashrc, so i can use a command like
notify “Compiling is ready”
A command like this i can put at the end of a command or in a script file at the end.
make && make install && notify “compile ready”

What does it do when executed?

  • Send a mqtt message to the broker
  • Node-red will read this message and:
    • Send a message to my display on my desk – Sound and message notification. (See another post how i made this )
    • Send a message to a script on my Domoticz instance downstairs.
      • This will use a script to get a speech file from google, and play this on some small speakers in my livingroom
    • Send a pushover message to my phone
    • Display a message on my TV ( not in code below )

How?

At the end of your .bashrc

function notify() {
    if [ -z "$1" ]; then
        echo "Usage: $0 \"message\"";
        exit 1;
    fi
    mosquitto_pub -h 10.1.0.17 -t notify/bashscript -m "$1"
}

Scripts on my Domoticz instance

Python script

#!/usr/bin/python
import paho.mqtt.client as mqttClient
import time
import os
import subprocess
import shlex
Connected = False

def on_connect(client, userdata, flags, rc):
    if rc == 0:
        print("Connected to broker")
        global Connected
        Connected =True
    else:
        print("Connection failed")

def on_message(client, userdata, message):
    print "Message received: " + message.topic + " : " + message.payload
    fileName = "/home/pi/domoticz/scripts/speech" + " \"" + message.payload + "\""
    print fileName
    args = shlex.split(fileName)
    time.sleep(1)
    p = subprocess.Popen(args)


broker_address = "10.1.0.17"
port = 1883
#user = "user"
#password = "password"
client = mqttClient.Client("speechcmd")
#client.username_pw_set(user, password=password)
client.on_connect = on_connect
client.on_message = on_message

client.connect(broker_address, port=port)
client.loop_start()

while Connected != True:
    time.sleep(0.1)

client.subscribe('speech/cmd')

try:
    while True:
        time.sleep(1)

except KeyboardInterrupt:
    print "exiting"
    client.disconnect()
    client.loop_stop()

Caching speech script

This script will look for a cached audiofile with requested text, and uses that. Else it wil request a audio file from google, caches it and plays it though the speakers.

#!/bin/bash
INPUT=$*
input2=$(echo $INPUT | base64)
echo "$input2 = $INPUT" >> /home/pi/cache/files-text-relation
if [ -f /home/pi/cache/$input2.mp3 ] ; then
mpg123 -q /home/pi/cache/$input2.mp3 1>/dev/null 2>/dev/null
else
echo not cached
STRINGNUM=0

ary=($INPUT)
for key in "${!ary[@]}"
  do
    SHORTTMP[$STRINGNUM]="${SHORTTMP[$STRINGNUM]} ${ary[$key]}"
    LENGTH=$(echo ${#SHORTTMP[$STRINGNUM]})
    #echo "word:$key, ${ary[$key]}"
    #echo "adding to: $STRINGNUM"
    if [[ "$LENGTH" -lt "100" ]]; then
      #echo starting new line
      SHORT[$STRINGNUM]=${SHORTTMP[$STRINGNUM]}
    else
      STRINGNUM=$(($STRINGNUM+1))
      SHORTTMP[$STRINGNUM]="${ary[$key]}"
      SHORT[$STRINGNUM]="${ary[$key]}"
    fi
done

for key in "${!SHORT[@]}"
  do
    echo "Playing line: $(($key+1)) of $(($STRINGNUM+1))"
    NEXTURL=$(echo ${SHORT[$key]} | xxd -plain | tr -d '\n' | sed 's/\(..\)/%\1/g')
echo $NEXTURL
    mpg123 -w $input2 -q "http://translate.google.com/translate_tts?ie=UTF-8&client=tw-ob&q=$NEXTURL&tl=En-us"
    ffmpeg -i $input2 -codec:a libmp3lame -qscale:a 2 /home/pi/cache/$input2.mp3
mpg123 /home/pi/cache/$input2.mp3
done
fi

Node-red flow

function notify() {
    if [ -z "$1" ]; then
        echo "Usage: $0 \"message\"";
        exit 1;
    fi
    mosquitto_pub -h 10.1.0.17 -t notify/bashscript -m [
    {
        "id": "1442fca698589679",
        "type": "mqtt in",
        "z": "cb6f001b.721c3",
        "name": "",
        "topic": "notify/bashscript",
        "qos": "2",
        "datatype": "auto",
        "broker": "8c74c5f6.9a7a48",
        "nl": false,
        "rap": false,
        "inputs": 0,
        "x": 180,
        "y": 580,
        "wires": [
            [
                "ddf5744bb5b73d4d",
                "faa5c794652d7a57",
                "b4e0107399248fea",
                "443f960b5d1cf40e"
            ]
        ]
    },
    {
        "id": "ddf5744bb5b73d4d",
        "type": "mqtt out",
        "z": "cb6f001b.721c3",
        "name": "",
        "topic": "speech/cmd",
        "qos": "",
        "retain": "",
        "broker": "8c74c5f6.9a7a48",
        "x": 590,
        "y": 560,
        "wires": []
    },
    {
        "id": "e95e828451d83158",
        "type": "comment",
        "z": "cb6f001b.721c3",
        "name": "bash notify",
        "info": "",
        "x": 170,
        "y": 540,
        "wires": []
    },
    {
        "id": "faa5c794652d7a57",
        "type": "mqtt out",
        "z": "cb6f001b.721c3",
        "name": "",
        "topic": "mqttlcd/message",
        "qos": "",
        "retain": "",
        "broker": "8c74c5f6.9a7a48",
        "x": 570,
        "y": 720,
        "wires": []
    },
    {
        "id": "b4e0107399248fea",
        "type": "delay",
        "z": "cb6f001b.721c3",
        "name": "",
        "pauseType": "delay",
        "timeout": "5",
        "timeoutUnits": "seconds",
        "rate": "1",
        "nbRateUnits": "1",
        "rateUnits": "second",
        "randomFirst": "1",
        "randomLast": "5",
        "randomUnits": "seconds",
        "drop": false,
        "outputs": 1,
        "x": 480,
        "y": 640,
        "wires": [
            [
                "ac4faf30b8adbe3f"
            ]
        ]
    },
    {
        "id": "ac4faf30b8adbe3f",
        "type": "function",
        "z": "cb6f001b.721c3",
        "name": "Empty payload",
        "func": "msg.payload = \"\";\nreturn msg;",
        "outputs": 1,
        "noerr": 0,
        "initialize": "",
        "finalize": "",
        "libs": [],
        "x": 640,
        "y": 640,
        "wires": [
            [
                "faa5c794652d7a57"
            ]
        ]
    },
    {
        "id": "dfbe26c12fc5e742",
        "type": "pushover",
        "z": "cb6f001b.721c3",
        "name": "Alleen Henri",
        "device": "rmx1931",
        "title": "Node-Red-Pushover",
        "priority": "1",
        "sound": "pushover",
        "url": "",
        "url_title": "",
        "html": false,
        "x": 850,
        "y": 500,
        "wires": []
    },
    {
        "id": "443f960b5d1cf40e",
        "type": "function",
        "z": "cb6f001b.721c3",
        "name": "Set pushover payload",
        "func": "\nmsg.topic = \"Bash Notify\";\nmsg.priority = 1;\nmsg.sound = \"cosmic\";\nreturn msg;",
        "outputs": 1,
        "noerr": 0,
        "initialize": "",
        "finalize": "",
        "libs": [],
        "x": 660,
        "y": 500,
        "wires": [
            [
                "dfbe26c12fc5e742"
            ]
        ]
    },
    {
        "id": "8c74c5f6.9a7a48",
        "type": "mqtt-broker",
        "name": "10.1.0.17",
        "broker": "10.1.0.17",
        "port": "1883",
        "clientid": "",
        "usetls": false,
        "compatmode": true,
        "keepalive": "15",
        "cleansession": true,
        "birthTopic": "",
        "birthQos": "0",
        "birthPayload": "",
        "closeTopic": "",
        "closePayload": "",
        "willTopic": "",
        "willQos": "0",
        "willPayload": ""
    }
]