var system = require('system');
var page = require('webpage').create();
var url = system.args[1];
page.open(url, function () {
console.log(page.content);
phantom.exit();
});
Run phantomjs
phantomjs-2.1.1-linux-x86_64/bin/phantomjs printsource.js https://xxxxxxxx/show/xxxx > out
So now i got the rendered page, get mp3’s and titles, for this example
cat out | sed 'N;s/\n/,/' | cut -f2,7 -d\" | while read line ; do
mp3=$( echo $line | cut -f1 -d\")
title=$( echo $line | cut -f3 -d\> | tr -d '/<>[]]\!,;' | tr -d "'" | sed s/CDATA//g | sed s#title##g | sed s/:/-/g )
echo "$mp3 $title"
wget $mp3 -O "$title.mp3"
done
I’m using ZFS for my main fileserver, this pool was created over 10 years ago. Meanwhile i’ve: Swapped broken disks, switched disks for bigger ones and effectively resized my storage 2 or 3 times. Never had any corruption.
Yesterday i say a warning that one of the disks in the pool was OFFLINE. Today i replaced it using below command’s
Put the disk in OFFLINE mode (if needed, mine was already offline)
zpool offline tank sdb
Remove disk from system
echo 1 | sudo tee /sys/block/sdb/device/delete
Remove the disk physically
Insert the replacement disk. And copy headers/structure from another disk
sgdisk –replicate=/dev/sdb /dev/sda
sgdisk –randomize-guids /dev/sdb
Run the zpool replace command.
zpool replace tank /dev/sdb
Use online command to activate disk (no needed in my case, it already did that)
Tips:
# My labels with serials fell off :(
dd if=/dev/sdb of=/dev/null -> blinky led ..
# What is the serial?
sudo hdparm -i /dev/sdb | grep Serial
Output
root@latex:~# sgdisk --replicate=/dev/sdb /dev/sda
The operation has completed successfully.
root@latex:~# sgdisk --randomize-guids /dev/sdb
The operation has completed successfully.
root@latex:~# zpool replace tank /dev/sdb
root@latex:~# zpool status
pool: tank
state: DEGRADED
status: One or more devices is currently being resilvered. The pool will
continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
scan: resilver in progress since Wed May 18 11:31:21 2022
5.64T scanned out of 14.4T at 331M/s, 7h42m to go
1.88T resilvered, 39.16% done
config:
NAME STATE READ WRITE CKSUM
tank DEGRADED 0 0 0
raidz1-0 DEGRADED 0 0 0
sda ONLINE 0 0 0
replacing-1 REMOVED 0 0 0
old REMOVED 0 0 0
sdb ONLINE 0 0 0 (resilvering)
sdc ONLINE 0 0 0
errors: No known data errors
When running scripts which take a long time, i don’t want to wait for things to finish before i can start the next one.
For example, using my dedup script or compiling stuff. I wanna know when it is finished.
So i made some scripts
Maybe you can hear the spoken text in the background playing downstairs
I’ve put a function in .bashrc, so i can use a command like notify “Compiling is ready” A command like this i can put at the end of a command or in a script file at the end. make && make install && notify “compile ready”
What does it do when executed?
Send a mqtt message to the broker
Node-red will read this message and:
Send a message to my display on my desk – Sound and message notification. (See another post how i made this )
Send a message to a script on my Domoticz instance downstairs.
This will use a script to get a speech file from google, and play this on some small speakers in my livingroom
Send a pushover message to my phone
Display a message on my TV ( not in code below )
How?
At the end of your .bashrc
function notify() {
if [ -z "$1" ]; then
echo "Usage: $0 \"message\"";
exit 1;
fi
mosquitto_pub -h 10.1.0.17 -t notify/bashscript -m "$1"
}
This script will look for a cached audiofile with requested text, and uses that. Else it wil request a audio file from google, caches it and plays it though the speakers.
#!/bin/bash
INPUT=$*
input2=$(echo $INPUT | base64)
echo "$input2 = $INPUT" >> /home/pi/cache/files-text-relation
if [ -f /home/pi/cache/$input2.mp3 ] ; then
mpg123 -q /home/pi/cache/$input2.mp3 1>/dev/null 2>/dev/null
else
echo not cached
STRINGNUM=0
ary=($INPUT)
for key in "${!ary[@]}"
do
SHORTTMP[$STRINGNUM]="${SHORTTMP[$STRINGNUM]} ${ary[$key]}"
LENGTH=$(echo ${#SHORTTMP[$STRINGNUM]})
#echo "word:$key, ${ary[$key]}"
#echo "adding to: $STRINGNUM"
if [[ "$LENGTH" -lt "100" ]]; then
#echo starting new line
SHORT[$STRINGNUM]=${SHORTTMP[$STRINGNUM]}
else
STRINGNUM=$(($STRINGNUM+1))
SHORTTMP[$STRINGNUM]="${ary[$key]}"
SHORT[$STRINGNUM]="${ary[$key]}"
fi
done
for key in "${!SHORT[@]}"
do
echo "Playing line: $(($key+1)) of $(($STRINGNUM+1))"
NEXTURL=$(echo ${SHORT[$key]} | xxd -plain | tr -d '\n' | sed 's/\(..\)/%\1/g')
echo $NEXTURL
mpg123 -w $input2 -q "http://translate.google.com/translate_tts?ie=UTF-8&client=tw-ob&q=$NEXTURL&tl=En-us"
ffmpeg -i $input2 -codec:a libmp3lame -qscale:a 2 /home/pi/cache/$input2.mp3
mpg123 /home/pi/cache/$input2.mp3
done
fi
Sorting out my fileserver, i had the need for a deduplication script. Many files i’ve been copying from backup, clouds mobile devices and workstation. Inevitable to get many copies.
Below script walks a directory, using locate it tries to find files with same name. Using a md5sum it wil check if it is the same file, when found a simular file it stops searching, removes the one from the check-directory and checks the next one.
#!/bin/bash
# Copy this script to your to clean directory,
# when you got a copy on your fileserver from this script
# then the copy in your clean dir will be removed also.
# Dont want that? change
# find -type f |
# into
# find -type f | grep -v <nameofthisscript> |
# dont is current directory, skip these from locations
dont=$(pwd)
# Never start in /mnt ? uncomment below
# echo "$dont" | grep "^/mnt" && ( echo "start in tank" ; exit )
find -type f | while read file ; do
filemd5=$(md5sum "$file" | cut -f1 -d" ")
basenamefile=$(basename "$file")
echo "searching $basenamefile"
locate -i "/$basenamefile" | grep -v "$dont" | while read location ; do
if [ -f "$location" ] ; then
locatedfilemd5sum=$(md5sum "$location" | cut -f1 -d" ")
if [ "$filemd5" == "$locatedfilemd5sum" ] ; then
echo "found same md5sum at $location"
rm "$file"
break
fi
fi
done
done
# Remove empty dirs?
# find . -type d -empty -delete
Locate can be slow, sometimes it is better to put the locate DB in memory of on another fast storage system.
Script NO1: Below is a script which checks all sensors and switches available on my domoticz instance, and gives me information about last updates. For example when a device is out of reach or battery empty.
Usage:
Outputs to console, or you can use it in check_mk monitoring. (For the latter, create a script
vi /usr/lib/check_mk_agent/local/checkfrontdoor
chmod +x /usr/lib/check_mk_agent/local/checkfrontdoor
put in the script
#!/bin/bash
cd /path/to/script/
./belowscript "Frontdoor" 300 checkmk
Code:
#!/bin/bash
#set -x
# after running once check stateseconds for names
if [ $# -eq 0 ]; then
echo "$(basename $0) - without options .. getting states"
echo "use $(basename $0) "Sensorname" seconds" for check
echo "Getting all states"
: > stateseconds
curl -s -i -H "Accept: application/json" "http://127.0.0.1:8080/json.htm?type=devices&filter=all&used=true&order=Name" | egrep "Name|Last" | grep -v HardwareName |grep -vi levelnames> states
sed -i 'N;s/\n/,/' states
now=$(date +%s)
cat states | awk '{ print $3" "$4$7$8$9$10$11$12 }' | sed s/,,/,/g |rev | cut -c2- | rev | while read ; do
name="$(echo $REPLY | cut -f2 -d, )"
dater="$(echo $REPLY | cut -f1 -d, | sed s/\"//g)"
#echo $dater
seen=$(date -d "$dater" +%s)
#echo $seen
echo "$name $(( $now - $seen))" >> stateseconds
echo -n "."
done
fi
echo ""
if [[ ! -z "$1" && ! -z $2 ]]; then
checkold=$(cat stateseconds | grep "$1\"" | head -1 | awk '{ print $2 }')
total=$(( checkold - $2 ))
if [ -z $3 ] ; then
if [ $checkold -gt $2 ] ; then echo "$name lastseen longer than $2 seconds ago ($total sec)" ; exit 1 ;fi
else
if [ $checkold -gt $2 ] ; then echo "2 \"$1\" - Sensor older than $2 seconds" ; exit 1
else
echo "0 \"$1\" - Sensor age ok" ; exit 0
fi
fi
fi
I made two scripts, which take a movie, find out its lenght divide by 9. And generates a image montage of snapshots at certain times in the movie. ( A movie of 90 minutes, gets a snapshot every 10 minutes )
Example of snapshots of a movie
Below takes images as above and convert them into animated gifs. All movies in a path or directory structure gets a index page containing all animated gifs.
Example of generated directory index
I’ll post something about my media sorter also. A ajax website using drag drop viewing/sorting images (png/gifs/jpgs) and movies.
Doing my work from home via Jitsi (Jabra headset) and Listening to music on my speakers. Sometimes i want to switch output. For example spotify on my headset.
Using below code, i can easily switch between output sinks.
#!/bin/bash
if [ -z "$1" ]; then
echo "DEVICE"
pactl list sinks | egrep "^Sink|Description" | sed 'N;s/\n/,/' |sed -e 's/\t/ /g' | cut -f2,5- -d" "
echo "APPLICATIONS"
pactl list sink-inputs | egrep -ie "^Sink|application.name" | grep -v "\-application-name" | sed 'N;s/\n/,/' | sed -e 's/\t/ /g' | sed -e 's/ / /g' | cut -f3,6- -d" "
echo "Usage: $0 APPID DEVICEID" >&2
exit 1
fi
pactl move-sink-input $1 $2
It shows output devices, and applications which are using audio sinks. Just match them up.
"If something is worth doing, it's worth overdoing."