Trans: Latin prefix implying "across" or "Beyond", often used in gender nonconforming situations – Scend: Archaic word describing a strong "surge" or "wave", originating with 15th century english sailors – Survival: 15th century english compound word describing an existence only worth transcending.

Category: Featured (Page 1 of 12)

clipi CLI!

Find this project on my github here!

...post updated 06/22/2020
...and again 07/14/2020

clipi:

An efficient toolset for Pi devices

Emulate, organize, burn, manage a variety of distributions for Raspberry Pi.


Choose your own adventure....

Emulate:
clipi virtualizes many common sbc operating systems with QEMU, and can play with both 32 bit and 64 bit operating systems.

  • Select from any of the included distributions (or add your own to /sources.toml!) and clipi will handle the rest.

Organize:
clipi builds and maintains organized directories for each OS as well a persistent & convenient .qcow2 QEMU disk image.

  • Too many huge source .img files and archives? clipi cleans up after itself under the Utilities... menu.
  • additional organizational & gcc compilation methods are available in /kernel.py

Write:
clipi burns emulations to external disks! Just insert a sd card or disk and follow the friendly prompts. All files, /home, guest directories are written out.

  • Need to pre-configure (or double check) wifi? Add your ssid and password to /wpa_supplicant.conf and copy the file to /boot in the freshly burned disk.
  • Need pre-enabled ssh? copy /ssh to /boot too.
  • clipi provides options for writing from an emulation's .qcow2 file via qemu...
  • ...as well as from the source's raw image file with the verbatim argument

Manage:
clipi can find the addresses of all the Raspberry Pi devices on your local network.

  • Need to do this a lot? clipi can install itself as a Bash alias (option under the Utilities... menu, fire it up whenever you want.

Shortcuts:

Shortcuts & configuration arguments can be passed to clipi as a .toml (or yaml) file.

  • Shortcut files access clipi's tools in a similar fashion to the interactive menu:
# <shortcut>.toml
# you can access the same tools and functions visible in the interactive menu like so:
'Burn a bootable disk image' = true  
# same as selecting in the interactive cli
'image' = 'octoprint'
'target_disk' = 'sdc'  
  • clipi exposes many features only accessible via configuration file arguments, such as distribution options and emulation settings.
# <shortcut>.toml
# important qemu arguments can be provided via a shortcut file like so:
'kernel' = "bin/ddebian/vmlinuz-4.19.0-9-arm64"
'initrd' = "bin/ddebian/initrd.img-4.19.0-9-arm64"
# qemu arguments like these use familiar qemu lexicon:
'M' = "virt" 
'm' = "2048"
# default values are be edited the same way:
'cpu' = "cortex-a53"
'qcow_size' = "+8G"
'append' = '"rw root=/dev/vda2 console=ttyAMA0 rootwait fsck.repair=yes memtest=1"'
# extra arguments can be passed too:
'**args' = " -device virtio-blk-device,drive=hd-root \\
             -netdev user,id=net0 -no-reboot \\
             -monitor stdio \\
             -device virtio-net-device,netdev=net0 "
  • Supply a shortcut file like so:
    python3 clipi.py etc/find_pi.toml

  • take a look in /etc for some shortcut examples and default values

# clone:
git clone https://github.com/Jesssullivan/clipi
cd clipi

# preheat:
pip3 install -r requirements.txt
# (or pip install -r requirements.txt)

# begin cooking some Pi:
python3 clipi.py

Parse fdisk -l in Python

fdisk -l has got to be one of the more common disk-related commands one might use while fussing about with raw disk images. The fdisk utility is ubiquitous across linux distributions (also brew install gptfdisk and brew cask install gdisk, supposedly). The -l argument provides a quick look raw sector & file system info. Figuring out the Start, End, Sectors, Size, Id, Format of a disk image's contents without needing to mount it and start lurking around is handy, just the sort of thing one might want to do with Python. Lets write a function to get these attributes into a dictionary- here's mine:

import subprocess
import re

def fdisk(image):

    #  `image`, a .img disk image:
    cmd = str('fdisk -l ' + image)

    # read fdisk output- everything `cmd` would otherwise print to your console on stdout
    # is instead piped into `proc`.
    proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, shell=True)

    # the raw stuff from stdout is not parseable as is, so we read into a string:
    result = proc.stdout.read().__str__()

    # figure out what type we should iterate with when looking via file / part contained within image.  I have no idea if anything besides .img will work- YMMV, but YOLO xD
    if '.iso' in result:
        iter = '.iso'
    if '.qcow2' in result:
        iter = '.qcow2'
    else:  
        iter = '.img'

    # chop up fdisk results by file / partition-
    # the resulting `parts` are equivalent to fdisk "rows" in the shell
    parts = re.findall(r'' + iter + r'\d', result)

    # dictionary `disk` contains each "row" from `parts`:
    disk = {}
    for p in parts:
        # sub dictionary 'part' contains the handy fdisk output values:
        part = {}
        # get just the number words with regex sauce:
        line = result.split(p)[1]
        words = re.split(r'\s+', line)
        # place each word into 'part':
        part['Start'] = words[1]
        part['End'] = words[2]
        part['Sectors'] = words[3]
        part['Size'] = words[4]
        part['Id'] = words[5]
        part['Format'] = words[6].split('\\n')[0]
        # stick this part into 'disk', move onto next disk part:
        disk[p] = part
    return disk

The eBird API & regionCode

get this script and other GIS bits here on github

The Ebird dataset is awesome. While directly handling data as a massive delimited file- as distributed by the eBird people- is cumbersome at best, the ebird api offers a fairly straightforward and efficient alternative for a few choice bits and batches of data.

  • The eBird AWK tool for filtering the actual delimited data can be found over here:

    install.packages("auk")

It is worth noting R + auk (or frankly any R centered filtering method) will quickly become limited by the single-threaded approach of R, and how you're managing memory as you iterate. Working and querying the data from a proper database quickly becomes necessary.

Most conveniently, the eBird API already exists- snag an key over here.

...The API package for R is over here:
install.packages("rebird")

...There is also a neat Python wrapper over here:
pip3 install ebird-api

Region Codes:

I'm not sure why, but some methods use normal latitude / longitude in decimal degrees while some others use "regionCode", which seems to be some kind of eBird special. Only ever seen this format in ebird data.

For example, recent observations uses regionCode:

# GET Recent observations in a region:
# https://api.ebird.org/v2/data/obs/{{regionCode}}/recent

...But nearby recent observations uses latitude / longitude:

# GET Recent nearby observations:
# https://api.ebird.org/v2/data/obs/geo/recent?lat={{lat}}&lng={{lng}}

Regardless, lets just write a function to convert decimal degrees to this regionCode thing. Here's mine:

#!/usr/bin/env python3
"""
# provide latitude & longitude, return eBird "regionCode"
Written by Jess Sullivan
@ https://transscendsurvival.org/
available at: 
https://raw.githubusercontent.com/Jesssullivan/GIS_Shortcuts/master/regioncodes.py
"""
import requests
import json

def get_regioncode(lat, lon):

    # this municipal api is a publicly available, no keys needed afaict
    census_url = str('https://geo.fcc.gov/api/census/area?lat=' +
                     str(lat) +
                     '&lon=' +
                     str(lon) +
                     '&format=json')

    # send out a GET request:
    payload = {}
    get = requests.request("GET", census_url, data=payload)

    # parse the response, all api values are contained in list 'results':
    response = json.loads(get.content)['results'][0]

    # use the last three digits from the in-state fips code as the "subnational 2" identifier:
    fips = response['county_fips']

    # assemble and return the "subnational type 2" code:
    regioncode = 'US-' + response['state_code'] + '-' + fips[2] + fips[3] + fips[4]
    print('formed region code: ' + regioncode)
    return regioncode

Prius, Printers

Add EV only mode button for 2009 Prius-

Pinouts and wiring reference here:
http://www.calcars.org/prius-evbutton-install.pdf

Big shiny EV mode button in the Prius!

Fusion 360 files here: https://a360.co/2zOACJJ

Files uploaded to thingiverse here: https://www.thingiverse.com/thing:4422091

xposted to prius chat too:
https://priuschat.com/threads/3d-printed-ev-mode-button-xd.216774/

PLA & Carbon Polycarbonate button housings:


While we're at it....

See more notes on D&M 3d Printer stuff on github here:
https://github.com/Jesssullivan/Funmat-HT-Notes
...and here:
https://github.com/Jesssullivan/AeroTaz5_hotfix

« Older posts