transscendsurvival.org

Week Notes: Indeterminism, Passkeys, and Spring

Sat, 21 Mar 2026 00:00:00 GMT

![Spring grilling and stuff](/images/posts/IMG_2147.jpg) The grill is out for first-of-the-spring burgers and I spotted my first groundhog of the year peeking out this morning. Spring is real- also happy belated (ie, right on time) st. Patricks day dear reader, from a real life Sullivan ^w^ *corned beef curtesy of my lovely partner* ### Indeterminism Bugs me. Therefore.... Now that much of my personal computing life is largely governed by determinism -- Nix, Haskell, Dhall, starting most sentences refering to f-coalgebra concepts, hermetic builds everywhere -- I've gotten genuinely excited about ***indeterminism*** in large legacy systems, as this > giving up and crying. The convergence vs. congruence distinction from [Promise Theory (2004)](https://dl.acm.org/doi/fullHtml/10.5555/2666018.2666021) describes this tension I've been acutely aware of this winter in my automation work in higher ed: congruent systems repeat an ordered recipe of steps (Ansible, shell scripts, most deployment pipelines), while convergent systems declare a desired state and let the system figure out how to get there (Nix, NixOS, the platonic ideal of what Ansible originally marketed itself as, unfortunately). Most tools claim to be convergent while actually being congruent. The distinction matters enormously when you're trying to automate something that fights back, or a large army of things that fight back AND light your money on fire. - [AmbientTalk paper](https://drops.dagstuhl.de/storage/00lipics/lipics-vol134-ecoop2019/LIPIcs.ECOOP.2019.27/LIPIcs.ECOOP.2019.27.pdf) - [OOPSLA 2016 Distinguished Paper](https://dl.acm.org/doi/10.1145/2983990.2984000) - [IEEE SANER 2019 survey](https://ieeexplore.ieee.org/document/8919181/) - I've been dredging up public remnants of [AmbientTalk - Google Code Archive]([https://soft.vub.ac.be/amop/](https://code.google.com/archive/p/ambienttalk/)) and have had a fun time swimming around the work of [Coen De Roover](https://soft.vub.ac.be/~cderoove/), his students and the [AMOP research group](https://soft.vub.ac.be/amop/) at VUB. - - - ## Listening Rec from my dear friend Lena: [*The Colours of Chloe*](https://tidal.com/album/13466598/u) ([Apple Music](https://music.apple.com/us/album/the-colours-of-chlo%C3%AB/1696074362)) -- weird German ECM-style jazz from 1974. Eberhard Weber. The kind of record that sounds like it was recorded in a cathedral made of felt. Absolutely gorgeous. This [YoungArts Week performance](https://youtu.be/UgnuMd7HaNw) -- the comments section is at least as good as the music. Been revisiting my Miss May I albums again, including the re-recordings. I find the redux releases amusing. The originals have what I can only describe as "dithered by accident because the grooves go just that hard" -- a kind of analog overdriven chaos in the production that is actually part of the experience. The re-recordings lose that and introduce their own wonky production artifacts that are, honestly, just as charming in a completely different way. ## Spring Legs Ran 20km this week and did some lead climbing. It's spring and I'm itching to get back on the bike once it warms up a bit more. #### Random FIDO2 stuff from this week feat. teletype 2026 edition - I've also started investigating direct [CTAP2](https://fidoalliance.org/specs/fido-v2.1-ps-20210615/fido-client-to-authenticator-protocol-v2.1-ps-20210615.html) integration as an alternative to expanding the application's required entitlements from Apple -- fewer entitlements, more portable, less platform coupling -- on my [fork](https://github.com/Jesssullivan/cmux). - Here is my MVP [CTAP2 FFI lib in Zig here - link](https://github.com/Jesssullivan/zig-ctap2) gotta go fast! - **[cmux](https://cmux.com/)** -- I quite like this shell multiplexer and contributed a [captcha handling fix](https://github.com/manaflow-ai/cmux/pull/1877) that merged, and I've been experimenting with better FIDO2 supportability options for current and future cmux iterations / discussion. [The current FIDO2 development path has some limitations around attestation flows I am musing about](https://github.com/manaflow-ai/cmux/pull/1876#issuecomment-4104749274) given I think there are some important issues here around how a foundational tool such as a terminal multiplexer considers fork culture in the long run. - **qutebrowser** -- Improved FIDO2 supportability on my glab tinyland qutebrowser flake (which is tightly coupled with my local solr and searxng instances) which are fairly deviated from upstream at this point. [Here is the FIDO2 PR and discussion upstream from coderkun](https://github.com/qutebrowser/qutebrowser/pull/8642) Iterating on some similar work in the keepassxc codebase as well. Basically, this passkey work has been somewhat spurred on because I picked up the new Apple [Neo](https://www.apple.com/macbook-neo/) to serve as a dedicated "teletype" for the wide array of servers in the basement in which (and on which, in the case of the BCI server) I do my work, contracting and projects. As is tradition, all my widgets have been ceremoniously velcroed to this little macbook- between cmux, qutebrowser and my nix-based intellij gateway setup the little screen is all I need. FIDO2 support makes it way easier to traverse orgs / glab instances / signing keys / this key store or that keystore / this server or that server etc. As far as the Neo goes, I'll begrudgingly say is a genuinely pleasant experience, total AlphaSmart vibes. Just a terminal multiplexer with a good screen, a new battery, a good keyboard and a slab of velcro with little keys, two SDRs, 2 KVMs, 3 keystores, 3 hardware keys and a few adapters.

WinRM Quotas, Plugin Confusion, and Why PSRP Has Been the Answer Since 2018

Fri, 13 Mar 2026 00:00:00 GMT

--- ## [Find the full demo repo here - link](https://github.com/Jesssullivan/winrm-molecule-forkbomb-demo) ### How this started *(Well, my first error was agreeing to touch Windows VMs, but anyway)* As part of the grind to move the Sun and Moon around a bit and automate an EMS upgrade + deployment on campus (and the pleathora of integrations that integrate with this this crazy thing) I've been running batches of pytests, molecule tests etc as part of the slog of ansible fact gathering --> do a bunch of stuff --> evaulate what went wrong arc. Unfortunately, my quest for more ways to parallelize led me to getting me locked out, here, there everywhere on campus. :eyes: This is what Ansible gives you when *WinRM* runs out of capacity: ``` fatal: [win-target]: UNREACHABLE! => { "msg": "Task failed: ntlm: the specified credentials were rejected by the server" } ``` "Credentials rejected." My keys are fine damnit! What's actually happening is that the Windows Remote Management service ran out of room- but the error it sends back through the NTLM handshake looks identical to a real auth failure. This matters enormously because Active Directory counts each of these as a failed login attempt. The typical lockout threshold is 5 failures in 15 minutes. I sent 41 in a single burst. It took me a while to piece together what was going on. Most people know WinRM has a quota system. What I didn't know- and I suspect most Ansible-on-Windows shops don't know either- is that there are actually three quota layers stacked on top of each other. The effective limit for any connection is the minimum across all of them. ```mermaid flowchart TD A["Incoming WinRM Connection"] --> B{"Service Layer
MaxConnections: 300"} B -->|pass| C{"Shell Layer
MaxShellsPerUser: 30
MaxConcurrentUsers: 10"} C -->|pass| D{"Plugin Layer
MaxShellsPerUser: 25
MaxConcurrentUsers: 5"} D -->|pass| E["Shell Created ✓"] B -->|exceeded| F["Connection Refused"] C -->|exceeded| G["'Credentials Rejected'
(misleading error)"] D -->|exceeded| G style D fill:#c62828,color:white style G fill:#e65100,color:white style E fill:#2e7d32,color:white ``` That bottom layer- the plugin layer at `WSMan:\localhost\Plugin\microsoft.powershell\Quotas`- is the one that got me. It sits underneath the shell-level quotas everyone googles, and its defaults are actually *lower*: | Setting | Shell Default | Plugin Default | **Effective** | |---------|:---:|:---:|:---:| | MaxShellsPerUser | 30 | 25 | **25** | | MaxConcurrentUsers | 10 | 5 | **5** | | MaxProcessesPerShell | 25 | 15 | **15** | So you can go raise `MaxShellsPerUser` to 100 at the shell level and still get blocked by the plugin's `MaxConcurrentUsers` of 5. These defaults haven't changed since WinRM 2.0 shipped with Server 2008 R2- [seventeen years](https://transscendsurvival.org/winrm-molecule-forkbomb-demo/winrm-quota-research/#1-default-quota-values-by-windows-version) of the same values across every Windows Server version through 2025. I put together a fairly thorough [quota behavior writeup](https://transscendsurvival.org/winrm-molecule-forkbomb-demo/winrm-quota-research/#2-quota-behavior) covering the exact error codes, SOAP faults, and the surprisingly confusing relationship between `Set-Item WSMan:\` and service restarts. ## Why pywinrm makes this a forkbomb Ok so this is the big one. The default `ansible.builtin.winrm` connection plugin uses [pywinrm and pywinrm creates a **new WinRM shell with a fresh NTLM authentication** for every single Ansible task. No connection pooling. No session reuse. No buffering or piping. ```mermaid flowchart LR subgraph "One Ansible Fork" T1["Task 1"] --> S1["New Shell + NTLM"] T2["Task 2"] --> S2["New Shell + NTLM"] T3["Task 3"] --> S3["New Shell + NTLM"] TN["..."] --> SN["New Shell + NTLM"] end S1 --> W["WinRM Service"] S2 --> W S3 --> W SN --> W style S1 fill:#e65100,color:white style S2 fill:#e65100,color:white style S3 fill:#e65100,color:white style SN fill:#e65100,color:white ``` The math gets bad fast: ``` parallel_molecule_processes × ansible_forks × tasks_per_role = total_shell_attempts 4 × 5 × 15 = 300 ``` Three hundred shell creation attempts against `MaxConcurrentUsers=5`. Five get through, the rest come back as "credentials rejected," and each of those is a real NTLM auth failure against Active Directory. One shared service account across all your managed Windows hosts means one lockout touches everything. This per-task connection model also means that any [credential plugin](https://transscendsurvival.org/winrm-molecule-forkbomb-demo/plugin-quota-analysis/#connection-to-keepassxc-credential-plugin-issues) you're using during execution- KeePassXC lookups, SOPS decryption, 1Password CLI- has to resolve on every single connection rather than once per session. Under parallel load that overhead compounds quickly. ## Reproducing it I put together a [demo repo](https://github.com/Jesssullivan/winrm-molecule-forkbomb-demo) to reproduce this in a controlled way. One thing that tripped me up initially (and I am not proud of how long this took)- Ansible's `forks` setting only controls parallelism *across hosts*. With a single target host you can set `forks=50` and everything still runs serially. The trick is a pressure test inventory with 50 entries all pointing at the same machine: ```yaml # ansible/inventory/pressure-test.yml pressure_targets: hosts: pressure-01: {} pressure-02: {} # ... 48 more pressure-50: {} vars: ansible_host: localhost ansible_connection: winrm ansible_port: 15986 ``` With the shell-level quotas set to defaults (`MaxConcurrentUsers=10`): | | Result | |--------|--------| | Total connections | 50 | | **SUCCESS** | **9** | | **UNREACHABLE** | **41** | | AD lockout threshold | 5 | Nine connections got through- roughly matching `MaxConcurrentUsers=10`. The other 41 went straight to AD as failed auth attempts. That's 8x the lockout threshold in a single burst. ## Finding PSRP After staring at the [forks vs quotas problem](https://transscendsurvival.org/winrm-molecule-forkbomb-demo/winrm-quota-research/#ansible-forks-vs-winrm-quotas) for a while, I stumbled onto [`ansible.builtin.psrp`](https://docs.ansible.com/projects/ansible/latest/collections/ansible/builtin/psrp_connection.html). From the docs- > Run commands or put/fetch on a target via PSRP (WinRM plugin). This is similar to the `ansible.builtin.winrm` connection plugin which uses the same underlying transport but instead runs in a PowerShell interpreter. This solves many of my core qualms with `ansible.builtin.winrm`- specifically, buffering and piping are inherently possible with PSRP. And critically for this problem- it allows for plugin-level connection pooling. One authenticated connection per fork, multiplexing all commands over a persistent PowerShell Runspace Pool. No per-task shell creation, no per-task NTLM handshake. Same pressure test, same 50 connections, but with `ansible_connection=psrp`: ```bash $ ansible -i inventory/pressure-test.yml pressure_targets -m win_ping -f 50 \ -e ansible_connection=psrp -e ansible_psrp_auth=ntlm ``` | | pywinrm | pypsrp | |--------|:---:|:---:| | Successes | 9 | 24 | | UNREACHABLE (auth failure) | 41 | **0** | | AD lockout risk | **HIGH** | **None** | Zero authentication failures. The remaining PSRP failures were TCP timeouts from my SSH tunnel- an infrastructure bottleneck, not an auth problem. The connection pooling also means that credential plugin resolution (my [KeePassXC lookups](https://transscendsurvival.org/winrm-molecule-forkbomb-demo/plugin-quota-analysis/), SOPS, whatever you're using) happens once per connection rather than once per task. The `psrp` plugin has been in `ansible.builtin` (ansible-core) since [Ansible 2.7](https://github.com/ansible/ansible/pull/41729)- October 2018. Same author as pywinrm- [Jordan Borean](https://github.com/jborean93). It's been sitting there for seven years. Better late to the party than never. ```yaml # group_vars/windows.yml ansible_connection: psrp ansible_psrp_auth: ntlm ansible_psrp_protocol: https ansible_psrp_cert_validation: false ``` The quota limits are still possible issues with enough forks, but given I have admin credentials I can [set those on the fly](https://transscendsurvival.org/winrm-molecule-forkbomb-demo/winrm-quotas/#toggleable-quota-tool) during development time. The authentication flood problem- the thing that actually locks you out of AD- that's just gone with PSRP. ## The other footgun I found While resetting quotas to Windows defaults for benchmarking, I tried restarting WinRM over WinRM: ```yaml - name: restart winrm ansible.windows.win_service: name: WinRM state: restarted ``` This is, in retrospect, obviously a bad idea. The service stops (killing the connection I'm using to issue the restart), fails to come back up properly, and I'm left with a box that won't accept remote management at all. `Start-Service WinRM` from RDP also failed- the service was genuinely corrupted, not just stopped. Full OS reboot was the only way back. The good news is that WSMan quota changes [take effect immediately](https://transscendsurvival.org/winrm-molecule-forkbomb-demo/winrm-quota-research/#2-quota-behavior) on new connections without a restart. I didn't need the handler at all. Removed it, documented the finding, moved on. ## yo check it out Everything I found during this investigation- the quota research, the benchmark data, the Ansible roles for managing all of this- is in a [demo repo](https://github.com/Jesssullivan/winrm-molecule-forkbomb-demo) with a [companion docs site](https://transscendsurvival.org/winrm-molecule-forkbomb-demo/). A few things in there that might be useful if you're running into similar problems: - A [`winrm_quota_config` role](https://github.com/Jesssullivan/winrm-molecule-forkbomb-demo/tree/main/ansible/roles/winrm_quota_config) that manages both shell-level and plugin-level quotas idempotently - A [`winrm_monitoring` role](https://github.com/Jesssullivan/winrm-molecule-forkbomb-demo/tree/main/ansible/roles/winrm_monitoring) that deploys Prometheus metrics for active shell counts and quota utilization - [Dhall-typed benchmark profiles](https://github.com/Jesssullivan/winrm-molecule-forkbomb-demo/tree/main/dhall) for systematic forkbomb reproduction - The full [quota research](https://transscendsurvival.org/winrm-molecule-forkbomb-demo/winrm-quota-research/) covering defaults by Windows version, GPO override behavior, registry paths, and what the DISA STIGs actually say (spoiler- they don't constrain quota values at all) ### References - [`ansible.builtin.psrp` docs](https://docs.ansible.com/projects/ansible/latest/collections/ansible/builtin/psrp_connection.html) - [pywinrm#277](https://github.com/diyan/pywinrm/issues/277) — the thread safety issue underlying the per-task connection model - [ansible.windows#597](https://github.com/ansible-collections/ansible.windows/issues/597) — community discussion of WinRM failures at scale - [ansible#41729](https://github.com/ansible/ansible/pull/41729) — the original PSRP PR from August 2018 - [Microsoft WinRM Quotas](https://learn.microsoft.com/en-us/windows/win32/winrm/quotas) - [Plugin quota analysis](https://transscendsurvival.org/winrm-molecule-forkbomb-demo/plugin-quota-analysis/) — the hidden layer and its implications for credential plugins --- -Jess

Updated Resume & CV — March 2026

Thu, 12 Mar 2026 00:00:00 GMT

# Updated Resume & CV I finally got around to a proper update of both my resume and full CV. It's been a while since the last revision, and a lot has happened. ## What's new The big additions: - **Publications section** — The [Northeastern Naturalist paper](https://doi.org/10.1656/045.026.0213) documenting interspecific parental feeding between Black-capped Chickadees and Hermit Thrush nestlings, and my recent [NVMe SSD recovery paper](/papers/recovery-paper.pdf) on bypassing USB bridge firmware opcode whitelists with XRAM injection. - **Research section** — Binary analysis and reverse engineering work (Ghidra, Frida, Zig), heterogeneous compute research with WebGPU and Futhark, and ongoing functional programming explorations with ESDT monads. - **Restructured ventures** — [Tinyland.dev](https://github.com/tinyland-inc) now leads as the primary venture, with xoxd.ai positioned as its ML/AI research arm. Cleaner framing of what we're actually building. - **New FOSS contributions** — Added rspamd, Budgie DE, Mason, and Chapel-lang to the list alongside the usual suspects. - **AAG presentation** — Finally gave the [2019 AAG poster](/aag) its own permalink. Web GIS work presented at the Annual Meeting in Washington, DC. ## The resume Embedded below, or grab it from the [dedicated CV page](/cv) where you can switch between the full CV and condensed resume. Download Resume PDF · Download Full CV PDF ## The full CV The full CV is the longer-form document with the cover letter, fun facts, detailed project descriptions, and all the photography/bartending/bagel history. It's the one with the header image. You can view it on the [CV page](/cv) or grab it directly above. Both documents are compiled from LaTeX source via [Tectonic](https://tectonic-typesetting.github.io/) in CI — the TeX source is [on GitHub](https://github.com/Jesssullivan/jesssullivan.github.io/tree/main/cv) if you're into that sort of thing.

From Bricked to Recovered: The Story of Hacking an NVMe SSD Back to Life

Wed, 04 Mar 2026 00:00:00 GMT

```mermaid graph LR A["Host (Zig)"] -->|"0xE5 XDATA Write"| B["ASM2362 XRAM
0xB000-0xB1FF"] B -->|"Admin SQ Entry"| C["NVMe Controller"] C -->|"Format NVM"| D["SSD Recovered"] style A fill:#2d3748,color:#fff style B fill:#c53030,color:#fff style C fill:#2b6cb0,color:#fff style D fill:#276749,color:#fff ``` It started with the most insidious kind of bug: a disk that lies to you. I was doing routine cleanup on a Lenovo Yoga laptop running Rocky Linux 10. A 256GB Silicon Power NVMe SSD sat in a USB enclosure, connected via USB 3.1. I needed to wipe it and repurpose it. Simple enough. I have done this a thousand times. ```bash $ sudo dd if=/dev/zero of=/dev/sdb bs=512 count=1 conv=fsync oflag=direct 512 bytes copied, 0.000313659 s, 1.6 MB/s ``` Success. Or so `dd` told me. I read back sector zero to confirm the wipe: ```bash $ sudo xxd -l 64 /dev/sdb 00000000: 33c0 8ed0 bc00 7c8e... # MBR boot code - NOT ZEROS ``` The MBR was still there. Every byte intact. Untouched. I blinked, ran it again. Same result. Tried `wipefs`: ```bash $ sudo wipefs --all --force /dev/sdb /dev/sdb: 8 bytes were erased at offset 0x200 (gpt) /dev/sdb: 2 bytes were erased at offset 0x1fe (PMBR) ``` Two messages confirming erasure. Read it back: ```bash $ sudo xxd -s 512 -l 8 /dev/sdb 00000200: 4546 4920 5041 5254 # "EFI PART" still present ``` The GPT header was still there. The MBR magic number `0x55AA` still at offset `0x1FE`. Every signature, every partition entry, every byte -- unchanged. I sat there for a long moment. In twenty years of working with Linux, I have never seen this. Tools do not lie. `dd` does not lie. If it says it wrote data, it wrote data. Unless the disk itself is the liar. --- ## Act 1: The Mystery ### When Your Tools Report Success and Nothing Changes I went through every diagnostic I knew. `blockdev --getro` returned 0 -- the kernel said the device was writable. The SCSI mode pages reported WP=0. Every single indicator a Linux system provides said this drive was read-write and healthy. But nothing I wrote to it stuck. I tried `fdisk`, `gdisk`, `blkdiscard`, `sg_format`, `sg_sanitize`, `sg_write_same`. Some of these tools had the decency to fail with actual errors. But the ones that reported success were the terrifying ones -- they were lying, or more precisely, they were being lied to. The definitive test is embarrassingly simple: write a known pattern, read it back, compare. I wrote zeros to sector 2 (a GPT partition entry containing "Microsoft" in UTF-16), read it back, and found the partition entry byte-for-byte identical to what was there before: ``` BEFORE dd: 00000430: 0000 0000 0000 0000 4d00 6900 6300 7200 ........M.i.c.r. $ sudo dd if=/dev/zero of=/dev/sdb bs=512 count=1 seek=2 conv=fsync oflag=direct 512 bytes copied, 0.00578528 s, 88.5 kB/s AFTER dd: 00000430: 0000 0000 0000 0000 4d00 6900 6300 7200 ........M.i.c.r. ``` This is "silent write failure." The drive accepts write commands at the SCSI protocol level, returns Good status, and quietly drops them on the floor. No errors in `dmesg`, no SCSI check conditions, no I/O failures. Every standard Linux disk tool -- from `dd` to `parted` to `mkfs` -- would report success while accomplishing nothing. ### The Error Trail The tools that failed openly were more instructive. `smartctl -d sntasmedia -i /dev/sdb` returned: ``` Read NVMe Identify Controller failed: scsi error no medium present ``` `sg_sanitize` returned SCSI sense data with a pattern that became depressingly familiar: | Field | Value | Meaning | |-------|-------|---------| | Sense Key | 0x02 | NOT READY | | ASC | 0x3A | MEDIUM NOT PRESENT | | ASCQ | 0x00 | -- | "Medium not present." Like an empty CD-ROM tray. But this was a soldered-down NVMe SSD. There was no media to remove. And in `dmesg`, the kernel was not happy either: ``` sd 1:0:0:0: [sdb] Media removed, stopped polling sd 1:0:0:0: [sdb] tag#5 uas_eh_abort_handler 0 uas-tag 1 inflight: CMD sd 1:0:0:0: [sdb] tag#5 CDB: Read(10) 28 00 00 00 00 00 00 00 01 00 usb 1-2.3: reset high-speed USB device number 23 using xhci_hcd ``` ### Understanding the Stack To understand what was happening -- and more importantly, why every tool lied to me about it -- you need to understand the hardware stack: ```mermaid graph TB subgraph host ["Linux Host"] LINUX["UAS / usb-storage driver
/dev/sdb"] end subgraph bridge ["USB Bridge: ASMedia ASM2362"] BRIDGE["8051 CPU @ 114 MHz
64KB XRAM
VID:PID 174c:2362
Opcode whitelist here"] end subgraph ssd ["NVMe SSD: Silicon Power 256GB"] SSD["Phison PS5012-E12 controller
FTL corrupted → read-only mode
SMART Critical Warning bit 3"] end LINUX -->|"USB 3.1 Gen 2
SCSI commands"| BRIDGE BRIDGE -->|"M.2 PCIe Gen3 x2
NVMe commands (filtered!)"| SSD style bridge fill:#ff6b6b22,stroke:#ff6b6b style ssd fill:#ffa50022,stroke:#ffa500 ``` Three layers of hardware. Three layers of firmware. Three layers of potential misunderstanding. The NVMe SSD at the bottom speaks PCIe/NVMe. The ASMedia ASM2362 USB bridge in the middle translates between NVMe and USB/SCSI. Linux at the top talks SCSI to what it thinks is a normal USB mass storage device. Every command I sent had to traverse this entire stack. Every response had to travel back up. And as I was about to discover, the middle layer had opinions about which commands deserved to be forwarded and which deserved to be silently swallowed. ### The Root Cause I traced the origin to a Windows hibernate event weeks earlier. This laptop had been running Windows with the SSD as a secondary drive. During hibernation, Windows suspends writes to all volumes and dumps system state to disk. If power is lost during that process -- a battery death, an accidental unplug -- the NVMe controller's Flash Translation Layer (FTL) can be left in an inconsistent state. The FTL is the firmware component that maps logical block addresses (the addresses your OS uses) to physical NAND flash locations (the actual cells on the silicon). It is the controller's internal bookkeeping. When it gets corrupted, the controller cannot safely write new data without risking overwriting valid data it can no longer track. So it does the responsible thing: it enters read-only mode. This is documented in the NVMe specification as SMART/Health Critical Warning Bit 3 (mask 0x08): > "The media has been placed in read only mode. Vendor specific recovery may be required." That last sentence is doing a tremendous amount of heavy lifting. "Vendor specific recovery" means there is no standard way to fix this. Every NVMe controller manufacturer implements their own recovery mechanism, and most of them do not publish the details. ### Why This Matters This exact failure mode affects a staggering number of consumer NVMe SSDs. Academic research from UC San Diego documented 50-75% bit error rates during power-loss events on consumer drives. A study by Ohio State and HP Labs found that 13 out of 15 consumer SSDs lost data in power fault testing. Enterprise SSDs have supercapacitors to flush pending writes during power loss. Consumer SSDs do not. If your consumer NVMe SSD loses power during a write-heavy operation -- a Windows hibernate, a database commit, a large file copy -- there is a meaningful chance your FTL will corrupt and your drive will enter this silent read-only mode. Professional data recovery for NVMe SSDs runs $300 to $1,500 or more, and even professionals sometimes cannot help if the FTL is truly destroyed. I decided to see if I could do it myself. I had a $15 USB enclosure, a Linux box, and time. --- ## Act 2: The Rabbit Hole ### Building the Tool I needed to send raw SCSI commands and inspect the results at the byte level. Existing tools like `sg_raw` and `nvme-cli` got me started, but I quickly hit limitations. I needed custom CDB construction, hex-level introspection of sense data, and eventually direct memory manipulation of the bridge chip. No existing tool combined all of those capabilities. I chose Zig. Not because it was trendy, but because the problem demanded it. I needed direct control over memory layout with `packed struct` for exact binary protocol matching. I needed the ability to reason about endianness at the byte level -- NVMe uses little-endian internally, the ASM2362's PCIe TLP registers use big-endian, and SCSI CDBs are big-endian. I needed zero runtime overhead for timing-sensitive hardware interactions. And I needed `comptime` for building CDBs with compile-time-known layouts. The first module was `sg_io.zig` -- a thin wrapper around Linux's SG_IO ioctl interface. SG_IO lets you send raw SCSI commands to any device managed by the kernel's SCSI subsystem. You fill in an `sg_io_hdr` structure with your Command Descriptor Block (CDB), a data buffer, a sense buffer for error information, and a transfer direction flag. Then you call `ioctl(fd, SG_IO, &hdr)` and the kernel sends your exact bytes to the device. The second module was `passthrough.zig` -- the ASMedia ASM2362's proprietary 0xe6 CDB format. This 16-byte CDB tunnels NVMe admin commands through the USB bridge: ``` Byte 0: 0xe6 # ASMedia passthrough opcode Byte 1: NVMe opcode # (0x06=Identify, 0x02=GetLog, etc.) Byte 3: CDW10[7:0] # NVMe Command Dword 10, low byte Byte 6: CDW10[23:16] Byte 7: CDW10[31:24] Bytes 8-11: CDW13 # Big-endian Bytes 12-15: CDW12 # Big-endian ``` I implemented all eight NVMe opcodes that I had found in USB traffic captures of SP Toolbox, Silicon Power's vendor utility: Identify (0x06), Get Log Page (0x02), Format NVM (0x80), Sanitize (0x84), Set Features (0x09), Get Features (0x0A), Security Send (0x82), and Security Receive (0x81). ### First Breakthrough: The Diagnostic Commands Work The `probe` and `identify` commands worked. On the broken drive, I got the "Medium not present" error -- but I already knew the NVMe controller was in protection mode. More importantly, when I tested against a healthy drive, I could query the NVMe controller through the USB bridge and get back valid Identify Controller data: ``` VID: 0x1E4B (Phison Electronics) Model: SPCC M.2 PCIe SSD Firmware: H211011a Serial: 00000000000000001387 OACS: 0x00 ``` VID 0x1E4B confirmed the controller: Phison PS5012-E12, one of the most common NVMe controllers in consumer SSDs. OACS (Optional Admin Command Support) was 0x00, meaning the controller's own Identify data claimed it did not support Format NVM or Security commands through this interface. An ominous sign. The SMART data, when I could read it from a reference check, painted a picture of a drive that had lived a hard life: 270 unsafe shutdowns, 292 media integrity errors, only 1% available spare capacity. The hibernate power loss was the final straw. ### First Disappointment: Recovery Commands Go Nowhere With diagnostics working, the plan was straightforward: send a Format NVM or Sanitize command through the 0xe6 passthrough to wipe the corrupted FTL and force the controller to rebuild it from scratch. Data would be lost, but the drive would be functional again. I sent Format NVM (opcode 0x80). SCSI status: Good. No errors. Duration: 2 milliseconds. That was suspicious. A real NVMe format on a 256GB drive takes minutes, not milliseconds. I checked the drive state. Nothing changed. Sent it again. Same result -- instant "success," zero effect. Tried Sanitize. Same thing. Set Features to clear write protection. Security Send with ATA password protocols. Every single command: SCSI Good status, zero measurable effect on the drive. I spent two full days debugging my CDB construction. Checked every byte position. Verified endianness. Compared my implementation byte-for-byte against the smartmontools `sntasmedia_device` class. My CDBs were correct. The problem was not in my code. ### The Eureka Moment Then I found [cyrozap's USB-to-PCIe reverse engineering repository](https://github.com/cyrozap/usb-to-pcie-re). This was the work of someone who had fully decompiled the ASM2362's 8051 firmware and documented its internals. And buried in the notes was the answer that reframed everything: **The ASM2362 has an opcode whitelist.** The bridge's 8051 microcontroller firmware parses byte 1 of the 0xe6 CDB -- the NVMe opcode -- and checks it against an internal allowlist. Exactly two NVMe admin opcodes are forwarded to the NVMe controller. Everything else is silently dropped: | NVMe Opcode | Command | Passes Through? | |-------------|---------|-----------------| | 0x02 | Get Log Page | **Yes** | | 0x06 | Identify | **Yes** | | 0x09 | Set Features | **Silently dropped** | | 0x0A | Get Features | **Silently dropped** | | 0x80 | Format NVM | **Silently dropped** | | 0x81 | Security Receive | **Silently dropped** | | 0x82 | Security Send | **Silently dropped** | | 0x84 | Sanitize | **Silently dropped** | Six of my eight commands were being swallowed by the bridge firmware. The bridge returned SCSI Good status because from its perspective the SCSI transaction completed successfully -- it just never forwarded the NVMe command to the actual drive. The "Medium not present" error I was getting for some commands was the bridge's catch-all SCSI error for "I do not know what to do with this NVMe opcode." The bridge was not transparent. It had opinions about which commands I was allowed to send. And it expressed those opinions by lying. ### The Catalog of Dead Ends With the 0xe6 path closed off, I took stock of everything that had failed: **Wine + SP Toolbox**: Silicon Power's vendor recovery tool (SP Toolbox) is a .NET Windows application. Wine does not implement `IOCTL_SCSI_PASS_THROUGH_DIRECT` -- the Windows equivalent of Linux's SG_IO. The GUI launches, the device list populates, but every device operation returns `STATUS_NOT_SUPPORTED`. No SSD vendor tool works under Wine for direct hardware access. **Set Features 0x84 (Namespace Write Protection)**: The NVMe spec defines a Write Protection feature (Feature ID 0x84) that the host can toggle. I initially thought this was how the controller locked the drive. It is not. SMART Critical Warning Bit 3 reflects autonomous firmware state stored in the controller's NAND service area. Feature 0x84 is a completely separate mechanism -- host-initiated namespace write protection, defined in NVMe 1.4+. Clearing it would not affect firmware-level read-only mode. **0xe6 passthrough for all recovery commands**: Format NVM, Sanitize, Security Send, Security Receive, Set Features, Get Features -- all silently dropped by the whitelist. The bridge only permits diagnostic reads. You can look at the drive through this interface, but you cannot touch it. Each dead end cost a day or more. The 0xe6 passthrough implementation alone was several hundred lines of carefully constructed Zig code -- format.zig and sanitize.zig, complete with unit tests and proper error handling. Perfectly correct code for commands that could never reach the drive. I archived them and moved on. --- ## Act 3: The Breakthrough ### The Vendor Commands Nobody Talks About Cyrozap's reverse engineering revealed that the ASM2362 does not just have the 0xe6 command. It exposes an entire family of vendor-specific SCSI opcodes: | Opcode | Command | Purpose | |--------|---------|---------| | 0xE0 | Read Config | 128 bytes of bridge configuration | | 0xE1 | Write Config | Modify bridge configuration | | 0xE2 | Flash Read | Read SPI flash contents | | 0xE3 | Firmware Write | Flash new firmware | | 0xE4 | **XDATA Read** | Read 1-255 bytes from bridge XRAM | | 0xE5 | **XDATA Write** | Write a single byte to bridge XRAM | | 0xE6 | NVMe Admin | NVMe passthrough (whitelisted) | | 0xE8 | **Reset** | CPU reset or PCIe link reset | The 0xE4 and 0xE5 commands provide direct read and write access to the ASM2362's internal XRAM -- the 64KB memory space of the 8051 microcontroller. This is the chip's working memory. Its scratch pad. Its brain. And here is the critical detail: the NVMe Admin Submission Queue lives in that XRAM, at addresses 0xB000 through 0xB1FF. This is the ring buffer where the bridge CPU stages NVMe commands before pushing them across the PCIe bus to the NVMe controller. The theory formed immediately. The 0xe6 handler parses your NVMe command, checks the opcode against its whitelist, and only builds a queue entry for approved commands. But what if I skipped the 0xe6 handler entirely? What if I wrote the NVMe Submission Queue Entry directly into XRAM using 0xE5? The bytes would land in the queue *after* any filtering. The bridge firmware would never know a non-whitelisted command had been inserted. It would just see a queue entry appear and push it to the controller. I would be writing NVMe commands directly into the bridge chip's brain. ### The UAS Obstacle The first 0xE4 read command failed before it reached the bridge. Linux returned `errno -75` (`EOVERFLOW`) with `host_status=0x07` (`DID_ERROR`). The command was rejected at the USB transport layer. The culprit was UAS -- USB Attached SCSI protocol. The Linux kernel had loaded the `uas` driver for the ASM2362, which uses USB stream endpoints for command queuing and higher throughput. UAS is the modern, high-performance USB storage driver. It is also pickier about CDB formats. The ASM2362's vendor commands use non-standard CDB sizes (6 bytes for 0xE4/0xE5, 12 bytes for 0xE8) and non-standard data transfer patterns. UAS expects standard SCSI command structures and chokes on the vendor deviations. ```mermaid flowchart TD START["Vendor SCSI command
(0xE4 / 0xE5 / 0xE8)"] --> CHECK{Which USB
driver?} CHECK -->|"UAS
(default)"| UAS["UAS driver rejects
non-standard CDB"] UAS --> FAIL["DID_ERROR
errno -75 EOVERFLOW"] CHECK -->|"usb-storage
(BOT mode)"| BOT["BOT passes CDB
without inspection"] BOT --> OK["Command reaches
ASM2362 bridge"] SWITCH["modprobe usb-storage
quirks=174c:2362:u"] -.->|"Switch driver"| BOT style FAIL fill:#ff6b6b22,stroke:#ff6b6b style OK fill:#51cf6622,stroke:#51cf66 style SWITCH fill:#4ecdc422,stroke:#4ecdc4 ``` The fix was to force the device into BOT mode -- Bulk-Only Transport, the older, simpler USB Mass Storage protocol. BOT is dumber but more permissive. It passes CDBs through without scrutinizing their format: ```bash sudo bash -c ' echo "1-3" > /sys/bus/usb/drivers/usb/unbind rmmod uas rmmod usb_storage modprobe usb-storage quirks=174c:2362:u echo "1-3" > /sys/bus/usb/drivers/usb/bind ' ``` The `quirks=174c:2362:u` flag tells the `usb-storage` module to ignore UAS capability for this specific USB VID:PID combination. The device re-enumerated as `/dev/sdc`. I verified the driver binding -- `readlink` on the sysfs driver path now pointed to `usb-storage` instead of `uas`. ### First XRAM Read With BOT mode active, I sent my first 0xE4 read: ``` CDB: E4 40 00 B0 00 00 ^ ^ ^ ^ ^ ^ | | | | | +-- padding | | | | +-- address low byte: 0x00 | | | +-- address high byte: 0xB0 (address = 0xB000) | | +-- padding | +-- length: 64 bytes +-- opcode: XDATA Read ``` And back came 64 bytes of real data from inside the bridge chip's memory. The first NVMe Submission Queue Entry, left there by the bridge firmware during power-on initialization: ``` B000: 09 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| B010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| B020: 00 00 00 00 00 00 00 00 06 00 00 00 00 00 00 00 |................| B030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| ``` Opcode 0x09 at byte 0: Set Features. The bridge firmware's own initialization command, sitting in the queue like a fossil in amber. I dumped all 512 bytes of the Admin SQ and found four populated entries: ``` [0] 0xB000: OPC=0x09 (Set Features) CID=0 CDW10=0x00000006 [1] 0xB040: OPC=0x09 (Set Features) CID=1 CDW10=0x00000007 [2] 0xB080: OPC=0x05 (Create I/O CQ) CID=2 [3] 0xB0C0: OPC=0x01 (Create I/O SQ) CID=3 [4] 0xB100: (empty) [5] 0xB140: (empty) [6] 0xB180: (empty) [7] 0xB1C0: (empty) ``` The firmware had initialized four commands during boot: two Set Features calls (configuring Number of Queues and Interrupt Coalescing), then Create I/O Completion Queue and Create I/O Submission Queue. The standard NVMe controller initialization sequence, frozen in XRAM. I was reading the bridge chip's brain. Now I needed to write to it. ### The CDB Format Bug This part cost me half a day. My initial 0xE5 write CDB had the byte positions wrong. I had guessed the layout based on the 0xe6 format, which was wrong: ``` My initial guess: E5 [00] [value] [addr_hi] [addr_lo] [00] # WRONG Actual format: E5 [value] [00] [addr_hi] [addr_lo] [00] # CORRECT ``` Byte 1 is the value to write, not byte 2. The data byte is embedded directly in the CDB -- there is no data transfer phase at all. The 0xE5 command is a 6-byte CDB where the payload is inside the command itself. This is unusual for SCSI (most write commands have a separate data-out phase), but it makes sense for single-byte writes. Getting this wrong meant writing the wrong value to the wrong address. In XRAM. On a chip with no memory protection and no firmware signature verification. If I had written garbage to the wrong MMIO register, I could have bricked the bridge controller itself -- turning a recoverable SSD problem into a dead USB enclosure. After correcting the CDB format from cyrozap's Python reference implementation, I wrote 0x42 to address 0xB100 (the first byte of slot 4), read it back via 0xE4, and got 0x42. First confirmed XRAM write. Byte-level control of the bridge chip's internal memory, over USB, from Linux. --- ## Act 4: The Injection ### Writing an NVMe Command, One Byte at a Time An NVMe Submission Queue Entry is a 64-byte little-endian structure defined by the NVMe Base Specification. For a Sanitize Block Erase, the critical fields are: ```zig pub fn craftSanitizeEntry(sanact: u3, command_id: u16) NvmeSqEntry { return .{ .cdw0 = @as(u32, 0x84) | (@as(u32, command_id) << 16), .nsid = 0, // Sanitize operates on entire controller .cdw10 = @as(u32, sanact), // 2 = Block Erase }; } ``` CDW0 packs the opcode (0x84 for Sanitize) in the low byte and a 16-bit Command ID in the high word. CDW10 contains the Sanitize Action -- 2 for Block Erase, which erases all user data blocks at the NAND level. NSID is 0 because Sanitize applies to the whole controller, not individual namespaces. All other fields (PRP pointers, metadata, CDW11-CDW15) are zero -- Sanitize does not transfer data. The entry serializes to 64 bytes. I wrote it to the Admin SQ using 0xE5 -- one SCSI command per byte, 64 individual write operations, each followed by a read-back verification. That is 128 SCSI round-trips through the USB stack to inject one NVMe command. At roughly 80ms per round-trip, the full write-and-verify took about five seconds. ```bash $ sudo ./zig-out/bin/asm2362-tool inject \ --inject-cmd=sanitize-block --dry-run /dev/sdc Phase 1: Reading Admin SQ state... Phase 2: Finding SQ slot... Using slot 4 at XRAM 0xB100 Phase 3: Writing 64 bytes to XRAM... Phase 4: Verifying... All 64 bytes verified. Phase 5: SKIPPED (dry-run) -- doorbell NOT rung ``` Every byte was in place. The NVMe command was sitting in queue memory, correctly formatted, fully verified. But it was just data in RAM. The NVMe controller did not know it was there. ### The Doorbell Problem NVMe uses a producer-consumer model for command submission. The host writes commands to the Submission Queue in memory, then writes a new "tail" pointer to a hardware register called the Submission Queue Tail Doorbell. This doorbell tells the controller: "I have added entries up to this index in the queue. Please process them." On native PCIe, the doorbell is a memory-mapped I/O register at a fixed offset from the controller's BAR0 (Base Address Register 0). For the Admin SQ, the doorbell lives at BAR0 + 0x1000. But I was not on native PCIe. I was going through a USB bridge. The NVMe controller's MMIO registers are on the PCIe side of the bridge, not the USB side. I could not just write to BAR0 + 0x1000 from Linux. Cyrozap's reverse engineering came through once more. The ASM2362 has a PCIe TLP (Transaction Layer Packet) engine accessible through XRAM registers at 0xB200-0xB296. This engine can construct and transmit arbitrary PCIe transactions across the bridge's PCIe port. By writing the right values to the TLP header registers, data register, and control/status register, I could synthesize a PCIe memory write -- a doorbell ring -- from the USB side. ```mermaid sequenceDiagram participant Host as Linux Host participant Bridge as ASM2362 Bridge
(XRAM) participant TLP as TLP Engine
(0xB200) participant NVMe as NVMe Controller Note over Host,NVMe: XRAM Injection Sequence Host->>Bridge: 0xE4: Read Admin SQ (0xB000) Bridge-->>Host: Current queue state (4 entries) Host->>Bridge: 0xE5: Write 64 bytes to slot 0 Note right of Bridge: 64 individual byte writes
each verified via 0xE4 Host->>Bridge: 0xE4: Full readback verify Bridge-->>Host: All 64 bytes match Host->>TLP: Write new tail value to 0xB220 Host->>TLP: Write TLP header to 0xB210 Host->>TLP: Trigger (0x0F → 0xB254) TLP->>NVMe: PCIe Memory Write TLP
(BAR0 + 0x1000 = doorbell) Note right of NVMe: Controller processes
injected command Host->>Bridge: 0xE4: Read Admin CQ (0xBC00) Bridge-->>Host: CID=0x4242, Status=Success ``` The sequence, ported from cyrozap's Python `pcie_gen_req()` function into Zig: ``` 1. Read BAR0 via PCIe Config Read (bus=1, dev=0, fn=0, offset=0x10) 2. Compute doorbell address: BAR0 + 0x1000 3. Write new tail value (big-endian u32) to XRAM 0xB220 4. Write TLP header (3 x big-endian u32) to XRAM 0xB210: - 0x40000001 (Memory Write, 1 DW payload) - Byte-enable mask - Target address (doorbell, masked to 4-byte boundary) 5. Write 0x01 to XRAM 0xB296 (clear timeout flag) 6. Write 0x0F to XRAM 0xB254 (trigger TLP generation) 7. Poll XRAM 0xB296 bit 2 (wait for ready) 8. Write 0x04 to XRAM 0xB296 (send the TLP) ``` Each step required multiple 0xE5 writes (four bytes per u32, one SCSI command per byte). The full doorbell sequence was about 50 SCSI commands. But a posted memory write TLP has a beautiful property: the USB connection stays alive. Unlike a PCIe reset, which disconnects USB and forces device re-enumeration, a posted write is fire-and-forget. The NVMe controller processes it, and I can immediately read back results. BAR0 came back as `0x00D00000`. Admin SQ Tail Doorbell address: `0x00D01000`. ### First Live Attempt -- And Silence I started conservatively. Format NVM before Sanitize. Slot 4 (the first empty slot). Tail value 5 (one past the new entry). The `--force` flag to enable the doorbell: ```bash $ sudo ./zig-out/bin/asm2362-tool inject \ --inject-cmd=format --slot=4 --tail=5 --force /dev/sdc ``` The doorbell write succeeded. The USB connection stayed alive. No disconnects, no kernel errors. That alone was a minor victory -- earlier experiments using the 0xE8 PCIe reset as a crude doorbell substitute had crashed the USB link every time. But the NVMe controller did not process the command. No completion entry appeared in the Completion Queue. The drive state was unchanged. I tried different slots. Different tail values. Multiple commands in a row. Format, then Sanitize, then Set Features. All correctly written to XRAM, all verified byte-for-byte. The doorbell rang successfully each time. And nothing happened. ### The Admin CQ Discovery The breakthrough came from looking at the Completion Queue -- or rather, from looking at the *right* Completion Queue. I had been reading the Admin CQ at XRAM 0xB800, which is where I expected it based on the memory map layout. But 0xB800 turned out to be the I/O Completion Queue. The Admin CQ was at 0xBC00. When I read the correct address, the picture clarified instantly: ```bash $ sudo ./zig-out/bin/asm2362-tool admin-cq /dev/sdc Admin Completion Queue (0xBC00, 8 entries) ============================================= [0] SQHD=4 SQID=0 CID=0x0003 P=1 SCT=0 SC=0x00 DNR=0 (Success) [1] SQHD=3 SQID=0 CID=0x0002 P=1 SCT=0 SC=0x00 DNR=0 (Success) [2] SQHD=2 SQID=0 CID=0x0001 P=1 SCT=0 SC=0x00 DNR=0 (Success) [3] SQHD=1 SQID=0 CID=0x0000 P=1 SCT=0 SC=0x00 DNR=0 (Success) [4] (empty) [5] (empty) [6] (empty) [7] (empty) ``` Four completions. CIDs 0 through 3, matching the four firmware-initialized commands. And the critical field: **SQHD (Submission Queue Head) was 4**. The controller had consumed exactly 4 entries and stopped. The queue depth was 4, not 8. The XRAM had 512 bytes allocated for the Admin SQ -- room for 8 entries of 64 bytes each. But the bridge firmware had configured the NVMe controller with a queue size of 4 during initialization. Slots 4 through 7 existed in XRAM memory, but they were outside the controller's configured queue boundary. My carefully crafted, perfectly verified NVMe commands were sitting in memory that the controller would never read. It was a classic off-by-one-class bug. Not in my code, but in my assumptions about the hardware. I had inferred queue depth from buffer size. The buffer was 8 entries. The queue was 4. --- ## Act 5: The Recovery ### The Winning Command Armed with the correct queue depth, I reformulated everything. Slot 0 instead of slot 4. Tail value 1 instead of 5. Sanitize Block Erase with SANACT=2 (not Format NVM, which had proven ineffective even when correctly delivered). And a distinctive Command ID -- 0x4242 -- so I could unmistakably identify the completion entry: ```bash $ sudo ./zig-out/bin/asm2362-tool inject \ --inject-cmd=sanitize-block \ --slot=0 --tail=1 --cid=0x4242 \ --force /dev/sdc XRAM INJECTION -- EXPERIMENTAL Command: Sanitize Block Erase (0x84, SANACT=2) OPC=0x84, NSID=0x00000000, CDW10=0x00000002, CID=0x4242 Explicit slot: 0 Explicit tail: 1 Phase 1: Reading Admin SQ state... Phase 2: Finding SQ slot... Using explicit slot 0 Phase 3: Writing 64 bytes to XRAM... Phase 4: Verifying... All 64 bytes verified. Phase 5: Ringing doorbell with tail=1... BAR0 = 0x00D00000 Doorbell addr = 0x00D01000, writing tail = 1 Doorbell rung successfully (USB alive) Injection result: Slot used: 0 Bytes written: 64 Verified: true Doorbell rung: true ``` I held my breath and read the Admin CQ: ``` [0] SQHD=1 SQID=0 CID=0x4242 P=1 SCT=0 SC=0x00 DNR=0 (Success) ``` **CID 0x4242. Phase bit 1. Status code 0x00. Success.** The NVMe controller had accepted and was executing the Sanitize Block Erase command. Not a silent drop. Not a firmware refusal. Not a whitelist rejection. The command had been injected into the queue below the firmware's filtering layer, the doorbell had been rung via a synthesized PCIe memory write, and the NVMe controller had processed it. ### Watching the Sanitize I tried running Identify Controller to check the drive state. The response came back as all zeros. This was expected -- during an active sanitize operation, the NVMe specification says the controller may return empty Identify data and should report "Sanitize In Progress" for most admin commands. The sanitize was running. The USB link stayed alive throughout. No disconnects, no kernel errors, no UAS abort handlers. The bridge was peacefully unaware that a non-whitelisted command was being executed on the other side of its PCIe bus. For a 256GB drive, Block Erase is relatively fast -- it operates at the NAND erase-block level, which is a bulk physical operation. After about a minute, I needed to force the controller to re-initialize with fresh state. I sent a CPU reset: ```bash $ sudo ./zig-out/bin/asm2362-tool reset --reset-type=0 /dev/sdc ``` The USB device dropped and re-enumerated. I switched back to BOT mode (the reset had caused the kernel to reload UAS), then ran Identify again: ```bash $ sudo ./zig-out/bin/asm2362-tool identify /dev/sdc ``` Full Identify Controller data came back. Not "Medium not present." Not zeros. Real, valid data: ``` VID: 0x1E4B (Phison) Model: SPCC M.2 PCIe SSD Serial: 00000000000000001387 Firmware: H211011a ``` The controller was alive. The FTL had been rebuilt from scratch. The firmware protection mode was gone. ### The Moment of Truth I wrote a test string to the drive: ```bash $ echo "HIBERPOWER WRITE TEST" | sudo dd of=/dev/sdc bs=512 conv=fsync $ sudo xxd -l 32 /dev/sdc 00000000: 4849 4245 5250 4f57 4552 2057 5249 5445 HIBERPOWER WRITE 00000010: 2054 4553 540a 0000 0000 0000 0000 0000 TEST........... ``` It was there. I read it back and the data matched. For the first time in weeks, this drive had accepted a write and actually committed it to NAND. I ran a more rigorous test -- 1MB of random data, written and read back with comparison: ```bash $ sudo dd if=/dev/urandom of=/dev/sdc bs=1M count=1 conv=fsync oflag=direct 1048576 bytes (1.0 MB) copied, 0.0592 s, 17.7 MB/s $ sudo dd if=/dev/sdc of=/tmp/readback bs=1M count=1 iflag=direct $ cmp /dev/urandom_seed /tmp/readback # (no output -- files identical) ``` 17.7 MB/s write throughput. Full read-back verification. Every byte correct. I checked the Sanitize Status log (NVMe Log Page 0x81): ``` SSTAT: 0x0001 (Most Recent Sanitize: Completed Successfully) SCDW10: 0x00000002 (Sanitize Action: Block Erase) ``` The sanitize had completed successfully. Block Erase. The FTL was clean. ### Formatting for Use The drive was blank -- 256GB of erased NAND. I formatted it: ```bash $ sudo mkfs.vfat -F 32 -n RECOVERED /dev/sdc mkfs.fat 4.2 (2021-01-31) ``` A 239GB FAT32 volume, labeled "RECOVERED." I copied files to it, unmounted, remounted, verified the files. Everything worked. The drive was recovered. --- ## What We Learned Six weeks of work distilled into eight lessons. Some obvious in hindsight, some genuinely surprising. ### 1. USB Bridges Are Not Transparent This is the most important takeaway for anyone debugging storage issues on USB-attached drives. You are not talking to the drive. You are talking to a bridge chip that has its own 8051 CPU, its own firmware, and its own opinions about which commands deserve to be forwarded. The ASM2362 silently drops six of eight NVMe admin opcodes while returning SCSI Good status. It generates SCSI error codes ("Medium not present") for a protocol concept (removable media) that does not exist in NVMe. If your tool reports success but nothing changes, consider the possibility that a middleman is lying to you. ### 2. Silent Failures Are the Worst Debugging Experience An error message, even a bad one, at least tells you something went wrong. When `dd` reports "512 bytes copied" and the bytes were never copied, you have entered a reality distortion field. You start questioning your tools, your kernel, your understanding of how block devices work. The problem is in none of those places. It is in the firmware of a bridge chip that decided your command was not worth forwarding. Loud failures are gifts. Silent ones are traps. ### 3. "Medium Not Present" Is a SCSI Lie ASC 0x3A means "MEDIUM NOT PRESENT" in SCSI terminology. It was designed for removable media -- CD-ROM drives with empty trays, tape drives with no cartridge. NVMe SSDs have no concept of removable media. The ASM2362 generates this error when the NVMe controller does not respond as expected during initialization, or when it receives an opcode it does not recognize. The medium IS present. The bridge just cannot (or will not) talk to it through the requested channel. If you see "Medium not present" on a fixed-media device, the error is probably being generated by a translation layer, not by the storage device itself. ### 4. XRAM Is the Real Attack Surface The 0xe6 passthrough is the "official" API for NVMe commands through the bridge. It is also the filtered, sanitized, whitelisted API. The 0xE4/0xE5 vendor commands provide raw access to the bridge's internal working memory -- no filtering, no whitelist, no firmware opinions. The NVMe Admin Submission Queue is just a data structure in that XRAM. Write the right 64 bytes to the right address and you have injected an arbitrary NVMe admin command. The bridge firmware does not know it happened. The NVMe controller does not care how the command got there. ### 5. NVMe Queue Depths Vary I assumed the Admin SQ had 8 entries because the XRAM buffer was 512 bytes (8 times 64 bytes). The actual queue depth was 4. The firmware configured a 4-entry queue during initialization, and the extra XRAM was unused padding. Commands written to slots 4-7 were in valid memory but outside the controller's queue boundary. Always read the Completion Queue to determine actual queue depth. The SQHD (Submission Queue Head) field in CQ entries tells you exactly how many entries the controller has consumed. ### 6. BOT Mode vs. UAS Mode Matters for Vendor Commands Modern USB storage devices negotiate UAS for better performance. UAS is faster but stricter about command formats. The ASM2362's vendor SCSI commands (0xE4, 0xE5, 0xE8) use non-standard CDB sizes and transfer patterns that UAS cannot handle. You must force the device into Bulk-Only Transport mode with `modprobe usb-storage quirks=174c:2362:u` for vendor commands to work. Standard SCSI commands (INQUIRY, READ, WRITE) work fine under either protocol. The vendor commands are the ones that need BOT. ### 7. Sanitize Block Erase Is More Powerful Than Format NVM I tried Format NVM (opcode 0x80) first. Even when correctly injected and accepted by the controller (the CQ showed success), it did not clear the FTL corruption. Sanitize Block Erase (opcode 0x84, SANACT=2) did. The difference: Format NVM performs a logical format of individual namespaces -- it reinitializes the namespace metadata but does not necessarily erase the underlying NAND or rebuild the FTL. Sanitize Block Erase performs a physical erase of all NAND blocks across the entire controller, forcing a complete FTL rebuild from scratch. For firmware-level write protection caused by FTL corruption, Sanitize is the right tool. Format is insufficient. ### 8. The Entire Recovery Was Done Through USB No M.2 slot required. No soldering. No opening the laptop case. No borrowing a desktop with a spare NVMe slot. The USB bridge that was filtering my commands also provided the attack surface (XRAM access via 0xE4/0xE5) to bypass its own filtering, and the mechanism (PCIe TLP engine at 0xB200) to ring the NVMe doorbell without resetting the USB link. Sometimes the obstacle is the path. The $15 USB enclosure that caused all the problems also contained all the tools needed to solve them. --- ## What This Means for You If you have a "bricked" NVMe SSD in a USB enclosure -- one that reads but does not write, or reports "Medium not present," or shows zero capacity -- it might be recoverable using this technique. The prerequisites: 1. **Your USB enclosure uses an ASMedia ASM236x bridge chip.** Check with `lsusb` for VID `174c`. The ASM2362 (PID `2362`) is the most common in M.2 NVMe enclosures, but the entire ASM236x family uses the same vendor SCSI commands. 2. **The bridge firmware supports 0xE4/0xE5 vendor commands.** All known ASM236x firmware versions do, but always test with a safe read-only probe first. 3. **The failure is FTL corruption, not physical NAND damage.** If the NAND cells themselves are worn out or physically damaged, no software command can help. But if the controller entered read-only mode due to FTL corruption -- the most common failure after power loss -- a Sanitize Block Erase can reset it. 4. **You do not need the data.** Sanitize Block Erase destroys everything. This is a drive recovery technique, not a data recovery technique. The goal is to make the hardware functional again, not to retrieve the old contents. Professional NVMe recovery costs $300 to $1,500 and typically requires specialized hardware like the PC-3000 SSD. This approach costs time and a willingness to read firmware documentation. The tool is open source: approximately 4,000 lines of Zig with 35 unit tests. --- ## Technical Appendix ### The Winning Command Sequence Three commands. That is all it takes once you know the right parameters: ```bash # 1. Switch to BOT mode (required for vendor SCSI commands) sudo bash -c ' echo "" > /sys/bus/usb/drivers/usb/unbind rmmod uas; rmmod usb_storage modprobe usb-storage quirks=174c:2362:u echo "" > /sys/bus/usb/drivers/usb/bind ' # 2. Inject Sanitize Block Erase into Admin SQ slot 0 sudo asm2362-tool inject \ --inject-cmd=sanitize-block \ --slot=0 --tail=1 --cid=0x4242 \ --force /dev/sdX # 3. CPU reset to force NVMe re-initialization sudo asm2362-tool reset --reset-type=0 /dev/sdX ``` ### How to Switch to BOT Mode Find your USB port number with `lsusb -t` or check `dmesg` for the ASM2362 enumeration. The port ID (e.g., "1-3") is the USB topology address. The `quirks` parameter format is `VID:PID:flags` where `u` means "ignore UAS, use BOT." Verify the switch worked: ```bash $ readlink /sys/devices/.../1-3:1.0/driver # Should show: usb-storage (not uas) ``` ### Key XRAM Addresses | Address Range | Size | Contents | |---------------|------|----------| | 0x07F0 | 6B | Firmware version string | | 0xA000-0xAFFF | 4KB | NVMe I/O Submission Queue | | 0xB000-0xB1FF | 512B | NVMe Admin Submission Queue (8 slots, depth may be less) | | 0xB200-0xB296 | ~150B | PCIe TLP engine registers | | 0xBC00-0xBC7F | 128B | NVMe Admin Completion Queue | | 0xF000-0xFFFF | 4KB | NVMe data buffer | ### References - [cyrozap/usb-to-pcie-re](https://github.com/cyrozap/usb-to-pcie-re) -- ASM2362 firmware reverse engineering, the foundation this work is built on - [NVMe Base Specification 2.0](https://nvmexpress.org/specifications/) -- SMART Critical Warning, Sanitize command, SQ/CQ structure - [smartmontools](https://github.com/smartmontools/smartmontools/blob/master/smartmontools/scsinvme.cpp) -- Reference `sntasmedia_device` 0xe6 implementation - [smx-smx/ASMTool](https://github.com/smx-smx/ASMTool) -- ASMedia firmware dumper - [Phison PS5012 Recovery Tools](https://www.usbdev.ru/files/phison/ps5012reinitialtool/) -- Vendor-specific recovery for Phison controllers --- ## Research Paper The full technical paper covers the NVMe specification details, XRAM memory layout, PCIe TLP doorbell mechanism, and complete injection workflow in IEEE conference format. Download PDF --- *This post documents a recovery performed in January through March 2026. The drive was a 256GB Silicon Power NVMe SSD with a Phison PS5012-E12 controller that entered firmware read-only mode after a Windows hibernate power loss event. Total cost of recovery: $0 and six weeks of evenings. The drive is currently formatted and in active use as removable storage.*

Week Notes: Cyborgs, Servers, and Sending

Wed, 04 Mar 2026 00:00:00 GMT

![Cyborg lab: goggles, server guts, drives](/images/posts/IMG_2125.jpg) ## Trash Computing [T7810A34.exe](https://www.dell.com/support/home/en-us/drivers/driversdetails?driverid=8h1nw) and trash computing: bifurcation woes and continued server bending on a jumped up Dell 7810. The struggles of lane splitting for multiple NVMe drives via [80G5N breakout card](https://www.dell.com/support/home/en-us/drivers/driversdetails?driverid=8h1nw) have been more complex than I expected. I already modified this server to run multiple PSUs (went from ~685 watt PSU to drawing 2300 watts at full bore). This machine's job is split between being a BCI HID development box (interfaces with 100 channels of analog serial IO for sensors and measurement, some clocking hardware, a 9070 XT GPU to do graphics things with the goggles, a 82599ES for networking) and participating in various ridiculous network experiments. ## BCI Goggles [BS2e](https://www.bigscreenvr.com/) goggles are laced up to the BCI server and my face! I am developing against the BS2e headset in conjunction with [OpenBCI Ganglion](https://openbci.com/) and [Babble](https://github.com/SummerSigh/ProjectBabble) VR mouth tracking project to bring my evil plans to reality, or something. I do not play videogames. Continued development on **XoxdWM**, a completely bespoke WM/DE for cyborgs inspired by [Stanislav Aleksandrov's VR development for the KDE ecosystem](https://invent.kde.org/plasma/kwin/-/merge_requests/8671). Originally forked from [EXWM](https://github.com/ch11ng/exwm). ## Norwegian Metal in the Office - [Firestarter mixtape](https://youtu.be/Igg__Pmj-nk?si=0y8pXlgEd7HpNkyE) — a classic! - [Sirena — *The 13th Floor*](https://tidal.com/album/167019555/u) (album) - [Immortal — *Northern Chaos Gods*](https://tidal.com/album/107059333/u) (album) ## Tuesday Evening: Indoor Rock Climbing - Warmed up on a fun chimney climb. - Iterated on a 5.13 top rope project. - Rest of the evening on lead belay duty with the [Petzl Neox](https://www.petzl.com/US/en/Sport/Belay-Devices-And-Descenders/technical-content-product/NEOX). ## tcfs After great consternation and headbanging, my TummyCryptFileSystem (tcfs) now has native, notarized [FileProvider](https://developer.apple.com/documentation/fileprovider) support for Darwin targets, whoot whoot. Tcfs now graces the darned work Macs in my life, no macFUSE needed. ## Dhall Since grabbing [Dhall](https://dhall-lang.org/) by the bullhorns a few weeks ago, I've refactored down over 4 thousand lines of Ansible and Tofu code this week. ## Reverse Engineering The use of [Ghidra](https://ghidra-sre.org/) and [Frida](https://frida.re/) to completely disassemble and instrument corpo products for fun will continue until morale improves.

XRAM Injection: Bypassing USB Bridge Whitelists to Recover NVMe Drives

Wed, 04 Mar 2026 00:00:00 GMT

![The ASM2362 USB-NVMe bridge enclosure under investigation](/images/posts/IMG_2127.jpg) ## Introduction If you have ever tried to send a Format NVM or Sanitize command to an NVMe drive connected through a USB enclosure, you have probably discovered an unpleasant truth: the USB bridge does not let you. The ASMedia ASM2362, the most common USB-to-NVMe bridge chip on the market, maintains a firmware-level opcode whitelist that silently drops any NVMe admin command it does not approve of. Out of the entire NVMe admin command set, only two opcodes are forwarded to the NVMe controller: Identify (0x06) and Get Log Page (0x02). Everything else -- Format NVM, Sanitize, Set Features, Security Send/Receive -- is acknowledged at the SCSI layer with a "success" status and then quietly discarded before it ever reaches the drive. This is a critical problem when your NVMe SSD has entered a firmware-level read-only protection mode (SMART Critical Warning bit 3) due to FTL corruption, and the only way to recover it is to send a Format or Sanitize command. If the drive is soldered into a laptop or its M.2 slot is otherwise inaccessible, the USB enclosure may be your only interface. The standard passthrough path is a dead end. **XRAM injection** is a technique that bypasses this whitelist entirely. Instead of asking the bridge firmware to forward an NVMe command through its whitelisted passthrough path, we write the NVMe command directly into the bridge chip's internal XRAM -- the 64KB memory space where the NVMe Admin Submission Queue physically resides. We place a fully-formed 64-byte NVMe Submission Queue entry directly into the hardware queue, then ring the NVMe controller's doorbell via a PCIe Transaction Layer Packet (TLP) write. The bridge firmware never sees the command in its passthrough logic. It is already sitting in the queue, ready for the NVMe controller to fetch via DMA. This post is a complete technical reference for the technique. It covers the ASM2362 hardware architecture, the vendor SCSI commands that provide XRAM access, the NVMe queue structures and their byte layouts, the PCIe TLP doorbell mechanism, and a step-by-step recovery procedure with troubleshooting guidance. All code examples reference the open-source [asm2362-tool](https://github.com/Jesssullivan/hiberpower-ntfs) written in Zig. > **Warning**: The techniques described here involve direct manipulation of hardware bridge memory and NVMe controller queues. Incorrect use can brick drives, corrupt data, or render devices permanently unresponsive. This material is intended for firmware engineers, security researchers, and data recovery specialists who understand NVMe and SCSI at the register level. Only apply these techniques to hardware you own or have explicit authorization to access. ### When You Need This - Your NVMe SSD is stuck in read-only mode (SMART Critical Warning bit 3 set) - The drive reports "Medium not present" for NVMe admin commands via USB - You need to send Format NVM, Sanitize, Security Send, or Set Features commands - The drive is connected through an ASMedia ASM236x-based USB enclosure - You cannot access the drive's M.2 slot directly ### Prerequisites - **Linux** with SG_IO support (any modern distribution; tested on Rocky Linux 10, kernel 6.12.x) - **Root access** or `CAP_SYS_RAWIO` capability - An **ASM236x-based USB-to-NVMe enclosure** (USB VID:PID `174c:2362` or similar) - The drive must be connected through **usb-storage (BOT mode)**, not UAS - **Zig 0.13.0+** to build the tool from source - Basic familiarity with NVMe, SCSI, and hexadecimal notation --- ## The ASM2362 Architecture The ASMedia ASM2362 is a USB 3.1 Gen 2 (10 Gbps) to PCIe Gen 3 x2 bridge chip. It translates SCSI commands (over USB) into NVMe commands (over PCIe). Understanding its internals is the foundation for XRAM injection. ### The 8051 Core The ASM2362 runs an **8051-compatible microcontroller** at approximately 114 MHz. This venerable architecture (dating to Intel's 1980 design) manages USB transport, SCSI command parsing, NVMe queue management, PCIe link training, and the opcode whitelist we need to bypass. ### 64KB XRAM Address Space The 8051 has access to a 64KB external RAM (XRAM) address space, mapped from 0x0000 to 0xFFFF. This memory region contains everything the bridge needs to operate: CPU registers, firmware variables, NVMe queue structures, PCIe controller registers, and data buffers. Unlike the NVMe controller's host memory (which would be system RAM on a native PCIe connection), this XRAM is on the bridge chip itself and is directly accessible through vendor SCSI commands. Here is the XRAM memory map as determined through firmware reverse engineering (cyrozap/usb-to-pcie-re) and our own empirical probing: ```mermaid graph TD cpu["0x0000 - 0x07FF
CPU Registers & Firmware Variables (2 KB)"] fw["0x07F0
Firmware Version (6 bytes)"] usb["0x9000 - 0x9FFF
USB/SCSI Control Registers (~4 KB)"] iosq["0xA000 - 0xAFFF
NVMe I/O Submission Queue (4 KB)"] adminsq["0xB000 - 0xB1FF
Admin Submission Queue (512 B)
READ/WRITE"] pcie["0xB200 - 0xB7FF
PCIe MMIO / TLP Engine (1.5 KB)"] iocq["0xB800 - 0xBBFF
I/O Completion Queue (1 KB)"] admincq["0xBC00 - 0xBFFF
Admin Completion Queue (1 KB)"] data["0xF000 - 0xFFFF
NVMe Data Buffer (4 KB)"] cpu --> usb --> iosq --> adminsq --> pcie --> iocq --> admincq --> data cpu --> fw style adminsq fill:#51cf6622,stroke:#51cf66 style pcie fill:#4ecdc422,stroke:#4ecdc4 ``` | Address Range | Size | Contents | Access | |---------------|------|----------|--------| | `0x0000-0x07FF` | 2 KB | CPU registers, firmware variables | Read | | `0x07F0-0x07F5` | 6 B | Firmware version string | Read | | `0x9000-0x9FFF` | ~4 KB | USB/SCSI control registers | Read | | `0xA000-0xAFFF` | 4 KB | NVMe I/O Submission Queue (SQID=1) | Read | | `0xB000-0xB1FF` | 512 B | NVMe Admin Submission Queue (SQID=0) | **Read/Write** | | `0xB200-0xB7FF` | 1.5 KB | PCIe controller MMIO / TLP engine registers | Read/Write* | | `0xB800-0xBBFF` | 1 KB | NVMe I/O Completion Queue (SQID=1) | Read | | `0xBC00-0xBFFF` | 1 KB | NVMe Admin Completion Queue (SQID=0) | Read | | `0xF000-0xFFFF` | 4 KB | NVMe generic data buffer (Identify cache, etc.) | Read/Write | (*The PCIe MMIO registers at 0xB200+ are technically writable, but writing to the wrong address can crash the bridge or disconnect USB. We only write to the specific TLP engine registers needed for doorbell operations.) The critical insight: **the Admin Submission Queue at 0xB000-0xB1FF is both readable and writable** via vendor SCSI commands. This is where NVMe commands live before the controller fetches them over DMA. If we can write a properly-formed command there and then tell the controller to process it, we have bypassed the firmware's whitelist entirely. ### No Firmware Signature Verification The ASM2362 does not verify its firmware cryptographically. Flash read (0xE2) and write (0xE3) are unauthenticated. The chip was designed for manufacturability and debug access, not security -- which is why the vendor SCSI commands are so permissive. ### Additional Hardware Details The downstream PCIe Gen 3 x2 link connects to the NVMe SSD. The bridge's XRAM queues are DMA-mapped so the NVMe controller can fetch commands: the Admin SQ at 0xB000 maps to physical address 0x00800000. For hardware debugging, UART is available at 921600 baud, 8N1, 3.3V. --- ## SCSI Vendor Commands The ASM2362 responds to several vendor-specific SCSI opcodes beyond the standard command set and the well-documented 0xE6 NVMe passthrough. Three of these are the foundation of XRAM injection. ### 0xE4: XDATA Read The XDATA Read command retrieves bytes from the bridge's XRAM address space. ``` CDB (6 bytes): Byte 0: 0xE4 Opcode Byte 1: length Number of bytes to read (1-255) Byte 2: 0x00 Padding Byte 3: addr_hi XRAM address high byte Byte 4: addr_lo XRAM address low byte Byte 5: 0x00 Padding Direction: Device -> Host (SG_DXFER_FROM_DEV) Transfer size: length bytes ``` This command reads `length` bytes starting at the 16-bit XRAM address formed by `(addr_hi << 8) | addr_lo`. The entire 64KB address space is readable -- unmapped regions simply return zeros. The maximum single read is 255 bytes, so larger reads must be performed in chunks. In our Zig implementation: ```zig pub fn buildXdataReadCdb(address: u16, length: u8) [6]u8 { return .{ XDATA_READ_OPCODE, // 0xE4 length, // 1-255 bytes 0x00, // padding @truncate(address >> 8), // addr_hi @truncate(address & 0xFF), // addr_lo 0x00, // padding }; } ``` The SG_IO direction must be set to `SG_DXFER_FROM_DEV` (-3), and a data buffer of at least `length` bytes must be provided. ### 0xE5: XDATA Write The XDATA Write command writes a **single byte** to an XRAM address. ``` CDB (6 bytes): Byte 0: 0xE5 Opcode Byte 1: value The byte value to write Byte 2: 0x00 Padding Byte 3: addr_hi XRAM address high byte Byte 4: addr_lo XRAM address low byte Byte 5: 0x00 Padding Direction: None (SG_DXFER_NONE) Transfer size: 0 bytes ``` This is critically important to understand: **the byte value is embedded directly in the CDB itself, not transmitted in a data buffer**. The SG_IO direction must be `SG_DXFER_NONE` (-1), and no data buffer is used. This means writing a 64-byte NVMe Submission Queue entry requires 64 separate SCSI commands, each carrying a single byte in CDB byte 1. ```zig pub fn buildXdataWriteCdb(address: u16, value: u8) [6]u8 { return .{ XDATA_WRITE_OPCODE, // 0xE5 value, // byte to write 0x00, // padding @truncate(address >> 8), // addr_hi @truncate(address & 0xFF), // addr_lo 0x00, // padding }; } ``` After every write, we perform a readback verification using 0xE4 to confirm the byte was stored correctly. This catches bus errors, firmware interference, and our own bugs. The overhead is significant -- each verified byte write costs two SCSI round-trips -- but correctness matters more than speed when you are injecting commands into hardware queues. ### 0xE8: Reset The Reset command triggers a bridge-level reset. ``` CDB (12 bytes): Byte 0: 0xE8 Opcode Byte 1: type 0x00 = CPU reset, 0x01 = PCIe soft reset Bytes 2-11: 0x00 Padding Direction: None (SG_DXFER_NONE) ``` Two reset types are available: - **CPU reset (type 0x00)**: Full 8051 restart, re-initializing the NVMe controller from scratch. Useful after a Sanitize command completes, as the bridge needs to re-enumerate the drive. The USB connection typically survives this reset. - **PCIe soft reset (type 0x01)**: Triggers PCIe link re-negotiation. This frequently causes the USB connection to drop entirely, requiring the device to be physically unplugged and reconnected. We avoid this reset type in normal operation. --- ## Important: UAS vs BOT Mode Before any vendor SCSI commands will work, the drive must be connected through **usb-storage (Bulk-Only Transport / BOT mode)**, not UAS (USB Attached SCSI). Linux defaults to UAS for USB 3.x devices, but UAS cannot handle the non-standard transfer semantics of the vendor commands -- specifically, 0xE5 embeds its data byte in the CDB with no data-out phase. Under UAS, vendor commands return `DID_ERROR` (host_status=0x07, errno -75 `EOVERFLOW`). This was the project's most time-consuming debugging problem: three days of investigation into "correct" commands that failed, resolved by a one-line driver switch. ### Switching to BOT Mode ```bash # Identify your USB port (check dmesg or lsusb -t) USB_PORT="1-3" sudo bash -c " # Unbind device from current driver echo '$USB_PORT' > /sys/bus/usb/drivers/usb/unbind # Remove both drivers to ensure clean state rmmod uas 2>/dev/null rmmod usb_storage 2>/dev/null # Reload usb-storage with quirk flag forcing BOT mode modprobe usb-storage quirks=174c:2362:u # Rebind device echo '$USB_PORT' > /sys/bus/usb/drivers/usb/bind " ``` The `quirks=174c:2362:u` parameter tells the usb-storage driver to claim the ASM2362 device (USB VID:PID `174c:2362`) and apply the `:u` quirk flag, which forces BOT mode even though the device advertises UAS support. After rebinding, the device re-enumerates (possibly as a different `/dev/sdX`). Verify with `readlink /sys/bus/usb/devices/$USB_PORT/*/driver` -- it should show `usb-storage`, not `uas`. --- ## NVMe Submission Queue Entry Format Each entry in the NVMe Admin Submission Queue is a 64-byte structure encoded in **little-endian** byte order. This is the exact binary layout you must write into XRAM at 0xB000 (slot 0), 0xB040 (slot 1), 0xB080 (slot 2), or 0xB0C0 (slot 3): ``` Offset Size Field Description ------ ---- ----- ----------- 0x00 4B CDW0 Opcode[7:0] | FUSE[9:8] | PSDT[15:14] | CID[31:16] 0x04 4B NSID Namespace ID 0x08 4B Reserved Must be zero 0x0C 4B Reserved Must be zero 0x10 4B MPTR low Metadata Pointer (low 32 bits) 0x14 4B MPTR high Metadata Pointer (high 32 bits) 0x18 4B PRP1 low Physical Region Page 1 (low 32 bits) 0x1C 4B PRP1 high Physical Region Page 1 (high 32 bits) 0x20 4B PRP2 low Physical Region Page 2 (low 32 bits) 0x24 4B PRP2 high Physical Region Page 2 (high 32 bits) 0x28 4B CDW10 Command-specific dword 10 0x2C 4B CDW11 Command-specific dword 11 0x30 4B CDW12 Command-specific dword 12 0x34 4B CDW13 Command-specific dword 13 0x38 4B CDW14 Command-specific dword 14 0x3C 4B CDW15 Command-specific dword 15 ``` For commands that do not transfer data (Format NVM, Sanitize), PRP1, PRP2, and MPTR are all zero. The command-specific behavior lives entirely in CDW0 through CDW15. **CDW0** is the most important dword. Its low byte (bits 7:0) contains the NVMe opcode, and bits 31:16 contain the Command ID (CID) -- a tag you choose to correlate submissions with completions. Always use a distinctive CID (like 0x4242) that will not collide with the bridge firmware's own commands. Our Zig representation mirrors the spec exactly: ```zig pub const NvmeSqEntry = struct { cdw0: u32, // Opcode[7:0], FUSE[9:8], PSDT[15:14], CID[31:16] nsid: u32, reserved8: u32 = 0, reserved12: u32 = 0, mptr_lo: u32 = 0, mptr_hi: u32 = 0, prp1_lo: u32 = 0, prp1_hi: u32 = 0, prp2_lo: u32 = 0, prp2_hi: u32 = 0, cdw10: u32 = 0, cdw11: u32 = 0, cdw12: u32 = 0, cdw13: u32 = 0, cdw14: u32 = 0, cdw15: u32 = 0, }; ``` When serializing to XRAM, each u32 field is stored little-endian. For example, CDW0 = 0x42420084 becomes bytes `84 00 42 42` at offsets 0x00-0x03. --- ## Admin Completion Queue Entry Format The Admin CQ lives at XRAM **0xBC00** (not 0xB800, which is the I/O CQ -- an important distinction). Each entry is 16 bytes: ``` Offset Size Field Description ------ ---- ----- ----------- 0x00 4B DW0 Command-specific result 0x04 4B Reserved 0x08 2B SQHD SQ Head Pointer (how far the controller has consumed) 0x0A 2B SQID SQ Identifier (0 = Admin, 1 = I/O) 0x0C 2B CID Command ID (correlate with your injected command) 0x0E 2B Status Phase[0] | SC[8:1] | SCT[11:9] | CRD[13:12] | More[14] | DNR[15] ``` The **Phase bit** (bit 0 of the Status field) flips each time the controller wraps around the CQ. By tracking phase transitions, you can determine which entries are new completions. The **Status Code** (SC) and **Status Code Type** (SCT) together describe success or failure: SC=0x00 with SCT=0x00 means "Successful Completion." The **CID field** is how you confirm that a completion corresponds to your injected command. If you injected with CID=0x4242 and see a CQ entry with CID=0x4242, that is your response. Common status codes you may encounter: | SCT | SC | Meaning | |-----|-----|---------| | 0x0 | 0x00 | Successful Completion | | 0x0 | 0x01 | Invalid Command Opcode | | 0x0 | 0x02 | Invalid Field in Command | | 0x0 | 0x0B | Invalid Namespace or Format | | 0x0 | 0x1D | Sanitize In Progress | | 0x1 | 0x0D | Feature Not Changeable | | 0x1 | 0x0F | Feature Not Saveable | --- ## The Critical Discovery: Admin SQ Depth is 4 The NVMe specification allows variable Submission Queue depths. The ASM2362 firmware creates the Admin SQ with a **depth of 4**, not 8. Although the 512-byte XRAM region at 0xB000-0xB1FF physically holds 8 entries (8 x 64 = 512), the firmware only configured the NVMe controller to process entries at indices 0 through 3. **The evidence**: reading the Admin CQ after firmware initialization shows SQHD (SQ Head Pointer) values that wrap at 4, never exceeding index 3. The NVMe controller fetches commands from slots 0-3 when the doorbell is rung, but slots 4-7 are invisible to it -- outside the configured queue boundary. This means **you must inject commands into slots 0-3 only** and set the doorbell tail value accordingly. Injecting into slots 4-7 will produce no effect: the NVMe controller will never fetch those entries, no matter how many times you ring the doorbell. In practice, we typically: 1. Read the current Admin SQ to see what the firmware placed there (usually Set Features x2, Create I/O CQ, Create I/O SQ in slots 0-3) 2. Overwrite slot 0 with our injected command (these firmware commands have already executed during initialization; overwriting them has no side effect) 3. Ring the doorbell with tail=1 --- ## PCIe TLP Doorbell After writing a command to the Admin SQ in XRAM, you need to **ring the doorbell** to tell the NVMe controller there is a new command to process. The Admin SQ Tail Doorbell is a memory-mapped register in the NVMe controller's BAR0 space at offset 0x1000. The ASM2362's 8051 CPU cannot directly write to PCIe MMIO registers. Instead, it has a **TLP engine** -- a set of XRAM-mapped registers that let you construct and send raw PCIe Transaction Layer Packets. This is the same mechanism the firmware uses internally for its own NVMe operations; we are simply using it directly. ### Step 1: Read BAR0 via PCIe Config Space First, determine BAR0's address. The NVMe device sits at PCIe bus 1, device 0, function 0 (as seen from the bridge). We read BAR0 using a Type 1 Configuration Read TLP: ``` TLP type: 0x05 (Type 1 Configuration Read) Address: (bus=1 << 24) | (dev=0 << 19) | (fn=0 << 16) | offset=0x10 Result: 0x00D00000 (on our specific hardware) ``` BAR0's low nibble contains type bits, which we mask off: `bar0 = raw_value & 0xFFFFFFF0`. ### Step 2: The TLP Engine Registers The TLP engine is controlled through four XRAM register groups: | Register | Address | Size | Purpose | |----------|---------|------|---------| | TLP Header | `0xB210` | 12 B | Three big-endian u32s forming the TLP header | | TLP Data | `0xB220` | 4 B | Big-endian u32 payload for memory writes | | Operation Trigger | `0xB254` | 1 B | Write 0x0F to initiate TLP send | | Control/Status (CSR) | `0xB296` | 1 B | Bit 0: timeout, Bit 1: completion done, Bit 2: ready | Note the endianness: TLP headers and data are stored in **big-endian** format, even though the 8051 and NVMe structures elsewhere use little-endian. Each 32-bit value written to these registers requires four separate 0xE5 commands (one byte at a time, most-significant byte first). ### Step 3: The Doorbell Sequence ```mermaid sequenceDiagram participant Host as Linux Host
(0xE5 writes) participant XRAM as Bridge XRAM
(TLP registers) participant PCIe as PCIe Bus participant NVMe as NVMe Controller Note over Host,NVMe: Doorbell Ring Sequence Host->>XRAM: Write tail value → 0xB220
(4 bytes, big-endian) Host->>XRAM: Write TLP header → 0xB210
(12 bytes: 0x40000001, BE mask, addr) Host->>XRAM: Clear timeout → 0xB296 = 0x01 Host->>XRAM: Trigger → 0xB254 = 0x0F loop Poll ready Host->>XRAM: Read 0xB296 XRAM-->>Host: Check bit 2 end Host->>XRAM: Send TLP → 0xB296 = 0x04 XRAM->>PCIe: Posted Memory Write TLP PCIe->>NVMe: Write to BAR0 + 0x1000
(Admin SQ Tail Doorbell) Note right of NVMe: Controller fetches
command from SQ slot ``` ```python # 1. Write the new tail value (big-endian u32) to the data register # This is what the NVMe controller will read as the new SQ tail index write_be32(0xB220, new_tail) # 2. Write the 12-byte TLP header (three big-endian u32s) # First DW: fmt_type=0x40 (Posted Memory Write, 32-bit), length=1 DW write_be32(0xB210, 0x40000001) # Second DW: byte enable mask (0x0F = all 4 bytes of a dword write) write_be32(0xB214, 0x0000000F) # Third DW: target address (BAR0 + 0x1000, must be dword-aligned) write_be32(0xB218, 0x00D01000) # 3. Clear the timeout bit in the CSR write_byte(0xB296, 0x01) # 4. Trigger the TLP operation write_byte(0xB254, 0x0F) # 5. Poll CSR for ready (bit 2 set) while (read_byte(0xB296) & 0x04) == 0: pass # Busy-wait; typically completes within 1ms # 6. Send the TLP write_byte(0xB296, 0x04) # 7. Done. For posted writes (type 0x40), no completion TLP is generated. # The USB connection stays alive -- no disconnection! ``` Each `write_be32()` expands to four 0xE5 SCSI commands. Adding the CSR reads for polling and the trigger/send bytes, the entire doorbell sequence requires approximately 20-25 SCSI commands over USB. On our hardware, this completes in about 50-100 milliseconds. The critical design choice: we use a **posted memory write** (TLP format type 0x40). Posted writes fire-and-forget without generating a completion TLP, which means the USB connection is never disrupted. Early in development, we experimented with PCIe reset (0xE8 type=1) as a crude doorbell substitute -- this disconnected USB every time, and the first "format" it triggered caused the drive to report 0 bytes capacity until a power cycle. The TLP doorbell is the correct mechanism. In Zig, our `ringDoorbell` function encapsulates the entire sequence: ```zig pub fn ringDoorbell(device_path: []const u8, new_tail: u32) XramError!void { // Read BAR0 from PCIe config space (bus=1, dev=0, fn=0, offset=0x10) const bar0 = try readBar0(device_path); // Posted memory write to Admin SQ Tail Doorbell (BAR0 + 0x1000) const doorbell_addr = bar0 + 0x1000; _ = try pcieGenReq(device_path, 0x40, doorbell_addr, new_tail, 4); } ``` --- ## Crafting NVMe Commands With the delivery mechanism understood, we can craft specific NVMe admin commands for drive recovery. Each command is a 64-byte Submission Queue entry, and only the non-zero fields need deliberate setting -- all others default to zero. ### Sanitize Block Erase Sanitize with Block Erase (SANACT=2) erases all user data by writing a vendor-determined pattern to every user data location. This is the most commonly supported sanitize action and the command that ultimately succeeded in our recovery. ``` CDW0 = 0x42420084 CID=0x4242, OPC=0x84 (Sanitize) NSID = 0x00000000 Applies to all namespaces (per NVMe spec, NSID is unused for Sanitize) CDW10 = 0x00000002 SANACT=2 (Block Erase) All other fields = 0 ``` In little-endian bytes, CDW0 becomes: `84 00 42 42`. ```zig pub fn craftSanitizeEntry(sanact: u3, command_id: u16) NvmeSqEntry { return .{ .cdw0 = @as(u32, 0x84) | (@as(u32, command_id) << 16), .nsid = 0, .cdw10 = @as(u32, sanact), }; } // Usage: craftSanitizeEntry(2, 0x4242) ``` ### Sanitize Crypto Erase Crypto Erase (SANACT=4) changes the media encryption keys, rendering all previously written data unrecoverable. This is the fastest sanitize action on drives with hardware encryption. ``` CDW0 = 0x42420084 CID=0x4242, OPC=0x84 (Sanitize) NSID = 0x00000000 All namespaces CDW10 = 0x00000004 SANACT=4 (Crypto Erase) ``` ### Format NVM Format NVM (opcode 0x80) reinitializes the NVM media. The LBAF field selects the sector size, and the SES (Secure Erase Setting) field controls whether user data is erased during the format. ``` CDW0 = 0x01000080 CID=0x0100, OPC=0x80 (Format NVM) NSID = 0xFFFFFFFF All namespaces CDW10 = 0x00000200 LBAF=0 (512-byte sectors), SES=1 (User Data Erase) ``` CDW10 bit layout: `LBAF[3:0] | SES[11:9]`. SES=1 at bits 11:9 equals `0x200`. ```zig pub fn craftFormatNvmEntry(nsid: u32, lbaf: u4, ses: u3, command_id: u16) NvmeSqEntry { const cdw10: u32 = @as(u32, lbaf) | (@as(u32, ses) << 9); return .{ .cdw0 = @as(u32, 0x80) | (@as(u32, command_id) << 16), .nsid = nsid, .cdw10 = cdw10, }; } ``` ### Set Features: Clear Write Protection If the NVMe controller supports Namespace Write Protection (Feature ID 0x84), clearing it may remove firmware-imposed write restrictions. This is worth trying before the more destructive Format/Sanitize commands. ``` CDW0 = 0x03000009 CID=0x0300, OPC=0x09 (Set Features) NSID = 0x00000001 Namespace 1 CDW10 = 0x80000084 FID=0x84 (Write Protect Config), SV=1 (save across power cycles) CDW11 = 0x00000000 Value=0 (no write protection) ``` ```zig pub fn craftSetFeaturesEntry(fid: u8, nsid: u32, value: u32, save: bool, command_id: u16) NvmeSqEntry { var cdw10: u32 = @as(u32, fid); if (save) cdw10 |= (1 << 31); // SV bit return .{ .cdw0 = @as(u32, 0x09) | (@as(u32, command_id) << 16), .nsid = nsid, .cdw10 = cdw10, .cdw11 = value, }; } ``` --- ## Complete Injection Workflow XRAM injection proceeds in six phases. Each phase produces observable output, making it possible to diagnose failures at every step. ```mermaid flowchart LR P1["Phase 1
Read Admin SQ"] --> P2["Phase 2
Select Slot"] P2 --> P3["Phase 3
Write 64 Bytes"] P3 --> P4["Phase 4
Verify Readback"] P4 --> P5{"Phase 5
Dry run?"} P5 -->|"--dry-run"| SKIP["Skip doorbell
(safe)"] P5 -->|"--force"| RING["Ring TLP
Doorbell"] RING --> P6["Phase 6
Check CQ"] style SKIP fill:#ffa50022,stroke:#ffa500 style RING fill:#ff6b6b22,stroke:#ff6b6b style P6 fill:#51cf6622,stroke:#51cf66 ``` ### Phase 1: Read Admin SQ State Read the 512-byte Admin Submission Queue from XRAM 0xB000-0xB1FF using chunked 0xE4 reads. Parse the raw bytes into 8 `NvmeSqEntry` structures. This reveals the bridge firmware's own NVMe commands -- the ones it issued during initialization: ``` Admin Submission Queue (0xB000-0xB1FF, 8 entries): [0] 0xB000: OPC=0x09 (Set Features) CID=1 NSID=0x00000000 CDW10=0x80000007 [1] 0xB040: OPC=0x09 (Set Features) CID=2 NSID=0x00000000 CDW10=0x8000000B [2] 0xB080: OPC=0x05 (Create I/O CQ) CID=3 NSID=0x00000000 CDW10=0x003F0001 [3] 0xB0C0: OPC=0x01 (Create I/O SQ) CID=4 NSID=0x00000000 CDW10=0x003F0001 [4] 0xB100: (empty) [5] 0xB140: (empty) [6] 0xB180: (empty) [7] 0xB1C0: (empty) ``` Slots 0-3 contain the firmware's initialization commands; slots 4-7 are empty. But remember: only slots 0-3 are within the Admin SQ depth. ### Phase 2: Select Slot We must target a slot within the active queue depth (0-3). Use `--slot=0` explicitly, overwriting the firmware's first Set Features command. Since these commands have already executed during boot, overwriting them has no side effect on bridge operation. ### Phase 3: Write Command Bytes Serialize the `NvmeSqEntry` to 64 little-endian bytes, then write each byte to XRAM via 0xE5 with read-back verification. This generates 128 SCSI commands (64 writes + 64 verification reads): ``` Phase 3: Writing 64 bytes to XRAM... 0xB000: write 0x84, readback 0x84 OK (CDW0 byte 0: opcode) 0xB001: write 0x00, readback 0x00 OK (CDW0 byte 1) 0xB002: write 0x42, readback 0x42 OK (CDW0 byte 2: CID low) 0xB003: write 0x42, readback 0x42 OK (CDW0 byte 3: CID high) ... (60 more bytes) ... ``` ### Phase 4: Full Readback Verification After all 64 bytes are written individually, read back the entire 64-byte entry in one 0xE4 read and compare against the intended byte array using `memcmp`. This catches bits that may have flipped between individual verification reads and the final state, or any firmware interference that modified the entry after we wrote it. If verification fails, injection aborts immediately. We never ring the doorbell with an unverified command in the queue. ### Phase 5: Ring Doorbell If not in dry-run mode, execute the PCIe TLP doorbell sequence with `new_tail` set to the slot index + 1 (e.g., tail=1 for slot 0). In dry-run mode, this phase is skipped entirely: ``` Phase 5: SKIPPED (dry-run) -- doorbell NOT rung ``` Dry-run mode writes the command to XRAM (phases 1-4) but does not trigger execution. This lets you verify the entire write path and inspect the injected bytes without risking any NVMe controller action. ### Phase 6: Check Completion Queue Read the Admin CQ at XRAM 0xBC00 to look for a completion entry matching our CID. Also re-read the Admin SQ to confirm the command was consumed (the slot may be zeroed by the controller, or SQHD may have advanced). A successful completion: ``` Admin Completion Queue: [0] SQHD=1 SQID=0 CID=0x4242 P=1 SCT=0 SC=0x00 DNR=0 (Success) ``` An error response: ``` [0] SQHD=1 SQID=0 CID=0x4242 P=1 SCT=0 SC=0x01 DNR=0 (Invalid Command Opcode) ``` --- ## The Recovery Procedure (Step by Step) ### Preparation ```bash # Build the tool (requires Zig 0.13.0+) cd /path/to/hiberpower-ntfs zig build # Switch to BOT mode (adjust USB_PORT for your system) USB_PORT="1-3" sudo bash -c " echo '$USB_PORT' > /sys/bus/usb/drivers/usb/unbind rmmod uas 2>/dev/null rmmod usb_storage 2>/dev/null modprobe usb-storage quirks=174c:2362:u echo '$USB_PORT' > /sys/bus/usb/drivers/usb/bind " # Identify the new device path (may have changed) lsblk dmesg | tail -10 ``` ### Diagnostic Phase ```bash # 1. Verify bridge detection sudo ./zig-out/bin/asm2362-tool probe /dev/sda # 2. Check NVMe controller identity (may fail with "Medium not present") sudo ./zig-out/bin/asm2362-tool identify /dev/sda # 3. Probe XRAM capabilities (safe, completely read-only) sudo ./zig-out/bin/asm2362-tool xram-probe /dev/sda # 4. Read Admin CQ to understand controller state sudo ./zig-out/bin/asm2362-tool admin-cq /dev/sda # 5. Dump the Admin SQ to see firmware initialization commands sudo ./zig-out/bin/asm2362-tool xram-dump --addr=0xB000 --len=512 /dev/sda ``` ### Dry Run (Always Do This First) ```bash sudo ./zig-out/bin/asm2362-tool inject \ --inject-cmd=sanitize-block \ --slot=0 \ --tail=1 \ --cid=0x4242 \ --dry-run \ /dev/sda ``` Expected output: ``` XRAM INJECTION -- EXPERIMENTAL Command: Sanitize Block Erase (0x84, SANACT=2) OPC=0x84, NSID=0x00000000, CDW10=0x00000002, CID=0x4242 Explicit slot: 0 Explicit tail: 1 Phase 1: Reading Admin SQ state... Phase 2: Finding SQ slot... Using explicit slot 0 Using slot 0 at XRAM 0xB000 Phase 3: Writing 64 bytes to XRAM... Phase 4: Verifying... Phase 5: SKIPPED (dry-run) -- doorbell NOT rung Phase 6: Reading post-injection state... Injection result: Slot used: 0 Bytes written: 64 Verified: true Doorbell rung: false Duration: 3842ms ``` ### Live Injection If the dry run succeeded (verified=true), execute the live injection: ```bash sudo ./zig-out/bin/asm2362-tool inject \ --inject-cmd=sanitize-block \ --slot=0 \ --tail=1 \ --cid=0x4242 \ --force \ /dev/sda ``` ### Post-Injection Verification ```bash # Wait for sanitize to complete (60-120 seconds typical for 256GB) sleep 90 # Check the completion queue for our CID sudo ./zig-out/bin/asm2362-tool admin-cq /dev/sda # Reset the bridge to re-enumerate the drive sudo ./zig-out/bin/asm2362-tool reset --reset-type=0 /dev/sda # After re-enumeration, verify write capability sudo dd if=/dev/urandom of=/dev/sda bs=1M count=1 sudo dd if=/dev/sda bs=1M count=1 | md5sum # If the hash matches data written, writes are working # Format the drive for use sudo mkfs.vfat -F 32 -n RECOVERED /dev/sda ``` --- ## Troubleshooting Every failure mode encountered during development and testing, with root causes and fixes: | Symptom | Cause | Fix | |---------|-------|-----| | `DID_ERROR` / errno -75 (EOVERFLOW) | Using UAS mode instead of BOT | Switch to usb-storage with `quirks=174c:2362:u` | | "XRAM read failed" | Wrong `/dev/sdX` path after mode switch | Re-check device path with `lsblk` or `dmesg` | | "Permission denied" | Not running as root | Use `sudo` or grant `CAP_SYS_RAWIO` | | Injected command not in CQ | Wrote to slot 4-7 (outside queue depth) | Use `--slot=0 --tail=1` (slots 0-3 only) | | CQ shows "Invalid Command Opcode" (SC=0x01) | NVMe opcode not supported by controller | Check OACS/SANICAP bits in Identify data | | CQ shows "Invalid Field" (SC=0x02) | Bad CDW10/CDW11 parameters | Verify command encoding against NVMe spec | | CQ shows "Sanitize In Progress" (SC=0x1D) | Previous sanitize still running | Wait 2-5 minutes, then check CQ again | | USB disconnects after doorbell | Used PCIe reset (0xE8 type=1) as doorbell | Use `ringDoorbell()` with TLP posted write | | Identify returns all zeros after sanitize | Controller metadata wiped during sanitize | Normal; wait for completion, then CPU reset | | Command accepted but drive still read-only | Firmware-level protection beyond NVMe spec | See "The Phison Barrier" below | | Doorbell times out (PcieTimeout) | PCIe link may be down | Try CPU reset (type=0), wait 3 seconds, retry | | Write verification fails on specific bytes | Bus error or firmware interference | Retry; if persistent, try a different slot | ### The Phison Barrier In our testing against a Phison PS5012-E12 controller, we reached a frustrating plateau: injected NVMe commands were delivered to the controller and returned **success status** in the Completion Queue, but the FTL write protection persisted. The controller acknowledged Format NVM and Sanitize commands without actually executing them. This is a firmware-level protection mechanism in the Phison PS5012-E12 that operates below the NVMe admin command layer. The controller's internal state machine inspects destructive operations and refuses to execute them when the FTL is in a corrupted state, even though the NVMe interface reports success. The XRAM injection technique itself works correctly -- commands are delivered to the controller and processed through the NVMe admin command path. The limitation is at the NVMe controller firmware level, not the bridge bypass. Potential paths forward from this barrier: 1. **Phison Reinitial Tool** (ECFM22.6 from usbdev.ru) -- a vendor-specific recovery utility designed for this exact failure mode, with explicit ASM2362 bridge support 2. **Direct M.2 PCIe connection** -- bypassing the USB bridge entirely to use native NVMe reset mechanisms (CC.EN toggle, PCIe Function Level Reset) 3. **Vendor-specific Phison commands** -- proprietary command sequences that exist in recovery tooling but are not publicly documented --- ## Security Considerations XRAM injection exposes a significant security gap in USB-NVMe bridge designs: - **No authentication.** The vendor SCSI commands require no passwords or cryptographic challenges. Physical USB access + root = full XRAM control. - **No firmware verification.** Flash read (0xE2) and write (0xE3) are equally unauthenticated. - **Full admin command access.** Any NVMe admin command can be injected: Format, Sanitize, Security Send/Receive, Firmware Download. - **Practical implication:** Physical USB access to an ASM236x enclosure equals unrestricted control over the enclosed drive, including drives with full-disk encryption (Sanitize Crypto Erase rotates keys). USB enclosures are not a security boundary. **Ethical use.** Apply this technique only to drives you own or have explicit authorization to access. --- ## Implementation Notes The tool is written in Zig for zero-overhead C interop (SG_IO struct layout must exactly match the kernel), comptime structure validation, static linking (no runtime dependencies), and explicit error handling at every SCSI command boundary. ### Performance Characteristics | Operation | SCSI Commands | Approximate Time | |-----------|--------------|-----------------| | Read 64 bytes from XRAM | 1 | ~2 ms | | Read 512 bytes from XRAM | 3 chunks | ~8 ms | | Write 1 byte (with verify) | 2 | ~4 ms | | Write 64 bytes (with verify) | 128 | ~250 ms | | Ring doorbell (TLP sequence) | ~22 | ~50 ms | | Complete injection (dry-run) | ~135 | ~500 ms | | Complete injection (live) | ~157 | ~600 ms | The bottleneck is USB round-trip latency. Each SCSI command requires a USB bulk transfer; protocol overhead dominates for these small payloads. --- ## How We Got Here: The Dead Ends The path to XRAM injection was not a straight line. Understanding what failed and why is as valuable as understanding what works, because it reveals the assumptions that trip people up when working with USB-attached NVMe hardware. ### Dead End 1: The 0xE6 Passthrough We implemented all 8 NVMe opcodes from SP Toolbox USB captures through the 0xE6 CDB. All silently dropped by the bridge firmware -- and the SCSI layer returned **success status** for every one. No error, no timeout, no indication the command never reached the controller. Only SMART data comparison (unchanged after "successful" formats) revealed the truth. **Lesson:** USB bridges are not transparent tunnels. Always verify command delivery at the NVMe level, not just SCSI transport-layer success. ### Dead End 2: UAS Mode Three days of `DID_ERROR` on every 0xE4/0xE5 command. CDB format was correct, SG_IO layout was correct, everything was correct -- except the USB driver. UAS was silently corrupting vendor commands. One line fixed it: `modprobe usb-storage quirks=174c:2362:u`. **Lesson:** Check the USB driver before debugging SCSI. `readlink .../driver` should be your first step, not your last. ### Dead End 3: Slots 4-7 `findEmptySlot()` returned slot 4 -- the first empty slot. Perfectly formed commands, perfectly verified, perfectly ignored. The SQ depth was 4, not 8. The SQHD field in the CQ told us this plainly. **Lesson:** Read the Completion Queue. SQHD tells you the queue depth. Do not assume maximum capacity. ### Dead End 4: PCIe Reset as Doorbell Before the TLP doorbell, we used `resetBridge(.pcie)` to force the controller to re-examine the SQ. USB disconnected immediately. The triggered format reported 0 bytes capacity. Power cycle reverted it. **Lesson:** The TLP engine exists for a reason. A PCIe link reset is not a doorbell. --- ## References and Credits This work builds on foundational reverse engineering and documentation from the open-source community: - **[cyrozap/usb-to-pcie-re](https://github.com/cyrozap/usb-to-pcie-re)** -- The foundational firmware reverse engineering of ASMedia USB bridge chips. The XRAM memory map, vendor SCSI opcode documentation, CDB byte formats, and TLP engine register definitions all originate from this project. If you are working with ASMedia bridges in any capacity, start here. - **[smartmontools](https://www.smartmontools.org/)** -- The `sntasmedia_device` class in `os_linux.cpp` provided a reference implementation of the 0xE6 passthrough CDB format. While limited to Identify and SMART (the only two whitelisted opcodes), it confirmed CDB byte layout and SG_IO usage patterns. - **[NVMe Base Specification 2.0](https://nvmexpress.org/specifications/)** -- The definitive reference for Submission Queue entry format, Completion Queue entry format, doorbell register locations, and all NVMe admin command definitions. Freely available from the NVMe Express consortium. - **[Zig Programming Language](https://ziglang.org/)** -- Andrew Kelley and the Zig community. The language's zero-overhead C interop and explicit error handling made low-level hardware programming significantly less error-prone than equivalent C code would have been. - **[sg3_utils](https://sg.danny.cz/sg/sg3_utils.html)** -- The `sg_io_hdr` structure documentation and SG_IO ioctl reference material. Essential reading for anyone doing SCSI passthrough on Linux. - **usbdev.ru** -- The Russian USB device forensics and firmware community, which maintains Phison firmware recovery tools and extensive documentation of NVMe controller behavior under failure conditions. --- ## Appendix: Quick Reference Card ### SCSI Vendor Command CDB Formats ``` 0xE4 XDATA Read: E4 [len] 00 [addr_hi] [addr_lo] 00 SG_DXFER_FROM_DEV 0xE5 XDATA Write: E5 [val] 00 [addr_hi] [addr_lo] 00 SG_DXFER_NONE 0xE8 Reset: E8 [type] 00 00 00 00 00 00 00 00 00 00 SG_DXFER_NONE ``` ### Key XRAM Addresses ``` 0xB000 Admin SQ entry 0 (64 bytes) 0xB040 Admin SQ entry 1 0xB080 Admin SQ entry 2 0xB0C0 Admin SQ entry 3 0xB210 TLP header register (12 bytes, 3x u32 big-endian) 0xB220 TLP data register (4 bytes, u32 big-endian) 0xB254 TLP operation trigger (1 byte, write 0x0F) 0xB296 TLP control/status (1 byte: bit 0=timeout, 1=done, 2=ready) 0xBC00 Admin CQ entry 0 (16 bytes) 0xBC10 Admin CQ entry 1 0xF000 NVMe data buffer start (4 KB) ``` ### NVMe Admin Opcodes: Whitelist Status ``` 0x02 Get Log Page Whitelisted (works via 0xE6 passthrough) 0x06 Identify Whitelisted (works via 0xE6 passthrough) 0x09 Set Features BLOCKED -- requires XRAM injection 0x0A Get Features BLOCKED -- requires XRAM injection 0x80 Format NVM BLOCKED -- requires XRAM injection 0x81 Security Send BLOCKED -- requires XRAM injection 0x82 Security Receive BLOCKED -- requires XRAM injection 0x84 Sanitize BLOCKED -- requires XRAM injection ``` ### Minimal Injection Command ```bash # Dry run (writes to SQ, skips doorbell): sudo ./zig-out/bin/asm2362-tool inject \ --inject-cmd=sanitize-block --slot=0 --tail=1 --cid=0x4242 \ --dry-run /dev/sda # Live injection (rings doorbell!): sudo ./zig-out/bin/asm2362-tool inject \ --inject-cmd=sanitize-block --slot=0 --tail=1 --cid=0x4242 \ --force /dev/sda ```

Aperture and the Tagged-Device Identity Gap

Thu, 26 Feb 2026 00:00:00 GMT

*Figuring out why half our tailnet couldn't talk to our AI gateway.* --- ## Setting the scene We run a small fleet of AI agents on Kubernetes. Three agents (IronClaw, PicoClaw, HexStrike) make LLM calls to Anthropic's API. We wanted to route those calls through [Tailscale Aperture](https://tailscale.com/aperture) — their AI gateway — so we'd get identity-aware metering, usage dashboards, and a single place to manage API keys. The setup looked straightforward: ```mermaid graph LR Agent["K8s Agent Pod"] --> Egress["Egress Proxy
ExternalName Service"] Egress --> AI["Aperture
ai.ts.net"] AI --> Anthropic["api.anthropic.com"] style AI fill:#4a9,stroke:#2a7,color:white ``` The Tailscale Operator creates an egress proxy as a K8s Service. Agents set `ANTHROPIC_BASE_URL` to the in-cluster service. Traffic tunnels through the tailnet to Aperture. Aperture adds the real API key and forwards to Anthropic. We had the ACL rules in place (managed with [Dhall](https://dhall-lang.org), naturally): ```haskell -- fragments/aperture.dhall let acls = [ { action = "accept" , src = [ C.tag.dev, C.tag.k8s, C.tag.k8s_operator, ... ] , dst = [ "ai:*" ] } ] ``` Network connectivity: check. We could ping Aperture, we could see it respond. But every request came back **403 Forbidden**: `"access denied: no role granted"`. ## Two layers of auth It took us a while to realize Aperture has **two separate auth layers**, and we were only solving one. ### Layer 1: Tailnet ACL (network) This controls which devices can open TCP connections to Aperture. It's the standard Tailscale ACL — `src` tags, `dst` hosts, port wildcards. We had this right. Our Dhall config compiled to the correct policy. Connections succeeded. ### Layer 2: Aperture's internal roles (application) This is where it got interesting. Aperture uses Tailscale's WhoIs API to identify *who* is connecting. Then it checks its own `temp_grants` config — a separate JSON structure managed through Aperture's web UI or config API — to decide what that identity can do. These are **not** tailnet grants. There's no `tailscale.com/cap/aperture` capability domain (unlike Setec's `tailscale.com/cap/secrets` or the K8s Operator's `tailscale.com/cap/kubernetes`). Aperture manages its own authorization independently. Our config looked fine: ```json { "temp_grants": [ { "src": ["jess@sulliwood.org", "jsullivan2@gmail.com", "tagged-devices"], "grants": [{"role": "admin"}] }, { "src": ["tagged-devices"], "grants": [{"role": "user"}, {"providers": [...]}] } ] } ``` Admin access for our user accounts and `"tagged-devices"`. User + model access for `"tagged-devices"`. Should work, right? ## The identity gap Here's what we missed. When Aperture does a WhoIs lookup on a connection from a **tagged device**, it sees something like: ``` Machine: Name: yoga.example.ts.net Tags: tag:dev, tag:dollhouse, tag:qa, ... (no User field) ``` There's no `User.LoginName`. Tagged devices aren't owned by a user — they're owned by the tailnet itself. The string `"tagged-devices"` that shows up in `tailscale status` is a **display label**, not an identity field that Aperture matches on. Similarly, `"tag:dev"` as a string in `temp_grants.src` doesn't match anything. Aperture's identity matching only recognizes: | Pattern | Matches | |---------|---------| | `"jess@example.com"` | User-owned devices with that login | | `"*"` | Everything | | `"tagged-devices"` | Nothing (it's not a real identity) | | `"tag:dev"` | Nothing (Aperture doesn't check tags) | So our entire fleet of tagged K8s workloads — every agent, every operator proxy — was invisible to Aperture's role system. ## The chicken and the egg This created a fun bootstrapping problem: 1. We need to update Aperture's config to use `"*"` (wildcard) instead of `"tagged-devices"` 2. Aperture's config API (`PUT /api/config`) requires an admin role 3. We don't have an admin role because the config is wrong 4. The web UI (`http://ai/ui`) also requires a role We couldn't even fix the config because the broken config prevented us from accessing the API. Every active device on our tailnet was tagged. The only user-owned devices (phones, old laptops) were offline. We were locked out of our own AI gateway. The solution turned out to be surprisingly elegant. But that's [Part 2](/blog/bootstrapping-aperture-config-with-tsnet). --- ## Key takeaways - Aperture's auth is **not** part of the tailnet grants system — it's a separate internal config - `"tagged-devices"` is a display label, not a matchable identity - Tag strings like `"tag:dev"` don't work in Aperture's `temp_grants.src` - Only user emails and `"*"` wildcard are recognized - Network ACLs and Aperture roles are independent — passing one doesn't mean passing the other - If all your devices are tagged, you need a user-owned device to bootstrap Aperture *[Continue to Part 2: Bootstrapping Aperture config with tsnet](/blog/bootstrapping-aperture-config-with-tsnet)*

Bootstrapping Aperture Config with tsnet

Thu, 26 Feb 2026 00:00:00 GMT

*A throwaway Go program, an ephemeral node, and a five-minute escape hatch.* --- ## Where we left off [Part 1](/blog/aperture-and-the-tagged-device-identity-gap) ended with us locked out of Aperture. Every device on our tailnet was tagged, Aperture's role system doesn't recognize tagged identities, and we couldn't access the config API to fix it. We needed a way to talk to Aperture as a **user-owned** device — one that presents a real email address in WhoIs. ## What didn't work ### Attempt 1: curl from the host Our workstation (Yoga) is on the tailnet with `tag:dev`. Simple curl: ```bash $ curl http://ai/api/config {"error":"access denied: no role granted"} ``` Expected failure. Aperture sees the connection, does WhoIs, gets a tagged identity, finds no matching role. ### Attempt 2: Podman container with Tailscale We spun up a Tailscale container with an ephemeral auth key that had **no tags** — so it would register as user-owned: ```bash $ podman run --rm -d \ --cap-add=NET_ADMIN \ -e TS_AUTHKEY="$EPHEMERAL_KEY" \ tailscale/tailscale:latest \ tailscaled --tun=userspace-networking ``` The node registered correctly. `tailscale whois` showed it as `jess@sulliwood.org`. But: ```bash $ podman exec aperture-bootstrap \ sh -c 'ALL_PROXY=socks5://localhost:1055 wget -qO- http://ai/api/config' # HTTP/1.1 403 Forbidden ``` Still 403. The SOCKS5 proxy that tailscaled provides in userspace-networking mode doesn't preserve the WhoIs identity correctly. The TCP connection that reaches Aperture doesn't carry the right node credentials through the proxy layer. ### Attempt 3: tsnet This is the one that worked. ## Why tsnet is different [tsnet](https://pkg.go.dev/tailscale.com/tsnet) embeds a Tailscale node directly in your Go program. Instead of running `tailscaled` as a separate daemon, your process *is* the node. When you call `srv.HTTPClient()`, you get an `*http.Client` that makes connections **through the embedded Tailscale stack**. The key difference: these connections carry full node identity. Aperture's WhoIs sees exactly who's connecting — no proxy stripping, no SOCKS indirection. ```mermaid graph LR subgraph "Option A: SOCKS proxy (broken)" App1["wget/curl"] --> SOCKS["tailscaled
SOCKS5"] --> AI1["Aperture"] AI1 -->|"WhoIs: ???"| X["403"] end subgraph "Option B: tsnet (works)" App2["Go program
+ tsnet"] -->|"Direct Tailscale conn"| AI2["Aperture"] AI2 -->|"WhoIs: jess@..."| Y["200"] end style X fill:#f66,stroke:#900,color:white style Y fill:#4a9,stroke:#2a7,color:white ``` ## The bootstrap tool The whole program is about 80 lines of interesting code. Here's the essential structure: ```go srv := &tsnet.Server{ Hostname: "aperture-bootstrap", Ephemeral: true, AuthKey: os.Getenv("TS_AUTHKEY"), Dir: tmpDir, } defer srv.Close() // This is the key line. srv.Up() joins the tailnet. // srv.HTTPClient() returns a client that routes through // the embedded Tailscale stack with full WhoIs identity. srv.Up(ctx) client := srv.HTTPClient() // Now this works — Aperture sees our real user identity. resp, _ := client.Get("http://ai/api/config") ``` Read the config, modify `temp_grants` to use `"*"` wildcard, PUT it back with the OCC hash. The ephemeral node cleans itself up when the program exits. ## Creating the auth key The auth key is important. It must have **no tags** so the node registers as user-owned: ```bash curl -u "$TS_KEY:" \ -X POST "https://api.tailscale.com/api/v2/tailnet/$TAILNET/keys" \ -d '{ "capabilities": { "devices": { "create": { "reusable": false, "ephemeral": true, "preauthorized": true, "tags": [] } } }, "expirySeconds": 300 }' ``` The empty `tags: []` is what makes it user-owned. The key expires in 5 minutes. The node is ephemeral — it disappears when the program stops. ## Making it repeatable with Dhall Hard-coding JSON configs is fine for emergencies. For ongoing management, we use [Dhall](https://dhall-lang.org) to define the config with types: ```haskell -- config/types.dhall let Grant = < Role : { role : Text } | Providers : { providers : List ProviderAccess } | HookGrant : { hook : Hook } > let TempGrant = { src : List Text, grants : List Grant } ``` The default config template: ```haskell -- config/default.dhall let adminGrant = { src = [ "*" ], grants = [ Grant.Role { role = "admin" } ] } let userGrant = { src = [ "*" ], grants = [ Grant.Role { role = "user" }, ... ] } ``` `just render` compiles this to JSON. If we typo a field name, Dhall catches it at compile time instead of Aperture returning a cryptic error. ## The full workflow ```bash # One-time setup nix develop # Go + Dhall + just + jq export TS_KEY # your Tailscale management key export TAILNET="your-tailnet.ts.net" # Bootstrap just bootstrap ``` Under the hood: ```mermaid sequenceDiagram participant J as justfile participant TS as Tailscale API participant Go as tsnet node participant AI as Aperture J->>TS: POST /keys (ephemeral, no tags) TS-->>J: auth key J->>Go: TS_AUTHKEY=key go run . Go->>Go: tsnet.Server.Up() Go->>AI: GET /api/config AI-->>Go: {config, hash: "abc..."} J->>J: dhall-to-json config/default.dhall Go->>AI: PUT /api/config {config, hash} AI-->>Go: {success, hash: "def..."} Note over Go: Ephemeral node removed on exit ``` ## After bootstrapping With `"*"` wildcard grants and the network ACL restricting which devices can reach Aperture, the system is secure and functional: ```mermaid graph TB subgraph "Network ACL (Dhall)" direction TB ACL["aperture.dhall
tag:dev, tag:k8s, tag:k8s-operator → ai:*"] end subgraph "Aperture Config" direction TB Grants["temp_grants
src: * → admin, user, models"] end subgraph "Security Model" ACL -->|"Layer 1: who can connect"| AI["Aperture"] Grants -->|"Layer 2: what they can do"| AI end style ACL fill:#36a,stroke:#248,color:white style Grants fill:#4a9,stroke:#2a7,color:white ``` The wildcard in `temp_grants` isn't as scary as it looks. Aperture only sees connections that pass the tailnet ACL. If a device isn't in the `src` list of your `aperture.dhall` fragment, it can't reach Aperture at all. The two layers complement each other: - **Tailnet ACL**: which devices can connect (narrow, tag-based) - **Aperture grants**: what connected devices can do (broad, wildcard) ## What we learned 1. **Aperture and tailnet grants are separate systems.** Don't expect `tailscale.com/cap/aperture` to exist. Check the [app capabilities docs](https://tailscale.com/docs/features/access-control/grants/grants-app-capabilities) for what's actually supported. 2. **`"tagged-devices"` is not an identity.** It's a display name in `tailscale status`. Aperture's WhoIs doesn't match on it. 3. **Userspace networking breaks WhoIs.** The SOCKS5 proxy in containerized Tailscale doesn't preserve node identity for the receiving end's WhoIs lookup. Use tsnet instead. 4. **Ephemeral tsnet nodes are a great escape hatch.** Five-minute auth keys, auto-cleanup, no permanent infrastructure. Perfect for one-off admin tasks on WhoIs-authenticated services. 5. **Type your configs.** We caught two JSON typos when we moved to Dhall. The Dhall type checker is faster feedback than "push config, get 500, guess which field is wrong." ## Try it The [aperture-bootstrap](https://github.com/Jesssullivan/aperture-bootstrap) repo has everything: the Go tool, Dhall config types, a Nix flake for reproducible builds, and a justfile that ties it all together. ```bash git clone https://github.com/Jesssullivan/aperture-bootstrap cd aperture-bootstrap nix develop just bootstrap ``` --- *This is Part 2 of a two-part series. [Part 1](/blog/aperture-and-the-tagged-device-identity-gap) covers the identity gap problem. This post covers the tsnet solution.*

Ligature Test Fixture

Tue, 31 Dec 2024 00:00:00 GMT

This post exists to verify that Fira Code ligatures render in code blocks. ```typescript // Arrow functions with => const add = (a: number, b: number) => a + b; // Strict equality if (x !== y && a === b) { console.log("ligatures active"); } // Comparison operators const check = (a: number) => a >= 10 && a <= 100; // Not equal if (a != b) return false; // Type annotations const map: Map = new Map(); ``` Inline code with ligatures: `const fn = () => x !== y`

What have I been up to these last few months?

Wed, 22 May 2024 00:00:00 GMT

![Bates Sports - YouTube](https://yt3.googleusercontent.com/2pPw4acB2M-On6TW6Bjse6W0XXMJC_GsbJ-mEAzC22pMPVX26R_4pPp5j-j0Y7iaqzCJrjVvzg=s900-c-k-c0x00ffffff-no-rj)**I am off to do cool stuff at Bates College! 🚀** I am pleased to shared I have accepted position as a Systems Analyst at Bates. That’s right- I’ve moved from Ithaca to the middle of Maine, by way of Northampton, MA! * Picked up a passion for **Rock Climbing**! * I’ve learn how to make **bagels** and generally just had a terrific time playing bakery at [Tandem Bagel Co](https://www.tandembagelco.com/)! * Developed a slew of client projects in **SvelteKit**! * I continue to procrastinate rebuilding this blog of mine, which to be honest has contributed to the lack of recent activity. “ _Oh, I’ll write that post once I finish rebuilding the site_ ” I tell myself. Regardless, yes- at some point or another this site will migrate from wordpress to a sleeker SvelteKit [MDsveX](https://github.com/pngwn/MDsveX) blog. _Eventually_ 😸 * Playing lots of guitar! * Sporadically designing and 3d printing trivial and overly complicated things (7 versions of shower drain, 3 versions of dishrack clamp, bread shuriken, the usual tomfoolery) * **Reading** : Mostly been readying about economic systems and fascism: * * [The Rise and Fall of the 3rd Reich](https://en.wikipedia.org/wiki/The_Rise_and_Fall_of_the_Third_Reich) * [Consequences of Capitalism](https://www.goodreads.com/book/show/54546067-consequences-of-capitalism) * [the Dawn of Everything](https://en.wikipedia.org/wiki/The_Dawn_of_Everything) * [A Woman of No Importance](https://www.goodreads.com/book/show/40595446-a-woman-of-no-importance) * [Pirate Enlightenment, or the Real Libertalia ](https://en.wikipedia.org/wiki/Pirate_Enlightenment) * [Anarchy Works by Peter Gelderloos](https://theanarchistlibrary.org/library/peter-gelderloos-anarchy-works) * [Ur Fascism by ](https://theanarchistlibrary.org/library/umberto-eco-ur-fascism)[Umberto Eco](https://theanarchistlibrary.org/category/author/umberto-eco)

I wrote a mutual aid mental health service

Thu, 22 Feb 2024 00:00:00 GMT

Check out this project here: https://tetrahedron.gay/ Tetrahedron.gay is a mutual aid mental health group informally continuing the structured partial hospitalization Triangle Program. I wrote this intake / project page in SvelteKit; feel welcome to take a look through the [code here on github](https://github.com/Jesssullivan/tetrahedron/tree/main/app) if you are curious- PRs welcome. There are a few goals of this experimental humanitarian project: – recreate the structure group therapy offers; to do this, we host a daily wakeup call at ~8:30 every day via Discord. Any member with the ‘wakeup-call’ discord roll is empowered to initiate, join and call (and- if needed- delegate further wakeup calls, via an means necessary- phone call, discord, email, DMs, doordash coffee delivery to your peer, knock on their door, call their neighbors etc) . I intend to write a utility to automate wakeup phone calls and allow users to securely wake each other up without having their personal info (phone #, address etc). – recreate and extend the social and interpersonal aspects of a group therapy program through in-person activities as well as virtual, peer led group sessions – experiment with horizontal organization structures with minimal hierarchical organization in a traditionally stratified space; enable the trusting reliance of individuals to guide group moderation, while protecting personal info. To do this, function group size is to be limited at 13 individuals (wakeup call groups, digital and in-person group sessions). Additional function groups are to be spawned and destroyed as needed; this group size limit is to be reflected across Tetrahedron.gay projects.

AccuWix Report CLI

Tue, 02 Jan 2024 00:00:00 GMT

I wrote this lil command line application while riding on a bus to and from New York City- find the project repository here at [jesssullivan/AccuWixReport](https://github.com/Jesssullivan/AccuWixReport/tree/main). 😸 This utility generates a variety of concise, merged monthly financial superlative reports in raw markdown, drawing from transaction CSVs exported by Acuity and Wix. * **“Paid” transaction reports** are marked as “paid” by the respective platforms (automatically via credit card or manually upon receiving otherwise untracked currency) * **“Unpaid” transaction reports** were _not explicitly_ marked as “paid”; while the most likely scenario is that this transaction was **completed successfully** offline (via cash / venmo), an “Unpaid” status may imply the client did not settle up or the transaction was recorded erroneously due to cancellation, revision or other extenuating circumstances. ### Intended usage: _Setup:_ ```bash python3.12 -m venv accuwix_venv source accuwix_venv/bin/activate pip install -r requirements.txt ``` _Generate and print report for January:_ ``` python3 src/cmdline.py -m Jan ``` _Generate and save all reports to markdown file:_ ``` python3 src/cmdline.py -all > AccuWixReport.md ``` _Generate a sexy PDF report from markdown file:_ ```bash pandoc AccuWixReport.md -o output.pdf -V geometry:margin=1in ``` * * * _Display help screen:_ ``` usage: -h : print this message again -all : generate all reports -m : `month` (optional); specify a month to generate a superlative report` use any of the following month qualifiers: Jan, Feb, Mar, Apr, May, Jun, Jul, Aug, Sep, Oct, Nov, Dec ```

Lets write a simple, efficient and fast flask-based image server in an afternoon

Mon, 18 Dec 2023 00:00:00 GMT

….that uses lanczos resampling to serve optimized cached photos. Find this project on GitHub here: [jesssullivan/FastPhotoAPI](https://github.com/Jesssullivan/FastPhotoAPI) Interact with this API graphically here (hosted on [koyeb](https://www.koyeb.com/docs) via Docker): https://api.birdphoto.website/ **Structure:** This application adopts the factory pattern; `flask run` instantiates the built-in development server by executing `create_app()` at the root of the `app/` package, while `python application.py` creates a new production application, served by waitress. ```bash git clone https://github.com/Jesssullivan/FastPhotoAPI && cd FastPhotoAPI python3.12 -m venv fast_photo_venv source fast_photo_venv/bin/activate pip install -r requirements.txt ``` **Structure:** ``` ├── app ├── __init__.py # create and serve development application └── main ├── config │ │ └── config.cfg # set directories, max image dimensions, etc │ ├── fullsize │ │ └── routes.py # Blueprint routing for serving verbatim image files │ ├── __init__.py # `create_app()` entrypoint │ ├── resampled │ │ ├── model.py # Image resampling methods │ │ └── routes.py # Blueprint routing for `/image/` │ └── static │ └── routes.py # Blueprint routing for `/static/` ├── application.py # create and serve production application w/ waitress ├── cache # resampled images are dynamically generated adn stored here ├── Dockerfile # currently deployed at Koyeb ├── pictures # full res pictures go here ├── README.md # you are here ├── static │ └── style.css # index styling └── templates ├── index.html └── upload.html ``` **Build** : _Locally**:**_ ``` _# dev WSGI:_ flask run # 0.0.0.0:5000 # _waitress WSGI:_ flask run # 0.0.0.0:8000 _Production via Docker:_ ## build production docker image: # docker build -t . ## serve production docker image locally: docker run -d -p 8000:8000 :latest ## stop local image: # docker ps # docker stop ## push image to a container registery: # docker push ``` **Basic Usage: ** – Fetch a resampled & cached image ``/image/`` – Fetch the original, unmodified image ``/full/``

Turkey Probe

Mon, 27 Nov 2023 00:00:00 GMT

![Turkey Probe](/images/posts/esp_Turkey.jpeg) _A spur-of-the moment turkey thermometer with dubious accuracy, questionable code and unhelpful complexity made of random gahbage._ Uses Steinhart-Hart equation to estimate the temperature from the 8266's analog pin (this board has a single channel 10 bit ADC) produced by a 100k 104GT-2/104NT-4 thermistor using 10k pullup resistor. The ESP 8266 serves a real-time temperature readout via a websocket connection over the local network. Find this ridiculous project of mine on Github here: https://github.com/Jesssullivan/turkeyprobe ![Turkey Probe in action during thanksgiving](/images/posts/esp_Turkey.jpeg) | ![Just moments before thanksgiving](/images/posts/testESP.jpeg) ---|---

Fusion 360 for 3d Printing w/ Jess!

Wed, 01 Mar 2023 00:00:00 GMT

_Next class: April 3rd, 6pm - 8pm_ **RSVP Here!** ### Draw and print your mechanical daydreams! Watertight threads? Anneal-fit tolerances? Learn to design and print beautiful hardware and parts in this exciting workshop with Jess! **_Functions and features we will touch may include:_** * Offset, trim, point to point move * Mirror, pattern, split * BREP booleans, chamfer, fillet * Minimalist sketch management & construction planes * Press pull, sweep, extrude as boolean * Scaling, threading & design considerations for FDM 3d printing * Utilize the McMaster-Carr library & manipulate meshes as BREP * Workflow interoperability with Tinkercad, Inkscape, MeshMixer, a360 & myhub services * Versioning, Git & public collaboration Please share your enthusiasm, ask questions and join the conversation in IG’s lively Discord server.

This and That!

Mon, 23 May 2022 00:00:00 GMT

_updated on 5/23/22_ **_Letsee...._** * Built out a lil web dashboard for DLA Makerspace- find the space's calendar, general information, technical documentation, printer livestreams for students and access the discord ticketing system over here: https://dlamaker.space/dashboard/home * Superduper excited to be hopping over to the Department of Landscape Architecture to explore and expand the [DLA Makerspace at Cornell](https://cals.cornell.edu/landscape-architecture/facilities-resources/fabrication-laboratory)! * I have been busy at the [Ithaca Generator](https://ithacagenerator.org/)\- building up our new website and writing new member mangegment & accounting software. I've taken over as IG's 3D Printer Area Captain too! 😸 *Some images from the original WordPress post are no longer available.* _See a need, fill a need!_ | _An exciting new venture!_ | _Chemicals and experiments oh my!_ ##### [Magic Fusion 360 for 3D Printing Class](https://ithacagenerator.org/home/fusion-360-class) * _Draw and print your mechanical daydreams! Watertight threads? Anneal-fit tolerances? Learn to design and print beautiful hardware and parts in this exciting workshop with Jess!_ ##### [Weekly 3D Printer Party w/ Jess & whoppingpochard!](https://ithacagenerator.org/home/weekly-3d-printer-party-w-jess) Tuesdays 5 - 8pm * _All are welcome! Come build Vorons, Doomcubes and other advanced 3D printers with us and explore the cutting edge in open source, community-fueled hardware design!_ * [Join us on Discord!](https://discord.gg/HADJeKhTMC) * [Support whoppingpochard’s amazing work @ Lightweight Labware!](https://www.lightweightlabware.com/) * [Support our printer friends @ DFH!](https://deepfriedhero.in/) * [Join the Voron Discord!](https://discord.com/invite/voron) * [Join the Doomcube community on Discord!](https://discord.gg/K26gvetF2A) ##### [Noise Night](https://ithacagenerator.org/home/noise-night) * _Sunday nights 6pm to 8pm beginning March 20th_ _~~~ Come enjoy a low-key evening of casual jamming, music and noisy projects with Jess!_ **_Some lil random printed projects:_** * [Headset Right Cup Brace - PC3XX](https://www.thingiverse.com/thing:5341547) * [Printable Huion tablet nib](https://www.thingiverse.com/thing:5334247) * [Ongoing C2H4nTiO2 container experiments](https://github.com/DaytimeDoting/XoXDust-Containers) * [Ongoing experiments with soy wax pitchers](https://www.thingiverse.com/thing:5334265)

Running Cornell's DLA Makerspace

Sun, 15 May 2022 00:00:00 GMT

From 2021 through 2022 I managed the fabrication lab for Cornell's Department of Landscape Architecture in CALS. The lab had a pair of Ultimaker S3s (Ada and Greta), an S5 (Fender), a ShopBot Buddy 32 CNC router, and two Epilog Edge 50W CO2 laser cutters (Freddie and Vaux). Yes, I named all the machines. ![Dual-material PVA and PLA print](/images/posts/PVA_PLA_Xinke.webp) ![ShopBot CNC router work](/images/posts/shopbot_tinkering.webp) ![CNC tooling and blade holder](/images/posts/blade_holder.webp) ![Multidimensional laser cut work](/images/posts/multidimensional_lasercuts.webp) ![Planar fabrication demos](/images/posts/planar_demos.webp) I developed curricula covering rapid fabrication from digital design through machine setup, material selection, and iteration. I also built [dlamaker.space](https://dlamaker.space) as the lab's home base and set up a GitHub org, Discord, and custom Cura profiles for our machines. During the same period I was serving as 3D Printing Captain at Ithaca Generator. ![RailCore printer](/images/posts/railcore.webp) ![Small-scale 3D printing projects](/images/posts/tiny_printer_projects.webp)

Makerspace financial reporting w/ ipython

Sun, 06 Feb 2022 00:00:00 GMT

[_Visit this project on github here_](https://github.com/ithacagenerator/MembershipWorks-Migration) * Merge PayPal & Membershipworks members in a sorta intelligent way to create kinda accurate financial reports * Convert a PayPal transaction export into an upsert-able csv to import into membershipworks * Keep tabs on PayPal memberships as they become deprecated (note, you'll need [nbconvert, pandoc, TeX to write to pdf](https://nbconvert.readthedocs.io/en/latest/)) #### _...If all goes well, the output will look something like:_ ``` loaded **** paypal records loaded ** existing membershipworks records converted Date column to datetime objects kept *** records processed between 01/01/22 and 22/11/21; discarded **** records discarded **'s Donation Payment record for **, continuing... discarded **'s PreApproved Payment Bill User Payment record for **, continuing... ** ** already in member list! continuing... discarded ** **'s Payment Refund record for -**.00, continuing... discarded ** **'s Donation Payment record for **.00, continuing... exported: ** Members! - ** Standard Members - ** Offline Standard Members - ** Extra Members - ** Offline Extra Members ...to a membershipworks-readable format at ./csv/membershipworks_import.csv ```

Datasets, Plots, Graphs, Charts

Sat, 01 Jan 2022 00:00:00 GMT

_Pinned from November 11, 2021_ ## Merlin Sound ID has been released! * [...Check out the awesome new Machine Learning Blog at Macaulay Library!](https://www.macaulaylibrary.org/machine-learning/) * [...Download Merlin!](https://merlin.allaboutbirds.org/download/) ##### ...A small web api by 6 letter species code describing how a species is represented in Merlin Sound ID: * [**ai.columbari.us/reports/norcar**](https://ai.columbari.us/reports/norcar) * [**ai.columbari.us/reports/daejun**](https://ai.columbari.us/reports/daejun) ##### **...Boxes, Boxes, Boxes!** *Some images from the original WordPress post are no longer available.*

This and that, bits and bobs…

Tue, 30 Nov 2021 00:00:00 GMT

##### Afternoon scribbles & prints: * [Nezuko as a coaster = NezuCoaster](https://www.thingiverse.com/thing:5141629) * [Big hooks for warm winter coats](https://www.thingiverse.com/thing:5141632) * [Drain plug to end all plugs](https://www.thingiverse.com/thing:5150859) * [Wyze Gal, e.g. _don't_ perv my printer](https://www.thingiverse.com/thing:5150863) ##### ...Continuing to fix Jess and Suisei's New York nest with PLA: *Some images from the original WordPress post are no longer available.* ##### _…Experimental microphone hardware for Merlin Sound ID:_ YMMV, but YOLO: | 3d printing @ IG repo and docs: ---|--- | [![](https://avatars.githubusercontent.com/u/3699732?s=200&v=4)](https://ithacagenerator.github.io/IG-3DP-Profiles/) ##### ...A variety of improvements and updates to the chrome remote desktop [automatic patching repo for Ubuntu Budgie](https://github.com/Jesssullivan/chrome-remote-desktop-budgie) * [Release v.4](https://github.com/Jesssullivan/chrome-remote-desktop-budgie/releases/tag/v.4) includes a variety of improvements and bug fixes- now supports fully remote setup with host authorization over ssh # get this tool: ```bash git clone https://github.com/Jesssullivan/chrome-remote-desktop-budgie/ && cd chrome-remote-desktop-budgie # launch, permiss: python3 chrome-remote-desktop sudo chmod +s /opt/google/chrome-remote-desktop/user-session # add the service file to start the daemon on boot: sudo chmod +x addsystemd.sh && sudo ./addsystemd.sh # or, if you'd prefer to launch the daemon manually: sudo --user=$USER /usr/bin/python3 /usr/local/bin/chrome-remote-desktop ``` * * *

Morning Metal 4.22.21 “Sleepish”

Fri, 23 Apr 2021 00:00:00 GMT

...The usual racket... #### Audio: [Jess S.](https://soundcloud.com/jesssullivan "Jess S.") · [Morning Metal 4.22.21](https://soundcloud.com/jesssullivan/morning-metal-42221 "Morning Metal 4.22.21")

Bits & Bobs, Mushstools & Toadrooms

Thu, 18 Mar 2021 00:00:00 GMT

...Despite being a chilly & wintery March up here in the White Mountains, there is no shortage of fun birds and exciting projects! * So many [Redpolls](https://www.allaboutbirds.org/guide/Common_Redpoll/id) are keeping the [Juncos](https://www.allaboutbirds.org/guide/Dark-eyed_Junco/id) company this year! Two pairs of Hooded Mergansers moved in just next door last week! * **_Use_** [**_AWS S3 as a Joplin sync target!_**](https://github.com/laurent22/joplin/pull/4675/files#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5R266) * Local [Sharp-shinned hawks](https://www.allaboutbirds.org/guide/Sharp-shinned_Hawk/id) & [Fluffy Red Foxes](https://en.wikipedia.org/wiki/Red_fox) have been busy careening around town gobbling up prey left and right- they seem to know Spring is right around the corner! #### Merlin AI pipeline for Mushroom identification! _It's happening, and its going to be awesome_ [**_visit this project over here on GitHub_**](https://github.com/Jesssullivan/image-identifer) ```bash git clone https://github.com/Jesssullivan/image-identifer/ && cd image-identifer ``` **_Overview:_** * **Setup** * **Artifacts** * **Preprocess** * **Artifacts** * **Train** * **Structures** * **Notes** #### ### _Setup:_ ``` # venv: python3 -m venv mushroomobserver_venv source mushroomobserver_venv/bin/activate pip3 install -r requirements.txt ``` #### **_Artifacts:_** | [_train.tgz_](https://mo.columbari.us/static/train.tgz) | [_test.tgz_](https://mo.columbari.us/static/test.tgz) ---|---|--- [_images.tgz_](https://mo.columbari.us/static/images.tgz) | [_images.json_](https://mo.columbari.us/static/images.json) | [_gbif.zip_](https://mo.columbari.us/static/gbif.zip) * * * #### ### _Preprocess:_ ```bash python3 preprocess ``` * Fetches & saves off gbif archive to `./static/` * Checks the archive, tries loading it into memory etc * Fetches Leaflet Annotator binary & licenses from [JessSullivan/MerlinAI-Interpreters](https://github.com/Jesssullivan/MerlinAI-Interpreters); Need to commit annotator _(as of 03/16/21)_ , still fussing with a version for Mushroom Observer * Generates an `images.json` file from the 500 assets selected by Joe & Nathan * Downloads, organizes the 500 selected assets from _images.mushroomoberver.org_ at `./static/images//.jpg` * writes out images archive * More or less randomly divvies up testing & training image sets * writes out example testing/training archives; (while training it'll probably be easier to resample directly from images.tgz from keras) ### _Train:_ ```bash python3 train ``` * Fetches, divvies & shuffles train / validation sets from within Keras using archive available at [_mo.columbari.us/static/images.tgz_](https://mo.columbari.us/static/images.tgz) * More or less running Google's demo transfer learning training script in `train/training_v1.py` as of _03/17/21_ , still need to bring in training operations and whatnot from merlin_ai/ repo --> experiment with Danish Mycology Society's ImageNet v4 notes **_Google Colab:_** * [@gvanhorn38](https://github.com/gvanhorn38/) pointed out Google Colabs's neat Juptyer notebook service will train models for free if things are small enough- I have no idea what the limits are- fiddle with their [**_intro to image classification on Google Colab here_**](https://colab.research.google.com/github/tensorflow/docs/blob/master/site/en/tutorials/images/classification.ipynb), its super cool! **_Jupyter:_** * One may also open and run notebooks locally like this: * [rendered pdf version available over here](https://github.com/Jesssullivan/image-identifer/blob/main/train/notebook/training_v1.pdf) * rename ipython notebook: cp train/notebook/training_v1.ipynb.bak train/notebook/training_v1.ipynb ``` * launch jupyter: jupyter notebook * or without authentication: jupyter notebook --ip='*' --NotebookApp.token='' --NotebookApp.password '' ``` * * - #### ### _Structures:_ * _Leaflet Annotator`images.json` Structure:_ * **id** : _taxonID_ The MO taxon id * **category_id** : The binomen defined in the `./static/sample_select_assets.csv`; for directories and URIs this is converted to snake case. * **url** : Temporary elastic ip address this asset will be available from, just to reduce any excessive / redundant traffic to _images.mushroomobserver.org_ * **src** : _imageURL_ The asset's source URL form Mushroom Observer [{ ``` "id": "12326", "category_id": "Peltula euploca", "url": "https://mo.columbari.us/static/images/peltula_euploca/290214.jpg" "src": "https://images.mushroomobserver.org/640/290214.jpg" }] ``` * _Selected asset directory structure:_ ├── static ``` ├── gbif.zip ├── images | ... │ └── peltula_euploca │ ├── 290214.jpg │ ... │ └── 522128.jpg │ ... ├── images.json ├── images.tgz ├── js │ ├── leaflet.annotation.js │ └── leaflet.annotation.js.LICENSE.txt └── sample_select_assets.csv ... ``` * * - #### #### _Notes:_ _Fiddling with the archive:_ * `MODwca.gbif[1].id`: Integer: This is the Mushroom Observer taxon id, e.g. * `https://mushroomobserver.org/13` * `https://images.mushroomobserver.org/640/13.jpg` * `MODwca.gbif[1].data:`: Dictionary: DWCA row data, e.g. * `MODwca.gbif[1].data['http://rs.gbif.org/terms/1.0/gbifID']` = `13` * `MODwca.gbif[1].data['http://rs.tdwg.org/dwc/terms/recordedBy']` = `Nathan Wilson`

Merlin AI Demos!

Wed, 17 Feb 2021 00:00:00 GMT

#### [_...Notes_](https://github.com/Jesssullivan/tmpUI#readme) #### [_....Even more demos @ ai.columbari.us_](https://ai.columbari.us/classify/server) --- | --- | --- ---|---|--- Vue UI: handling multiple clients- demo of MLD-1514 | Leaflet UI: handling multiple clients- demo of MLD-1514 | user trust schema experiments etc | | Jest x Puppeteer Demos | Web Interpreter: upload / record, (re)crop & classify | Leaflet Annotator bits & bobs | *The photograph that was here has since disappeared from the web.* | [_miscellaneous dregs, bits, bobs, demos in this playlist on youtube_](https://www.youtube.com/playlist?list=PL6y8N_vP4OopcvDfwcEyvx1Nz4MVSZK7R) * [handling multiple annotator clients- Vue UI, 2/17/21](https://www.youtube.com/watch?v=dxnAx0CXUqU) * [handling multiple annotator clients- Leaflet UI, 2/7/21](https://youtu.be/8aoNftMP4w8) * [Jest x Puppeteer, testing annotator UI](https://youtu.be/E1WcZ2yOC10) * [Bits from 1.21.21](https://youtu.be/xdog2U2-c04) * [preformance updates to core leaflet annotator, 1/17/21](https://youtu.be/uxrnf8oPsF8) * [mongodb, trust schema, drag & drop 12/7/20](https://youtu.be/5rsRts7-_Xs) * [web tools, bbox classify, ridiculous CLI 12/1/20](https://www.youtube.com/watch?v=TKKjo1ypSPY&feature=youtu.be) * [audio & photo annotators 11/23/20](https://youtu.be/rKzl5aQmJ-Y) * [what this swiftui thing 10/26/20](https://youtu.be/3btWzARKYKM)

Chindōgu ASCII art

Sat, 23 Jan 2021 00:00:00 GMT

_A ridiculous_ [_Chindōgu_](https://en.wikipedia.org/wiki/Chind%C5%8Dgu) _utility prompt & CLI for_ [_fetching private releases & files from GitHub & BitBucket_](https://github.com/Jesssullivan/LeafletSync) ```bash curl https://raw.githubusercontent.com/Jesssullivan/LeafletSync/main/LeafletSync --output LeafletSync && chmod +x LeafletSync && ./LeafletSync ``` * Fetch, unpack, extract specific releases & files or a complete master branch from a private GitHub repo with an api access token * Fetch and extract specific files or complete branches from a private BitBucket account with user's git authentication * Prefill default prompt values with a variety of console flags * Save & load default prompt values with a file of environment variables, see templates [`FetchReleasegSampleEnv_GitHub`](https://github.com/Jesssullivan/LeafletSync/blob/main/FetchReleaseSampleEnv_GitHub), [`FetchFilegSampleEnv_BitBucket`](https://github.com/Jesssullivan/LeafletSync/blob/main/FetchFileSampleEnv_BitBucket), [`FetchEverythingSampleEnv_BitBucket`](https://github.com/Jesssullivan/LeafletSync/blob/main/FetchEverythingSampleEnv_BitBucket), [`FetchEverythingSampleEnv_GitHub`](https://github.com/Jesssullivan/LeafletSync/blob/main/FetchEverythingSampleEnv_GitHub); pass as an argument with the ` -e ` flag, (`./LeafletSync -e YourEnvFile`) or provide one on launch. ./LeafletSync ``` LeafletSync: Do you want to load values from a file? If so, enter one now...:[Nope!]: _ __ _ _ _____ | | / _| | | | / ___| | | ___ __ _| |_| | ___| |_ \ `--. _ _ _ __ ___ | | / _ \/ _` | _| |/ _ \ __| `--. \ | | | '_ \ / __| | |___| __/ (_| | | | | __/ |_ /\__/ / |_| | | | | (__ \_____/\___|\__,_|_| |_|\___|\__| \____/ \__, |_| |_|\___| \ _____________________/ | \ Fetch from Github: / α wιρ Σ ♥ |_@__Jess /───────────────────/ \ Your API Token | -t | --token | Required | = | Your Handle | -u | --user | Required | = | Source Repo | -r | --repo \ Required \ = | Repository Owner | -a | --author \ Required \ = | Release Version | -v | --version | Optional | = Fetch Everything / Output Directory | -o | --out / Optional / = ./dist/ /─────────────────────────/ \ Fetch from BitBucket: / \──────────────────────/ \ Your Handle / -bu / --b-user / ~Required | = \ Your Passhrase / -bp / --b-pass / ~Required / = \ Source Branch / -bb / --b-branch / ~Optional / = master \ Source File / -bf / --b-file / ~Optional / = \────────────/ Your Handle []: Source Repo []: Repo Owner []: Host: GitHub | BitBucket [GitHub]: Your Token []: Release to fetch: []: Output to fetch (e.g. /dist/*): []: ... ```

naive distance measurements with opencv

Sat, 02 Jan 2021 00:00:00 GMT

* _visit these notes_ [_over here on github!_](https://github.com/Jesssullivan/misc-roi-distance-notes/) Knowing both the Field of View (FoV) of a camera's lens and the dimensions of the object we'd like to measure (Region of Interest, ROI) seems like more than enough to get a distance. Note, [opencv has an extensive suite of actual calibration tools and utilities here.](https://docs.opencv.org/4.5.0/d9/db7/tutorial_py_table_of_contents_calib3d.html) ...But without calibration or much forethought, could rough measurements of known objects even be usable? Some notes from a math challenged individual: ```bash # clone: git clone https://github.com/Jesssullivan/misc-roi-distance-notes && cd misc-roi-distance-notes ``` Most webcams don't really provide a Field of View much greater than ~50 degrees- this is the value of a MacBook Pro's webcam for instance. Here's the plan to get a Focal Length value from Field of View: `Focal Length = (ImageDimension / 2) * tan(FieldOfView / 2)` So, thinking along the lines of similar triangles: * Camera angle `FoV / 2` forms the angle between the _hypotenuse_ side (one edge of the FoV angle) and the _adjacent_ side * Dimension `ImageDimension / 2` is the _opposite_ side of the triangle we are using to measure with. * ^ This makes up the first of two ["similar triangles"](https://byjus.com/maths/similar-triangle-construct/) * Then, we start measuring: First, calculate the opposite ROI Dimension using the arbitrary Focal Length value we calculated from the first triangle- then, plug in the Actual ROI Dimensions. * Now the adjacent side of this ROI triangle should hopefully be length, in the the units of ROI's Actual Dimension. source a fresh venv to fiddle from: ```bash # venv: python3 -m venv distance_venv source distance_venv/bin/activate # depends are imutils & opencv-contrib-python: pip3 install -r requirements.txt ``` The opencv people provide a bunch of [prebuilt Haar cascade models](https://github.com/opencv/opencv/tree/master/data/haarcascades), so let's just snag one of them to experiment. Here's one to detect human faces, we've all got one of those: ```bash mkdir haar wget https://raw.githubusercontent.com/opencv/opencv/master/data/haarcascades/haarcascade_frontalface_alt2.xml -O ./haar/haarcascade_frontalface_alt2.xml ``` Of course, an actual thing with fixed dimensions would be better, like a stop sign! Let's try to calculate the _distance_ as the _difference_ between an actual dimension of the object with a detected dimension- here's the plan: `Distance = ActualDimension * (FocalLength / ROIDimension)` #### _YMMV, but YOLO:_ ```python # `python3 measure.py` import math from cv2 import cv2 DFOV_DEGREES = 50 # such as average laptop webcam horizontal field of view KNOWN_ROI_MM = 240 # say, height of a human head # image source: cap = cv2.VideoCapture(0) # detector: cascade = cv2.CascadeClassifier('./haar/haarcascade_frontalface_alt2.xml') while True: ``` # Capture & resize a single image: _, image = cap.read() image = cv2.resize(image, (0, 0), fx=.7, fy=0.7, interpolation=cv2.INTER_NEAREST) # Convert to greyscale while processing: gray_conv = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) gray = cv2.GaussianBlur(gray_conv, (7, 7), 0) # get image dimensions: gray_width = gray.shape[1] gray_height = gray.shape[0] focal_value = (gray_height / 2) / math.tan(math.radians(DFOV_DEGREES / 2)) # run detector: result = cascade.detectMultiScale(gray) for x, y, h, w in result: dist = KNOWN_ROI_MM * focal_value / h dist_in = dist / 25.4 # update display: cv2.rectangle(image, (x, y), (x + w, y + h), (255, 0, 0), 2) cv2.putText(image, 'Distance:' + str(round(dist_in)) + ' Inches', (5, 100), cv2.FONT_HERSHEY_SIMPLEX, 1, (255, 255, 255), 2) cv2.imshow('face detection', image) if cv2.waitKey(1) == ord('q'): break ``` ``` _run demo with:_ ```bash python3 measure.py ``` #### Results: ...With only Field of View and a ROI of known dimensions, the accuracy of this "similar triangles" solution thus far has only been limited by image resolution and the quality of the detector- pretty cool! *This image has been lost to time. The original was hosted on WordPress.* -Jess

some boilerplate code, ?, etc

Wed, 23 Dec 2020 00:00:00 GMT

## ?: Some "freezing-cold-New-Hamshire-Winter" morning metal: [Jess S.](https://soundcloud.com/jesssullivan "Jess S.") · [Morning Metal 12.17.20](https://soundcloud.com/jesssullivan/morning-metal-121720 "Morning Metal 12.17.20") ## ## ?: **Organization for Flask + React + Typescript + MongoDB using the nifty[Blueprints](https://flask.palletsprojects.com/en/1.1.x/blueprints/) library.** * this project on [github is over here](https://github.com/Jesssullivan/Flask-Mongo-Authenticate) __ _Setup:_ __ __ __ ```bash # clone: git clone https://github.com/Jesssullivan/Flask-Mongo-Authenticate/ && cd Flask-Mongo-Authenticate  # venv: python3 -m venv api_venv source api_venv/bin/activate pip3 install -r requirements.txt  # node: npm install  # permiss: sudo chmod +x setup run  # configure (default values are provided too): ./setup  # have at it: ./run ``` * * * ### _Structure:_ ```sql ├── api  ├── main  ├── auth  └── token authentication methods  ├── config  └── the ./setup script populates a new config.cfg file for Flask,  using the ##FIELDS## provided in config.cfg.sample  ├── tools  └── utilities for date/time, expression matching, the like  └── user  └── models.py defines the User() class  └── routes.py implements User() methods api/routes as a blueprint  (registered at /user/) ├── public  └── all hot reloading and whatnot is done from react-scripts at index.html └── src  └── insert client-side source here, hack away xD  the thinking is one deal with compiling & serving production code elsewhere ``` * * * _Notes:_ * spawned from [Luke Peters work here](https://github.com/LukePeters/flask-mongo-api-boilerplate) * Only tested on Ubuntu with GNU utilities, YMMV * On Mac, please use GNU `sed`, see `./setup` for details # MongoDB & gnu sed for Mac: brew install gnu-sed brew tap mongodb/brew brew install mongodb-community@4.4

Client-side, asynchronous HTTP methods- TypeScript

Wed, 28 Oct 2020 00:00:00 GMT

* [**_Example in action!_**](https://tmpui.herokuapp.com/crop_post) * [_Client source!_](https://github.com/Jesssullivan/tmpUI/blob/master/demos/spec_record_crop_post.ts#L40) | [_Server source!_](https://github.com/Jesssullivan/tmpUI/blob/master/app.py#L35) Despite the ubiquitousness of needing to make a POST request from a browser (or, perhaps for this very reason) there seems to be just as many [ways](https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest), [methods](https://www.w3schools.com/js/js_ajax_http_send.asp), [libraries](https://www.npmjs.com/package/axios), and [standards](https://developer.mozilla.org/en-US/docs/Web/API/Request) of implementing http functions in JavaScript as there are people doing said implementing. Between the adoption of the [fetch api](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API) in browsers and the prevalence and power of [Promises in JS](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Using_promises), asynchronous http needn't be a hassle! ```typescript /* ...happily processing some data in a browser, when suddenly... ....panik! you need to complete a portion of this processing elsewhere on some server...: */ const handleBlobPOST = (blob: Blob, destination: string) => { /* `blob` is some data you've been processing available in the DOM (such as a photo or audio) . some blob objects to upload: */ const file = blob; // file is wrapped in `formData` for POST: const formData = new FormData(); /* `name: 'file'` is the ID the server will use to find and parse the POSTs contents of fileName: 'fluffy.chonk': */ formData.append('file', file, 'fluffy.chonk'); // make the POST w/ fetch, no one should be using IE anyway xD: fetch(destination, { method: 'POST', body: formData }) // make, then handle the Promise: .then(response => { /* we can .then wait for the Promise Object `response`... */ response.json().then(data => { /* ...and .then once we have the the `response` Object, take only the important json part: */ console.log('received JSON!'); /* but wait- json is unstructured, how can we continue working with this data? zing the response json (here just as key: value pairs) into an Array: */ let ix; let results = []; for (ix in data) { results.push([ix, data[ix]]); } /* sort the Array by descending value: while the json returned is to remain unstructured, with this Array we can preform all sorts of nifty operations, like so: (key: value pairs in this case are string: number, so sort pairs by decending value) */ results = results.sort((a, b) => b[1] - a[1]); // print the sorted scores: let i; for (i in results) { console.log(i + ' ' + results[i]); } }); }) .catch(error => { console.error(error); }); }; /* Now that we've got everything sorted by Promise, we can use this as an async function: */ async function postClassifyWaveform() { // posts blob directly: return handleBlobPOST(someBlobs, '/cool_endpoint'); } ``` _Some extra notes spawned from web demos with Merlin AI:_ * [...check out more notes on this project here](https://jesssullivan.github.io/tmpUI/) * [...and the repo over here](https://github.com/Jesssullivan/tmpUI) -Jess

...In other news...

Wed, 28 Oct 2020 00:00:00 GMT

## 🎵: ## 🎵: [Jess S.](https://soundcloud.com/jesssullivan "Jess S.") · [Evening Metal 9.14.20](https://soundcloud.com/jesssullivan/evening-metal-91420 "Evening Metal 9.14.20") **_..also, check out the sporadically maintained Terrific Music list :)_**

Obliterate non-removable MDM profiles enforced by Apple's Device Enrollment Program

Sun, 06 Sep 2020 00:00:00 GMT

_Or, when life gives you apples, use Linux_ Seemingly [harder to remove](https://gist.github.com/sghiassy/a3927405cf4ffe81242f4ecb01c382ac) with every eye-glazing [gist and thread](https://gist.github.com/henrik242/65d26a7deca30bdb9828e183809690bd)... A mac plagued with an _`is_mdm_removable=false`_ [Mobile Device Management](https://support.apple.com/en-us/HT204142) profile: **the worst!** 🙂 First, [boot into recovery mode](https://support.apple.com/en-us/HT201314) by rebooting while holding down the `Command` & `R` keys. At this stage, you'll need to connect to the internet briefly to download the recovery OS. This provides a few tools including like disk utility, support, an osx reinstaller- at the top menu, you'll find an option to access a terminal. Once in there, you'll want to: Disable [SIP](https://support.apple.com/en-us/HT204899): ```bash csrutil disable ``` Then reboot: ```bash reboot now ``` While holding down `Command` \+ `Option` \+ `P` \+ `R` to start afresh with cleared [NVRAM](https://support.apple.com/en-us/HT204063#:~:text=How%20to%20reset%20NVRAM,Mac%20might%20appear%20to%20restart). Reboot once again while holding down the `Command` & `R` keys to return to the recovery OS. Reinstall whatever version of OSX it offers- instead of trying to deal with the slippery, network connected DEP plists & binaries contained within the various `LaunchAgents` and `LaunchDaemons` found in the `/System/Library` directories directly, we'll let Apple finish with the `ConfigurationProfiles` first, _then_ sneak in and remove them. While this stuff is cooking, get yourself a usb stick and a [penguin](https://en.wikipedia.org/wiki/Tux_$mascot$), such as [Budgie](https://ubuntubudgie.org/downloads/): ```bash wget -nd http://cdimage.ubuntu.com/ubuntu-budgie/releases/20.04.1/release/ubuntu-budgie-20.04.1-desktop-amd64.iso umount /dev/sdc 2>/dev/null || true sudo dd if=ubuntu-budgie-20.04.1-desktop-amd64.iso of=/dev/sdc bs=1048576 && sync ``` Boot up again, this time holding the `Option` key for the bootloader menu. Once in the live usb system, make sure you can read Apples HFS filesystem: ```bash sudo apt-get install hfsprogs ``` For me at least, I needed to run a quick `fsck` to fix up the headers before I could mount the osx filesystem living at `/dev/sda2` (`sda1` is the `efi` partition): ```bash sudo fsck.hfsplus /dev/sda2 ``` Now, lets go in there and remove those ConfigurationProfiles: ```bash mkdir badapple sudo mount -o force /dev/sda2 badapple cd badapple sudo rm -rf private/var/db/ConfigurationProfiles/* ``` 🙂

KVM does Fruit: Xcode from QEMU

Sat, 05 Sep 2020 00:00:00 GMT

### The digression starts over here: #### https://github.com/kholia/OSX-KVM *Some images from the original WordPress post are no longer available.* _Hmmmm....._ #### _?_ **_...In the mean time, here is my`./OpenCore-Boot.sh:`_** ```bash #!/usr/bin/env bash # setup tap0 if haven't already for $session: sudo ip tuntap add dev tap0 mode tap sudo ip link set tap0 up promisc on sudo ip link set dev virbr0 up sudo ip link set dev tap0 master virbr0 REPO_PATH="./" OVMF_DIR="." args=( -enable-kvm -m 24000 -cpu Penryn,kvm=on,vendor=GenuineIntel,+invtsc,vmware-cpuid-freq=on,+pcid,+ssse3,+sse4.2,+popcnt,+avx,+aes,+xsave,+xsaveopt,check -machine q35 -smp 4,cores=2,sockets=1 -device usb-ehci,id=ehci -device usb-kbd,bus=ehci.0 -device usb-mouse,bus=ehci.0 -device nec-usb-xhci,id=xhci -device isa-applesmc,osk="ourhardworkbythesewordsguardedpleasedontsteal(c)AppleComputerInc" -drive if=pflash,format=raw,readonly,file="$REPO_PATH/$OVMF_DIR/OVMF_CODE.fd" -drive if=pflash,format=raw,file="$REPO_PATH/$OVMF_DIR/OVMF_VARS-1024x768.fd" -smbios type=2 -device ich9-intel-hda -device hda-duplex -device ich9-ahci,id=sata -drive id=OpenCoreBoot,if=none,snapshot=on,format=qcow2,file="$REPO_PATH/OpenCore-Catalina/OpenCore-nopicker.qcow2" -device ide-hd,bus=sata.2,drive=OpenCoreBoot -device ide-hd,bus=sata.3,drive=InstallMedia -drive id=InstallMedia,if=none,file="$REPO_PATH/BaseSystem.img",format=raw -drive id=MacHDD,if=none,file="$REPO_PATH/mac_hdd_ng.img",format=qcow2 -device ide-hd,bus=sata.4,drive=MacHDD -netdev tap,id=net0,ifname=tap0,script=no,downscript=no -device vmxnet3,netdev=net0,id=net0,mac=52:54:00:c9:18:27 -vga vmware ) qemu-system-x86_64 "${args[@]}" ```

Purple Prius Parts

Sun, 23 Aug 2020 00:00:00 GMT

# _In other news..._ * The prius parts are purple * A Pi manages car audio & PID interface ### _...Cleese will sum up the rest of this project:_ ![drawing](https://i.chzbgr.com/full/8543536128/h60BDA31C/and-now-for-something-completely-different ) ![drawing](/images/posts/unnamed.jpg) ![drawing](/images/posts/unnamed-1.jpg)

...Ever tried to Chrome Remote ➡️ Ubuntu Budgie? xD

Thu, 23 Jul 2020 00:00:00 GMT

[_Check out this project on my Github over here 🙂_](https://github.com/Jesssullivan/chrome-remote-desktop-budgie) #### Fully automated patching for Chrome Remote Desktop on Ubuntu Budgie. Chrome remote desktop is fantastic, but often clashes with Xorg nuances from a variety of desktop environments in Ubuntu. This `chrome-remote-desktop` script extends and replaces the version automatically installed by Google in `/opt/google/chrome-remote-desktop/chrome-remote-desktop`. This stuff is only relevant for accessing your Ubuntu machine from elsewhere _(e.g. the "server", the client machine should not be installing anything, all it needs is a web browser)_. **_Set up the server:_** Before patching anything or pursuing other forms of delightful tomfoolery, follow the [installation instructions provided by Google](https://remotedesktop.google.com/access/). Set up everything normally- install Google's .deb download with dpkg, set up a PIN, etc. The trouble comes when you are trying to remote in- some problems you may encounter include: * none of the X sessions work, each immediately closing the connection to the client * the remote desktop environment crashes or becomes mangled * odd scaling issues or flaky resolution changes **_Patch it up:_** ```bash # get this script: wget https://raw.githubusercontent.com/Jesssullivan/chrome-remote-desktop-budgie/master/chrome-remote-desktop # or: git clone https://github.com/Jesssullivan/chrome-remote-desktop-budgie/ cd chrome-remote-desktop-budgie # behold: python3 chrome-remote-desktop # ...perhaps, if you are keen (optional): # sudo chmod u+x addsystemd.sh # sudo ./addsystemd.sh ``` **_What does this do?_** We are primarily just enforcing the use of existing instances of X and correct display values as reported by your system. * This version keeps a persistent version itself in `/usr/local/bin/` in addition updating the one executed by Chrome in `/opt/google/chrome-remote-desktop/`. * A mirror of this script is also maintained at `/usr/local/bin/chrome-remote-desktop.github`, and will let the user know if there are updates. * The version distributed by google is retained in `/opt/` too as `chrome-remote-desktop.verbatim`. * Each of these versions are compared by md5 hash- this way our patched version of `chrome-remote-desktop` will always make sure it is where it should be, even after Google pushes updates and overwrites everything in `/opt/`.

This, That, etc

Wed, 15 Jul 2020 00:00:00 GMT

_....Updated 07/19/2020_ **_Bits & bobs, this & that of late:_** ...In effort to thwart the recent heat and humidity here in the White Mountains (or, perhaps just to follow the philosophy of circuitous overcomplication... 🙂 ) here are some sketches of quick-release exhaust fittings of mine for a large, wheeled AC & dehumidifier unit (these have been installed throughout my home via window panels). ![](/images/posts/unnamed.jpg) ...Sketching out a severely overcomplicated "computer shelf", rapid-fab style: _(plasma cut / 3d printed four-post server rack ==[RepRapRack](https://reprap.org/wiki/RepRap)?? xD)_ 🙂 ...Also, Ryan [@ V1Engineering](https://www.v1engineering.com/) recently released his [new MPCNC Primo here](https://www.v1engineering.com/mpcnc-primo-is-live/), should anyone be keen. Long Live the MPCNC! 🙂 [...Oodles of fun everyday over in the `clipi` project- check it out!](https://github.com/Jesssullivan/clipi) ...A really short morning metal: ..._[All Morning Metal tracks here](https://soundcloud.com/jesssullivan/tracks), a couple [videos here](https://www.youtube.com/playlist?list=PL6y8N_vP4OooPyza8EOIrz2XC0cck9Fae)_ xD

clipi CLI!

Sat, 11 Jul 2020 00:00:00 GMT

[**_Find this project on my github here!_**](https://github.com/Jesssullivan/clipi) _...post updated 07/19/2020_ **_An efficient toolset for Pi devices_** _Emulate, organize, burn, manage a variety of distributions for Raspberry Pi._ * * * ### Choose your own adventure.... **_Emulate:_** `clipi` virtualizes many common sbc operating systems with QEMU, and can play with both 32 bit and 64 bit operating systems. * _Select from any of the included distributions (or add your own to[/sources.toml](https://github.com/Jesssullivan/clipi/blob/master/etc/sources.toml)!) and `clipi` will handle the rest._ **_Organize:_** `clipi` builds and maintains organized directories for each OS as well a [persistent & convenient .qcow2](https://www.qemu.org/docs/master/interop/qemu-img.html) QEMU disk image. * Too many huge source _.img_ files and archives? `clipi` cleans up after itself under the ``Utilities...`` menu. * additional organizational & gcc compilation methods are available in [/kernel.py](https://github.com/Jesssullivan/clipi/tree/master/kernel.py) **_Write:_** `clipi` burns emulations to external disks! Just insert a sd card or disk and follow the friendly prompts. All files, `/home`, guest directories are written out. * _Need to pre-configure (or double check) wifi? Add your ssid and password to[/wpa_supplicant.conf](https://github.com/Jesssullivan/clipi/blob/master/wpa_supplicant.conf) and copy the file to `/boot` in the freshly burned disk._ * _Need pre-enabled ssh? copy[/ssh](https://github.com/Jesssullivan/clipi/blob/master/ssh) to `/boot` too._ * _`clipi` provides options for writing from an emulation's `.qcow2` file via qemu..._ * _...as well as from the[source's](https://github.com/Jesssullivan/clipi/blob/master/etc/sources.toml) raw image file with the `verbatim` argument_ **_Manage:_** `clipi` can find the addresses of all the Raspberry Pi devices on your local network. * _Need to do this a lot?`clipi` can install itself as a Bash alias (option under the ``Utilities...`` menu, fire it up whenever you want._ **_Shortcuts:_** Shortcuts & configuration arguments can be passed to `clipi` as a [.toml](https://github.com/toml-lang/toml) ([or yaml](https://yaml.org/)) file. * _Shortcut files access clipi's tools in a similar fashion to the interactive menu:_ # .toml ``` # you can access the same tools and functions visible in the interactive menu like so: 'Burn a bootable disk image' = true # same as selecting in the interactive cli 'image' = 'octoprint' 'target_disk' = 'sdc' ``` * _`clipi` exposes many features only accessible via configuration file arguments, such as distribution options and emulation settings._ # .toml ``` # important qemu arguments can be provided via a shortcut file like so: 'kernel' = "bin/ddebian/vmlinuz-4.19.0-9-arm64" 'initrd' = "bin/ddebian/initrd.img-4.19.0-9-arm64" # qemu arguments like these use familiar qemu lexicon: 'M' = "virt" 'm' = "2048" # default values are be edited the same way: 'cpu' = "cortex-a53" 'qcow_size' = "+8G" 'append' = '"rw root=/dev/vda2 console=ttyAMA0 rootwait fsck.repair=yes memtest=1"' # extra arguments can be passed too: '**args' = """ -device virtio-blk-device,drive=hd-root \ -no-reboot -monitor stdio """ # additional network arguments can be passed like so: # (clipi may automatically modify network arguments depending on bridge / SLiRP settings) 'network' = """ -netdev bridge,br=br0,id=net0 \ -device virtio-net-pci,netdev=net0 """ ``` * _Supply a shortcut file like so:_ ``python3 clipi.py etc/find_pi.toml`` * _take a look in[/etc](https://github.com/Jesssullivan/clipi/tree/master/etc) for some shortcut examples and default values_ * * * #### _TODOs & WIPs:_ _bridge networking things:_ * working on guest --> guest, bridge --> host, host only mode networking options. as of 7/17/20 only SLiRP user mode networking works, see branch [broken_bridge-networking](https://github.com/Jesssullivan/clipi/tree/broken_bridge-networking) to see what is currently cooking here _kernel stuff:_ * automate ramdisk & kernel extraction- most functions to do so are all ready to go in /kernel.py * _other random kernel todos-_ * working on better options for building via qemu-debootstrap from chroot instead of debian netboot or native gcc * add git specific methods to sources.py for mainline Pi linux kernel * verify absolute binutils version * need to get cracking on documentation for all this stuff _gcp-io stuff:_ * formalize ddns.py & dockerfile * make sure all ports (22, 80, 8765, etc) can up/down as reverse proxy * * * # clone: ```bash git clone https://github.com/Jesssullivan/clipi cd clipi # preheat: pip3 install -r requirements.txt # (or pip install -r requirements.txt) # begin cooking some Pi: python3 clipi.py ```

OSS+ELC -> Monday

Mon, 29 Jun 2020 00:00:00 GMT

[Open Source Summit + Embedded Linux Conference!](https://events.linuxfoundation.org/open-source-summit-north-america/) _Jess's Monday:_ * * * _In other tangents..._ **_....Dover Microsystem's Genesys 2 FPGA is moving into it's new home, v6:_** _....A quick polyphase hub motor sketch around a Toyota wheel bearing:_

Dover's Enclosure

Wed, 24 Jun 2020 00:00:00 GMT

*Some images from the original WordPress post are no longer available.* A stylish demo enclosure for the Xilinx / Digilent Genesys 2 with a display panel. Check out what Dover Microsystems is up to here: https://www.dovermicrosystems.com/ Prototyping & production @ the D&M Makerspace- see what else we're up to: https://makerspace.plymouthcreate.net/ ### Electronics: **FPGA-[Digilent Xilinx Genesys 2 FPGA Reference](https://reference.digilentinc.com/reference/programmable-logic/genesys-2/reference-manual)** **The display and HDMI driver board from pimoroni-** [_The sketch for panel dimensions are shared over here too_](https://forums.pimoroni.com/t/cad-file-for-hdmi-8-ips-lcd-screen-kit-1024x768/12499/3?u=jesssullivan) #### BOM for version 6: [_You can find the V6 interactive Fusion 360 model over here_](https://a360.co/36RBUQ1) [_...additional V6 svg, stl layouts on tinkercad_](https://www.tinkercad.com/things/6H87w83xGPq) _Materials:_ Size | Type ---|--- 12"x12" | 1/4" (6.35mm) clear acrylic sheet 12"x12" | 3mm clear acrylic sheet 12"x12" | 3mm colored acrylic sheet ~45 grams | printer plastic (filament or resin) _Hardware:_ Qty | Size ---|--- 3 | m3x8 3 | m3x18 1 | m3x20 7 | m3 nut 2 | m2x14 2 | m2x10 4 | m2 nut #### What is this thing? Some words from founding scientist @gregsgit: *"We use the FPGA to prototype / emulate a "Soft Core" CPU with and without Dover's IP (logic) called CoreGuard. An FPGA can simulate (sometimes called "emulate") logical circuits, and is reprogrammable. So you can design circuitry that eventually will be fabricated in silicon, but you can work out bugs and try different designs using the FPGA "fabric".* *For demos, we synthesize to the Xilinx FPGA: a design for a RISC-V CPU, a simple UART (serial interface), an interface to the on-board DDR memory and flash memory, and a simple video output. We put some software in the on-board flash, then boot a working RISC-V system. We'll show how the software can be attacked, using I/O over the serial port to mimic what would typically take place over a network connection. Next, we show the same SoC (CPU + UART + memory) with CoreGuard logic added in. We run the same software and then show that the same attack is blocked by CoreGuard. We also use the FPGA to emulate the Arm CPU that we are interfacing with for our NXP customer."* *Some images from the original WordPress post are no longer available.*

Parse fdisk -l in Python

Wed, 24 Jun 2020 00:00:00 GMT

[`fdisk -l`](https://www.man7.org/linux/man-pages/man8/fdisk.8.html) has got to be one of the more common disk-related commands one might use while fussing about with raw disk images. The fdisk utility is ubiquitous across linux distributions (also `brew install gptfdisk` and `brew cask install gdisk`, supposedly). The ` -l` argument provides a quick look raw sector & file system info. Figuring out the `Start, End, Sectors, Size, Id, Format` of a disk image's contents without needing to mount it and start lurking around is handy, just the sort of thing one might want to do with Python. Lets write a function to get these attributes into a dictionary- here's mine: ```python import subprocess import re def fdisk(image): # `image`, a .img disk image: cmd = str('fdisk -l ' + image) # read fdisk output- everything `cmd` would otherwise print to your console on stdout # is instead piped into `proc`. proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, shell=True) # the raw stuff from stdout is not parseable as is, so we read into a string: result = proc.stdout.read().__str__() # figure out what type we should iterate with when looking via file / part contained within image. I have no idea if anything besides .img will work- YMMV, but YOLO xD if '.iso' in result: iter = '.iso' if '.qcow2' in result: iter = '.qcow2' else: iter = '.img' # chop up fdisk results by file / partition- # the resulting `parts` are equivalent to fdisk "rows" in the shell parts = re.findall(r'' + iter + r'\d', result) # dictionary `disk` contains each "row" from `parts`: disk = {} for p in parts: # sub dictionary 'part' contains the handy fdisk output values: part = {} # get just the number words with regex sauce: line = result.split(p)[1] words = re.split(r'\s+', line) # place each word into 'part': part['Start'] = words[1] part['End'] = words[2] part['Sectors'] = words[3] part['Size'] = words[4] part['Id'] = words[5] part['Format'] = words[6].split('\\n')[0] # stick this part into 'disk', move onto next disk part: disk[p] = part return disk ```

The eBird API & regionCode

Sun, 14 Jun 2020 00:00:00 GMT

[_get this script and other GIS bits here on github_](https://github.com/Jesssullivan/GIS_Shortcuts) The [Ebird dataset](https://ebird.org/science/download-ebird-data-products) is awesome. While directly handling data as a **massive** delimited file- as distributed by [the eBird people-](https://ebird.org/data/download) is cumbersome at best, the [ebird api](https://documenter.getpostman.com/view/664302/S1ENwy59?version=latest#e18ea3b5-e80c-479f-87db-220ce8d9f3b6) offers a fairly straightforward and efficient alternative for a few choice bits and batches of data. * The eBird `AWK` tool for filtering the actual delimited data can be [found over here](https://cornelllabofornithology.github.io/auk/): ``install.packages("auk")`` It is worth noting R + `auk` (or frankly any R centered filtering method) will quickly become limited by the single-threaded approach of R, and how you're managing memory as you iterate. Working and querying the data from a proper database quickly becomes necessary. Most conveniently, the [eBird API already exists-](https://documenter.getpostman.com/view/664302/S1ENwy59?version=latest#e18ea3b5-e80c-479f-87db-220ce8d9f3b6) snag an [key over here](https://ebird.org/api/keygen). ...The API package for R is [over here](https://cran.r-project.org/web/packages/rebird/index.html): ``install.packages("rebird")`` ...There is also a neat Python wrapper [over here](https://pypi.org/project/ebird-api/): ``pip3 install ebird-api`` **_Region Codes:_** I'm not sure why, but some methods use normal latitude / longitude in decimal degrees while some others use `"regionCode"`, which seems to be some kind of eBird special. Only ever seen this format in ebird data. For example, recent observations uses `regionCode`: ```text # GET Recent observations in a region: # https://api.ebird.org/v2/data/obs/REGIONCODE/recent ``` ...But nearby recent observations uses latitude / longitude: ```text # GET Recent nearby observations: # https://api.ebird.org/v2/data/obs/geo/recent?lat=LAT&lng=LNG ``` Regardless, lets just write a function to convert decimal degrees to this `regionCode` thing. Here's mine: ```python #!/usr/bin/env python3 """ # provide latitude & longitude, return eBird "regionCode" Written by Jess Sullivan @ https://transscendsurvival.org/ available at: https://raw.githubusercontent.com/Jesssullivan/GIS_Shortcuts/master/regioncodes.py """ import requests import json def get_regioncode(lat, lon): ``` # this municipal api is a publicly available, no keys needed afaict census_url = str('https://geo.fcc.gov/api/census/area?lat=' + str(lat) + '&lon=' + str(lon) + '&format=json') # send out a GET request: payload = {} get = requests.request("GET", census_url, data=payload) # parse the response, all api values are contained in list 'results': response = json.loads(get.content)['results'][0] # use the last three digits from the in-state fips code as the "subnational 2" identifier: fips = response['county_fips'] # assemble and return the "subnational type 2" code: regioncode = 'US-' + response['state_code'] + '-' + fips[2] + fips[3] + fips[4] print('formed region code: ' + regioncode) return regioncode ``` ```

Prius, Printers

Thu, 04 Jun 2020 00:00:00 GMT

**_Add EV only mode button for 2009 Prius-_** Pinouts and wiring reference here: http://www.calcars.org/prius-evbutton-install.pdf Big shiny EV mode button in the Prius! *Some images from the original WordPress post are no longer available.* Fusion 360 files here: https://a360.co/2zOACJJ Files uploaded to thingiverse here: https://www.thingiverse.com/thing:4422091 xposted to prius chat too: https://priuschat.com/threads/3d-printed-ev-mode-button-xd.216774/ PLA & Carbon Polycarbonate button housings: *Some images from the original WordPress post are no longer available.* * * * **While we're at it....** See more notes on D&M 3d Printer stuff on github here: https://github.com/Jesssullivan/Funmat-HT-Notes ...and here: https://github.com/Jesssullivan/AeroTaz5_hotfix

While at a safe distance...

Mon, 18 May 2020 00:00:00 GMT

...Playing with [Bandlab's Sonar reboot](https://www.bandlab.com/products/cakewalk) \--> [morning metal](https://soundcloud.com/jesssullivan/tracks) ....Frankly the whole suite (yes, Melodyne, the whole nine yards) is way better than when it was with the late Cakewalk, and its all free now. PSA! ...Unexpected success with [Nylon 680 FDA](https://taulman3d.com/680-features.html) {3mm @ .8} for some rather delicate parts: *Some images from the original WordPress post are no longer available.* ..._Yet_ another improved pi monitoring sketch, currently in production w/ polycarbonate & 1/4"... ...or to quote Mad-eye Moody, "CONSTANT VIGILANCE!" 🙂 xD

oes

Mon, 27 Apr 2020 00:00:00 GMT

# **_Interactive 3D Views-_** #### _Without OES Suggestions:_ _tap window to start navigation_ #### **_With OES Suggestions:_** _tap window to start navigation_ * * * *Some images from the original WordPress post are no longer available.* ![](/images/posts/JESSSOES-03.png) ![](/images/posts/JESSSOES-04.png) ![](/images/posts/JESSSOES-05.png) ![](/images/posts/JESSSOES-06.png) ![](/images/posts/JESSSOES-07.png) ![](/images/posts/JESSSOES-08.png) ![](/images/posts/JESSSOES-09.png) ![](/images/posts/JESSSOES-10.png) ![](/images/posts/JESSSOES-11.png) ![](/images/posts/JESSSOES-12.png) ![](/images/posts/JESSSOES-13.png) ![](/images/posts/JESSSOES-14.png) ![](/images/posts/JESSSOES-15.png) ![](/images/posts/JESSSOES-16.png) ![](/images/posts/JESSSOES-17.png) ![](/images/posts/JESSSOES-18.png)

Install Adobe Applications on AWS WorkSpaces

Sat, 25 Apr 2020 00:00:00 GMT

By default, the browser based authentication used by Adobe’s Creative Cloud installers will fail on AWS WorkSpace instances. Neither the installer nor Windows provide much in the way of useful error messages- here is how to do it! [Charlie Brown Good Grief GIF](https://tenor.com/view/good-grief-charlie-brown-gif-10296718) from [Charliebrown GIFs](https://tenor.com/search/charliebrown-gifs) Open Server Manager. Under “Local Server”, open the “Internet Explorer Enhanced Security Configuration”- *(mercy!)* - and turn it off. *An image described as "Good Lord" was here, but has since vanished from the web.* ##### Tada! The sign on handoff from the installer→Browser→ back to installer will now work fine. xD *Some images from the original WordPress post are no longer available.*

Convert .heic -> .png

Thu, 09 Apr 2020 00:00:00 GMT

[_on github here_](https://github.com/Jesssullivan/misc/blob/master/etc/heic_png.sh), _or just get this script:_ ```bash wget https://raw.githubusercontent.com/Jesssullivan/misc/master/etc/heic_png.sh ``` Well, following the current course of Apple’s corporate brilliance, iOS now defaults to .heic compression for photos. Hmmm. Without further delay, let's convert these to png, here from the sanctuary of Bash in [♡Ubuntu Budgie♡](https://ubuntubudgie.org/downloads/). [_Libheif is well documented here on Github BTW_](https://github.com/strukturag/libheif) ```bash #!/bin/bash # recursively convert .heic to png # by Jess Sullivan # # permiss: # sudo chmod u+x heic_png.sh # # installs heif-convert via ppa: # sudo ./heic_png.sh # # run as $USER: # ./heic_png.sh command -v heif-convert >/dev/null || { echo >&2 -e "heif-convert not intalled! \nattempting to add ppa...."; if [[ $EUID -ne 0 ]]; then echo "sudo is required to install, aborting." exit 1 fi add-apt-repository ppa:strukturag/libheif apt-get install libheif-examples -y apt-get update -y exit 0 } # default behavior: for fi in *.heic; do echo "converting file: $fi" heif-convert $fi $fi.png # FWIW, convert to .jpg is faster if png is not required # heif-convert $fi $fi.jpg done ```

D&M Shields - Fusion 360

Sun, 05 Apr 2020 00:00:00 GMT

As of 4/4/20, we are busy 3d printing our rigid shield design, efficiently hacked into its current form by Bret here at D&M. [click here to visit or download the Fusion files!](https://a360.co/39I4iE0) The flat, snap-fit nature of this design can easily be lasercut as well- the varied depths of the printed model are just an effort to minimize excess plastic and print time. More to come on the laser side of things- in addition to the massive time savings- like <20 seconds vs. >3 hours per shield- we can use far cheaper and varied materials with the addition of our sterilizable and durable UV resins and coatings. Similarly, lasercut stock + resin offers the possibility quick adaptation and derivative design, such as [flexible](https://a360.co/2UFKRHM) UV cured forms.

ppe & whatnot

Thu, 02 Apr 2020 00:00:00 GMT

Yep, we too are busy cooking up protective medical devices....... **¯_(ツ)_/¯** *Some images from the original WordPress post are no longer available.* ## & whatnot: **_Prototyping bits, bobs for an ada motorsports startup-_** *Some images from the original WordPress post are no longer available.* **_Fast Pi camera stand sketch:_** **_Quick pass at a low friction filament spool holder for some very fragile materials:_** *Some images from the original WordPress post are no longer available.*

Some GDAL shell macros from R instead of rgdal

Mon, 23 Mar 2020 00:00:00 GMT

[also here on github](https://github.com/Jesssullivan/GIS_Shortcuts#rmacros) _it's not R sacrilege if nobody knows_ Even the little stuff benefits from some organizational scripting, even if it’s just to catalog one’s actions. Here are some examples for common tasks. Get all the source data into a R-friendly format like csv. `ogr2ogr` has a nifty option `-lco GEOMETRY=AS_WKT` (Well-Known-Text) to keep track of spatial data throughout abstractions- we can add the WKT as a cell until it is time to write the data out again. ```python # define a shapefile conversion to csv from system's shell: sys_SHP2CSV <-function(shp) { csvfile <-paste0(shp, '.csv') shpfile <-paste0(shp, '.shp') if (!file.exists(csvfile)) { # use -lco GEOMETRY to maintain location # for reference, shp --> geojson would look like: # system('ogr2ogr -f geojson output.geojson input.shp') # keeps geometry as WKT: cmd <-paste('ogr2ogr -f CSV', csvfile, shpfile, '-lco GEOMETRY=AS_WKT') system(cmd) # executes command } else { print(paste('output file already exists, please delete', csvfile, 'before converting again')) } return(csvfile) } ``` Read the new csv into R: ``` # for file 'foo.shp': foo_raw <-read.csv(sys_SHP2CSV(shp='foo'), sep = ',') ``` One might do any number of things now, some here lets snag some columns and rename them: ```r # rename the subset of data "foo" we want in a data.frame: foo <-data.frame(foo_raw[1:5]) colnames(foo) <-c('bar', 'eggs', 'ham', 'hello', 'world') ``` We could do some more careful parsing too, here a semicolon in cell strings can be converted to a comma: ``` # replace ` ; ` to ` , ` in col "bar": foo$bar <-gsub(pattern=";", replacement=",", foo$bar) ``` Do whatever you do for an output directory: ``` # make a output file directory if you're into that # my preference is to only keep one set of output files per run # here, we'd reset the directory before adding any new output files redir <-function(outdir) { if (dir.exists(outdir)) { system(paste('rm -rf', outdir)) } dir.create(outdir) } ``` Of course, once your data is in R there are countless "R things" one could do... ``` # iterate to fill empty cells with preceding values for (i in 1:length(foo[,1])) { if (nchar(foo$bar[i]) < 1) { foo$bar[i] <-foo$bar[i-1] } # fill incomplete rows with NA values: if (nchar(foo$bar[i]) < 1) { foo[i,] <-NA } } # remove NA rows if there is nothing better to do: newfoo <-na.omit(foo) ``` Even though this is totally adding a level of complexity to what could be a single `ogr2ogr` command, I've decided it is still worth it- I'd definitely rather keep track of everything I do over forget what I did.... xD ```python # make some methods to write out various kinds of files via gdal: to_GEO <-function(target) { print(paste('converting', target, 'to geojson .... ')) system(paste('ogr2ogr -f', " geojson ", paste0(target, '.geojson'), paste0(target, '.csv'))) } to_SHP <-function(target) { print(paste('converting ', target, ' to ESRI Shapefile .... ')) system(paste('ogr2ogr -f', " 'ESRI Shapefile' ", paste0(target, '.shp'), paste0(target, '.csv'))) } # name files: foo_name <-'output_foo' # for table data 'foo', first: write.csv(foo, paste0(foo_name, '.csv')) # convert with the above csv: to_SHP(foo_name) ``` Cheers! -Jess

When it must be Windows....

Tue, 17 Mar 2020 00:00:00 GMT

_Edit 07/26/2020:_ [Check out the expanded GIS notes page here!](https://jesssullivan.github.io/GIS_Shortcuts/) **_Regarding Windows-specific software, such as ArcMap:_** _Remote Desktop:_ The greatest solution I've settled on for ArcMap use continues to be [Chrome Remote Desktop](https://remotedesktop.google.com/home), coupled with an [IT Surplus](https://www.plymouth.edu/webapp/itsurplus/) desktop purchased for ~$50. Once Chrome is good to go on the remote Windows computer, one can operate everything from a web browser from anywhere else (even reboot and share files to and from the remote computer). While adding an additional, dedicated computer like this may not be possible for many students, it is certainly the simplest and most dependable solution. _VirtualBox, Bootcamp, etc:_ [Oracle's VirtualBox](https://www.virtualbox.org/wiki/Downloads) is a longstanding (and free!) virtualization software. A Windows virtual machine is vastly preferable over [Bootcamp](https://support.apple.com/boot-camp) or further [partition tomfoolery](https://www.digitalocean.com/community/tutorials/how-to-partition-and-format-storage-devices-in-linux). One can start / stop the VM only when its needed, store it on a usb stick, avoid insane pmbr issues, etc. * Bootcamp will consume at least 40gb of space at all times before even attempting to function, whereas even a fully configured Windows VirtualBox VDI will only consume ~22gb, and can be moved elsewhere if not in use. * There are better (not free) virtualization tools such as [Parallels](https://www.parallels.com/), though any way you slice it a dedicated machine will almost always be a better solution. **Setup & Configure VirtualBox:** * [Install VirtualBox- link](https://www.virtualbox.org/wiki/Downloads) * [Download a Windows 10 ISO- link](https://www.microsoft.com/en-us/software-download/windows10ISO) There are numerous sites with VirtualBox guides, so I will not go into detail here. _Extra bits on setup-_ * [Guest Additions](https://www.virtualbox.org/manual/ch04.html) are not necessary, despite what some folks may suggest. * Dynamically Allocated VDI is the way to go as a virtual disk. There is no reason not to set the allocated disk size to the biggest value allowed, as it will never consume any more space than the virtual machine actually needs. * Best to click through all the other machine settings just to see what is available, it is easy enough to make changes over time. * There are many more levels of convoluted not worth stooping to, ranging from ArcMap via [AWS EC2](https://aws.amazon.com/ec2/) or [openstack](https://www.openstack.org/) to [KVM/QEMU](https://www.linux-kvm.org/page/Main_Page) to [WINE](https://www.winehq.org/about). _Take it from me_ **_xD_**

QEMU for Raspian ARM!

Wed, 30 Oct 2019 00:00:00 GMT

_Updated 11/6/19_ Visit the repo on Github here- https://github.com/Jesssullivan/QEMU-Raspian _tested on Mac OSX 10.14.6_ Emulates a variety of Raspian releases on proper ARM hardware with QEMU. **_Prerequisites:_** QEMU and wget (OSX homebrew) ```bash brew install qemu wget ``` Get the Python3 CLI in this repo: ```bash wget https://raw.githubusercontent.com/Jesssullivan/USBoN/master/QEMU_Raspian.py ``` * * * **_Usage:_** After the first launch, it will launch from the persistent .qcow2 image. With no arguments & in a new folder, Raspian "stretch-lite" (no desktop environment) will be: * downloaded as a zip archive with a release kernel * unarchived --> to img * converted to a Qcow2 with 8gb allocated as disk * launched from Qcow2 as administrator sudo python3 QEMU_Raspian.py **_Optional Arguments:_** * `` -h `` prints CLI usage help * `` -rm `` removes ALL files added in dir with QEMU_Raspian.py * `` stretch `` uses standard graphical stretch release with GUI * ``stretchlite `` for stretchlite release [default!] * `` buster `` for standard graphical buster release [YMMV] * ``busterlite`` for busterlite release [YMMV] # examples: ```bash sudo python3 QEMU_Raspian.py busterlite python3 QEMU_Raspian -h # print help ``` * * * **_Burn as .img:_** ```bash qemu-img convert -f qcow2 -O raw file.qcow2 file.img ``` *The image of "Four Pi Emulations ala QEMU" is no longer available.*

Small Updates to GDAL Notes

Mon, 07 Oct 2019 00:00:00 GMT

# GIS Shortcuts ![codecogseqn-1](/images/posts/CodeCogsEqn-1.png) * * * **GDAL on OSX, Ubuntu & Windows WSL** Bash is great: It is a shell scripting language that is baked into every new Apple computer and many, many Linux distributions too. It is fun to learn and easy to find your “insert repetitive task here” to Bash translation on Stack Exchange, just a search away. As you uses the command prompt more and more for increasingly cool GIS tasks- be it for Python QGIS OR ESRI, R language for data cleaning and analysis, or just because you noticed you can get “mad amounts of work done” at increasingly faster rates- your vocabulary in the UNIX shell and bash will naturally grow. I wanted to make a GIS post for Mac OS because it is both under-represented (for great reasons) in GIS and arguably the number 1 choice for any discerning consumers of computer hardware. Many Linux OS options are faster for much of this “UNIX for GIS”, as quite a few of the things we need are already included with many Debian / Ubuntu distros, and come forms that have been stable for a long, long time. If you are looking to setup a system primarily for GIS / data science (disregarding ESRI of course), See my initial notes on the Ubuntu variant Pop_OS by System76. If you like the ChromeOS vibe for multitasking and simplicity and the familiarity of Mac OS, it is a keeper (and also a sleeper, seeing how many folks are still on Windows for GIS…….). **Open Terminal** (The less you have in your way on your screen, the faster you can go!) xD Note: in my opinion, homebrew and macPorts are good ideas- try them! If you don’t have it, get it now: ```bash /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" ``` (….However, port or brew installing QGIS and GDAL (primarily surrounding the delicate links between QGIS / GDAL / Python 2 & 3 / OSX local paths) can cause baffling issues. If possible, don’t do that. Use QGIS installers from the official site and build from source!) if you need to resolve issues with your GDAL packages via removal: on MacPorts, try similar: ```bash sudo port uninstall qgis py37-gdal # on homebrew, list then remove (follow its instructions): brew list brew uninstall gdal geos gdal2 ``` _!!! NOTE: I am investigating more reliable built-from-source solutions for gdal on mac._ Really! There are numerous issues with brew-installed gdal. Those I have run into include: * linking issues with the crucial directory “gdal-data” (libraries) * linking issues Python bindings and python 2 vs. 3 getting confused * internal raster library conflicts against the gdal requirements * Proj.4 inconsistencies (see source notes below) * OSX Framework conflicts with source / brew / port (http://www.kyngchaos.com/software/frameworks/) * Linking conflicts with old, qgis default / LTR libraries against new ones * Major KML discrepancies: expat standard vs libkml. brew install gdal ``` # # brew install qgis can work well too. At least you can unbrew it! # ``` Next, assuming your GDAL is not broken (on Mac OS this is rare and considered a miracle): ``` # double check CLI is working: gdalinfo --version # “GDAL 2.4.0, released 2018/12/14” gdal_merge.py # list of args ``` # Using Ubuntu GDAL on Windows w/ WSL [LINK: get the WSL shell from Microsoft](https://docs.microsoft.com/en-us/windows/wsl/install-win10) ```bash # In the WSL shell: sudo apt-get install python3.6-dev -y sudo add-apt-repository ppa:ubuntugis/ppa && sudo apt-get update sudo apt-get install libgdal-dev -y sudo apt-get install gdal-bin -y # See here for more notes including Python bindings: # https://mothergeo-py.readthedocs.io/en/latest/development/how-to/gdal-ubuntu-pkg.html ``` # In a new Shell: ``` # Double check the shell does indeed have GDAL in $PATH: gdalinfo --version ``` **_To begin- try a recent GIS assignment that relies on the ESRI mosaic system and lots of GUI and clicking but use GDAL instead._** Data source: ftp://ftp.granit.sr.unh.edu/pub/GRANIT_Data/Vector_Data/Elevation_and_Derived_Products/d-elevationdem/d-10m/ !! Warning! These files are not projected in a way ESRI or GDAL understands. They WILL NOT HAVE A LOCATION IN QGIS. They will, however, satisfy the needs of the assignment. ```bash # wget on mac is great. This tool (default on linux) lets us grab GIS data from # most providers, via FTP and similar protocols. brew install wget ``` make some folders ```bash mkdir GIS_Projects && cd GIS_Projects ``` use wget to download every .dem file (-A .dem) from the specified folder and sub-folders (-r) ```bash wget -r -A .dem ftp://ftp.granit.sr.unh.edu/pub/GRANIT_Data/Vector_Data/Elevation_and_Derived_Products/d-elevationdem/ cd ftp.granit.sr.unh.edu/pub/GRANIT_Data/Vector_Data/Elevation_and_Derived_Products/d-elevationdem ``` make an index file of only .dem files. (If we needed to download other files and keep them from our wget (more common) this way we can still sort the various files for .dem) ```bash ls -1 *.dem > dem_list.txt ``` use gdal to make state-plane referenced “Output_merged.tif” from the list of files in the index we made. it will use a single generic "0 0 255" band to show gradient. ```bash gdal_merge.py -init "0 0 255" -o Output_Merged.tif --optfile dem_list.txt ``` copy the resulting file to desktop, then return home ```bash cp Output_Merged.tif ~/desktop && cd ``` if you want (recommended): ```bash rm -rf GIS_Projects # remove .dem files. Some are huge! ``` In Finder at in ~/desktop, open the new file with QGIS. A normal photo viewer will NOT show any detail. Need to make something like this a reusable script? In Terminal, just a few extra steps: ```bash mkdir GIS_Scripts && cd GIS_Scripts ``` open an editor + filename. Nano is generally pre-installed on OSX. ```bash nano GDAL_LiveMerge.sh ``` COPY + PASTE THE SCRIPT FROM ABOVE INTO THE WINDOW * ctrl+X , then Y for yes make your file runnable: ```bash chmod u+x GDAL_LiveMerge.sh ``` run with ./ ```bash ./GDAL_LiveMerge.sh ``` You can now copy + paste your script anywhere you want and run it there. scripts like this should not be exported to your global path / bashrc and will only work if they are in the directory you are calling them: If you need a global script, there are plenty of ways to do that too. _See /Notes_GDAL/README.md for notes on building GDAL from source on OSX_ [**_Click here to visit this repo_**](https://github.com/Jesssullivan/GIS_Shortcuts) # 🙂

Decentralized Pi Video Monitoring w/ motioneye & BATMAN

Mon, 30 Sep 2019 00:00:00 GMT

[Visit the me here on Github](https://github.com/Jesssullivan/motioneye-mesh) _Added parabolic musings 10/16/19, see below_ **_...On using motioneye video clients on Pi Zeros & Raspbian over a BATMAN-adv Ad-Hoc network_** [link: motioneyeos](https://github.com/ccrisan/motioneyeos/wiki) [link: motioneye Daemon](https://github.com/ccrisan/motioneye/wiki) [link: Pi Zero W Tx/Rx data sheet:](https://www.cypress.com/file/298756/download#page=28&zoom=100,0,184) [link: BATMAN Open Mesh](https://www.open-mesh.org/projects/open-mesh/wiki) This implementation of motioneye is running on Raspbian Buster (opposed to motioneyeos). **Calculating Mesh Effectiveness w/ Python:** Please take a look at dBmLoss.py- the idea here is one should be able to estimate the maximum plausible distance between mesh nodes before setting anything up. It can be run with no arguments- ```bash python3 dBmLoss.py ``` ...with no arguments, it should use default values (Tx = 20 dBm, Rx = |-40| dBm) to print this: ``` you can add (default) Rx Tx arguments using the following syntax: python3 dBmLoss.py 20 40 python3 dBmLoss.py 57.74559999999994 ft = max. mesh node spacing, @ Rx = 40 Tx = 20 ``` _Regarding the Pi: _The Pi Zero uses an onboard BCM43143 wifi module. See above for the data sheet. We can expect around a ~19 dBm Tx signal from a BCM43143 if we are optimistic. Unfortunately, "usable" Rx gain is unclear in the context of the Pi. _Added 10/16/19:_ Notes on generating an **accurate** parabolic antenna shape with FreeCAD’s Python CLI: For whatever reason, (likely my own ignorance) I have been having trouble generating an accurate parabolic dish shape in Fusion 360 (AFAICT, Autodesk is literally drenching Fusion 360 in funds right now, I feel obligated to at least try). Bezier, spline, etc curves are not suitable! If you are not familiar with FreeCAD, the general approach- geometry is formed through fully constraining sketches and objects- is quite different from Sketchup / Tinkercad / Inventor / etc, as most proprietary 3d software does the “constraining” of your drawings behind the scenes. From this perspective, you can see how the following script never actually defines or changes the curve / depth of the parabola; all we need to do is change how much curve to include. A wide, shallow dish can be made by only using the very bottom of the curve, or a deep / narrow dish by including more of the ever steepening parabolic shape. ```python import Part, math # musings derived from: # https://forum.freecadweb.org/viewtopic.php?t=4430 # thinking about units here: tu = FreeCAD.Units.parseQuantity def mm(value): return tu('{} mm'.format(value)) rs = mm(1.9) thicken = -(rs / mm(15)) # defer to scale during fitting / fillet elsewhere m=App.Matrix() m.rotateY(math.radians(-90)) # create a parabola with the symmetry axis (0,0,1) parabola=Part.Parabola() parabola.transform(m) # get only the right part of the curve edge=parabola.toShape(0,rs) pt=parabola.value(rs) line=Part.makeLine(pt,App.Vector(0,0,pt.z)) wire=Part.Wire([edge,line]) shell=wire.revolve(App.Vector(0,0,0),App.Vector(0,0,1),360) # make a solid solid=Part.Solid(shell) # apply a thickness thick=solid.makeThickness([solid.Faces[1]],thicken,0.001) Part.show(thick) Gui.SendMsgToActiveView("ViewFit") """ # Fill screen: Gui.SendMsgToActiveView("ViewFit") # Remove Part in default env: App.getDocument("Unnamed1").removeObject("Shape") """ ``` **FWIW, here is my Python implimentation of a Tx/Rx "Free Space" distance calulator-** _dBmLoss.py:_ ```python from math import log10 from sys import argv ''' # estimate free space dBm attenuation: # ...using wfi module BCM43143: Tx = 19~20 dBm Rx = not clear how low we can go here d = distance Tx --> Rx f = frequency c = attenuation constant: meters / MHz = -27.55; see here for more info: https://en.wikipedia.org/wiki/Free-space_path_loss ''' f = 2400 # MHz c = 27.55 # RF attenuation constant (in meters / MHz) def_Tx = 20 # expected dBm transmit def_Rx = 40 # (absolute value) of negative dBm thesh def logdBm(num): return 20 * log10(num) def maxDist(Rx, Tx): dBm = 0 d = .1 # meters! while dBm < Tx + Rx: dBm = logdBm(d) + logdBm(f) - Tx - Rx + c d += .1 # meters! return d # Why not use this with arguments Tx + Rx from shell if we want: def useargs(): use = bool try: if len(argv) == 3: use = True elif len(argv) == 1: print('\n\nyou can add (default) Rx Tx arguments using the following syntax: \n \ python3 dBmLoss.py 20 40 \n \ python3 dBmLoss.py \ \n') use = False else: print('you must use both Rx & Tx arguments or no arguments') raise SystemExit except: print('you must use both Rx & Tx arguments or no arguments') raise SystemExit return use def main(): if useargs() == True: arg = [int(argv[1]), int(argv[2])] else: arg = [def_Rx, def_Tx] print(str('\n ' + str(maxDist(arg[0], arg[1])*3.281) + \ ' ft = max. mesh node spacing, @ \n' + \ ' Rx = ' + str(arg[0]) + '\n' + \ ' Tx = ' + str(arg[1]))) main() ```

Persistent, Live Ubuntu for College

Sun, 04 Aug 2019 00:00:00 GMT

**Below is are live mirrors of my "PSU Hacking Club" Ubuntu repos.** * * * [_https://github.com/psu-hacking/iso-gen_](https://github.com/psu-hacking/iso-gen) * * * [_https://psuhacking.club/_](https://psuhacking.club/) [_https://github.com/psu-hacking/static-site_](https://github.com/psu-hacking/static-site) # Simple File Hosting * * * **_README yet to be updated with Makerspace-specific formatting_** * * * Static site built quickly with [Hugo CLI](https://gohugo.io/getting-started/quick-start/) ```bash # on OSX # get hugo brew install hugo # clone site git clone https://github.com/psu-hacking/static-site cd static-site # Compile and compress public directory hugo zip -r site-archive.zip public # upload and host with sftp & ssh sftp user@yoursite.net > cd yoursite.net > put site-archive.zip # new terminal window ssh user@yoursite.net # check your remote filesystem- the idea is: > unzip site-archive.zip > rm -rf yoursite.net/site-archive.zip ``` [visit us](https://psuhacking.club) * * * #### [Also, check out the evolving PSU Hacking Club wiki here!](https://github.com/psu-hacking/home-wiki/wiki/Developer-Student-Software) xD - Jess

Summer 2019 Update!

Fri, 05 Jul 2019 00:00:00 GMT

**GIS Updates:** Newish Raster / DEM image → STL tool in the Shiny-Apps repo: [https://github.com/Jesssullivan/Shiny-Apps](https://github.com/Jesssullivan/Shiny-Apps) *Some images from the original WordPress post are no longer available.* See the (non-load balanced!) live example on the Heroku page: [https://kml-tools.herokuapp.com/](https://kml-tools.herokuapp.com/) Summarized for a forum member here too: [https://www.v1engineering.com/forum/topic/3d-printing-tactile-maps/](https://www.v1engineering.com/forum/topic/3d-printing-tactile-maps/) *Some images from the original WordPress post are no longer available.* **CAD / CAM Updates:** Been revamping my CNC thoughts- Basically, the next move is a complete rebuild (primarily for 6061 aluminum). I am aiming for: * Marlin 2.x.x around either a full-Rambo or 32 bit Archim 1.0 ([https://ultimachine.com/](https://ultimachine.com/)) * Dual endstop configuration, CNC only (no hotend support) * 500mm2 work area / swappable spoiler boards (~700mm exterior MPCNC conduit length) * Continuous compressed air chip clearing, shop vac / cyclone chip removal * Two chamber, full acoustic enclosure (cutting space + air I/O for vac and compressor) * Full octoprint networking via GPIO relays **FWIW** : Sketchup MPCNC: [https://3dwarehouse.sketchup.com/model/72bbe55e-8df7-42a2-9a57-c355debf1447/MPCNC-CNC-Machine-34-EMT](https://3dwarehouse.sketchup.com/model/72bbe55e-8df7-42a2-9a57-c355debf1447/MPCNC-CNC-Machine-34-EMT) Also TinkerCAD version: [https://www.tinkercad.com/things/fnlgMUy4c3i](https://www.tinkercad.com/things/fnlgMUy4c3i) **Electric Drivetrain Development:** BORGI / Axial Flux stuff: [https://community.occupycars.com/t/borgi-build-instructions/37](https://community.occupycars.com/t/borgi-build-instructions/37) Designed some rough coil winders for motor design here: [https://community.occupycars.com/t/arduino-coil-winder/99](https://community.occupycars.com/t/arduino-coil-winder/99) Repo: [https://github.com/Jesssullivan/Arduino_Coil_Winder](https://github.com/Jesssullivan/Arduino_Coil_Winder) *Some images from the original WordPress post are no longer available.* Also, an itty-bitty, skate bearing-scale axial flux / 3-phase motor to hack upon: [https://www.tinkercad.com/things/cTpgpcNqJaB](https://www.tinkercad.com/things/cTpgpcNqJaB) Cheers- -- Jess

Succession, Warblers, and GIS

Wed, 01 May 2019 00:00:00 GMT

For the past couple of field seasons I have been working with Dr. Leonard Reitsma at Plymouth State University studying warbler species distribution in managed forest patch cuts up in Canaan, NH. The work sits right at the intersection of GIS, ecology, and a lot of early mornings standing in regenerating clear cuts trying to pin down territorial birds with a handheld GPS. The basic question we were after: how do different warbler species partition space in and around patch cuts at various stages of forest succession? The focal species were Common Yellowthroat (COYE), Chestnut-sided Warbler (CSWA), Magnolia Warbler (MAWA), and Black-throated Blue Warbler (BTBW). Each of these species has different habitat preferences tied to vegetation structure, so the patchwork of regenerating cuts, edges, and intact forest creates a natural experiment in how birds sort themselves out across a landscape. The data collection workflow was straightforward but tedious. We tracked individual birds (many of them color-banded) using a combination of a handheld Garmin GPS and an iPhone running Compass 55 for quick waypoint drops. Every observation of a territorial male got a GPS point with notes on behavior, band combo if visible, and what the bird was doing. Over the course of a season this adds up to hundreds of points per site. All the spatial data lived as KML files, which is Google's XML-based format for geographic data. KML is convenient for field collection and visualization in Google Earth, but it is not the friendliest format for analysis. I put together a companion tutorial on converting KML point data to CSV using QGIS and R, which is available as a [PDF here](/papers/kml-qgis-r-tutorial.pdf). The short version: import the KML as a vector layer in QGIS, merge and export to CSV, then use R to query and filter the data by description fields. This made it much easier to pull out subsets of observations for specific birds or behaviors. For the spatial analysis itself, I used ArcMap to run kernel density estimates on the point data, which gives you a heat map of where each species was concentrating its activity. More interesting was the convex hull analysis -- drawing minimum bounding polygons around all the observations of a single bird to approximate its territory, then overlaying those territories on the patch cut polygons. This let us look at how individual territories related to the cut boundaries and whether birds were setting up shop inside the cuts, along the edges, or in the surrounding mature forest. The results lined up with what you would expect from the ecology. COYE and CSWA, both early-successional specialists, were concentrated inside the younger patch cuts with dense shrubby regrowth. MAWA showed up more in intermediate-aged cuts where young conifers were filling in. BTBW, a mature forest bird, mostly stayed out in the surrounding intact canopy but occasionally pushed into the edges of older cuts. The kernel density maps made these patterns visually obvious in a way that point maps alone do not. This work fed directly into a presentation at the 2019 AAG Annual Meeting in Washington, DC. You can find the [poster and related materials on the AAG page](/aag). The research was also connected to some other collaborative work with Dr. Reitsma -- around the same time, we published a paper in the Northeastern Naturalist documenting Black-capped Chickadees feeding Hermit Thrush nestlings (Reitsma, Burns & Sullivan, 2019), which came out of observations during the same field seasons in Canaan. The full GIS presentation is embedded below: Looking back, this project was where I really started to appreciate how much of field ecology is actually data management. The birds are the fun part, but the hours spent cleaning KML files, troubleshooting coordinate reference systems in ArcMap, and writing R scripts to wrangle waypoint descriptions into usable data frames -- that is where most of the work happens. It also got me thinking seriously about better tooling for field data collection, which eventually led to some of the software projects I picked up later. -- Jess

Warbler Trillers of the Charles

Thu, 18 Apr 2019 00:00:00 GMT

The first ones to arrive in MA, brush up! **Palm warbler** [songs](https://www.allaboutbirds.org/guide/Palm_Warbler/sounds) This is usually the first one to arrive. Gold bird, medium sized warbler, rufus hat. When they arrive in MA they are often found lower than usual / on the ground looking for anything they can munch on. Song is a rapid trill. More “musical / pleasant” than a fast chipping sparrow, faster than many Junco trills. **Pine warbler** [songs](https://www.allaboutbirds.org/guide/Pine_Warbler/sounds) Slimmer than palm, no hat, very slim beak, has streaks on the breast usually. Also a triller. They remain higher in the trees on arrival. **Yellow-rumped warbler** [songs](https://www.allaboutbirds.org/guide/Yellow-rumped_Warbler/sounds) Spectacular bird, if it has arrived you can’t miss it- also they will arrive by the dozen so worth waiting for a good visual. These also trill, which is another reason it is good to get a visual. The trill is slow, very “sing-song”, and has a downward inflection at the end. If there are a bunch sticking around for the summer, try to watch some sing- soon enough you will be able to pick out this trill from the others. -- [Yellow warbler](https://www.allaboutbirds.org/guide/Yellow_Warbler/id) says “sweet sweet sweet, I’m so Sweet!” and can get a bit confusing with Yellow-rumped warbler -- [Chestnut-sided warbler](https://www.allaboutbirds.org/guide/Chestnut-sided_Warbler/id) says “very very pleased to meet ya!” and can get a bit confusing with Yellow warbler **Black-and-white warbler** [songs](https://www.allaboutbirds.org/guide/Black-and-white_Warbler/sounds) Looks like a zebra - always acts like a nuthatch (clings to trunk and branches). This one trills like a rusty wheel. It can easily be distinguished after a bit of birding with some around. **American Redstart** [songs](https://www.allaboutbirds.org/guide/American_Redstart/sounds) Adult males look like a late 50’s hot-rodded American muscle car: long, low, two tone paint job. Matte/luster black with flame accents. Can’t miss it. The females and young males are buff (chrome, to keep in style I guess) with yellow accents. Look for behavior- if a “female” is getting beaten up while trying to sing a song in the same area, it is actually a first year male failing to establish a territory due to obviously being a youth. Cheers, -- Jess

Open Source Collaboration and Higher Education

Mon, 15 Apr 2019 00:00:00 GMT

I have been thinking a lot lately about how much the open source world reminds me of being at university. Not the tuition part, obviously, and not the part where you have to take that one required course that has nothing to do with your interests. I mean the actual good parts -- the knowledge sharing, the mentorship, the sense that everyone is working on something bigger than themselves. There is a quote I came across from Quint-Rapoport that stuck with me: the open source development process mimics the academic knowledge creation process, where gift economies are central to the social system. That framing really clicked. When I push something to a public repo, I am not expecting payment. I am contributing to a shared pool of knowledge, the same way a researcher publishes a paper so other people can build on it. The currency is not money -- it is the work itself, and the fact that someone else might find it useful. At Plymouth State, I have been lucky enough to be in an environment where interdisciplinary thinking is encouraged. That has made the parallels between university culture and open source culture even more obvious to me. Both are fundamentally about people with different specialties coming together to figure things out. The CS student helps the biology researcher automate their data pipeline. The GIS person (that would be me) contributes spatial analysis tools that someone in environmental science ends up using. It is the same dynamic you see on GitHub every day -- people with different skills converging on shared problems. ### What makes a good open source project Regardless of what a project actually does, the ones that work well tend to share a few things in common. First, there is clear documentation. This is honestly the most important part. If someone cannot look at your project and understand what it does and how to get started, you have already lost them. Providing an "in" for people who are less experienced is crucial, and it is something that good teachers do instinctively. Second, there is version control. Using Git (or something like it) is table stakes for any software team at this point. The thing I find interesting about VCS from an educational perspective is how it encourages a particular kind of independence. You work on your own copy, you figure things out, you document what you did and why. Then you share it back. That cycle of independent learning followed by contribution back to the group is basically the ideal seminar class, except it happens asynchronously and across time zones. Third, there is a way to ask questions and give advice. Issues, forums, mailing lists -- the format matters less than the culture around it. The best projects are the ones where asking a question is welcomed, not punished. Sound familiar? That is what a good classroom is supposed to be. ### The university connection Peters wrote about how universities can act as clusters that promote regional development through collaboration with government, business, and the local community. I think the open source world has been doing something similar, just without the institutional overhead. Open source communities are already global, already interdisciplinary, already built on the principle that sharing knowledge freely produces better outcomes than hoarding it. The interesting thing is that this did not come out of nowhere. The norms of open source development were established largely by people who came out of academic environments. They brought university values -- peer review, citation, collaborative inquiry -- into the software world. The README file is basically an abstract. A pull request is a peer review. Filing an issue is raising your hand in class. ### Where this is going I think the natural next step is for universities to lean into this more deliberately. Not just teaching students to use Git (though that matters), but recognizing that the open source community has built a functioning model of the knowledge economy that higher education has always aspired to be. It is a model without admission costs, without paywalls, without most of the social biases that make traditional academia inaccessible to a lot of people. That does not mean universities are irrelevant. Far from it. But it does mean that the wall between "academic work" and "open source contribution" is thinner than most people realize. For students like me, working in both spaces at once, that is a pretty exciting place to be.

Notes on a Free and Open Source Notes App: Joplin

Thu, 21 Mar 2019 00:00:00 GMT

Joplin for all your Operating Systems and devices As a lifelong IOS + OSX user (Apple products), I have used many, many notes apps over the years. From big name apps like OmniFocus, Things 3, Notes+, to all the usual suspects like Trello, Notability, Notemaster, RTM, and others, I always eventually migrate back to Apple notes, simply because it is always available and always up to date. There are zero “features” besides this convenience, which is why I am perpetually willing to give a new app a spin. Joplin is free, open source, and works on OSX, Windows, Linux operating systems and IOS and Android phones. Find it here: [https://joplin.cozic.net/](https://joplin.cozic.net/) `brew install joplin` The most important thing this project has nailed is cloud support and syncing. I have my iPhone and computers syncing via Dropbox, which is easy to setup and works…. really well. Joplin folks have added many cloud options, so this is unlikely to be a sticking point for users. Here are some of the key features: * Markdown is totally supported for straightforward and easy formatting * External editor support for emacs / atom / etc folks * Extra bit to troubleshoot possible issue for hardcore emacs users: * [https://discourse.joplin.cozic.net/t/note-for-emacs-users/623](https://discourse.joplin.cozic.net/t/note-for-emacs-users/623) * Layout is clean, uncluttered, and just makes sense * Built-in markdown text editor and viewer is great * Notebook, todo, note, and tags work great across platforms * Browser integration, E2EE security, file attachments, and geolocation included Hopefully this will be helpful. Cheers, -- Jess

Querying KML Point Data with QGIS and R

Sun, 10 Mar 2019 00:00:00 GMT

This semester I have been working with Dr. Leonard Reitsma at Plymouth State University on warbler territory research, and one of the recurring tasks is wrangling GPS point data collected in the field. We have roughly 800 point observations of banded birds stored as KML files -- the kind of thing Google Earth spits out -- and I needed a reliable way to pull specific subsets of those observations for analysis. The full tutorial PDF is embedded below, but here is the gist of the workflow. ## KML to CSV via QGIS KML is great for visualization but not so great for programmatic querying. The first step is getting the data into a flat format. In QGIS, you can drag a KML file straight into the layers panel, then right-click the layer and export it as a CSV. Make sure to check the "Add saved file to map" option so you can verify the export looks right. The attribute table should preserve all the descriptive fields from the original KML -- bird ID, band colors, sex, pairing status, location notes, and so on. Nothing fancy here, just `Layer > Export > Save Features As... > CSV`. The key thing is that the geometry (lat/lon) gets written into the CSV columns so you can reference it later if needed. ## Loading and Querying in R Once you have a CSV, R makes quick work of filtering. I have been using `data.table::fread` because it is fast and handles messy field data well: ```r library(data.table) library(stringr) obs <- fread("warbler_observations.csv") ``` From there, say you want to find all observations of paired females. The description fields in these KML exports tend to be free-text, so pattern matching is the way to go. You can use `stringr::str_detect` for this: ```r paired_females <- obs[str_detect(Description, "paired") & str_detect(Description, "female"), ] ``` Or if you prefer staying within `data.table` syntax, the `%like%` operator works: ```r paired_females <- obs[Description %like% "paired" & Description %like% "female"] ``` Both approaches get you the same result. I tend to reach for `str_detect` when the patterns get more complex (regex support is cleaner), but `%like%` is perfectly fine for simple substring matches. ## Writing Filtered Results Once you have your subset, writing it back out is straightforward: ```r library(readr) write_csv(paired_females, "paired_females_obs.csv") ``` That filtered CSV can go right back into QGIS or ArcGIS for mapping, or into whatever downstream analysis you are running. In our case, we were feeding these subsets into territory mapping workflows -- more on that in the [succession warblers and GIS post](/blog/succession-warblers-and-gis) from later that spring. ## Why This Matters When you are dealing with field data at this scale -- hundreds of GPS points with free-text attribute descriptions -- you need a pipeline that lets you slice the data quickly without manually scrolling through spreadsheets. The QGIS-to-R workflow here is simple but it saved a lot of time during the research season. It also made it easy to generate specific datasets for the [AAG poster](/aag) we put together. The full step-by-step tutorial is below if you want to follow along with your own KML data. Cheers, -- Jess

Musings On Chapel Language and Parallel Processing

Wed, 20 Feb 2019 00:00:00 GMT

View below the readme mirror from my Github repo. Scroll down for my Python3 evaluation script. ....Or visit the page directly: [https://github.com/Jesssullivan/ChapelTests ](https://github.com/Jesssullivan/ChapelTests) # ChapelTests Investigating modern concurrent programming ideas with Chapel Language and Python 3 **See here for dupe detection:** [/FileChecking-with-Chapel](https://github.com/Jesssullivan/ChapelTests/tree/master/FileChecking-with-Chapel) **Iterating through all files for custom tags / syntax:** [/GenericTagIterator](https://github.com/Jesssullivan/ChapelTests/tree/master/GenericTagIterator) added 9/14/19: The thinking here is one could write a global, shorthand / tag-based note manager making use of an efficient tag gathering tool like the example here. Gone are the days of actually needing a note manager- when the need presents itself, one could just add a calendar item, todo, etc with a global tag syntax. The test uses $D for date: `$D 09/14/19` ``` // Chapel-Language // // non-annotated file @ /GenericTagIterator/nScan.chpl // use FileSystem; use IO; use Time; config const V : bool=true; // verbose logging, currently default! module charMatches { var dates = {("")}; } // var sync1$ : sync bool=true; not used in example- TODO: add sync$ var back in!! proc charCheck(aFile, ref choice, sep, sepRange) { // note, reference argument (ref choice) is needed if using Chapel structure "module.domain" try { var line : string; var tmp = openreader(aFile); while(tmp.readline(line)) { if line.find(sep) > 0 { choice += line.split(sep)[sepRange]; if V then writeln('adding '+ sep + ' ' + line.split(sep)[sepRange]); } } tmp.close(); } catch { if V then writeln("caught err"); } } coforall folder in walkdirs('check/') { for file in findfiles(folder) { charCheck(file, charMatches.dates, '$D ', 1..8); } } ``` # Get some Chapel: In a (bash) shell, install Chapel: Mac or Linux here, others refer to: https://chapel-lang.org/docs/usingchapel/QUICKSTART.html ```bash # For Linux bash: git clone https://github.com/chapel-lang/chapel tar xzf chapel-1.18.0.tar.gz cd chapel-1.18.0 source util/setchplenv.bash make make check #For Mac OSX bash: # Just use homebrew brew install chapel # :) ``` # Get atom editor for Chapel Language support: ```bash #Linux bash: cd sudo apt-get install atom apm install language-chapel # atom [yourfile.chpl] # open/make a file with atom # Mac OSX (download): # https://github.com/atom/atom # bash for Chapel language support apm install language-chapel # atom [yourfile.chpl] # open/make a file with atom ``` # Using the Chapel compiler To compile with Chapel: ``` chpl MyFile.chpl # chpl command is self sufficient # chpl one file class into another: chpl -M classFile runFile.chpl # to run a Chapel file: ./runFile ``` # Now Some Python3 Evaluation: ``` # Ajacent to compiled FileCheck.chpl binary: python3 Timer_FileCheck.py ``` Timer_FileCheck.py will loop FileCheck and find the average times it takes to complete, with a variety of additional arguments to toggle parallel and serial operation. The iterations are: ``` ListOptions = [Default, Serial_SE, Serial_SP, Serial_SE_SP] ``` * Default - full parallel * Serial evaluation (--SE) but parallel domain creation * Serial domain creation (--SP) but parallel evaluation * Full serial (--SE --SP) # [](https://github.com/Jesssullivan/ChapelTests/tree/master/ChapelTesting-Python3#output-is-saved-as-time_filecheck_resultstxt)Output is saved as Time_FileCheck_Results.txt * Output is also logged after each of the (default 10) loops. The idea is to evaluate a "--flag" -in this case, Serial or Parallel in FileCheck.chpl- to see of there are time benefits to parallel processing. In this case, there really are not any, because that program relies mostly on disk speed. ## Evaluation Test: ```python # Time_FileCheck.py # # A WIP by Jess Sullivan # # evaluate average run speed of both serial and parallel versions # of FileCheck.chpl -- NOTE: coforall is used in both BY DEFAULT. # This is to bypass the slow findfiles() method by dividing file searches # by number of directories. import subprocess import time File = "./FileCheck" # chapel to run # default false, use for evaluation SE = "--SE=true" # default false, use for evaluation SP = "--SP=true" # no coforall looping anywhere # default true, make it false: R = "--R=false" # do not let chapel compile a report per run # default true, make it false: T = "--T=false" # no internal chapel timers # default true, make it false: V = "--V=false" # use verbose logging? # default is false bug = "--debug=false" Default = (File, R, T, V, bug) # default parallel operation Serial_SE = (File, R, T, V, bug, SE) Serial_SP = (File, R, T, V, bug, SP) Serial_SE_SP = (File, R, T, V, bug, SP, SE) ListOptions = [Default, Serial_SE, Serial_SP, Serial_SE_SP] loopNum = 10 # iterations of each runTime for an average speed. # setup output file file = open("Time_FileCheck_Results.txt", "w") file.write(str('eval ' + str(loopNum) + ' loops for ' + str(len(ListOptions)) + ' FileCheck Options' + "\n\\")) def iterateWithArgs(loops, args, runTime): for l in range(loops): start = time.time() subprocess.run(args) end = time.time() runTime.append(end-start) for option in ListOptions: runTime = [] iterateWithArgs(loopNum, option, runTime) file.write("average runTime for FileCheck with "+ str(option) + "options is " + "\n\\") file.write(str(sum(runTime) / loopNum) +"\n\\") print("average runTime for FileCheck with " + str(option) + " options is " + "\n\\") print(str(sum(runTime) / loopNum) +"\n\\") file.close() ```

Deploy Shiny R apps along Node.JS

Sun, 25 Nov 2018 00:00:00 GMT

**Find the tools in action on Heroku as a node.js app!** [https://kml-tools.herokuapp.com/](https://kml-tools.herokuapp.com/) **See the code on GitHub:** [https://github.com/Jesssullivan/Shiny-Apps](https://github.com/Jesssullivan/Shiny-Apps) After many iterations of ideas regarding deployment for a few research Shiny R apps, I am glad to say the current web-only setup is 100% free and simple to adapt. I thought I'd go through some of the Node.JS bits I have been fussing with. **The Current one:** Heroku has a free tier for node.js apps. See the pricing and limitations here: [https://www.heroku.com/pricing](https://www.heroku.com/pricing) as far as I can tell, there is little reason to read too far into a free plan; they don’t have my credit card, and thy seem to convert enough folks to paid customers to be nice enough to offer a free something to everyone. Shiny apps- [https://www.shinyapps.io/](https://www.shinyapps.io/)\- works straight from RStudio. They have a free plan. Similar to Heroku, I can care too much about limitations as it is completely free. The reasons to use Node.JS (even if it just a jade/html wrapper) are numerous, though may not be completely obvious. If nothing else, Heroku will serve it for free…. Using node is nice because you get all the web-layout-ux-ui stacks of stuff if you need them. Clearly, I have not gone to many lengths to do that, but it is there. Another big one is using node.js with Electron. [https://electronjs.org/](https://electronjs.org/) The idea is a desktop app framework serves up your node app to itself, via the chromium. I had a bit of a foray with Electron- the node execa `npm install execa` package let me launch a shiny server from electron, wait a moment, then load a node/browser app that acts as a interface to the shiny process. While this _mostly_ worked, it is definitely overkill for my shiny stuff. Good to have as a tool though. -Jess

Recycled Personal \"Cloud Computing\" under NAT

Mon, 05 Nov 2018 00:00:00 GMT

As many may intuit, I like the AWS ecosystem; it is easy to navigate and usually just works. ...However- more than 1000 dollars later, I no longer use AWS for most things.... 🙁 **** **My goals:** Selective sync: I need a unsync function for projects and files due to the tiny 256 SSD on my laptop (odrive is great, just not perfect for cloud computing. Shared file system: access files from Windows and OSX, locally and remote Server must be headless, rebootable, and work remotely from under a heavy enterprise NAT (College) Needs more than 8gb ram Runs windows desktop remotely for gis applications, (OSX on my laptop) Have as much shared file space as possible: 12TB+ Server: recycled, remote, works-under-enterprise-NAT: **Recycled Dell 3010 with i5:**[ https://www.plymouth.edu/webapp/itsurplus/](https://www.plymouth.edu/webapp/itsurplus/) - Cost: $75 (+ ~$200 in windows 10 pro, inevitable license expense) - free spare 16gb ram laying around, local SSD and 2TB HDD upgrades - Does Microsoft-specific GIS bidding, can leave running without hampering productivity **Resilio (bittorrent) Selective sync:** [https://www.resilio.com/individuals/](https://www.resilio.com/individuals/) - Cost: $60 - p2p Data management for remote storage + desktop - Manages school NAT and port restrictions well (remote access via relay server) **Drobo 5c:** Attached and syncs to 10TB additional drobo raid storage, repurposed for NTFS * Instead of EBS (or S3) What I see: front end- **Jump VNC Fluid service:** [https://jumpdesktop.com/](https://jumpdesktop.com/) - Cost: ~$30 - Super efficient Fluid protocol, clients include chrome OS and IOS, (with mouse support!) - Manages heavy NAT and port restrictions well - GUI for everything, no tunneling around a CLI * Instead of Workspaces, EC2 **Jetbrains development suite:** [https://www.jetbrains.com/](https://www.jetbrains.com/) (OSX) - Cost: FREE as a verified GitHub student user. - PyCharm IDE, Webstorm IDE * Instead of Cloud 9 **Total (** extra**) spent:** ~$165 (Example: my AWS bill for only October was $262) -Jess

Github UPDATE 9.12.18: Shiny Apps

Wed, 12 Sep 2018 00:00:00 GMT

**View the tools here:** http://kml.jessdev.org Three of my KML tools are now stable and in Github. These are actually displayed via the static site generator Hugo (read about the[ Hugo CLI here](https://gohugo.io/commands/)), which is sitting in the shiny server (port 3838) next to the apps. Messy, but it will do for now. https://github.com/Jesssullivan/Shiny-Apps -Jess

Quick fix: 254 character limit in ESRI Story Map?

Wed, 12 Sep 2018 00:00:00 GMT

https://gis.stackexchange.com/questions/75092/maximum-length-of-text-fields-in-shapefile-and-geodatabase-formats https://en.wikipedia.org/wiki/GeoJSON https://gis.stackexchange.com/questions/92885/ogr2ogr-converting-kml-to-geojson If you happened to be working with.... KML data (or any data with large description strings) and transitioning it into the ESRI Story Map toolset, there is a very good chance you hit the the dBase 254 character length limit with the ESRI Shapefile upload. Shapefiles are always a terrible idea. the solution: with GDAL or QGIS (alright, even in ArcMap), one can use GeoJSON as an output format AND import into the story map system- with complete long description strings! **QGIS:** Merge vector layers -> save to file -> GeoJSON **arcpy:** import arcpy import os arcpy.env.workspace = "/desktop/arcmapstuff" arcpy.FeaturesToJSON_conversion(os.path.join("outgdb.gdb", "myfeatures"), "output.json") **GDAL: **< ogr2ogr -f GeoJSON output.json input.kml

Early Hardware and Maker Projects

Sat, 01 Sep 2018 00:00:00 GMT

Looking back at 2017 and 2018, I was juggling a lot of threads at once. I had just started as the First Fellow at the D&M Makerspace at Plymouth State University, I was picking up GIS contracting work, and I was starting to get curious about machine learning. But the through-line for those two years was really the hands-on hardware stuff -- startups, shop projects, and a whole lot of trial and error at the bench. Here is a photo roundup of some of the projects from that stretch. ## Dover Micro In 2017 I spent time working with Dover Micro, a startup focused on FPGA-based hardware. We were prototyping board layouts and working through mockups for what the product might actually look like in someone's hands. Most of the early work was about figuring out form factor and packaging constraints before committing to a real board spin. Learned a lot about how much of hardware product development is just iterating on physical layouts before you ever get to firmware. ![Dover Micro FPGA mockup](/images/posts/dovermicro_fpga_mockup.webp) ## Adaptive Motorsport In 2018 I moved on to another startup, Adaptive Motorsport. The focus here was adaptive vehicle controls -- making motorsport more accessible. The demo unit we built was a solid proof of concept, and it was rewarding to work on something where the engineering had a direct impact on who could participate in the sport. ![Adaptive Motorsport demo unit](/images/posts/adaptive_motorsport_demo.webp) ## DIY CNC Machine I built a CNC machine from scratch during this period. This was a natural extension of the MPCNC work I had been doing earlier -- bigger frame, beefier spindle, and a lot of time squaring everything up. Having a working CNC in the shop opened up a whole category of projects that would have been impractical by hand. It became one of the most-used tools in the makerspace. ![DIY CNC machine](/images/posts/diy_cnc.webp) ## PWM Foam Cutter Hot wire foam cutting is one of those techniques that sounds simple until you try to get clean, repeatable cuts. I built a PWM-controlled foam cutter that let me dial in the wire temperature precisely. Too hot and the foam melts unevenly; too cold and you are just dragging the wire through. PWM control made it possible to find the sweet spot and stay there. ![PWM foam cutter](/images/posts/pwm_foam_cutter.webp) ## Thixotropic Sonicator This one is a bit niche. I built an ultrasonic sonicator for working with thixotropic materials -- fluids that change viscosity when agitated. The device uses ultrasonic energy to temporarily reduce viscosity for processing. It was a fun intersection of electronics and materials science, and I got to learn a lot about piezoelectric transducers along the way. ![Thixotropic Sonicator](/images/posts/ThixotropicSonicator.webp) ## Heavy Yoke Fabrication Sometimes a project is just about making a solid, functional piece of metal. The heavy yoke was a fabrication job that required careful layout, welding, and finishing. Nothing glamorous, but there is a real satisfaction in building something load-bearing and having it come out square and strong. ![Heavy yoke fabrication](/images/posts/heavy_yoke.webp) ## Low Friction Feeder The low friction feeder was designed to move material smoothly without binding or jamming. Getting the geometry right on something like this is all about reducing contact points and choosing the right bearing surfaces. It went through several iterations before it fed reliably, which is pretty much how every mechanical project goes. ![Low friction feeder](/images/posts/low_friction_feeder.webp) ## Looking Ahead: COVID PPE A couple years after most of these projects, the COVID-19 pandemic hit and the makerspace pivoted hard to producing PPE for New Hampshire EMS workers. The fabrication skills and shop infrastructure from all this earlier work turned out to be directly useful when it mattered most. But that is a story for another post. ![COVID PPE face shields for NH EMS](/images/posts/NH_EMS_shield.webp) --- That stretch at the D&M Makerspace was formative. I was learning to move between disciplines -- electronics, mechanical fabrication, software, materials -- and figuring out that the interesting problems usually live at the boundaries between them. The startup work taught me how to ship under constraints, and the shop projects taught me how to build things that actually work.

Shiny App-specific Repo

Sat, 28 Jul 2018 00:00:00 GMT

**New Shiny App specific Repo now live...** https://github.com/Jesssullivan/Shiny-Apps***This space once held an image. The WordPress CDN has since gone quiet.*** With KML Search and Convert now fully functional (along with the tiny app "clean") , live shiny apps of mine now have a repo of their own. Check it out!*Image from the original post is no longer available -- nothing on the internet is forever.* *This image has been lost to time. The original was hosted on WordPress.* -Jess

New App: KML Search and Convert

Fri, 27 Jul 2018 00:00:00 GMT

Written in R; using GDAL/EXPAT libraries on Ubuntu and hosted with AWS EC2. **New App: KML Search and Convert** Here is an simple (beta) app of mine that converts KML files into Excel-friendly CSV documents. It also has a search function, so you can download a subset of data that contains keywords. 🙂 The files will soon be available in Github. I'm still working on a progress indicator; it currently lets you download before it is done processing. Know a completely processed file is titled with "kml2csv_.csv". ...YMMV. xD

GDAL for R Server on Ubuntu - KML Spatial Libraries and More

Mon, 23 Jul 2018 00:00:00 GMT

**GDAL for R Server on Red Hat Xenial Ubuntu - KML Spatial Libraries and More** If you made the (possible mistake) of running with a barebones Red Hat Linux instance, you will find it is missing many things you may want in R. I rely on GDAL (the definitive Geospatial Data Abstraction Library) on my local OSX R setup, and want it on my server too. GDAL contains many libraries you need to work with KML, RGDAL, and other spatial packages. It is massive and usually take a long time to sort out on any machine. These notes assume you are already involved with a R server (usually port 8787 in a browser). I am running mine from an EC2 instance with AWS. ! Note this is a fresh server install, using Ubuntu; I messed up my original ones while trying to configure GDAL against conflicting packages. If you are creating a new one, opt for at least a T2 medium (or go bigger) and find the latest Ubuntu server AMI. For these instructions, you want an OS that is as generic as possible. On Github: https://github.com/Jesssullivan/rhel-bits **From Bash:** # SSH into the EC2 instance: (here is the syntax just in case) #ssh -i "/Users/YourSSHKey.pem" ec2-user@yourAWSinstance.amazonaws.com sudo su - apt-get update apt-get upgrade nano /etc/apt/sources.list #enter as a new line at the bottom of the doc: deb https://cloud.r-project.org/bin/linux/ubuntu xenial/ #exit nano wget https://raw.githubusercontent.com/Jesssullivan/rhel-bits/master/xen-conf.sh chmod 777 xen-conf.sh ./xen-conf.sh **Or...** From SSH: # SSH into the EC2 instance: (here is the syntax just in case) ssh -i "/Users/YourSSHKey.pem" ec2-user@yourAWSinstance.amazonaws.com # if you can, become root and make some global users- these will be your access to # RStudio Server and shiny too! sudo su – adduser # Follow the following prompts carefully to create the user apt-get update nano /etc/apt/sources.list # enter as a new line at the bottom of the doc: deb https://cloud.r-project.org/bin/linux/ubuntu xenial/ # exit nano # Start, or try [bash](https://github.com/Jesssullivan/rhel-bits): apt-get install r-base apt-get install r-base-dev apt-get update apt-get upgrade wget http://download.osgeo.org/gdal/2.3.1/gdal-2.3.1.tar.gz tar xvf gdal-2.3.1.tar.gz cd gdal-2.3.1 # begin making GDAL: this all takes a while ./configure [if your need proper kml support (like me), search on configuring with expat or libkml. There are many more options for configuration based on other packages that can go here, and this is the step to get them in order...] sudo make sudo make install cd # Try entering R now and check the version! # Start installing RStudio server and Shiny apt-get update apt-get upgrade sudo apt-get install gdebi-core wget https://download2.rstudio.org/rstudio-server-1.1.456-amd64.deb sudo gdebi rstudio-server-1.1.456-amd64.deb # Enter R or go to the graphical R Studio installation in your browser R # Authenticate if using the graphical interface using the usr:pwd you defined earlier # this will take a long time install.packages(“rgdal”) # Note any errors carefully! Then: install.packages(“dplyr”) install.packages(c("data.table", "tidyverse”, “shiny”) # etc ##### Well, there you have it! -Jess **Extras:** **##Later, ONLY IF you NEED Anaconda, FYI:** # Get Anaconda: this is a large package manager, and is could be used for patching up missing # dependencies: #Use "ls" followed by rm -r (fill in with ls results) to remove conflicting conda # installers if you have any issue there, I am starting fresh: mkdir binconda # *making a weak attempt at sandboxing the massive new package manager installation* cd binconda wget [http](https://repo.continuum.io/archive/Anaconda2-5.2.0-Linux-x86_64.sh)[://repo.continuum.io/archive/Anaconda2-4.3.0-Linux-x86_64.sh](https://repo.continuum.io/archive/Anaconda2-5.2.0-Linux-x86_64.sh) # install and follow the prompts bash Anaconda2-5.2.0-Linux-x86_64.sh # Close the terminal window completely and start a new one, and ssh back to where you left # off. Conda install requires this. # open and SSH back into your instance. You should now have either additional flexibility in # either patching holes in dependencies, or created some large holes in your server. YMMV. ### Done **Red Hat stuff:** Follow these AWS instructions if you are doing something else: https://aws.amazon.com/blogs/big-data/running-r-on-aws/ See my notes on this here: https://transscendsurvival.org/2018/03/08/how-to-make-a-aws-r-server/ and notes on Shiny server: https://transscendsurvival.org/2018/07/16/deploy-a-shiny-web-app-in-r-using-aws-ec2-red-hat/ GDAL on Red Hat:- Existing threads on this: https://gis.stackexchange.com/questions/120101/building-gdal-with-libkml-support/120103#120103 This is a nice short thread about building from source: https://gis.stackexchange.com/questions/263495/how-to-install-gdal-on-centos-7-4 neat RPM package finding tool, just in case: https://rpmfind.net/linux/rpm2html/ Info on the LIBKML driver if you end up with issues there: http://www.gdal.org/drv_libkml.html I hope this is useful- GDAL is important and best to set it up early. It will be a pain, but so is losing work while trying to patch it in later. xD -Jess

INFO: Deploy a Shiny web app in R using AWS (EC2 Red Hat)

Mon, 16 Jul 2018 00:00:00 GMT

**Info on deploying a Shiny web app in R using AWS (EC2 Redhat)** As [a follow-up to my post on how to create an AWS RStudio server](/blog/how-to-make-a-aws-r-server/), the next logical step is to host some useful apps you created in R for people to use. A common way to do this is the R-specific tool Shiny, which is built in to RStudio. Learning the syntax to convert R code into a Shiny app is rather subtle, and can be hard. I plan to do a more thorough demo on this- particularly the use of the $ symbol, as in “input$output”- later. 🙂 It turns out hosting a Shiny Web app provides a large number of opportunities for things to go wrong…. I will share what worked for me. All of this info is accessed via SSH, to the server running Shiny and RStudio. I am using the AWS “Linux 2” AMI, which is based on the Red Hat OS. For reference, here is some extremely important Red Hat CLI language worth being familiar with and debugging: “**sudo yum install** ” and “**wget** ” are for fetching and installing things like shiny. Don’t bother with instructions that include “apt-get install”, as they are for a different Linux OS! “**sudo chmod -R 777** ” is how you change your directory permissions for read, write, and execute (all of those enabled). This is handy if your server disconnecting when the app tries to run something- it is a simple fix to a problem not always evident in the logs. The default root folder from which shiny apps are hosted and run is “**/srv/shiny-server** ” (or just “**/srv** ” to be safe). “**nano /var/log/shiny-server.log** ” is the location of current shiny logs. “**sudo stop shiny-server** ” followed by “**sudo start shiny-server** ” is the best way to restart the server- “sudo restart shiny-server” is not a sure bet on any other process. It is true, other tools like a node.js server or nginx could impact the success of Shiny- If you think nginx is a problem, “**cd /ect/nginx** ” followed by “**ls** ” will get you in the right direction. Others have cited problems with Red Hat not including the directories and files at “/etc/nginx/sites-available”. You do not need these directories. (though they are probably important for other things). “**sudo rm -r** ” is a good way to destroy things, like a mangled R studio installation. Remember, it is easy enough to start again fresh! 🙂 “**sudo nano /etc/shiny-server/shiny-server.conf** ” is how to access the config file for Shiny. The fresh install version I used _did not work_! There will be lots of excess in that file, much of which can causes issues in a bare-bones setup like mine. One important key is to ensure Shiny is using a root user- see my example file below. I am the root user here (jess)- change that to mirror- at least for the beginning- the user defined as root in your AWS installation. See my notes HERE on that- that is defined in the advanced settings of the EC2 instance. BEGIN CONFIG FILE: *(the download link for this config file has been lost with the WordPress site)* *Download is properly indented ` # Define user: this should be the same user as the AWS root user! # run_as jess; # # Define port and where the home (/) directory is # Define site_dir/log_dir - these are the defaults # server{ listen 3838; location / { site_dir /srv/shiny-server; log_dir /var/log/shiny-server; directory_index on; } } ` END CONFIG FILE Well, the proof is in the pudding. At least for now, you can access a basic app I made that cleans csv field data files that where entered into excel by hand. They start full of missing fields and have a weird two-column setup for distance- the app cleans all these issues and returns a 4 column (from 5 column) csv. Download the test file here: *the test CSV file is no longer available -- it was hosted on the old WordPress site* And access the app here: [ Basic Shiny app on AWS!](http://ec2-34-228-197-7.compute-1.amazonaws.com:3838/sample-apps/Clean/) Below is an iFrame into the app, just to show how very basic it is. Give it a go! -Jess

Off-Grid File Sharing with SAMBA / GL.iNet

Mon, 25 Jun 2018 00:00:00 GMT

_Note: SMB / SharePoint is surely better with a proper server/computer. A Raspberry Pi running OpenMediaVault (Debian) is a more common and robust option (still 5v low power)._ _If you are actually in an "it must done in OpenWRT" scenario, Click Here for my Samba config file:*the config file download, once hosted on WordPress, is no longer available* and see below. Also, please use a NTFS or EX4 format. 🙂_ ...While my sharing method wasn't actually adopted by others, I still think it is good to know! -Jess

How To Generate convex hull polygons and centroids from KML

Sun, 24 Jun 2018 00:00:00 GMT

Click here for the PDF: *The PDF that was hosted here has been lost with the WordPress site.*

How to Query KML point data as CSV using QGIS and R

Sun, 10 Jun 2018 00:00:00 GMT

How to Query KML point data as CSV using QGIS and R ![](/images/posts/Screen-Shot-2018-06-10-at-8.28.16-AM.png) Here you can see more than 800 points, each describing an observation of an individual bird. This data is in the form of KML, a sort of XML document from Google for spatial data. I want to know which points have “pair” or “female” in the description text nodes using R. This way, I can quickly make and update a .csv in Excel of only the paired birds (based on color bands). Even if there was a description string search function in Google Earth Pro (or other organization-centric GIS/waypoint software), this method is more robust, as I can work immediately with the output as a data frame in R, rather than a list of results. First, open an instance of QGIS. I am running ~2.8 on OSX. Add a vector layer of your KML. ![](/images/posts/Screen-Shot-2018-06-10-at-8.28.54-AM.png) “Command-A” in the point dialog to select all before import! ![](/images/posts/Screen-Shot-2018-06-10-at-8.30.07-AM.png) Next, under “Vector”, select “Merge vector layers” via Data Management Tools. Select CSV and elect to save the file instead of use a temporary/scratch file (this is a common error). **Open your csv in Excel for verification!** ![Excel CSV verification](/images/posts/Screen-Shot-2018-06-10-at-8.32.27-AM.png) **The R bit:** ```r _# query for paired birds #EDIT: Libraries library(data.table) library(tidyverse)_ data <- data.frame(fread("Bird_CSV.csv")) pair_rows <- contains("pair", vars = data$description) fem_rows <- contains("fem", vars = data$description) result <- combine(pair_rows, fem_rows) result <- data[result,] write_csv(result, "Paired_Birds.csv") ``` **Tada!** ![](/images/posts/Screen-Shot-2018-06-10-at-5.55.15-PM.png) **-Jess**

Research Year Two: Three Photos

Thu, 31 May 2018 00:00:00 GMT

[![](/images/posts/IMG_8381-Edit.jpg)](/images/posts/IMG_8381-Edit.jpg) Male Common Yellowthroat Warbler The field season has officially started in Northern NH! Male Common Yellowthroat warbler (COYE): This fellow is defending a small territory in a patch of open thicket. These warblers rely on early succession forest- patches of substrate that haven't really grown in yet- to build cryptic, ground-level nests. They develop complex systems to divert/confuse predators away from their nests. Female Black-throated Blue Warbler (BTBW): I was lucky to see this female. She is paired with a male who defends a large mature forest territory. They have quite a few BTBW neighbors, which makes for a lot of skirmishes among the males over land. The females are often silent and move very fast... Male Mourning Warbler (MOWA): This is a rare bird here. Even more amazing, it is defending a territory in our research site- and trying to chase out a male COYE while doing so. The two species "share" resources, which means thy can't stand each other. 🙂 Each time the male COYE sings near the MOWA, it gets berated and chased away- and vice versa. It appears the COYE isn't budging either, probably because it hasn't had this domestic, neighborly problem before. -Jess [![](/images/posts/IMG_9176-Edit-2.jpg)](/images/posts/IMG_9176-Edit-2.jpg) Female Black-throated Blue Warbler [![](/images/posts/IMG_8895-Edit-2.jpg)](/images/posts/IMG_8895-Edit-2.jpg) Male Mourning Warbler

Solar upgrades!

Thu, 31 May 2018 00:00:00 GMT

Solar upgrades! Incredibly, the hut we are working from actually had another solar panel just laying around. 🙂 This 50w square panel had a junction box with MC4 connectors, the standard for small scale solar installations. As I was unsure how to know when we are running low on electricity reserves, I decided to make some adjustments. [![](/images/posts/IMG_3355-1.jpg)](/images/posts/IMG_3355-1.jpg) Additional 50w solar panel (Everything is still solder, hot glue, alligator clips, and zip-ties I’m afraid…) I traded my NEMA / USA two-prong connection with two MC4 splitters, such that both panels can run in parallel (into a standard USA 110v extension cord that goes into our hut). This way we should make well over one of the two 35ah batteries-worth of electricity a day. [![](/images/posts/IMG_3356.jpg)](/images/posts/IMG_3356.jpg) Dual MC4 splitters to extension cord I also added a cheap 12v battery level indicator. It is not very accurate (as it fluctuates with solar input) but it does give us some insight about how much "juice" we have available. (I also wired and glued the remote-on switch to the back of the input for stability.) [![](/images/posts/IMG_3357.jpg)](/images/posts/IMG_3357.jpg) Added battery indicator and button 🙂 -Jess

Gathering point data using Compass 55 on Apple iOS

Mon, 28 May 2018 00:00:00 GMT

Keeping track of birds is tricky! Attached is our team's workflow with Compass 55. From the Kml, we go into Google Earth Pro - ArcGIS Desktop (arcmap). QGIS is sometimes used too. Cheers, -Jess

840 Watts of Solar Power!

Thu, 24 May 2018 00:00:00 GMT

Equipment used: [Inverter/PWM Controller: http://a.co/fdl9YzI](http://a.co/fdl9YzI) [2x 35ah Batteries: http://a.co/5JBIxTC](http://a.co/5JBIxTC) [100w solar panel: http://a.co/5JBIxTC](http://a.co/5JBIxTC) We need power! While doing bird research in the wilds of northern NH, it became evident we needed electricity to power computers, big cameras, and phones/GPS units. Below is a table of the system and our expected electricity needs: System | Solar 100w | 35ah universal (x2) | | ---|---|---|---|--- Ah per day: | 33.33333333 | 35 | TOTAL Ah Reserve: | 70 V | 12 | 12 | Parallel wiring: | 12v Wh in: | 400 | 420 | TOTAL Wh Reserve: | 840 W | 100 | | | Cost | $105.00 | $64.00 | | ah/$ | | 2 | | Sun Hour / Multiplier | 4 | 2 | | Need/Day | Wh | multiplier | consump. in Wh = | 259.36 Computer | 100 | 2.5 | 250 | iPhone | 1.7 | 2 | 3.4 | AAs | 11.2 | 0.3 | 3.36 | Camera | 2.6 | 1 | 2.6 | **The milk crate system below can charge a 100 watt MacBook Pro around 8-9 times from being completely empty.** **Remember: V\*A=W, W/V=A, and Watts over time is Wh.** -Jess [![](/images/posts/IMG_3311-1.jpg)](/images/posts/IMG_3311-1.jpg) +/- relates to size of standard prongs [![](/images/posts/IMG_3312-1.jpg)](/images/posts/IMG_3312-1.jpg) [![](/images/posts/IMG_3313-1.jpg)](/images/posts/IMG_3313-1.jpg) Parallel maintains 12v but doubles Ah. (Series would go to 24v at 35ah)

Birding at Plumb Island!

Wed, 23 May 2018 00:00:00 GMT

While somewhat of a historical entry: Below are some of the few photos I took while birding on may 5, 2018- GLOBAL BIG DAY! Be sure to read about it here: https://ebird.org/news/global-big-day-5-may-2018 Myself and my father contributed 64 species, including the below Northern waterthrush, Female Rose-breasted Grosbeak, Northern Parula, and Common Yellowthroat warbler. -Jess [![](/images/posts/IMG_7092-Edit.jpg)](/images/posts/IMG_7092-Edit.jpg) [![](/images/posts/IMG_7227-Edit.jpg)](/images/posts/IMG_7227-Edit.jpg) [![](/images/posts/IMG_7265-Edit.jpg)](/images/posts/IMG_7265-Edit.jpg) [![](/images/posts/IMG_7320-Edit.jpg)](/images/posts/IMG_7320-Edit.jpg)

Signs of Thoughts of Possible Spring?

Wed, 18 Apr 2018 00:00:00 GMT

Spring is (possibly) coming people! 🙂 ![img_6256-edit](/images/posts/IMG_6256-Edit.jpg) Scituate, MA - Osprey ![img_6256-edit](/images/posts/IMG_6256-Edit.jpg) Scituate, MA - Killdeer *Some images from the original WordPress post are no longer available.* Plymouth, NH - (FOY photo for me) Hermit Thrush *Some images from the original WordPress post are no longer available.* Scituate, MA - Brant *Some images from the original WordPress post are no longer available.* Plymouth, NH - Savannah Sparrow

Intro to the AWS Cloud 9 IDE

Tue, 10 Apr 2018 00:00:00 GMT

The Cloud 9 IDE is the fastest way I have come up with to develop web-based or otherwise "connected" programs. Because it lives on a Linux-based EC2 server on AWS, running different node, html, etc programs that rely on a network system just work- it is all already on a network anyway. 🙂 There is no downtime trying to figure out your WAMP, MAMP, Apache, or localhost situation. Similarly, other network programs work just as well- I am running a MySQL server over here (RDS), storage over there (S3), and have various bits in Github and locally. Instead of configuring local editors, permissions, and computer ports and whatnot, you are modifying the VPC security policies and IAM groups- though generally, it just works. **Getting going: The only prerequisite is you have an AWS account.** Students: get $40 EC2 dollars below: https://aws.amazon.com/education/awseducate/ *The image that was here didn't survive the WordPress migration.* Open the cloud 9 tab under services. Setup is very fast- just know if others are going to be editing to, understand the [IAM policies](https://aws.amazon.com/iam/) and what [VPC settings](https://aws.amazon.com/vpc/) you actually want. Know this ideally a browser-based service; I have tried to come up with a reason a SSH connection would be better and didn't get any where. For one person, micro is fine. Know these virtual "RAMs" and "CPUs" are generous.... The default network settings are set up for you. This follows good practice for one person; more than that (or if you are perhaps a far-travelling person) note these settings. They are always editable under the VPC and EC2 instance tabs. *Some images from the original WordPress post are no longer available.* That's it! Other use things to know: This is a linux machine maintained by Amazon. Packages you think should work and be up to date (arguably like any other linux machine I guess...) may not be. Check your basics like the NPM installer and versions of what your going to be working on, it very well may be different than what you are used to. *Some images from the original WordPress post are no longer available.* **In the editor:** You have two panels of workspace in the middle- shown is node and HTML. Everything is managed by tabs- all windows can have as much stuff as you want this way. Below there is a "runner" (shown with all the default options!) and a terminal window. Off to the left is a generic file manager. I hope this is useful, it sure is great for me. -Jess

Using ESRI ArcGIS / ArcMap in the AWS Cloud

Thu, 05 Apr 2018 00:00:00 GMT

Selling AWS to... myself 🙂 Why struggle with underpowered local machines and VMs or watered-down web platforms for heavy lifting, learning and work? In addition to using ESRI software on mac computers, I am a big fan of the AWS WorkSpaces service (in addition to all their other developer tools, some of which are map-relevant: RDS for SQL and EC2 Redhat servers for data management for example ). Basically, for between ~$20 and ~$60 a month (Max, and not factoring in EDU discounts!), a user gets to use a well-oiled remote desktop. You can download and license desktop apps like ArcMap and GIS products, file managers, and more from any computer connected to the internet. This service is not very savvy; you make/receive a password and log right in. A big plus here of course is the Workspaces Application Manager (WAM); small sets of licenses can be administered in the same way desktops would, with extra easiness due to the "they are already really the same cloud thing any way". Another plus is any client- netbook, macbook, VM, etc- will work equally well. In this regard it can be a very cheap way to get big data work done on otherwise insufficient machines. Local storage and file systems work well with the client application, with the caveat being network speed. 🙂 https://docs.aws.amazon.com/workspaces/latest/adminguide/amazon-workspaces.html

Using ESRI ArcGIS / ArcMap on Mac OSX: 2 methods

Tue, 03 Apr 2018 00:00:00 GMT

_Edit 07/26/2020:_ [Check out the expanded GIS notes page here!](https://jesssullivan.github.io/GIS_Shortcuts/) **Using ESRI ArcGIS / ArcMap on Macs: 2 methods** I need to run ESRI products on my MacBook Pro. QGIS is always the prefered solution- open source, excellent free plugins, works on mac natively- but in a college / research environment, the only option that supports other people and school machines is ESRI. Despite the annoying bureaucracy and expense of the software, some things are faster (but not better!) in ESRI, like dealing with raster / multiband data. **First, you need a license.** I went about this two ways; My first solution was to buy an ESRI Press textbook on amazon. A 180 day trial for $50- when taken as a college course, this isn't to bad. 🙂 The book is slow and recursive, but a 180 days to play with all the plugins and whistles allows for way deeper learning via the internet. 🙂[![](/images/posts/Screen-Shot-2018-04-03-at-9.36.59-AM.png)](/images/posts/Screen-Shot-2018-04-03-at-9.36.59-AM.png) Do know there is a little-documented limit to the number of license transfers you may perform before getting either lock in or out of your software. I hit this limit, as I was also figuring out my virtual machine situation, which would occasionally need a re-installation. My current solution is “just buy a student license”. $100 per year is less than any adobe situation- so really not that bad. **Now you need a windows ISO.** [https://www.microsoft.com/en-us/software-download/windows10ISO](https://www.microsoft.com/en-us/software-download/windows10ISO) Follow that link for the window 10, 64 bit ISO. YOU DO NOT NEED TO BUY WINDOWS. It will sometimes complain about not having an authentication, but in the months of using windows via VMs, never have I been prohibited to do... anything. When prompted for a license when configuring your VM, click the button that says "I don't have a license". Done. **Option one: VirtualBox VM on a thumbdrive** [https://www.virtualbox.org/wiki/Downloads](https://www.virtualbox.org/wiki/Downloads) \- download for the VM software [http://a.co/4FEYMNY](http://a.co/4FEYMNY), [http://a.co/hanHYl1](http://a.co/hanHYl1) Suitable USBs. the VM will take up most of a 128gb flash drive- ~70 gb just for windows and all the stuff you'll want from a PC. Add ESRI software and allocated space for a cache (where your GIS project works!), bigger is better. Format all drives in disk utility as ExFat! this is important, any other file system either won't fly or could wreak havoc (other FAT based ones may have too small file allocations! I used two drives, a 128 and a 64- this is great because I can store all my work on the 64, so I can easily plug it into other (school) machines running windows ArcMap and keep going, without causing issues with the massive VM in the 128. Installation is straightforward, just install EVERYTHING on the usb drive and it will be fine. 🙂 [![](/images/posts/Screen-Shot-2018-04-03-at-9.52.43-AM.png)](/images/posts/Screen-Shot-2018-04-03-at-9.52.43-AM.png) [![](/images/posts/Screen-Shot-2018-04-03-at-9.52.38-AM.png)](/images/posts/Screen-Shot-2018-04-03-at-9.52.38-AM.png) **Problems** : Stability. Crashes, and python / some other script modules do not work well. This is a problem. ArcAdministrator gets confused about all kinds of things- FWIW, if you are googling to delete the FLEXnet folder to solve authentication file issues, move to option 2 🙂 Speed is down, but actually the ~same speed as our school "super" PCs- (though I happened to know they are essentially glorified "hybrid" VMs too!) . [![](/images/posts/Screen-Shot-2018-04-03-at-9.52.20-AM.png)](/images/posts/Screen-Shot-2018-04-03-at-9.52.20-AM.png) **Option two: OSX Bootcamp** [https://support.apple.com/boot-camp](https://support.apple.com/boot-camp) [https://support.apple.com/en-us/HT201468](https://support.apple.com/en-us/HT201468) This way, you will hit "option/alt" each time you restart/boot your computer to choose from win/osx. This is easy to install, as it is mac and mac = easy. **Big Caveat:** it is much harder to install windows externally (on a usb, etc) from bootcamp. I didn't succeed in my efforts, but there could be a way.... The thing is, it really wants to run everything like a normal intel based PC, with all installations in the usual place. This is good for the mac performance, but terrible for the tiny SSD hard drives we get as mac users. I have a 256gb SSD. I have an average of < 15 gb wiggle room here, and use every cloud service in the book. If you need to manage your cloud storage because of a itsy mac SSD, my solution is still ODrive. [https://www.odrive.com/](https://www.odrive.com/) I use Amazon cloud mostly with odrive, but I use a personal/school OneDrives, Dropboxes, Google, etc. with only the occasional hiccup. Also, all of the AWS tools are great and cheap- EC2, S3, Cloud 9, lambda, RDS.... Great way to do your work outside of your mac via the internet. **Result:** ArcMap and GIS stuff is **blazing** fast on my modest 2015 i5/8gb macbook pro. Comparing a huge, mega ATX+ school computer to my mac on boot camp, I am running large raster filtering operations significantly quicker than other folks doing the same type of work. That is GOOD. 🙂 -Jess

EBPP: My Epic Birding Prediction Project in R

Mon, 19 Mar 2018 00:00:00 GMT

GitHub Link: https://github.com/Jesssullivan/EBPP ## [...Epic Birding Predicting Project: Predicting good birding near you...](https://github.com/Jesssullivan/EBPP) ...Currently under development in R. 🙂 -Jess

Pages, fresh from the book

Tue, 13 Mar 2018 00:00:00 GMT

New Landing Page JessSullivan.co is LIVE! Additionally, my, "Epic Birding Prediction Project" is now on Github. Stay tuned! https://github.com/Jesssullivan/EBPP In the realm of tunes, the dorm soundcloud @ PSU stays fresh with a new Metal Ukulele track- see below.

How to make a AWS R server

Thu, 08 Mar 2018 00:00:00 GMT

When you need an R server and have lots of data to process, AWS is a great way to go. Sign up of the free tier and poke around! **Creating an AWS Rstudio server:** [https://aws.amazon.com/blogs/big-data/running-r-on-aws/](https://aws.amazon.com/blogs/big-data/running-r-on-aws/) \- using both the R snippet (works but the R core bits are NOT present and it will not work yet) and the JSON snippet provided [https://www.rstudio.com/products/rstudio/download-server/](https://www.rstudio.com/products/rstudio/download-server/) \- the suite being installed Follow most of the AWS blog AMI info, with the following items: **AMI** : Amazon Linux 2 (more packages and extras v. standard) * t2.micro (free tier) * IAM policy follows AWS blog JSON snippet * Security Policy contains open inbound ports 22, 8787, 3838 (the latter two for R server specific communication) * Append user, username:password in the blog post’s initial r studio install text (pasted into the “advanced” text box when completing the AMI setup **SSH into the EC2 instance** sudo yum install –y https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm sudo yum-config-manager --enable epel sudo yum repolist wget https://download2.rstudio.org/rstudio-server-rhel-1.1.423-x86_64.rpm sudo yum update -y sudo yum install -y R sudo rstudio-server verify-installation **Access the graphical R server:** In a web browser, tack on “:8787” to the end of the Instance’s public “connect” link. If it doesn’t load a login window (but seems to be trying to connect to something) the security policy is probably being overzealous…….. **Notes on S3-hosted data:** * S3 data is easiest to use if it is set to be public. * There are s3-specific tools for R, accessible as packages from CRAN directly from the R interface * Note data (delimited text at least) hosted in S3 will behave differently than it does locally, e.g. spaces, “na”, “null” need to be “cleaned” in R before use. There we have it! -Jess

Gallery of my \"proper photos\"- NH warbler research

Thu, 27 Jul 2017 00:00:00 GMT

[![](/images/posts/IMG_9803.jpg)](/images/posts/IMG_9803.jpg) [![](/images/posts/IMG_9788.jpg)](/images/posts/IMG_9788.jpg) [![](/images/posts/IMG_9757.jpg)](/images/posts/IMG_9757.jpg) [![](/images/posts/IMG_9435-Edit-Edit.jpg)](/images/posts/IMG_9435-Edit-Edit.jpg) [![](/images/posts/IMG_9468.jpg)](/images/posts/IMG_9468.jpg) [![](/images/posts/IMG_3553-Edit-Edit2.jpg)](/images/posts/IMG_3553-Edit-Edit2.jpg) [![](/images/posts/IMG_2227-Edit.jpg)](/images/posts/IMG_2227-Edit.jpg) [![](/images/posts/IMG_2075-Edit-Edit.jpg)](/images/posts/IMG_2075-Edit-Edit.jpg) [![](/images/posts/IMG_3163-Edit.jpg)](/images/posts/IMG_3163-Edit.jpg) [![](/images/posts/IMG_1350-Edit.jpg)](/images/posts/IMG_1350-Edit.jpg) [![](/images/posts/IMG_1207-Edit.jpg)](/images/posts/IMG_1207-Edit.jpg) [![](/images/posts/IMG_1244-Edit.jpg)](/images/posts/IMG_1244-Edit.jpg) [![](/images/posts/IMG_2366.jpg)](/images/posts/IMG_2366.jpg) [![](/images/posts/IMG_3299-Edit.jpg)](/images/posts/IMG_3299-Edit.jpg) [![](/images/posts/IMG_0995.jpg)](/images/posts/IMG_0995.jpg) [![](/images/posts/IMG_1025.jpg)](/images/posts/IMG_1025.jpg) [![](/images/posts/IMG_0967-Edit.jpg)](/images/posts/IMG_0967-Edit.jpg) [![](/images/posts/IMG_0944-Edit.jpg)](/images/posts/IMG_0944-Edit.jpg) [![](/images/posts/IMG_0962.jpg)](/images/posts/IMG_0962.jpg) [![](/images/posts/IMG_0895-Edit.jpg)](/images/posts/IMG_0895-Edit.jpg) [![](/images/posts/IMG_0848.jpg)](/images/posts/IMG_0848.jpg) [![](/images/posts/IMG_0536-Edit.jpg)](/images/posts/IMG_0536-Edit.jpg)

Gallery of \"Warblers in the Hand\"

Thu, 27 Jul 2017 00:00:00 GMT

*Some images from the original WordPress post are no longer available.* ![img_5324](/images/posts/IMG_5324.jpg) ![img_5324](/images/posts/IMG_5324.jpg) *Some images from the original WordPress post are no longer available.* ![img_5358](/images/posts/IMG_5358.jpg) *This image has been lost to time. The original was hosted on WordPress.* *Some images from the original WordPress post are no longer available.* ![img_5358](/images/posts/IMG_5358.jpg) ![img_5376](/images/posts/IMG_5376.jpg) *Some images from the original WordPress post are no longer available.* ![img_5376](/images/posts/IMG_5376.jpg)

MPCNC: It moves!

Tue, 25 Jul 2017 00:00:00 GMT

These are some photos of the current [MPCNC](https://www.vicious1.com/information/) project coming alive in the Sulliwood basement. The MPCNC is a (relatively) low cost, 3d printed CNC (computer numerical control=does stuff by itself)- featured here is the spindle (actually a drywall cutter) and a mashup mk8-style extruder. Below, you can see what I see before it is cutting time in the CAM module of Fusion 360. That object is half of a "trial run" pottery stand for someone's art show.... *Some images from the original WordPress post are no longer available.* The MPCNC awakens! *Some images from the original WordPress post are no longer available.* Patchwork MPCNC extruder and hot end... *Some images from the original WordPress post are no longer available.* Double-sided MPCNC gcode, just for the holder part..... ...For this design: *Some images from the original WordPress post are no longer available.*

Birding Beyond Binos: 5 Bird apps vs. “the Guide”.

Mon, 03 Jul 2017 00:00:00 GMT

We all have a favorite bird, animal or plant guide. Peterson is the best at drawing; Sibley takes the best pictures. Kauffman ties it all together; National Geographic makes a solid reference and Audubon is great for fast looks. While these books will always have a place on the shelf or table, the depth of content and portability of smartphone apps and _trustworthy_ (e.g. reaserch related or associated with a big bird organization you recognize) websites truly foster the next level of ecological acuity. [I will cover apps for iPhones and iPads- these are tools I have available and find to be indispensable.] Like the shelf of guides they can replace out in the field, there is always room for another guide- and, generally speaking, cost significantly less than the least expensive print guide on your shelf. 1. iBird PRO - This app does it all: view photos, range maps, sounds, and similar birds, and search by band code, Latin/common name. The sound recordings are pretty good and can be looped individually or as a species playlist (good for playback in research situations). Similar bird songs are playable at the bottom of each species- great for learning and verifying nuances between similar songs. The illustrations are “ok”- better than what I could do (obviously) but nothing quite like Peterson or Kauffman. There are two more (add-on) engines in this app I have not used: the local birds function by GPS (BAM) and a “humanized” search tool to pinpoint the bird you are looking for (Percevia). 2. Audubon Birds Audubon Birds has come a long way, and generally will offer more of a comprehensive written overview on each bird- going into feeding, behavior, breeding, and habitat discussions. They seem to have added eBird integration (far, far superior to their “nature share” tool) which allows for both a mobile search into the unfathomably large user-based data set for local birds and a way to add your own data to eBird (though traditionally, the best way to do that is from a computer). 3. Audubon Owls This app is only a small vignette on owls; there seems to be more info geared solely about owls here than in Audubon Birds- photos, videos, tips, and tricks 4. Merlin Bird ID Despite the hardcore Bird Photo ID algorithms and location-based searches, this is geared toward those who may be starting out, and want to up the ante. You fill in a few parameters about a bird _sighting_ (this will not help with bird sounds), then it will generate a list of probable birds. Supposedly, if you get a good photo of the bird on your phone (Digi scoping/Wi-Fi upload?) It can id the bird visually. 5. eBird If you are truly doing an eBird list for your trip, try this app for basic, quick additions- but I would not rely on it for media uploads or anything too crazy. You can upload your checklist from the field then edit it later, though it is unclear if that is really a good idea in the scheme of data collection. This is a list of the Bird apps I use on my phone, most getting use many times a week or even every day (iBird Pro). -Jess

Rugby Morning #3.6

Thu, 18 May 2017 00:00:00 GMT

...During #3 I got demolished by biting insects- ".6" times later I had purchased and applied a significant DEET and re-entered the fray! -Jess *Some images from the original WordPress post are no longer available.* *This image has been lost to time. The original was hosted on WordPress.* *Some images from the original WordPress post are no longer available.*

Rugby Morning #2

Wed, 17 May 2017 00:00:00 GMT

Guess where I went this morning? Breaking in the new spot. Additionally, I saw Magnolia, Yellow, and Common Yellowthroat warblers, and heard Black Throated Blue and Green warblers. Veery, Hermit, and Ovenbird thrushes were around, in addition to catbirds. -Jess [![](/images/posts/IMG_9544.jpg)](/images/posts/IMG_9544.jpg) ``` Killdeer in pouring rain (Scope out mission = a wash) ``` [![](/images/posts/IMG_9589.jpg)](/images/posts/IMG_9589.jpg) ``` Redstart singing ``` [![](/images/posts/IMG_9609.jpg)](/images/posts/IMG_9609.jpg) ``` redstart chilling ``` [![](/images/posts/IMG_9614.jpg)](/images/posts/IMG_9614.jpg) ``` Chestnut-Sided Warbler ``` [![](/images/posts/IMG_9631.jpg)](/images/posts/IMG_9631.jpg) ``` ????? ``` [![](/images/posts/IMG_9638.jpg)](/images/posts/IMG_9638.jpg) ``` Scarlet Tanager "Tree pose" ``` [![](/images/posts/IMG_9651.jpg)](/images/posts/IMG_9651.jpg) ``` Scarlet Tanager ``` [![](/images/posts/IMG_9660.jpg)](/images/posts/IMG_9660.jpg) ``` Baltimore Oriole (quick grab) ``` [![](/images/posts/IMG_9693.jpg)](/images/posts/IMG_9693.jpg) ``` Song sparrow ```

Wolf Pine @ Fox Park: Silence?

Fri, 12 May 2017 00:00:00 GMT

I have an extremely brief update on my Wolf Pine tree; I did my loop and heard nothing. All I found was an enormous explosion of Beech leaves. Yes, a peeper here and a Phobe's lone chip call there- but really, as the school year draws to a close, my 29th update on this area seemed to be telling me to just relax and enjoy the scene. So I did. 🙂 -Jess

New Business Site- Up And Running!

Mon, 08 May 2017 00:00:00 GMT

http://jessphoto.transscendsurvival.plymouthcreate.net/ Check out my new Jess S. photography business website! It is up and running, hosted by PSU's own Plymouth Create service. If you have questions or inquiries, feel free to contact me! If you are a student or faculty looking to build your website for free, you may contact me about that too (I work for the Plymouth Create team as well!) http://jessphoto.transscendsurvival.plymouthcreate.net/

Pre-dawn Fox Park Lot Walk (Birding by Ear)

Mon, 08 May 2017 00:00:00 GMT

Walking through the 'burbs in the dark can be exciting. About an 45 minutes before sunrise, I walked to the base area of Fox park and found these 15 birds. While I didn't see them, I could certainly hear them! Species | Count ---|--- Mourning Dove | 1 Downy Woodpecker | 1 Eastern Phoebe | 3 Blue Jay | 1 American Crow | 3 Black-capped Chickadee | 4 Tufted Titmouse | 3 House Wren | 2 Wood Thrush | 1 American Robin | 2 Northern Mockingbird | 1 Ovenbird | 2 Chipping Sparrow | 1 Song Sparrow | 1 Northern Cardinal | 1 -Jess

Hunting for Trees = BIRDS

Sun, 07 May 2017 00:00:00 GMT

Today, I went lurking about Langdon Woods in search of as many trees as possible. I took over 300 photos of bark, leaves, and twigs, aiming to highlight the growth patterns and key ID features of the trees on the PSU natural history final exam. This went well, and I will be posting these Tree-I-Dee's as soon as I get through the pictures. The following photos are the result of chance and some enthusiastic "pishing" I did to draw in the birds, so I would not need to get to off course. 1. Redstart Warbler. This is almost certainly a breeding male in full attire. The Redstart song is often heard through these New Hampshire forests these days. This fellow responded very well to "pish" sounds, and danced over to me to see what the fuss was about. *Some images from the original WordPress post are no longer available.* 2\. Black-and-White Warbler. These Warblers have a weezy, squeaky sound almost identical to a rusty wheel. They act like Nuthatches but "dance" up and down the tree more enthusiastically, which is often a good way to tell which is which. *This image has been lost to time. The original was hosted on WordPress.* 3\. Hermit Thrush. These amber-toned thrushes have a beautiful song, but the only thrush singing today was the large Ovenbird population. More characteristic to the forests on the sides of white mountains, they will all sing together about an half an hour before sunrise. The proper thrushes (not including robin) of NH seem to follow an altitude metric: Ovenbird lives at the bottom, Hermit lives in the low hills, Swainson's sings in the mossy forest below the krumholz, and Bicknell's rules them all, only breeding above four thousand feet. !!! *Some images from the original WordPress post are no longer available.* -Jess

MEGA LANGDON

Sun, 07 May 2017 00:00:00 GMT

Well. There comes a time when one remembering [the right things at the right time equates to a high-stakes venture in academia](https://en.wikipedia.org/wiki/Final_examination). Below is a gallery of photos taken today, comprised almost entirely out of bark, leaves, and twigs. This is my study guide for the upcoming natural history final exam. It is not near complete; but for a walk through the woods and a significant number of hours behind a camera, computer, and coffee cup, I think it will do for now. Frogs, tracks, and birds are not covered here. -Jess *Some images from the original WordPress post are no longer available.* ``` Another likely Bigtooth Aspen leaf… ``` *This image has been lost to time. The original was hosted on WordPress.* ``` Fruticose Old man’s beard. Symbiosis between fungus and algae = Lichen ``` *Some images from the original WordPress post are no longer available.* ``` American Redstart ``` ![img_9220-1](/images/posts/IMG_9220-1.jpg) ``` Black-And-White Warbler ``` ![img_9272-1](/images/posts/IMG_9272-1.jpg) ``` Hermit thrush ``` ![img_9220-1](/images/posts/IMG_9220-1.jpg) ``` Japanese Barberry ``` ![img_9272-1](/images/posts/IMG_9272-1.jpg) ``` Black Cherry Bark ``` *Some images from the original WordPress post are no longer available.* ``` Sugar Maple bark, *young ``` *This image has been lost to time. The original was hosted on WordPress.* ``` Grape Vine ``` *Some images from the original WordPress post are no longer available.* ``` Small quaking Aspen leaves ```

New Photo/Promotional Media Business pages in the works!

Fri, 05 May 2017 00:00:00 GMT

**I am working on bringing new and exciting services to my website(s). Stay tuned for exciting services, prices, and (of course) galleries!** **-Jess**

Early Morning Guitar Session. ala Laundry

Thu, 04 May 2017 00:00:00 GMT

**Guitar Practice:** Laundry is best done early in the morning. The resulting time between cycles lets me do all kinds of things- below is a short guitar "shred" song inspired by Jeff Beck's most recent album, Loud Hailer- recorded in its entirety between washing and drying. I recommend turning your volume down, modesty is not part if this song's vocabulary. *The audio recording that was embedded here has been lost with the WordPress site.* -Jess

Fox Park to Langdon: Morning Birds!

Thu, 04 May 2017 00:00:00 GMT

Without further ado: This morning, I went birding across the campus starting at sunrise. Below is the species list, and two ID shots- Ruby-crowned kinglet and Yellow warbler. 2 Canada Goose 2 Mourning Dove 1 Red-bellied Woodpecker 1 Downy Woodpecker 2 Eastern Phoebe 2 Blue Jay 2 American Crow 1 Common Raven 3 Black-capped Chickadee 2 Tufted Titmouse 1 House Wren 2 Ruby-crowned Kinglet 2 American Robin 1 Northern Mockingbird 1 Ovenbird 1 Black-and-white Warbler 1 Yellow Warbler 1 Black-throated Blue Warbler 1 Chipping Sparrow 2 White-throated Sparrow 1 Song Sparrow 1 Northern Cardinal 1 Common Grackle 1 House Finch 1 American Goldfinch 2 House Sparrow *1 Black-throated Green Warbler, *Yellow-rumped Warbler found later. Number of Taxa: 26 + 2 *Some images from the original WordPress post are no longer available.* Above is a Ruby-crowned Kinglet. Note the crown is not ruby colored. *Some images from the original WordPress post are no longer available.* Above is a deeply-hued Yellow Warbler. A pure sounding, "Sweet-sweet-sweet, so-so Sweet!"- heard all over the forest an parking lots alike. -Jess

The Big Len List

Wed, 03 May 2017 00:00:00 GMT

This morning at 7am, very few birds were singing. Behind the Rugby field in a solid rain, a small group of people stood with their noses to the sky. This is PSU's very own Len R. -led class on vertebrate zoology. Now, please note I do not take this class, but I know a thing or two about Len. Len is a bird master; this means vertebrate zoology in the springtime may just equate to an excellent excuse to find and learn about birds and warblers on the premise of a college class. Thank goodness warblers have backbones. The following list was compiled mostly by Len and another student (who also is not taking the course...). The louisiana waterthrush, black and white warbler, and a glowing male redstart (all of which are warblers, despite the different naming conventions) really hit this walk out of the park for me. *Some images from the original WordPress post are no longer available.* -Jess

Langdon Woods with Kurt: Plants!

Tue, 02 May 2017 00:00:00 GMT

Today- after an extremely productive 7am trip with the vertebrate zoology class mind you- I nipped over to Langdon Woods in the rain to learn about the the little plants growing around the forest floor at a rapid, hydrated rate. The ones I can remember off the top of my head include: **Bunchberry** : a ground covering plant with red berries clustered in a bunch. Edible! **Partridge berry** : a tiny plant with a leaf or two red berry on top. not dangerous to eat! **Goldenthread** : A little plant with three fan-shaped leaves. the deep orange root has numbing and diuretic properties. Useful! Probably not good nutrition though! **Purple trillium:** a flower with an exotic crimson color. It is also called "stinking benjamin" because it has an undesirable odor when in bloom. Nice to look at! **Starflower:** A distinctive, flat white flower with 7 blades. the leaves all grow from the same spot in a circle- a phenomenon known also in pine trees as a "whorl". Pretty! **Indian cucumber:** a plant that also grows leaves in a whorl shape, with a varying number of leaves. A prime root tastes and feels like a sweet carrot. Some think they can tell how developed the cucumber is by how many leaves are on the plant, though I do not know this to be true. Very tasty! **Wintergreen:** a small thick-leafed plant. The leaves are round and a bit waxy looking, but the point is it is a great consumable. Makes great tea! **Sensitive fern:** This is a fairly nondescript fern with one key feature: it leaves its fertile fronds attached to the plant for a while, making them easy to spot. Just look for brown "beaded" fronds sticking straight up **-** this is a clear indication of sensitive fern. Fun to ID! **Ostrich fern** : A big fern with large fiddleheads. Great sauteed! There are more plants we covered, but these are the ones I can remember the best. -Jess

Addendum to \"Secret Beach Area\" Natural History Class Walk

Thu, 20 Apr 2017 00:00:00 GMT

_"by[Christopher Myrick](http://www.plymouth.edu/courses/user/view.php?id=30487&course=36400) \- Thursday, April 20, 2017, 5:38 PM_ _Bomber day in the woods. We saw a dead frog and a dead turtle. Multiple little insect varieties and some eggs which I will raise as my own. They are in a Gatorade bottle next to an open window to keep the water cooler than room temperature (updates will follow). A great blue heron flew over head as well as some black birds and a hawk. Also Sean ate it in the river. "_ ...This is a brilliant depiction. Best of luck rearing the kiddos. I will add: hop-hornbeam was in there, with yellowish/crackly bark. Lots of silver maples in the puddly areas, and a sugar maple. Lots of black cherry trees. A few aspens with "sunscreen" bark. 2 river otters, a small crayfish, 2 pre-flight dragonflies. The 10 most notable birds: 1. Coopers hawk - Looked like a broadwing hawk or even a merlin (falcon) at first sight, but had a longer, more triangle shaped tail with more horizontal bars then the broadwing, and had straighter wings and a slower soaring flight pattern than a merlin. 2. Common loon pair - sits low to the water (them solid bones) with a giant, fish-gobbling head. 3. horned grebe - tiny diving bird with a fuzzy head. They often are seen (in my personal experience) where loons are floating. 4. Belted kingfisher - KA-KA-KA-KA-KA-KA... 5. goldfinches - 'potatochip" 6. song sparrows 7. downy woodpecker (NEIGH!) 8. flicker (HA-HA-HA-HA-HA-HA...) 9. red-bellied woodpecker (bwbwuuraack!) 10. GHB - in flight, looks like pterodactyl but isn't, trust me... -Jess

Fox Park #18: 4/19/17, In Class Roamings

Wed, 19 Apr 2017 00:00:00 GMT

At around 9:35 yesterday morning, the natural history class gathered in Fox Park under an amazing clear sky and a light wind. Instead of going into the level of detail as I did over the winter (there were quite frankly less details to be had over the winter) I will try to summarize the two most significant findings . * **Porcupines are everywhere.** It turns out these giant walking pinecones are leaving traces of themselves all over the place (though the chances of seeing one are still quite slim). In the winter, they climb up hemlock trees and nibble. Everything. They sort of just sit up there and eat the tree, trying to move as little as possible and thus will everything within reach. These foliage holes in hemlock trees are a dead giveaway of this activity, and will often have dangling branches with rodent-esque chop marks. I have seen a stands of hemlocks with large amounts of debris underneath at Fox Park, which is extremely indicative of a porcupine's munching habits. * **The magnolia warblers have arrived.** [These are magnificent little birds. ](https://www.allaboutbirds.org/guide/Magnolia_Warbler/id) Famous for their "necklace with pendants", these warblers have officially arrived, and in full getup. A pair of them were calling to each other (which I did not get right away- they have a few calls) then chasing each other through the forest. What fun! -Jess

Fox Park #16 and #17: 4/16/17,Big Toad: Small Vireo. Yay.

Mon, 17 Apr 2017 00:00:00 GMT

I was in and out of Fox park today as well as yesterday, so I will not put a time. The sun was hot (77F), the skies were clear, and the birds were singing. Loudly. I did a sit spot yesterday, which kind of rolled into today- there was not a peep yesterday. I do not have the foggiest idea why; regardless, it was soggy and drizzly, and I did not make any great achievements worth writing home about. *This image existed once on WordPress, but the web moves on.*I did, however, find this extremely large and incredibly dead American Toad. Observe it in all its massiveness. This fellow was around 6 (6!) inches long. Key things to note about a toad: * the bizarre patterns with no discernible regularity. This one has leopard print pants and a camo shirt. This seems to have to do with where it lives; forest floors where yummy worms and grubs reside are where these toads make their homes. * The poisons in the bumps behind the eyes are "not weak". Toads have toxic glands, excreting "bufotoxins" (bufo really just means toad) which are a sort of steroid chemically mangled with strange and hard-to-synthesise-in-the-lab compounds. The toxins in this American (and "eastern") toad are "weak" because they should only kill your small dog if eaten. 🙂 The even larger South American cousin however (Cane toad) can not only grow to have a 9 inch long body, but simply licking it will kill most humans. As a result, they are not commonly eaten in the wild, so toads are generally not endangered. Catching up to today: I will cut right to the chase: *An image was here before the migration. It didn't make the journey.*This is a Blue-headed Vireo, and the worst picture I could possibly take. Indeed, I took it by accident while looking through my lens to verify this bird was "too far away to identify". Only on my way back did I realize what I had captured. I thought at first it was a nashville warbler- so, in my confusion, I stood for over an hour baking in the sun in the field where I took this picture. I did not see it again. BUT: I heard it. A slow and clear, "see-boo?? I-See-you!! Want-tea-too??" (or something to that effect), emanating from the middle of the trees. This, coupled with the eye ring, fuzzy blue-grey head, wing bars, and buff yellow throat and body, I can say with much certainty this really is a blue-headed vireo. Huzzah! I believe there are real wood warblers here, now. I hear the odd "zeeZEE" and "BeeZoo" and "ze,zee,ZEE", but no clear songs yet. These are warbler sounds, but not songs. Today was a 23 species day, all at Fox Park. Things should get pretty interesting this week..... -Jess

Fox Park #14: 4/10/17, \"1 if by land, 16 if by ear\"

Wed, 12 Apr 2017 00:00:00 GMT

Just a quick sit spot walk through. Every bird was found be ear first, or only by ear. They are singing! Wah hoo! 1. Phoebe 2. Crow 3. Blue jay 4. Carolina wren 5. Chickadee 6. House finch 7. tufted titmouse 8. Downy woodpecker 9. Canada goose 10. Brown creeper 11. Robin 12. Mourning dove 13. Goldfinch 14. Hairy woodpecker 15. Chipping sparrow 16. Junco ...Getting through a sit spot backlog. Please excuse the short post! -Jess

Fox Park #15: 4/12/17, “1 if by land, 17 if by ear”

Wed, 12 Apr 2017 00:00:00 GMT

Again, Just getting through the small backlog of sit spots. All by ear, many with a visual confirmation at some point. Species Count: 1. Mourning Dove 1 2. Eastern Phoebe 2 3. Blue Jay 4 4. American Crow 3 5. Black-capped Chickadee 3 6. Tufted Titmouse 4 7. White-breasted Nuthatch 1 8. Carolina Wren 1 9. American Robin 5 10. Northern Mockingbird 1 11. Chipping Sparrow 7 12. Dark-eyed Junco 3 13. Song Sparrow 3 14. Northern Cardinal 2 15. Common Grackle 18 16. American Goldfinch 2 17. House Sparrow 4 Again, I am just recounting the notes I took with eBird. Other living things and systems are to come! Hurrah! -Jess

Bird Observations Today- Langdon Woods

Tue, 11 Apr 2017 00:00:00 GMT

This is in lieu of a wonderful 2 hour walk around PSU property during my natural history class. Firstly- open the below link to see the 25 species we encountered today: http://ebird.org/ebird/view/checklist/S35873502 I want to also point out how spectacular and special the 3+ courting yellow bellied sapsuckers were. These birds are relatively rare to find- here are the first things off the top of my head you all should know: * They do not suck sap. They eat bugs like all the other woodpeckers... * They are farmers. These are the only birds to my knowledge who literally farm for a living.... They peck a matrix of holes in trees with sap- often perfect rows and columns- into which bugs fall and get trapped. Then, at their leisure, the sapsucker will visit its "sticky bug fields" and gobble the bugs up. This not only makes their life of pecking and eating super laid back, it allows them to have personal property. 🙂 * They were pests not long ago. Because of the whole farm-the-tree thing, human farmers of high octane fruit trees and other trees pushing a large amount of sap would generally shoot the sapsuckers ASAP to avoid the sapsucker killing the tree. As scary as that is, the humans are correct- the sapsucker will win, One sapsucker doing some farming can spring a significant number of leaks in young trees, allowing bugs and parasites in, killing the unassuming orchard. * They not endangered. These birds are rare because, as any farmer will tell you, they have lots of work to do around the active bug-sap-hole farms. They each have their own space, and respect each other's trees and areas. This makes the concentration low, but the overall population number still healthy. * They are a wild card for at risk forests. Sapsuckers don't mean any harm, but fragile ecosystems with just a few sap filled trees can get a makeover after a few sapsuckers move in. Sometimes, this is fabulous: old trees die, allowing other animals and organisms to move in, including other species of woodpecker. New plants will grow, and the space will move on and evolve. On the other hand, as our orchard friends know, a tree bleeding sap is undoubtedly going to have a problem sooner or later. Woodpeckers are fine with that, but other species are not... ...So those who saw the sapsuckers today, consider yourself lucky; that was spectacular! -Jess

pt. 1: Reflecting On Stunt Culture (Theories and Frisbee)

Sat, 08 Apr 2017 00:00:00 GMT

There are only tough times finding academic craziness to get involved with during spring break, so I decided I should break out the unicycle and do a trick. It was a 180 unispin.... But that is unimportant. I think tricks, fitness, and the subsequent cultures they create are a valuable asset worth cherishing. From "grandmaster", 50+ year old women and men duking it out armed to the teeth with ultimate frisbee discs, knee braces, ankle support, and vast quantities of beer to 20-30-year-olds scaling famous rock faces with nothing but a trad rack of cams and a bag of snow white chalk, to [some french dudes doing unicycle ballet](https://youtu.be/02kRossyKMw) to the little lads and lasses on their skateboards and scooters, one can draw a clear conclusion: humans seem to benefit from physical and mental challenges that really don't fit in with most primary evolution-related characteristics. Oddly enough, even the most elite athletes in frisbee, climbing, and scootering will consider their sports part of their culture and a way to "chill" while also pushing limits. Does that make sense? I had a great friend a few years back- nicest fellow in town. Not only was he extremely well educated and respected in the doctoral-level health services community, he was one of the top competitive ultimate frisbee players in the country, heralding from Boston. This dude is the definitive quarterback of frisbee for crying out loud- upon entering a stadium where he was playing on evening (I had not realized his elite-ness yet, I was just going to a game to be nice), I quickly realized the crowd was chanting, screaming my friend's last name- when he came roaring into the field leading the team, I could feel the adrenaline in the crowd like helium in my lungs. Yet: this was just his game, not even a "sport" (like pro baseball) to him as far as I could tell... ...My friend had unearthed the "ultimate" way to deal with stress from his lofty academic and work positions. Despite the immense amount of time, energy, and failure put into a complicated, dangerous game (he definitely tore more ankles, shoulders, and labrums "playing" than he ever would working on his doctorate) he was able to find a balance between relaxation, play, and work while maintaining a cheery attitude and high octane, dedicated mindset. *Some images from the original WordPress post are no longer available.* Photo by Bob Durling – Ultiphotos.com We see here a balance of playing, chilling, and working in this multifaceted fellow. Why is this so important? It turns out the balance being struck here is neither trivial or even fully understood. researchers in the 80s convinced themselves all this play and relaxing in other animals was about preparing for adulthood, deciding the reason for all this extra work animals go through boiled down to a ultimatum of adult survival and reproductive success... ...These researchers were great and important in advancing this difficult-to-pin-down subject, but that idea definitely does not complete much of our story here. We can observe all animals who have play and chill in their student workbook will indeed play as a child, about the same amount. We will then observe the data correlating any aspects of play to adult success really doesn't provide amazing parallels or strong trends.... So it must be more fundamental than what simple 80's observation studies can show us. These boundary-pushing sports we engage with for fun teach us to teach ourselves things and support others doing the same thing; in this way, it is a self-serving cycle. This learning, teaching, and progressing through extreme sport is simply a way of living and experiencing life through a lens already suited for play, relaxation, and hard work. [Next up: Scooters + Youtube = ?](/blog/pt-2-reflecting-on-stunt-culture-scooters/)

pt. 2: Reflecting On Stunt Culture (Scooters)

Sat, 08 Apr 2017 00:00:00 GMT

Gone are the days of hoodies and hooligans (or rather, that crowd seems to be moving somewhere else) at the skate park. In their place is the new generation: 12 year olds tearing about on scooters. These kids are tuned into the vast, global network of scooter riders who wear their helmets, knee pads, and elbow pads and give strangers high-fives for doing a good job. The key factors of playing, chilling, and hard work are all present on the scooter. Indeed, it would appear after even a brief stroll through youtube, the amount of "work" put into this "chill" sport is astounding. Theories aside, I have included two videos I think highlight the new, revamped, youtube-generation scooter crowd below. Note how the ideas in pt. 1 apply... Here is Claudius and his mini-sized friend. Not much is known about where Claudius actually came from, aside from popping up on youtube with a bang a year or two ago. He can be easily identified in other youtuber/scooter rider's videos in the background with is iconic neon-everything gear and apparel and his exotic, titanium, and tennis-grip-gripped scooter (while doing wildly technical tricks while yelling in various languages with a thick German accent, often involving a backflip to signify he completed his run). The odd thing is he is seen in the background of videos from australia , germany, the UK and California.... Regardless, he seems to have a good influence on younger kids with his amusing sensibility (or lack of sensibility in general). He preaches things like knowing one's limits, "staying hydrated", using foam pits/resi/gymnastic gyms for safe practice, and always wearing protective gear. This fellow is based out of NYC, and is one of the "original" (and really only) flatland scooter riders. He has been actively working to keep the "chill factor" a big part of the scooter scene for the new young folks. He generally doesn't do wildly crazy tricks, instead focusing on cherishing practice, focus, and riding in fun spots with friends. This ethos is very important for sports like this, when one can easily start asking, "why on earth am I putting myself through this difficulty trick?". **Ideas to keep things in check - according to Jon, Claudius, and myself:** **Have relaxed expectations for a session.** Going big, whatever that may mean, usually implies pursuing the hormones and their precursors (Epinephrine and dopamine respectively) instead of the primary intent of the sport: to play, chill, and progress. **Understand why it is alright to expect some level of injury.** Getting hurt happens, and doing things that look somewhat dangerous and perhaps a little stupid probably are. Yet- safety is easy, almost as easy as getting hurt. With that relationship in mind, we can aim to only ever get "a little hurt". The level of progression should ideally match the level of safety precautions- for instance, there are some 12 year olds doing backflip 180 tricks (flairs) in concrete skatepark ramps. In most cases, this is actually fine (if they have knee, elbow, and head protection of course!) because they probably threw hundreds of attempts into a foam pit, then a few hundred more to a soft, spongy ramp, then a few hundred more in a smooth wood ramp on which they could slide safely down on their knee pads. That is a vast amount of safety measures to ensure every time they do that trick and subsequently go upside down, with the worst that could happen ending with them on their knee or elbow pads sliding down the ramp. Similarly, an intro ultimate frisbee player should learn to condition and stretch their shoulders (hammer throws), calves (sprints) quads (epic jumps) and do proper warm-ups on their feet and toes to strengthen the worst and most common frisbee injury: the "out-for-6-months-softball-sized-ankle-sprain" or worse ankle injuries. I see most older frisbee players with one or even both ankles wrapped firmly in a brace- not to say this is inevitable, but we should understand this is a huge danger and new players should be extremely careful with their fresh ankles, whatever sport they end up going into the deep end with. Reflecting on stunt culture - A few references and further content: Sharpe, Lynda. "So You Think You Know Why Animals Play..." _Scientific American Blog Network_. Scientific American, 06 Aug. 2013. Web. 23 Mar. 2017. Thorpe, Holly. "Sign In: Registered Users." _Action Sports, Social Media, and New Technologies_. Te Oranga School of Human Development and Movement Studies, Faculty of Education, University of Waikato, Hamilton, New Zealand, 22 Mar. 2016. Web. 16 Mar. 2017. Wood, L., et al. (2014). Dispelling Stereotypes… Skate Parks as a Setting for Pro-Social Behavior among Young People. Current Urban Studies, 2, 62-73. http://dx.doi.org/10.4236/cus.2014.21007

Preview: Learning Fusion 360 and Next-Level CAD Craziness

Tue, 04 Apr 2017 00:00:00 GMT

This is my trunnion table: behold! Here is the practical use in China: And here it comes back to a geek-desk near you:

Parking Lot/Sit Spot @ Fox Park #12: 4/3/17, Owling 2 Hours Before Sunrise

Mon, 03 Apr 2017 00:00:00 GMT

It was very dark when I left the parking lot variant of my sit spot, and still it still is. Was it worth it? Maybe. I entered the parking lot around 4:35am this morning. After spending a while just listening to the sounds of "nature", finishing my coffee and trying to not make sounds into what they weren't, I gave in and decided to play some screech owl trills. Unfortunately, an issue I have not yet addressed was beginning to get in my way for real: the highway. Even when I play calls from my phone, I could tell the white noise from the interstate not far away was cancelling the sonorous sounds of my owls. That part isn't a big deal, but I know my inferior human hearing will struggle to pick out a chatting owl even within my part of Fox Park. The frequencies are just too similar, often exhibiting a similar timbre. This means a sound carrying more energy (lower frequency rumbles and what not) will not only mask the weaker and more refined owl toots and hoots, but could "phase cancel" them out altogether. Phase cancellation is obviously not a standard concern of birders, but I happened to know from recording sounds in this frequency range (lower end of a medium grand piano and acoustic guitar for example) achieving a mini "Bose noise cancellation" is quite easy. All it takes is two sounds going the opposite direction and/or of similar magnitude or at least frequency (a distant truck with a Jake brake and closer GHO for example and whoops! there goes the owl hoot. I mention all this because in the ~50 minutes waffled around in the parking lot (10 degrees below freezing mind you), during which I played screech, saw-whet, and GHO, I heard lots of mumbles and whoos and blops... ...yet I can only take one seriously. One toot, that's all. I had played screech, then saw-whet, and screech once more at this point. The toot sounded much lower than a saw-whet toot, and there was just one. It was not dainty, and had a nice conviction and resonance. I have never been compelled to describe an automobile this way, so I can say with good faith this was an owl. But was it Barred or GHO? Both make single toots in this way sometimes. Indeed, I've seen it done on trips where the either owl may want to just put a small idea out there, a pleasantry maybe to the owl it listened to from a birders phone, or perhaps just to test the waters on who could call back. For whatever reason, more than half of my hearing/visual owl encounters involved a single toot instead of a full blown dissertation of whoos and haws. So, I will tentatively stick with the current idea this is a GHO, because my other evidence seems to support this. As I played some GHO after the toot, I quite honestly could not listen between the cars and trucks from, say, half a mile away. Thus, while the tooting owl was not in spitting distance of my mini encampment on a bit of ice in the parking lot, it could easily been in Fox Park or an adjacent landowner's pine tree and I would never have known. The saga continues... -Jess

Wolf Pine @ Fox Park #11: 4/1/17, It Is A Snow-Show, Debunking The Melanistic Dogamount

Sun, 02 Apr 2017 00:00:00 GMT

I slipped and slid my way into Fox park Saturday, 4/1/17 at about 4pm. About 7 inches of snow had appeared on the ground over the last 24 hours, which (for the second time) definitely stifled and spring-like activities for the critters and what not. Yet, the still powder-like snow was melting already. This stuff hadn't really had time to settle and compact, it just came down from the sky _just_ below freezing, then bobbed above freezing at about noon and rained. This made for perfect postholing snow. Indeed, I saw some dogs who took it hard- leaving postholes almost 3 feet deep. I heard some confused titmice and a lonely Hairy woodpecker over (almost) the whole time out, though a the crow crew started up yakking away just as I left. I had really come for the tracks in the snow, but because of the rain and rapidly melting cover, I could only make out big dogs. [![](/images/posts/unnamed.jpg)](/images/posts/unnamed.jpg) "Big Dog" Here we have one of these big dogs. things to note: * triangle shaped claws * very symmetrical * creates a distinct circle-oval shape * Can easily be broken into left, right, two leading toes and rear pad quadrants These traits are interesting, though they get way cooler and silly when we look at the crazy, unique, and very artistically rendered "black panther" prints I found in the PSU dining hall: I realize this is the worst iPhone-picture-while-scooping-ice-cream example, but... ...I do not think these prints are for a black panther. I do not think they are for a dog. These are the one of a kind "melanistic dogamount" prints! Here we have the local catamount (cougar) vs the dog (similar to the big dog I found). Remember, the PSU mascot is a [melanistic jaguar](https://en.wikipedia.org/wiki/Jaguar#Color_morphism) named "Pemi". Jaguar prints are anatomically very similar to the puma/cougar version that is theoretically in new england, if only on occasion. Indeed, these "uber crazy level" cats have an (average) range of about 300 square miles. Which is 192,000 acres, if you weren't so hot on math. 🙂 This range makes tracking a single cougar extremely difficult, and as far as I can tell, nobody has been particularly successful- thus, finding photos of actual paw prints is really, really hard, and makes the far larger melanistic jaguar prints impossible to find. Below is a cougar paw from captivity. ![unnamed](/images/posts/unnamed.jpg) *Image: paw of cougar (Puma concolor). Source: "No Place for Predators?" Gross L, PLoS Biology Vol. 6, No. 2, e40* So, what are the good and bad parts of the PSU sign? **Good:** * Toes are mostly in front of the pad. This is indicative of a cat. * Rear pad is wide, (almost) a cat trait in this example **Bad:** * Rear pad is too oval shaped. Real cougars and jaguars have deep scallops creating three distinct parts of the pad * **THE NON-RETRACTABLE CLAWS ARE TRIANGLE SHAPED, LIKE A DOG'S CLAWS!!!** All cats have retractable, grappling-hook shaped claws. These are rarely out and about when walking, as they are really best for catching one's balance and slicing stuff to shreds. They are usually seen as dots with a groove toward the toe on a paw print. Dog claws on the other hand are designed to be a permanent part of the foot, and are shaped like a wider "V" to generally help with transport. These are what we see, making this paw print completely and unforgivably wrong. That concludes today's sit spot observation. -Jess

Parking Lot/Sit Spot @ Fox Park #10: 3/31/17, Whoo's Clues

Fri, 31 Mar 2017 00:00:00 GMT

I entered the Fox Park parking lot at approximately 5:30, about an hour before sunrise. 32 degrees, partly cloudy, and very dark and supremely quiet. I didn't have to wait 5 minutes after settling into a comfortable standing position to hear the first of 2 fat clues about my owl buddy at Fox park. Three sonorous "whoos" reverberated across the surrounding fields and white pine trees, followed by some muffled humming and burbling over the drone from the highway about half a mile away. What luck! The thing to know about this scenario however is these whoos were higher pitched than "ye average" great horned owl, BUT were far from the "hawws" and other gurgles the barred owls make. The second clue about this sound (and I heard it one more about 10 minutes later) is how a classmate recently described exactly what I heard today to me. "It was saying Whoo! but it was started going up, then down to some quieter sounds." This was heard not far from Fox Park, near Langdon Woods. That forest has a great field used for light football training by humans, and critter hunting by birds of prey no doubt. This is well within an average great horned owls "zone"- in fact, owls have been seen occupying a 25 mile radius of space as a residence. That means no other GHOs are allowed to live there. Quite territorial, and have interesting family/land relationship patterns because of the vast zones required for a proper turf. This is almost entirely the reason the GHO is both**** widespread and thus "common" and essentially impossible to find, making it a treat to locate. So, I know this pattern is very likely a GHO after two pairs of ears have heard it and agree. So... ...Success! As usual, after the second "whoo" and maybe 10 minutes of standing in the parking lot a sole cardinal started singing. Then, one by one, the local crows woke up and decided the calling owl was a significant problem (they decide this every day) and started up with the tomfoolery we can expect from them. On that laural, I was sure the owl would be silent to give the crows a sporting chance at hide and seek, so I left, after a bit more than 30 minutes in the parking lot. -Jess

Beaver Dam @ Quincy Bog #1: 3/30/17, The World Has Gone Mad

Thu, 30 Mar 2017 00:00:00 GMT

The first year PSU natural history class wandered into the middle of Quincy Bog onto the local beaver lodge at about 10am today, 3/30/17. We had came to this spot originally to float about the area and gather fun and mildly interesting questions and about the "real" natural world (as opposed to the classroom). We found lichen to be a mutualistic symbiosis between algae and fungus; the bog is full of "leather leaf", but is not acidic enough to be completely full of this plant, as real bogs around here are; we also saw a few crows mobbing a raven. And: A peregrine falcon... ...And (what to my knowledge is) a short eared owl. There have been between three and five short eared owls in the main portion of [New hampshire in the last decade](http://ebird.org/ebird/map/sheowl?neg=true&env.minX=&env.minY=&env.maxX=&env.maxY=&zh=false&gp=false&ev=Z&mr=1-12&bmo=1&emo=12&yr=last10) (besides at the seashore near the northeast tip of Massachusetts). According to the eBird, we can see in 2013 and 2014 there were around two or three short eared owls migrating up to their summer home in far north Canada. After reviewing a few migration routes from around the web, I can say there is a good chance a few short eared owls will be coming from the "middle of the east half of the west"- likely farms and fields emanating from Tennessee- at this time of year. Obviously, this owl is vastly more important than anything else I could have done today, so it will occupy the remainder of this spot review. *Some images from the original WordPress post are no longer available.* Taken from: https://www.allaboutbirds.org/guide/Short-eared_Owl/id *Some images from the original WordPress post are no longer available.* Screenshot from a google image search. Not my image. Note the black slashes parallel to the body on the median and greater coverts, and a dark head. Firstly; things pushing against the evidence I do have for this owl. The owl like to float around 10 feet over the grasses in fields to snatch mice and voles. This owl was about 700 feet up in the air. Additionally, it is likely there were two of these birds seen up there soon after we arrived. After that initial glimpse however, there was just one. Reasons this is a short eared owl: * We were standing in a prime hunting spot for a migrating short eared owl-an iced over bog should be riddled with yummy critters. A group of 20 people milling about the middle of this bog could be a good reason to do some gliding at some distance, waiting for the people to leave and the mammals to emerge again. * The bird was a light grey color, with distinct black wing tips. That is the first mark we saw. A fat "buteo"-like (think a soaring red-tailed hawk) tail, creating almost a complete semicircle, was really built into the bird's body. This is totally different than a flared harrier tail, which is long. *Some images from the original WordPress post are no longer available.* REALLY LONG TAIL- Harrier. https://www.allaboutbirds.org/guide/Northern_Harrier/id *Some images from the original WordPress post are no longer available.* Look at the fat but short tail. https://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&ved=0ahUKEwjPtb-Qn__SAhWB7YMKHa00AnMQjhwIBQ&url=http%3A%2F%2Fthrumyeye.deviantart.com%2Fart%2FShort-Eared-Owl-in-Flight-317860358&bvm=bv.151325232,d.amc&psig=AFQjCNEO_XfgvoVIrwFXZPwIcwVJCdFp_A&ust=1490998137246086 Additionally-the real kicker in my opinion- are the black slashes on the medial and greater coverts on the short eared owl. The two photos I have included here are enough to show these unique field marks. I have never seen a bird with that pattern; just a flat matte white with two black marks parallel to the body, and black wing tips. Many birds of prey have black wingtips or interesting patterns/markers under the wing. In fact, this is an outstanding way to learn big birds who usually fly overhead/don't usually hang around on a perch. In summary: unless we find another bird with this wing shape and color pattern, **this is a short eared owl.** **...**(And it may have a migration buddy!) -Jess

Wolf Pine @ Fox Park #9: 3/26/17, Everyone Is On Vacation

Tue, 28 Mar 2017 00:00:00 GMT

I walked into Fox park Sunday afternoon, 3/26/17, after "spring" break. Please note, however, neither the suburbs of Boston or Fox Park have turned the ignition on the spring thing. So, I will not provide pictures today because the view is, for both flora and fauna alike, the same as the last time I took pictures. The weather was warmer than freezing, but there is evidence of chilly rain and wind slowly wearing away at the snow. the Beech leaves are also having a hard time staying attached this long into the cold season. they are rustling and falling off both because the the weather but also because we really cannot have too much longer before our warblers come through, song birds start really singing, buds and leaves ome out, etc, etc. Let us see where the warblers are today, ehh? Here we have the most up to date info on the Palm warblers. If you are not used to the [eBird species range map](https://ebird.org/ebird/map/), you should click the link and get used to it. This is the most efficient way to find where species are, assuming there are people around the areas in question to report sightings. Here, I narrowed the time frame to this year and this month. We can see the Palm warbler crew is still in Florida for the most part. This is where many Palm warblers go when they go south, the farthest ones only ferrying over the Cuba. Up the coast they go, but the leaders of the pack are not really in New England yet. *Some images from the original WordPress post are no longer available.* Palm warblers are an early warbler in my experience around here. They often will be showing up as the buds on the trees begin to get serious about leaves. They simply don't cross the Gulf of Mexico or the Caribbean, unlike many of their peers. Another early bird is the pine warbler. They don't really migrate much, but in the spring they wander up from the south, making for regular sightings in MA in NH. *Some images from the original WordPress post are no longer available.* But what about the real warbler crew? Blackburnians! Chestnut sided! Well, as you can see below, they are all still singing songs in portuguese and spanish, as far south as Ecuador (for the blackburnians) right now. *Some images from the original WordPress post are no longer available.* Remarkable! Both of those birds will fly between 2,000 and 3,000+ miles, just to visit us in NH! Special indeed. Despite the snow, rain, and cold winds at my sit spot, the anticipation for spring is getting into gear. -Jess

Boutique everything: When The Hobby Grows Up

Mon, 20 Mar 2017 00:00:00 GMT

Food. Clothes. Art. Musical Equipment. Consumer Design and Products. Can a mere citizen enter the fray of cutting edge design and production? As a hobbyist designer with a passion for, say, high end audio, the options for actually producing a quality, well executed product may seem lucrative and completely not worth while. “It’s just a hobby” some say, or, “The cost of manufacturing tools or a bid at the factory floor in China are way bigger than my love for sound”, or, “nobody would ever purchase my design, there are so many other companies who have done this longer than me”. These answers are all valid, but may not be the complete picture when it comes to local, boutique production. Can a passionate enthusiast use makerspace technology and peer support to bring small batches/limited runs of high quality products to a localized, niche market? Could a food connoisseur use networking services to construct a timely supply chain for seasonal meals at local restaurants or cafes? Would a local tailor be able to source materials and equipment to realise the material science and design they have always dreamed of for a coat in small batches? Using cutting-edge makerspaces and the subsequent networking opportunities, I believe producing small batches of high quality goods and utilizing a local business/niche marketing approach or distribution system could increase the innovation and quality of any given local economy. The idea of “group buys” is elementary in DIY audio circles. Folks going in on a board design for fabrication will often drum up some enthusiasm on the internet or elsewhere, in a move to offset the high entry price of board manufacture. I have noticed some folks take it a step further, and will not only complete the project they intended to, but perfect the project into a product and do a run of a few pieces to a few dozen and beyond. This model is actually a great asset to the developing maker; offsetting the cost (or even making a few coins in profit!) of larger projects inherently makes bigger and better projects feasible. The folks building audio equipment in their basement, garage, or bedroom are, in essence, artists exploring art through avenues otherwise devoid of artisan qualities. It is easy to reproduce sound commercially- Apple supplies those iBud-earPod-headBeats with every phone they sell. Yet, the people in DIY audio are taking on audio components exactly how a great potter would craft a new bowl or coffee cup; functional sculpture, art in one of its oldest forms. *Some images from the original WordPress post are no longer available.* Screenshot of Jazzman's blog (http://jazzman-esl-page.blogspot.com/) Below is a picture of one of the quasi-famous Jazzman ESL panels. A true labor of love and work of art, Charlie has pioneered the processes required to build a ultra-top-end electrostatic loudspeaker, in the confines of the home, job, and hobby budget. Now, Jazzman's speakers are built almost exclusively by hand, using careful measurement techniques to ensure tight tolerances instead of using machines that could do this automatically- making these panels really one of a kind and certainly not an option for even the most ambitious cottage industry entrepreneurs. I bring these panels up simply to show what home-brew audio (or any labor-of-love-hobby) is about: craftsmanship, dedication, and a desire to learn. falling right in with home-brew beer, local pottery, cooking, painting, tailoring, and more, one can see from this artisanal point of view the value in these kinds of work. Unlike some of these art forms found exclusively in art shows and galleries, only recently has there been an opportunity for individuals to reverse the commercialization of otherwise beautiful hobbies. **Commercialization and hobbies: can we have both?** You bet. As individuals get better at their craft and further down the hobbyist rabbit hole, (I personally) wonder where to draw the line as a hobby. Don't! We develop makerspaces to propel creation into hyperdrive; the next and last step in completing the artist's high-end project circle is selling the last project so the new batch can be justified. Because rapid fabrication and makerspaces are "a thing" now, people need to understand what comes next with all those creative and production juices flowing. I think many makers may not approach their custom brazed bikes, amazing wooden trinkets, or tube guitar amps from the view a painter would monetize paintings- but they (we) should. Art stores, art shows, audio meetups, DIY ecommerce sites, Etsy, craft conventions... These are real venues we should be adding to our vocabulary as makers. It is the last step to a full circle justification, and for me (in my hobby bird photography work for sure) it simply feels amazing to be at that stage of chatting it up with locals about where I took the picture of the merganser. It takes way more effort than I or my fellow artists will let on, (learning high-end home printing, commerce, getting a materials supplier, website, etc) and marketing/selling is not NEARLY as glamorous as hacking away at our craft. But, at the end of the day, this is the right thing to do. Showing others through commerce the true value of maker craft not only educates and enriches, but increases the value in our local economies and local-maker-wizardry.

Wolf Pine @ Fox Park #8: 3/16/17, Someone Bumped The Snow Machine

Fri, 17 Mar 2017 00:00:00 GMT

I decided to snowshoe into Fox Park at about 6:30am Thursday, 3/16/17. Firstly, the forest sounded like it is in shock; dead quiet. No crows or bluejays, only the odd cardinal or chickadee singing a questioning song to advertize its previous idea about making babies. The snow is at least a foot deep, all powder, deadening sound as well as the attitudes the local animals were gearing up for the so-close-but-so-far spring. I found no tracks- none, zero. The mammals are sure to be down below the snow again, grudgingly re-entering the "subnivean" lifestyle. My owl friend was not home either; as we know from the GHO talk from a previous post, the vertical depth challenge snow presents doesn't deter the GHO from hunting, but plenty of other factors can cause an owl to take a vacation in a different tree. Obviously, the clues I have been using with this "Strix" or "Bubo" buddy are way more annoying for the owl itself. Imagine being mobbed by "idiot bird brains" day in and day out, especially when they tell their cronies where your house is... Yes, sometimes it is time to take a break. The snow was definitely the "highlight" of this morning's sit spot, even as a snowshoed out of the park at 7:45.... For how long though? When will the spanish-speaking warbler team from Panama touch down? ?

Sit spot #N.1: The GHO Talk

Mon, 13 Mar 2017 00:00:00 GMT

Great horned owls. Except for the only exception feasible- [Which is of course the Great Grey Owl who has decided to move to southern NH from its previous home in frigid Canada](http://www.wmur.com/article/rare-great-gray-owl-spotted-several-times-in-newport/9104108)\- the GHO is the ultimate, TOTL, high-ender of the hunters in New England at the very least. There is a reason all the other members of the animal kingdom hate these "_Bubo_ " eagle-owls as much as they do. GHO's have every trick in the book, every bell, whistle, and gadget, making the whole evolution game seem wholly unfair to, say, an unassuming chipmunk. I wanted to give a quick rundown of the key toys and tools the GHO has at its immediate disposal, why I care, and why everyone else should care. *This image has been lost to time. The original was hosted on WordPress.* By shudrburg - http://www.flickr.com/photos/shudrbug/1502256414/in/set-72157594307880833/, CC BY 2.0, https://commons.wikimedia.org/w/index.php?curid=2882446 ** 1.** The "ears" GHO's ears are essentially their entire head. The poky things are literally there to throw folks off, though the idea was originally to emulate some bark or a pair of pine cones, some think... Though horns, ears, or party hats are probably ok too. As I say above, one could say with a fair amount of accuracy the _entire_ head _is_ a single, huge ear; Those pretty concentric eye rings? Chamfers and fillets on the face? these are funneling, _extracting_ every scuffle and heartbeat falling in the laser-like path of the big, round, swivel-face. Remember: these owls are seeing with their ears. The GHO is always sleepy during the day, even while other owls might be a bit active- ruling out light as a reliable system for vision. Below I snipped a good description of the GHO system. The asymmetrical face construction of a GHO also is used for "vertical" hearing- check this out: "An Owl uses these unique, sensitive ears to locate prey by listening for prey movements through ground cover such as leaves, foliage, or even snow. When a noise is heard, the Owl is able to tell its direction because of the minute time difference in which the sound is perceived in the left and right ear - for example, if the sound was to the left of the Owl, the left ear would hear it before the right ear. The Owl then turns it's head so the sound arrives at both ears simultaneously - then it knows the prey is right in front of it. Owls can detect a left/right time difference of about 0.00003 seconds (30 millionths of a second!)" (taken from: http://www.owlpages.com/owls/articles.php?a=6) Obviously, anything can hear something more in the left ear and less in the right ear and know roughly where it is. However, "roughly" isn't in the GHO vocabulary. Other studies have shown how owls crunch sounds at .00003 seconds; accuracy comes at the price of wildly complex brain structures that are solely used to draw auditory conclusions. Think; each ear has a set of pre-decision-making brain structures, analysing in parallel both the intensity of incoming sounds and the passage of time- synced perfectly to the other ear's set and the brain as a whole. Look at it this way; the GHO sensing system, with its multiple super-computing cores is physically 3 times the size of the one found in our usual "smartest local birds"- the crows and ravens. No wonder the owls are always being bothered by crows- they must be so jealous! (and GHOs are a unrivaled predator to crows if the tides turn nasty) 2\. The wings Firstly, our local owls are dialing in around 10lbs of lift capacity. This makes even fat wild bunnies a piece of cake, no pun intended. Supposedly, these wings are rather disproportional to the usual bird weight/wing lift ratio, though I wouldn't know. Just assume the owl can lift around 2.5 times its body weight, at least as far as a nearby pine tree to start snacking. More importantly however, these evidently powerful wings are dead silent. The legend goes the mouse has no idea about its rapidly nearing demise until it feels the claws come in from above. I personally believe this to be 100% accurate- every possible flight detail has been subject to evolutionary innovation, from the crinkled, broken shape of the beefy coverts and wrist to the micro-turbulent primary and secondary feather structures, all the way to those huge, fuzz-covered legs and feet. These oversized fluffy feet, by the way, have a clamping force beginning to enter young snapping turtle territory... ...You have been warned. The micro-turbulences generated by the wings has sparked much intrigue over the years. Each feather exhibits a subtle, diffusive, "spiky" shape- the idea being the "ripping" and whooshing of air you hear from most birds when they take off can be removed by softening the hard edges of the feathers and wing such that the overall acceleration and lift isn't hindered. This acoustic principle is really the opposite of how their faces work, diffusing sound instead of funneling it in. An intersting addendum in this GHO technology is how the coverts- the thick, leading edge of the wing- are formed. Many other predatory birds, like the local supercar of aviation, the peregrine falcon, bank on really sharp, hard curves and edges in the coverts to squeeze as much speed and maneuverability into these big important body parts. But not the GHO! Without sacrificing effective speed or agility, the coverts are sort of rounded and "broken up" into smaller edges and curves, directing the air and subsequently sound into a more diffuse pattern. Case to point: the mouse example. The general consensus on these coverts is these nubs are exactly the tool needed for the final swoop in to snatch the ground-dwelling prey. Even at a steep, speedy angle, the GHO can silently hurdle to the ground without spooking anyone. Amazing! 3\. Other gizmos and gadgets: The color and shape _is_ its favorite spot to sleep. The local white pine trees, especially the trunk, are prime real estate for sleepy GHOs after a night munching- so, the owl naturally looks like a white pine tree trunk (complete with two pine cones on the top). Despite these owls being huge, they are rather common (in theory). The chances of finding one with human vision is essentially impossible, so we must rely on other clues on its whereabouts. The digestion system is the best among owls. When the forest has been robbed of mice and chipmunks, GHOs can- and might even enjoy- eating frogs, big insects, reptiles, domestic pets... The trick is they simply eat the whole animal. There is no kerfuffling with fur here or teeth there; GHOs just go for it, 100% in. This may contribute to the widespread success in the north east, with our crazy weather and prohibitive geological extremes other species struggle with. In conclusion: I hope this has been both educational and convincing enough to be enthused about owling. Something this special _and_ this relavent in the northeast is too important to ignore. -Jess

Wolf Pine @ Fox Park #7: 3/12/17, Surprise! It's Cold

Sun, 12 Mar 2017 00:00:00 GMT

Arrived at Fox Park at about 1:30 today, 3/12/17, under a deceiving blue sky and some light scattered clouds. A prohibitive 10 degrees and ~0 degrees with wind chill (at most) set the tone of my walk, though it did make walking a bit easier in general- the snow, mud, and debris had been frozen solid, so I could comfortably walk my sit spot loop in sneakers and and number of pairs of socks. I spent the first 20 or so minutes wandering the base of Fox Park, going around from the usual parking lot, to the lower-level parking lot, around the artificial, square-shaped wet area, and up around the immediate road. I didn't find any owl-related clues, but I did find some other points of interest: a few turkey vultures and a few red wing black birds. I think it is apt to be a bit concerned (mostly for the red wing black birds) because they clearly thought it was spring time, but it actually is not. Ground foraging, insect eating birds who rely on marshy habitats do not seem suited for today's balmy 10 degrees. Moving into my sit spot, I heard a brown creeper singing and a muffled "beep!" from a hairy woodpecker. At least they seem happy. The local "hooligan crows" and their cronies (blue jays) were zipping around in little gangs occasionally. There was very little localization, so I think they were just rabble rousing and partaking in tomfoolery. I did not take photos today, despite hauling my equipment around. It seems like much of what I saw two days ago is solidly frozen in place from the time being.

Wolf Pine @ Fox Park #6: 3/10/17, Theoretical Owls and Real Tree Bark

Sat, 11 Mar 2017 00:00:00 GMT

I hit the trails at 5 am sharp Friday 3/10/17. Nice morning, some cloud cover but a reasonable temperature for prospective chipmunks for jaunt out of the subnivean environment. *Some images from the original WordPress post are no longer available.* This is where I started- within the vicinity of my theoretical big owl. The big murder of crows was there; that is a good start. Being around dawn-ish time however, the raucous birds dispersed within half an hour, perhaps implying my nocturnal friend either fell asleep in a huff or flew away for a less noisy and more welcoming environment (if there even is an environment that welcomes oversized, silent, essentially unrivalled killing machines...?). I now can see a distinct, GHO-likely trend. The crows are noisy at the time I know owls like to come back to a nice spot to go to bed, thus a time they are most easily bothered; the crow activity is extremely centralized around this stand of large, sheltering pine trees- the crows all seem to circle the trunks of the pines that are growing most close together. GHO's love pine tree trunks, and rarely will nap far from the center. I have noted the crows are never "bothering" a deciduous tree, where barred owls could be more likely found (than GHO). I heard a few possible "whoos"the first day owling in response to screech owl calls, which is common with the GHOs. I played barred too around then, so I wasn't sure (the sounds I heard were very muted and did not complete any full call, but were unique owl-ish sounds nonetheless). *Some images from the original WordPress post are no longer available.* Here we have two common sights: red, "spear-like" Beech buds and the lingering brittle beech leaves. These are everywhere on my way into my sit spot. *Some images from the original WordPress post are no longer available.* Here we have some white pines. These are the only species of pine I could find around my sit spot... Paper Birches? -Yes, but they are too easy. Here we have a grey birch (the grey colored one) and a yellow birch (the yellow-tinted one)- both of which are sort of near my wolf pine tree. *Some images from the original WordPress post are no longer available.* Hemlocks! Look at the "crunchy" bark. These are everywhere... *Some images from the original WordPress post are no longer available.* Red oaks. Look at the deep cracks exhibiting an almost reddish color..... *Some images from the original WordPress post are no longer available.* ...And some red oak leaves. Pointy, "fire-flame tipped" leaves. They are also reddish, which helps a bit. *Some images from the original WordPress post are no longer available.* What could these be? White oaks! These have this random pattern to the nubby bark, and have a "whitish green" lichen or fungus on it more often than not. The tree to the right is the best non-greenish bark I could find. *Some images from the original WordPress post are no longer available.* The obligatory white oak leaf, in with some beech leaves. these do not seem to be nearly as prevalent as the red oak leaves in terms of what is currently still on the ground. This is the only leaf I could find. *Some images from the original WordPress post are no longer available.* To conclude, here we have a striped maple and a red maple. I assure you: both maples are well into adulthood! Despite one being green and thin and the other looking old and broken, this is in fact "how they do". Distinct barks, but also easy with the "opposite" branching pattern (not shown). In addition, the red maples are not only opposite branches but branch in a neon crimson color. This helps I.D. quite a bit. -Jess

Wolf Pine @ Fox Park #4: 3/5/17, Noon-ish time

Sun, 05 Mar 2017 00:00:00 GMT

This is the kind of day it is today. Not a cloud in the light blue sky, the sun casting sharp shadows from the bare branches. A bit blustery, and cold- I'd guess 25 degrees, not including wind chill. The snow is very hard, and has a icy surface. I wandered into Fox park at around noon- all I could hear was a few high peeps from chickadees and the occasional crow yelling at something. Notably, the woodpeckers seemed absent on my way into the woods toward my pine tree- could the extra-frozen trees deter all but the most robust woodpeckers? Usually at least a downy will be somewhere, tapping away. *Some images from the original WordPress post are no longer available.* These two trees on the right exhibit this intersting "crinkly", wafer-like, "scaly" bark. Around these parts, I would wager a guess these are black cherry trees. Magnificent! One interesting feature: they are always alone! I have yet to see two of these "scaly" trees within eyesight of each other. Compare this to the gaggles of hemlocks, clubs of white pines, and stands of beeches... I really haven't the foggiest why such an impressive and dense tree would manage to populate itself so sparsely. *Some images from the original WordPress post are no longer available.* Someone has been doing house cleaning! This cavity in the tree is getting excavated, and upon further inspection, the space inside is enormous. The wood chips at the base look relatively fresh, and the wind hasn't blown the sawdust off the bark yet. *Some images from the original WordPress post are no longer available.* These green conk mushrooms I found are gnarly. Beginning and end of story. *Some images from the original WordPress post are no longer available.* As I left my sit spot, I found myself staring into the top of each pine tree I walked under. I am getting a gut feeling the owls are going to be getting restless for spring soon. So many more mammals will become breakfast, lunch, and dinner (for our GHO and barred owls especially) in a few weeks when they emerge. I think I will aim to do my few sit spots before sunrise, armed with the saw-whet call, and see if I can pick out who is living up there in the multitudes of pines. Bonus: I found these well-preserved, flash-frozen crow footsteps literally wandering out of the parking lot and into the park, following the "human trail". *This image has been lost to time. The original was hosted on WordPress.* -Jess

Wolf Pine @ Fox Park #3: 3/3/17, Noon-ish time

Sat, 04 Mar 2017 00:00:00 GMT

I walked into Fox Park on 3/3/17. Note how I could walk right in; the snow has settled and melted into a single layer that had been frozen the night before. The sun was shining, but an impressively chilled wind was blustering around. That morning at 6am, the temperature hit somewhere around 40 degrees, which was followed by a hour-and-a-half blizzard starting at around 9am, followed by blue skies similar to what I experienced at 6am but about 15 degrees colder. ??? I tried to document the interesting tree formations, issues, and patterns today, as I realized after more natural history class time I was taking these trees for granted and focusing on the more "immediately exciting stuff". First off, I was noticing these Beech leaves everywhere. They seem to be the only leave around that is so stubborn about staying glued to the tree almost indefinitely- until of course the lext version pops out to replace it. *Some images from the original WordPress post are no longer available.* It was noted in class these leaves stay attached to the tree all year, the idea being maybe these trees can get a quicker head start on photosynthesis come spring. *Some images from the original WordPress post are no longer available.* Coppicing? General bizarreness? Next, I was finding all sorts of tree species with this tightly knit organization, which implied they are possibly sharing a mega-root system. The following tree photos all exhibit this super weird "window" into each tree's heartwood. Fox park is riddled with this phenomenon... ...And I have absolutely no idea why. *This image has been lost to time. The original was hosted on WordPress.* *The photograph that was here has since disappeared from the web.* *Image no longer available -- a reminder that digital things are impermanent too.* *This image didn't survive the migration from WordPress. Such is the web.* *The original image, once hosted on WordPress, has been lost to the digital ether.* By the time I got to actual spot to do some serious sitting, I already had these questions percolating. it seems like most trees I encountered dealt with a sort of trauma at the base, and are trying to recover. Take a moment to observe the "Bull" pine, or "Wolf" pine. That is my spot- isn't in gnarly? First off, it is HUGE. Those birch trees in the foreground aren't exactly little. I met a few dog walkers soon after I arrived under its crazy branches, one of whom said there is a chance this was a shade tree for farm animals eons ago. We wondered if the branches grew into this oddly un-shady shape when the farming situation came to a halt, and other shade and competition for the sun forced the needles farther into the sky. Below are some close ups of the uber-gnarlies this tree has. Note the little holes about 3-4mm across. these look like bug activity. (no shredded bark from even the most careful woodpecker, and seem totally unorganized. That said, the quarter-sized cone shaped holes look like red-bellied activity, but was quickly abandoned. It seems this tree has some tough bark, and the bugs are just too hard to get at. *Some images from the original WordPress post are no longer available.* Looking out from my spot, I see the fungus/Scale beetle issue on the nearest Beech tree. I also found this basketball-sized hive, high up in a tree. Obviously, it is the epitome of fine construction techniques, because it is basketball sized, after a whole slew of crazy weather events. *Some images from the original WordPress post are no longer available.* These two conifer trees had me confused for a while. The one on the right is a more spares when it comes to needles, and the bark/stem pattern is more "expansive" and "flat". I dub this a hemlock, because my first gut instinct is to forage underneath for the highly flammable branches, which often have those thin, bendy fingers that ignite immediately and violently. The tree on the right however, did not evoke this gut response. That said, I think it is also a hemlock, just much younger - I think. *Some images from the original WordPress post are no longer available.* Bird activity: On the right, we have my favorite, all time greatest "big woodpecker" hole example tunk. Top left: Probably a Hairy woodpecker. Note the lack of a bezel around the hole. Also, I doubt the local Piliated would be able to fit its beak that far into a tree, this hole is just too small. Note also the Hairy does have a rather long beak. I have heard and seen a hairy woodpecker in this area. Top right: Ehh, this one may be a collaborative effort. I see the inner circle STARTS at the size the hairy woodpecker left off. I say red bellied at that point. I have heard the bubbly-squawk-like sound of this woodpecker here on occasion. Also- note what happened! The tell-tale funnel shape that starts about 3/4 inch seems to have given someone else some ideas.... Bottom: Crazy Piliated woodie, who, as far as I can tell, has literally killed at least 1 tree nearby, by obsessively boring huge channels through the trunk (and almost out the other side in some cases!!! The upper right hole looks like the pileated could have given that one a go, just to give it a test drive, before going back to work on the bottom hole. *Some images from the original WordPress post are no longer available.* Finally, I found an example of all 4 major league woodies in one sit spot visit! Here is the resident diminutive downy, going berserk on a dusty branch. Look at that head go! My shutter way fast, but evidently not fast enough to stop a speeding downy head. Until next time!!

Wolf Pine @ Fox Park #2 (2/27/17)

Thu, 02 Mar 2017 00:00:00 GMT

I wandered into my sit spot last Monday evening, before sun down- slightly overcast, well above freezing. Upon entering the park, significantly warmer air pockets dotted the trail- why, I don't have the foggiest idea. This seems important, but I will leave the cause of such intense heat-bubbles of what a question to gnaw away at later. **Observations from the Wolf Pine @ Fox Park:** Firstly, the birds. I did not wait long before I heard a few White Breasted Nuthatches tooting. Soon after, I heard some call and response with some Cardinals, a first this year for me. The pileated activity has not wavered, but I wonder if the bugs are more prevalent now with the hot air floating around. I think I am seeming more medium sized woodpecker holes, particularly those from the Red Bellied Woodpecker. Downy/Hairy activity hasn't changed much, still the odd one of those here and there. Most interesting is the Brown Creeper activity. These birds went totally under the radar until now. They do not migrate- but all of a sudden, they are singing , swooping, and creeping like mad! I wonder if their two-tone whistle is a mating call, like the Chickadee's (thanks for the info, Kurt!) The tell-tale, "only goes up the tree, never down" nuthatch-shaped bird is extremely well camouflaged. I wonder if that is more helpful in the winter, when they are going incognito? I mean, they are kind of obvious when they are burbling around the forest, As for plant life, some kind of shrub that has reddish tendril tips is getting ready for show time. the previously hard and dry bud tips are now moist and are easily squished. This looks like fair game for little bugs and subsequently gleaners.... I can't wait to see the forest unfold around me as we get deeper into spring. Also: A fellow student recently told me quite possibly the most important pine tree-related info I have ever heard. Ready? "White pine trees have bunches of 5 needles because the word "white" has 5 letters. So, logically, the red pine has bunches of 3 needles, because there are ONLY 3 LETTERS IN THE WORD "RED"." ...Crazy. Food for thought.

Wolf Pine @ Fox Park #1 +Bonus Winter Birds

Mon, 20 Feb 2017 00:00:00 GMT

Today and yesterday, 2.19.17 - 2.20.17, have officially kicked off my first real visits to my "sit spot" (required for all adventure ed students at PSU) and commutes around campus armed with my bird rig, ready for the warmer-weather inclined birds. **Observations from the Wolf Pine @ Fox Park:** Snowshoed into Fox Park around 2:15 on Sunday, 2.19.17. Weather: After repeated heavy snow falls, Sunday was the first day solidly above freezing- thus a large amount of dripping and snow-condensing was happening. My wolf pine was in a bit of a freezing puddle, with ~2 feet of snow accumulation surrounding its base. High pressure day, bluish-grey skies and scattered wispy clouds. Light breeze, and fairly quiet. Upon quieting myself and my raucous snow-hoverboards, it became apparent how few birds and squirrels were about. I could hear "whispers" and chips from passerines, but they sounded far away, likely to be lower on the hill, near the squishy earth and faux-pond. Squirrels maybe rustled a branch or two during my sit- note the trees where about half evergreen and probably not a food source for these little mammals. These trees would, however, provide good coverage from avian predators... I wonder if the squirrels have thought of that. *Some images from the original WordPress post are no longer available.* Perhaps the surrounding homes and intermittent (not on Sunday) construction sounds provided a safer space park wide. Owls and to a lesser extent hawks are irked to no end by these sounds and regular but unpredictable human activity. I have observed elsewhere in MA owls are not put off by circadian dog walkers at all; in fact, I would glean most of my "big bird" info from the unperturbed 2 - 3 times a day dog walkers of my neighborhood. Great horned families, bald eagles, and belted kingfisher pairs could care less about 2 dozen or more dogs pass under their homes a day, but the moment a motor boat, police cars, or loud parties occurred these unbelievable species would vanish. I make this digression because _this is a_ college town, and the park is surrounded by active dwellings of different sorts, including development sites. THUS: there were essentially no rodents/lagomorphs/etc. (easily findable ones that is) *Some images from the original WordPress post are no longer available.* Speaking of which, the tracks were tough to figure out. Heavy dogs? Yes. beyond that, the melting snow and dripping was creating a fairly non-descript blanket over any crazy prints. I noticed remarkable BIG woodpecker activity, i.e. Pileated and Red Bellied/flicker- especially on my way out of the park. Holy smokes are the pileated OCD around here! *Some images from the original WordPress post are no longer available.* Also Note the intersting spiraling growth pattern on this Wolf Pine limb. It is long dead, but appears and felt denser than "ye average" pine tree. ?? *Some images from the original WordPress post are no longer available.* I plan to get back to my spot ASAP for more warm weather observations. I believe this is the forecast all week! **BONUS WINTER BIRDS FROM MY COMMUTE THIS MORNING:** A loud house finch and a lovely Bohemian waxwing. *Some images from the original WordPress post are no longer available.*

Develop A Limitless & Habitual \"SoIThen_SoICould_\" Ethos

Wed, 21 Dec 2016 00:00:00 GMT

_Some call it yak shearing._ **Some call it the next step, and a method to never let the next step get in your way.** ...So I then I learned to make custom blown vacuum tubes at home so I could build an amazing artisanal [nixie tube clock](http://www.daliborfarny.com/). ...So I then looked up how to throw a [bigger flower pot](https://youtu.be/QmhMnQBMZ10) using wheel-thrown pottery techniques on youtube, so I could become a better and more refined artist. ...So I am building a [CNC mill](https://vicious1-com.myshopify.com/) so I can make nice looking wood headphone cups for the DIY planar build. Occasionally, I find myself looking into space and feeling like whatever the next steps may be in my various hobbies are remain unreachable, need to wait, cost more than I have, or is in someway not the right time to continue. While some of these may be true, it does not mean all that remains is slow, unyielding time. More often than not in these situations, I really am looking at a step that is unreachable or untimely, but is actually not the end-all and last step. There is usually more than meets the eye; there are tasks to be done, as small as they might be, that will help a process along. I think of this mathematician: *Some images from the original WordPress post are no longer available.* For example- I do not have access to all the tools I need to complete my projects; it is inherently exciting to think about the actual fabrication of evolving ideas. That does not mean the tools needed to complete the projects are needed to do the peripheral learning and knowledge-gaining and scheming and planning. We can easily see in writing, the cutting and buffing and bolting of things is just a tiny part of a huge commitment in time, learning, and design. Have I exhausted all learning and designing time? No! I can always add an extra step between me and a goal, no matter how small. Learn Autodesk Fusion 360? add that to the list. Complete a few of their 10 hour instructional webinars? That sounds important. Learn to roughly mill and glue wood into end-grain orientations for future CNC-ing for the future headphone cup-rings? I should do that too. **So...** ...Is your back hurting from your posture or your chair? If so, you should learn better posture and/or build a perfect chair (and learn fine wood luthiery to make a snazzy inlay of your name, Blender and CAM to mill unique chair legs, etc, etc....) **🙂**

Music Sketches 2: Rethinking \"Canon in D\" \"guitar\" Discipline & Tone pt. 1

Fri, 16 Dec 2016 00:00:00 GMT

*Some images from the original WordPress post are no longer available.* Between 300 and 400 years ago, this archaic string of melody, harmony, and straight forward rhythm came about. I don't read this kind of scribble, but I, like many others, have heard Pachelbel's Canon in D. Christmas tunes, 80's orchestras, video games and more have ripped up this beautiful idea and tried to glue it back together. Below is what I believe to be the most influential niche version, posing a striking composition highlighting the power of discipline and learning (read: Time, Standards, Challenge) in lieu of the internet (read: all "[hard skills](http://www.investopedia.com/terms/h/hard-skills.asp)" are free and readily available in a computer-box near you) This, to both my pair of ears and many pairs before me, is simply stunning. The deft skill and accuracy, the angelic warbling tone.... This is an example of almost complete mastery AFAICT. What granted [Lim Jeong-hyun](https://en.wikipedia.org/wiki/Lim_Jeong-hyun), generally referred to mononymously as funtwo, the ability to play at this level? [Like just about everything, it seems his track record on the internet plus his standards and time against the challenge of learning guitar equals... ...the above video. ](http://www.nytimes.com/2006/08/27/arts/television/27heff.html) Observe the 75% "slow" speed (thanks youtube and guitarteacherdotcom for access to this stuff): # ....Coming up next: Unpacking discipline and learning through funtwo's "guitar"

(beta) - How to build \"The Phone Charger\"

Thu, 15 Dec 2016 00:00:00 GMT

Imagine, any AA or 9v battery could charge your phone and other USB gadgets. Behold, the LM780! This is a voltage regulator chip. Stick between 5 and, say, 9 volts in one end and huzzah! (about) 5 volts pops out the other end. these cost a few dimes and can be had on ebay 10 for 4$. Below is the BOM: (to make 10 chargers!) Sourced via ebay: $4: 10x of LM780 5v $4: 10x of little toggle switches $4: 10x of 9v snap connectors (I used a 6v supply from AAs but as far as the chip is concerned it doesn't matter to much. (I am reading 4.~ volts and enough power to charge a phone out of mine now) $0: PCB- Technically they aren't even needed, but for our uses they make the soldering and building more straightforward. $1: 10x of USB A ports from China $?: Casing- be creative. I want to make one in a shrink-wrapped tube or 3d printed box or something. **First, lay the parts out like so:** ...notice how when facing the shiny side of the chip, the left leg is in line with the left hand side of the USB port when looking into the port: *Some images from the original WordPress post are no longer available.* Then arrange them like so: ...Notice I squished the power wires under the bent pins. RED GOES TO THE PIN ON THE RIGHT when facing the shiny side, and BLACK GOES TO THE MIDDLE ONE. ...The remaining leg is attached to the far left pin on the USB port (when facing the chip's shiny side remember). *Some images from the original WordPress post are no longer available.* Then it works!! YAY! *Some images from the original WordPress post are no longer available.* Here I verify it works by charging my commercial usb charger with my DIY duct tape one: *Some images from the original WordPress post are no longer available.* Good luck!

Building Electrostatic \"Earspeakers\"

Thu, 15 Dec 2016 00:00:00 GMT

Below, you will find my collection of notes regarding how to build the electrostatic headphones themselves. This is largely based on stuff found at head-fi, in Wachara C.'s numerous posts and accomplishments on the subject.

DIY Electrostatic Amp Options

Thu, 15 Dec 2016 00:00:00 GMT

Embedded below is my current document highlighting the schematics, BOM, and cost of building a diy electrostatic amp.

Music Sketches 1: Playing with rhythm, division, and jazzy chords

Thu, 15 Dec 2016 00:00:00 GMT

*The audio file that lived here didn't survive the migration from WordPress.* Piano and straight up 4/4 drums: Here we have an interesting chord progression idea. In the 4/4 groove, the piano plays the second chord on the "and" of 2... Very cool. The pickup for the second chord starts on two, as I visualized in the midi roll diagram. *Some images from the original WordPress post are no longer available.* With a root of C (what I played it in), this could be called C minor to D flat, major 7, while at the end of the fourth bar the progression finishes on G7.

Opportunistic birding: 14/12/2016

Wed, 14 Dec 2016 00:00:00 GMT

**Today, I am in a good mood and feel happy because:** The two Bohemian waxwings were still hanging around the museum of cedar waxwings, both of whom arrived yesterday. Bohemian waxwings special because they are far, far rarer than the cedar waxwings. The cedars are pretty snazzy-looking by themselves, and I look at them in my daily travels too. One can tell the bohemians apart by the crimson/burgundy/rufus color under the tail and around the face. They are also a bit bigger, but that's only helpful if one can compare a cedar with a bohemian at the same time (which is still useful because, as we can see, they like to travel around together). This was opportunistic because I "birded" only in transit, as my route to my dorm door takes me under the waxwing tree. *Some images from the original WordPress post are no longer available.* **What does this mean?** Opportunistic birding is a easily one of the most rewarding ways to enhance life on earth. Imagine deriving copious amounts of joy and justification every day, revolving around the day-to-day views of exotic birds. Are there really that many exotic birds wandering around a tree near you? Absolutely. Everywhere I have lived and traveled to, I will find at least one exciting bird –that is, if I tune in correctly. The first part of finding divine avian joy is simple; * Decide birds are fascinating, exciting, and scattered around you like a Where's Waldo? original * Allow the innate curiosity and "buzz" be way more important than immediate applicability * Subsequently decide to read and research as much as possible in large binges about local birds... * Join [E-bird](http://ebird.org/ebird/MyEBird?cmd=Start) * Download [Merlin](http://merlin.allaboutbirds.org/) (North America and growing) * [subscribe](http://ebird.org/ebird/alerts) to all the local bird lists and updates * Visit your local sanctuary, patch of woods, river, marsh, sea bog, neighbor's bird feeder Then, once you know a robin and from a heron, a "murder" of fish crows from a "tournament" of wintering white throated sparrows to a "museum" of mixed cedar waxwings, you are done (with part one...)... ...Now the real fun begins! You will know you have reached part two when: * You are walking to work or class and hear some chips and beeps... * it is winter: You are inland, in a light mixed habitat of new-growth trees and leafless shrubby plants: * 2+2=4: this must mean there are small passerines wintering, likely in a mixed flock * You stop for 30 seconds; You hear fleeting piece of a sonorous whistle, then a flit with vertical slate and white stripes * 4+1+1=6: You feel euphoric and grandiose, then continue walking to class, thinking happily about the slate-colored-junco group you just witnessed containing a single white-throated sparrow. You arrive to your destination on time, unharmed, and in a jovial condition. ...This is opportunistic birding. Good Luck! *This image has been lost to time. The original was hosted on WordPress.*

Example Photo Work

Tue, 13 Dec 2016 00:00:00 GMT

## Here are some examples of my work: ([Contact me](mailto:Jess@sulliwood.org) for a quote and full portfolio on professional jobs, photo or video! I also sell assorted bird photo cards, if that's your thing - I can send low res collections to choose and order from.) [![dsc09997](/images/posts/dsc09997.jpg)](/images/posts/dsc09997.jpg) [![img_6354](/images/posts/img_6354.jpg)](/images/posts/img_6354.jpg) [![img_5674](/images/posts/img_5674.jpg)](/images/posts/img_5674.jpg) [![img_2827](/images/posts/img_2827.jpg)](/images/posts/img_2827.jpg) [![mg_9479](/images/posts/mg_9479.jpg)](/images/posts/mg_9479.jpg) [![kq8q8111](/images/posts/kq8q8111.jpg)](/images/posts/kq8q8111.jpg) [![img_8419](/images/posts/img_8419.jpg)](/images/posts/img_8419.jpg) [![img_8723](/images/posts/img_8723.jpg)](/images/posts/img_8723.jpg) [![img_2209](/images/posts/img_2209.jpg)](/images/posts/img_2209.jpg) *11 additional images from this post were too large to survive the WordPress migration intact (originals were 10-26MB; only the first 1MB was archived).*

Required Custom Tools?

Tue, 13 Dec 2016 00:00:00 GMT

"First, we build the fusion reactor. then, upon creating the mini sun, we shine the light onto the prehistoric spores. We now evolve the subsequent grass to be healthy for the custom-wound sheep animal we designed. Soon we will harvest the sheep to make the leather, which will of course be used for making the most comfortable ear pads in the... ...WHAT??!! You wanted them in black velor and suede?!?!!" ### Regardless: In order to make the design and manufacturing processes faster and the most flexible a dorm environment (in addition to making the fabrication of bizarre custom objects that may or may not function possible), all outsourcing and use of specialized equipment must be cast aside. Instead, we must build very, very small versions of rapid prototyping tools. These tools are: * 3 axis CNC mill. It only needs to be ~150mm cubed on the x,y,z axis, but it must have enough kick to not break or overheat when cutting hard wood cups. This tool can be used to drill PCB (ES stators), make internal chassis pieces, and slice various materials to fabricate parts that can’t be 3d printed. There is much to learn about how to use this tool, such that assembling one one is only the beginning. * 3d printer. This is my go-to for housings, internal spacers and interior connectors. It is easy to use with some research, and a few hundred hours behind sketchup. CADs include Sketchup and many additional extensions, Meshlab, 123d products, Blender, Inkscape, and more. Often people will use multiple softwares to get the best results. These printers can be built cheaply, requiring only an extruder, a small brain, and stepper motors (which can be avoided if extra work goes into recycling old parts from the floppy disk era). The rest is up to the builder’s expertise in figuring out how to make the x,y,z axis function. * Membrane stretching jig and etching tank. These are about as cheap as an inner tube, a few buckets, and a battery… ...However, care must be exercised in figuring out how to get repeatable stretch percentages and solid etches without destroying other materials in the chemical process. * Transformer and voice coil winder. This will be a very custom and relatively unusual tool, but will be helpful in getting the optimum step up voltages in amps. Additionally, custom voice coils can be made for new dynamic driver designs. The electrical parts are all very similar to those of a 3d printer or CNC- think a single axis moving back and forth to position wire very carefully over a spinning transformer or coil.

The Audeasy Build

Tue, 13 Dec 2016 00:00:00 GMT

These are my notes on the current project, build a set of fabulous planar headphones from scratch. This window into my Google doc will auto-update here as I add content.

Waxwing Audio:

Tue, 13 Dec 2016 00:00:00 GMT

High-Brow Foreword: “I understand ultra high-end headphones and audio pieces are the epitome of superfluous consumerism, only for the .1% of the world’s population, and do not get positive remarks from spouses as good places to spend money. Luckily, none of these are what this venture is about. My goal is to meld hand-crafted art and design with an open source philosophy, culminating in unique audio pieces and headphones, as well as a completely open source build log and design resource for others.” To be frank: I am a dorm-dwelling college student, studying experiential education. Everything I make is made from scratch, in my dorm, when I’m not doing other things more pertinent to my BS in adventure experiential education. As we know now, the only tools anyone needs to make anything these days is access to the internet, time, and a deep desire to learn…