post updated 11/23/20

Demos, interpreter implementations & data ingress tools for annotating, interpreting, and deploying trained models.

Web:

demos
Visit audio demos on Heroku
demos
Visit Leaflet.annotation demos on Heroku


Watch environment setup here


Watch Leaflet.annotation demo here

Hack upon these demos:

# Clone:
git clone --branch=master --depth=1 https://github.com/Jesssullivan/tmpUI && cd tmpUI

## Follow the prompts: ##
npm run-script develop-web-demos
Demo Description
deploy/demos/spec_record_crop_dl
deploy/demos/spec_record_crop_post
Experiments with record --> crop --> classify --> download; both client-side & server-side classifications methods
deploy/demos/load_audio
deploy/demos/spec_display
Experiment with Macaulay audio sources --> spectrogram
deploy/demos/spec_record_crop_v3
deploy/demos/spec_record_v2
deploy/demos/spec_record_v2
Single page feature experiments
deploy/demos/webgl_init
deploy/demos/deploy/demos/webgl_float_test
deploy/demos/spec_record_v2
Evaluate web client's capability for classification

Notes:

Configure Flask in config.py:

# config.py

# `True` serves demos @ 127.0.0.1:5000 via node proxy (set `False` for production @ 0.0.0.0:80)
devel = True

# rebuild header + demo + footer html renders before serving anything (set `False` for production):
prerender = True
  • / runs webgl_init, which figures out if the browser can or cannot make classifications and routes the client accordingly.
    • classification options:
    • if browser cannot do classification (i.e. safari on mobile, webgl mediump not supported) recording is beamed up to /uploader_standard for processing
    • both POST destinations /uploader_select & /uploader_standard can also be operated from within browser as a multipart form

Leaflet.annotation @ tmpUI:

demos

Visit Leaflet.annotation Audio demo
demos

Visit Leaflet.annotation Photo demo
  • (Jess is still typifying Annotator source, hang tight)

Hack on Annotator:

#### develop-anno-demos:

# packs annotator demos
# generates unique openssl cert & key
# serves annotator demos on node http-server

npm run-script develop-anno-demos
# pack only tool definitions @ `./src/annotator_tool.js:
npm run-script build-anno-tool

# pack only implementations of audio annotator @ `./demos/annotator_audio.ts:
npm run-script build-anno-audio

# pack only implementations of photo annotator @ `./demos/annotator_photo.ts:
npm run-script build-anno-photo
Demo Description
deploy/demos/annotator_audio Leaflet.annotator tool implementations for generating, labeling, exporting mel spectrogams as annotation data
deploy/demos/annotator_photo Leaflet.annotator tool implementations for labeling & exporting photo annotations
deploy/src/annotator_tool epic Annotator_tool entrypoint, contains class handler functions

Swift Native:

# Hack on Swift stuff:
npm run-script develop-swift-demos
  • focusing on codepaths for:

    • tflite interpreter
    • generating mel spectrograms
  • make sure info.plist has permissions for microphone access

  • The entrypoint for Swift tests is ./swift/swift-pkgs-tmpui/swift-pkgs-tmpui/swift_pkgs_tmpuiApp.swift

  • Toggle various interpreter experiments from entrypoint

Other Linker Libraries:

Project:

$(inherited)
-force_load Pods/TensorFlowLiteSelectTfOps/Frameworks/TensorFlowLiteSelectTfOps.framework/TensorFlowLiteSelectTfOps
-force_load Pods/TensorFlowLiteC/Frameworks/TensorFlowLiteC.framework/TensorFlowLiteC
-force_load Pods/TensorFlowLiteC/Frameworks/TensorFlowLiteCCoreML.framework/TensorFlowLiteCCoreML
-force_load Pods/TensorFlowLiteC/Frameworks/TensorFlowLiteCMetal.framework/TensorFlowLiteCMetal
-ObjC
-l"c++"

Target:

-force_load Pods/TensorFlowLiteSelectTfOps/Frameworks/TensorFlowLiteSelectTfOps.framework/TensorFlowLiteSelectTfOps
-force_load Pods/TensorFlowLiteC/Frameworks/TensorFlowLiteC.framework/TensorFlowLiteC
-force_load Pods/TensorFlowLiteC/Frameworks/TensorFlowLiteCCoreML.framework/TensorFlowLiteCCoreML
-force_load Pods/TensorFlowLiteC/Frameworks/TensorFlowLiteCMetal.framework/TensorFlowLiteCMetal
-ObjC
-l"c++"
# niftily switch between xcode versions:
sudo xcode-select --switch ~/Downloads/Xcode-beta.app

Interpreter Operations:

Hack on fft functions:
[./etc/tone.py:](https://github.com/Jesssullivan/tmpUI/blob/master/etc/tone.py)

### copy from here:
cp etc/tone.py .

### generate some .wav files for testing fft things:
python3 tone.py

### ...you can also specify duration in seconds & frequency in Hz like so:
python3 tone.py 5 440

### ...or just duration:
python3 tone.py 2

some fft-related links

Scripts:

# See ./package.json & ./scripts/ for additional scripts

main scripts links:

local ssl:

# Generates local ssl certs for testing w/ node http-server:
npm run-script sslgen

# you can also provide a $DOMAIN argument like so:
npm run-script sslgen hiyori
# ...returns key `hiyori_key.pem` & cert `hiyori.pem`

# ...or:
sudo chmod +x scripts/sslgen.sh && ./scripts/sslgen.sh
# osx is a bit more finicky

tone generator:

### available from here:
cp etc/tone.py .

### generate some .wav files for testing fft things:
python3 tone.py

### ...you can also specify duration in seconds & frequency in Hz like so:
python3 tone.py 5 440

### ...or just duration:
python3 tone.py 2

removing stuff:

# clean up with:
npm run-script clean all
# ...and follow the instruction prompt

# ...demo bundles:
npm run-script clean-web-bundles

# ...demo renders:
npm run-script clean-web-renders

etc, etc etc

xD

-Jess