Skip to content
transscendsurvival.org

Merlin AI Demos!

· 2 min read
BirdingFeaturedIdeas

post updated 11/23/20

Demos, interpreter implementations & data ingress tools for annotating, interpreting, and deploying trained models.

Web:

The image “demos” didn’t survive the WordPress migration.
Visit audio demos on Heroku | “demos” — the original image is no longer hosted anywhere we can find.
Visit Leaflet.annotation demos on Heroku
---|---

Watch environment setup here |

Watch Leaflet.annotation demo here

Hack upon these demos:

Clone:

git clone —branch=master —depth=1 https://github.com/Jesssullivan/tmpUI && cd tmpUI

Follow the prompts:

npm run-script develop-web-demos

DemoDescription

deploy/demos/spec_record_crop_dl
deploy/demos/spec_record_crop_post | Experiments with record —> crop —> classify —> download; both client-side & server-side classifications methods
deploy/demos/load_audio
deploy/demos/spec_display | Experiment with Macaulay audio sources —> spectrogram deploy/demos/spec_record_crop_v3
deploy/demos/spec_record_v2
deploy/demos/spec_record_v2 | Single page feature experiments
deploy/demos/webgl_init
deploy/demos/deploy/demos/webgl_float_test
deploy/demos/spec_record_v2 | Evaluate web client’s capability for classification

Notes:

Configure Flask inconfig.py:

config.py

True serves demos @ 127.0.0.1:5000 via node proxy (set False for production @ 0.0.0.0:80)

devel = True

rebuild header + demo + footer html renders before serving anything (set False for production):

prerender = True

  • / runs webgl_init, which figures out if the browser can or cannot make classifications and routes the client accordingly.
    • classification options:
    • if browser cannot do classification (i.e. safari on mobile, webgl mediump not supported) recording is beamed up to /uploader_standard for processing
    • both POST destinations /uploader_select & /uploader_standard can also be operated from within browser as a multipart form

Leaflet.annotation @ tmpUI:

Where “demos” once appeared, only this note remains.

Visit Leaflet.annotation Audio demo | The image of “demos” is no longer available.

Visit Leaflet.annotation Photo demo
---|---

  • (Jess is still typifying Annotator source, hang tight)

Hack on Annotator:

develop-anno-demos:

packs annotator demos

generates unique openssl cert & key

serves annotator demos on node http-server

npm run-script develop-anno-demos

pack only tool definitions @ `./src/annotator_tool.js:

npm run-script build-anno-tool

pack only implementations of audio annotator @ `./demos/annotator_audio.ts:

npm run-script build-anno-audio

pack only implementations of photo annotator @ `./demos/annotator_photo.ts:

npm run-script build-anno-photo

DemoDescription
deploy/demos/annotator_audioLeaflet.annotator tool implementations for generating, labeling, exporting mel spectrogams as annotation data
deploy/demos/annotator_photoLeaflet.annotator tool implementations for labeling & exporting photo annotations
deploy/src/annotator_toolepic Annotator_tool entrypoint, contains class handler functions

Swift Native:

Hack on Swift stuff:

npm run-script develop-swift-demos

  • focusing on codepaths for:

    • tflite interpreter
    • generating mel spectrograms

  • make sure info.plist has permissions for microphone access

  • The entrypoint for Swift tests is./swift/swift-pkgs-tmpui/swift-pkgs-tmpui/swift_pkgs_tmpuiApp.swift

  • Toggle various interpreter experiments from entrypoint

Other Linker Libraries:

Project:

$(inherited) -force_load Pods/TensorFlowLiteSelectTfOps/Frameworks/TensorFlowLiteSelectTfOps.framework/TensorFlowLiteSelectTfOps -force_load Pods/TensorFlowLiteC/Frameworks/TensorFlowLiteC.framework/TensorFlowLiteC -force_load Pods/TensorFlowLiteC/Frameworks/TensorFlowLiteCCoreML.framework/TensorFlowLiteCCoreML -force_load Pods/TensorFlowLiteC/Frameworks/TensorFlowLiteCMetal.framework/TensorFlowLiteCMetal -ObjC -l”c++”

Target:

-force_load Pods/TensorFlowLiteSelectTfOps/Frameworks/TensorFlowLiteSelectTfOps.framework/TensorFlowLiteSelectTfOps -force_load Pods/TensorFlowLiteC/Frameworks/TensorFlowLiteC.framework/TensorFlowLiteC -force_load Pods/TensorFlowLiteC/Frameworks/TensorFlowLiteCCoreML.framework/TensorFlowLiteCCoreML -force_load Pods/TensorFlowLiteC/Frameworks/TensorFlowLiteCMetal.framework/TensorFlowLiteCMetal -ObjC -l”c++”

niftily switch between xcode versions:

sudo xcode-select —switch ~/Downloads/Xcode-beta.app

Interpreter Operations:

Hack on fft functions:
[./etc/tone.py:](https://github.com/Jesssullivan/tmpUI/blob/master/etc/tone.py)

copy from here:

cp etc/tone.py .

generate some .wav files for testing fft things:

python3 tone.py

…you can also specify duration in seconds & frequency in Hz like so:

python3 tone.py 5 440

…or just duration:

python3 tone.py 2

some fft-related links

Scripts:

See ./package.json & ./scripts/ for additional scripts

main scripts links:

local ssl:

Generates local ssl certs for testing w/ node http-server:

npm run-script sslgen

you can also provide a $DOMAIN argument like so:

npm run-script sslgen hiyori

…returns key hiyori_key.pem & cert hiyori.pem

…or:

sudo chmod +x scripts/sslgen.sh && ./scripts/sslgen.sh

osx is a bit more finicky

tone generator:

available from here:

cp etc/tone.py .

generate some .wav files for testing fft things:

python3 tone.py

…you can also specify duration in seconds & frequency in Hz like so:

python3 tone.py 5 440

…or just duration:

python3 tone.py 2

removing stuff:

clean up with:

npm run-script clean all

…and follow the instruction prompt

…demo bundles:

npm run-script clean-web-bundles

…demo renders:

npm run-script clean-web-renders

etc, etc etc

xD

-Jess

Related

Originally published at transscendsurvival.org

Related Posts

Merlin AI Demos!

tmpUI & the Merlin Sound ID Project

The eBird API & regionCode

Comments