RTNeural Extension

Overview

RTNeural is a light-weight ML inferencing library designed with the intention of being used in real-time systems, specifically real-time audio processing. You can find out more about the project on the RTNeural GitHub page.

We've create an experimental RTNeural extension for Switchboard allowing you to insert real-time ML interfence powered signal processing into any Switchboard audio engine.

Example

Below is at example where we define a Switchboard JSON graph to play a guitar sample with a WebAudio.BufferSource node, then pass the output into a RTNeural.MLProcessor node. In this particular example we're using an open-source guitar distortion model from the GuitarLSTM project, converted into RTNeural JSON modal format.

You can also use the Upload button in the UI to upload an RTNeural JSON model from your computer.

Switchboard Audio IDE Preview

Switchboard JSON

{
    "title": "RTNeural distortion",
    "nodes": [
        {
            "id": "BufferSource1",
            "type": "WebAudio.BufferSource"
        },
        {
            "type": "RTNeural.MLProcessor",
            "assets": {
                "model": "/audio-lab/models/distortion.json"
            }
        }
    ],
    "tracks": [
        {
            "label": "EMH",
            "BufferSource1": "/audio-lab/audio/music/guitar-48k.wav"
        }
    ]
}

info

The RTNeural.MLProcessor node currently support mono audio only, down-mixing stereo to mono before its passed into the interfence engine and then up-mixing back to stereo as its passed back out of the RTNeural inferencing engine.

Node types

The Switchboard RTNeural Extension provides the following audio nodes for a Switchboard SDK audio graphs:

Switchbaord JSON Node	Description
RTNeural.MLProcessor	A processor node that runs the RTNeural model and receives audio data with the specified pre-processing, then generates audio data with the specified post-processing. Ideal for applications in audio transformation, such as Noise Reduction, Source Separation, and Voice Conversion.

info

This feature is in Beta, contact us to apply for our early access testing program! Please try our Switchboard Audio IDE to quickly try and share Switchboard projects that use RTNeural.