Skip to main content

LLM Extension

The LLM Extension allows you to integrate large language models (LLMs) into your audio applications using the llama.cpp library. This extension enables you to load and run LLM models locally, supporting advanced natural language processing tasks such as text generation, summarization, and conversational AI directly within your audio workflows. By leveraging llama.cpp, the extension provides efficient, on-device inference, making it ideal for real-time and privacy-sensitive scenarios.

Nodes​

The LLM Extension provides the following audio nodes for a Switchboard SDK audio graph:

NodeDescription
LLMSends received text to the LLM model and outputs the generated response.

Download​

You can find the download links for this extension on our Downloads page.