Normally, there are 3 audio codecs for AI ASR and TTS, WAV(PCM) MP3 and Opus. Opus is now the default codec of WerRTC. It has some advantages for real time audio data transmission, such as short latency, automatic correction and so on.But it is also cost more CPU power. In xiaozhi-esp32 and xiaozhi-esp32-server projects, opus codecs are implemented in software. A comment in xiaozhi-esp32 project says: (In managed_components\78__esp-opus-encoder\opus_encoder.cc)

    // Complexity 5 almost uses up all CPU of ESP32C3
    SetComplexity(5);

In xiaozhi-esp32-server, it uses python lib opuslib_next to encode/decode opus. There is a test_page.html in xiaozhi-esp32-server, and it can also encode/decode opus. It uses a browser js lib libopus.js, which is in the same folder with the html file. But I can't find this lib in npm site, and this lib can't be used in node.js environment or miniprogram. This libopus.js is using webassemlby, maybe because opus needs more CPU efficiency. In Linux, opus is a 3rd lib/software just like ffmpeg.

Label: none

Add new comment