A protocol is used for communicating between master and slave device. A suggested minium protocol structure is: HEAD + LENGTH + MSG_TYPE + DATA + TAIL. You can add SEQUENCE if the lower tranmission layer can't guarantee correct data sequence. You can add CHECKSUM if the lower tranmission layer can't guarantee correct data content. HEAD is to determine where a package starts from. LENGTH and TAIL can work together to determine if a package is received completely. A large package could be splited into several small packages and sent seperately. For example, BLE mtu is 20 byte by default, which is very small to send big data.

The core concept is "Do not let the tax rate exceed 20%". Three ways, and it should follow this sequence:

  1. Salary. Salary is the most cost-effective way, if the salary of a year is lower than 36w, or 3w/month. Then the composite tax rate will be lower than 20%.
  2. Annual bonus. If the boss want to get more than 36w/year, annual bonus is the next choice. Annual bonus should less than 14.4w, then the composite tax rate will be equal to 10%. Or 3.6w's tax rate is equal to 3%. Do not pay more than 14.4w, then the tax rate will be equal to 20%.
  3. Dividends for stockholders. Its tax rate is equal to 20%.

Normally, there are 3 audio codecs for AI ASR and TTS, WAV(PCM) MP3 and Opus. Opus is now the default codec of WerRTC. It has some advantages for real time audio data transmission, such as short latency, automatic correction and so on.But it is also cost more CPU power. In xiaozhi-esp32 and xiaozhi-esp32-server projects, opus codecs are implemented in software. A comment in xiaozhi-esp32 project says: (In managed_components\78__esp-opus-encoder\opus_encoder.cc)

    // Complexity 5 almost uses up all CPU of ESP32C3
    SetComplexity(5);

In xiaozhi-esp32-server, it uses python lib opuslib_next to encode/decode opus. There is a test_page.html in xiaozhi-esp32-server, and it can also encode/decode opus. It uses a browser js lib libopus.js, which is in the same folder with the html file. But I can't find this lib in npm site, and this lib can't be used in node.js environment or miniprogram. This libopus.js is using webassemlby, maybe because opus needs more CPU efficiency. In Linux, opus is a 3rd lib/software just like ffmpeg.

When you full deployed xiaozhi-esp32-server on a pubic linux, you may want to enable domain name and https access. Here is how: 0. Make sure your http and ws access to your server works.

  • Use browser to access http://your_ip:8001 , It should show the login page.
  • Use browser to access ota url: http://your_ip:8002/xiaozhi/ota/, It should show the correct message.
  • Use a websocket test tool, for example: Websocket test client in chrome web store, to test ws://your_ip:8000/xiaozhi/v1/, it should return the correct anwser.
  • You can use a real Xiaozhi device to access your own server. Change the OTA url in idf.py menuconfig and build + flash it. Try to configure a network and speak to it.
  • If you don't have a real device, you can use a test page. The offical test page https://2662r3426b.vicp.fun/test/ is not useable because the page is using https and can only access wss but ws. You should use the test page under the xiaozhi-esp32-server project path main/xiaozhi-server/test/test_page.html. Use a http-server( for example: npm install -g http-server then http-server) to serve the page and access it from the browser.
  1. Install nginx and run it as service;
  2. Install certbot, generate nginx public and private certs;
# Ubuntu/Debian
sudo apt-get install certbot python3-certbot-nginx

# generate certs
sudo certbot --nginx -d diy.esp32.cn

# automatic renew
sudo certbot renew --quiet

If certbot failed to add configuration to nginx.conf, you can do it by yourself. You need to copy the certs path and paste them into nginx.conf. For example:

  ssl_certificate /etc/letsencrypt/live/yourdomain.name/fullchain.pem;
  ssl_certificate_key /etc/letsencrypt/live/yourdomain.name/privkey.pem;

I'll show you in the next step.

  1. Config nginx. Add the following configuration to nginx.conf, change yourdomain.name to yours:

server {
  listen 80;
  server_name yourdomain.name;
  location / {
    proxy_pass http://localhost:8001;
  }
}
server {
  listen 443 ssl;
  server_name yourdomain.name;
  ssl_certificate /etc/letsencrypt/live/yourdomain.name/fullchain.pem;
  ssl_certificate_key /etc/letsencrypt/live/yourdomain.name/privkey.pem;
  
  location / {
    proxy_pass https://localhost:8001;
    
    # Normal proxy header, forward client's request information.
    proxy_set_header Host localhost:8001;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;

    # Set timeout as you wish
    #proxy_read_timeout 86400s;
  }
  location = /xiaozhi/v1/ {
    proxy_pass http://localhost:8000/xiaozhi/v1/;
    # Support WebSocket
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "Upgrade";
    
    
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;

    #proxy_read_timeout 86400s;
  }
}

Restart nginx by nginx -s reload or service nginx restart or systemctl restart nginx.

The ota url and wss url should work now. You can try to access https://yourdomain.name/xiaozhi/ota/ and wss://yourdomain.name/xiaozhi/v1/

  1. Change vue config file main/manager-web/vue.config.js, add a element https: true, under devServer:{}, like:
  devServer: {
    port: 8001,
    https: true,
    ...

Restart npm run serve, https://yourdomain.name/ should works. and you can also access it by ip: https://your_ip:8001. The http access is disabled at the same time.

  1. Change the ota and websocket config in the admin page of https://yourdomain.name/.

Set systemd service

Set systemd service redis, manager-api, manager-web, xiaozhi-server. manager-api should rely on redis, or it will fail to start. The ExecStart should use absolute path because systemd started earlier before PATH is loaded.

/etc/systemd/system/redis.service

[Unit]
Description=Redis Server
After=network.target
Wants=network.target

[Service]
Type=simple
ExecStart=/usr/sbin/service redis start
ExecStop=/usr/sbin/service redis stop
Restart=always
RestartSec=5
StandardOutput=append:/var/log/redis.log
StandardError=append:/var/log/redis.log

[Install]
WantedBy=multi-user.target

/etc/systemd/system/manager-api.service

[Unit]
Description=Manager API Service
After=redis.service
Requires=redis.service

[Service]
Type=simple
ExecStart=/www/server/java/jdk-21.0.2/bin/java -jar /root/s2/xiaozhi-esp32-server/main/manager-api/target/xiaozhi-esp32-api.jar
WorkingDirectory=/root/s2/xiaozhi-esp32-server/main/manager-api
Restart=no
RestartSec=5
StandardOutput=append:/var/log/manager-api.log
StandardError=append:/var/log/manager-api.log
SyslogIdentifier=manager-api

[Install]
WantedBy=multi-user.target

/etc/systemd/system/manager-web.service

[Unit]
Description=Manager Web Service
After=manager-api.service
Wants=manager-api.service

[Service]
Type=simple
Environment="PATH=/usr/local/bin:/usr/bin:/bin:/root/.nvm/versions/node/v22.16.0/bin"
ExecStart=/root/.nvm/versions/node/v22.16.0/bin/npm run serve
WorkingDirectory=/root/s2/xiaozhi-esp32-server/main/manager-web
Restart=always
RestartSec=5
StandardOutput=append:/var/log/manager-web.log
StandardError=append:/var/log/manager-web.log
SyslogIdentifier=manager-web

[Install]
WantedBy=multi-user.target

/etc/systemd/system/xiaozhi-server.service

[Unit]
Description=Xiaozhi Server
After=manager-web.service
Wants=manager-web.service

[Service]
Type=simple
Environment=XIAOZHI_HOME=/root/s2/xiaozhi-esp32-server/main
ExecStart=/www/server/pyporject_evn/xiaozhi/bin/python app.py
WorkingDirectory=/root/s2/xiaozhi-esp32-server/main/xiaozhi-server
Restart=always
RestartSec=5
StandardOutput=append:/var/log/xiaozhi-server.log
StandardError=append:/var/log/xiaozhi-server.log
SyslogIdentifier=xiaozhi-server

[Install]
WantedBy=multi-user.target

enable systemd service and check logs

systemctl enable redis manager-api manager-web xiaozhi-server
reboot

Reboot and check if services are active

systemctl status redis manager-api manager-web xiaozhi-server

# Check running log
journalctl -u redis
journalctl -u manager-api
journalctl -u manager-web
journalctl -u xiaozhi-server

# Check services' stdio output log
cat /var/log/redis.log
cat /var/log/manager-api.log
cat /var/log/manager-web.log
cat /var/log/xiaozhi-server.log
  1. 1
  2. 2
  3. 3
  4. 4