For a step-by-step video guide, click here.

To set up AllTalk within Text-generation-webui, follow either method:

1. Download AllTalk Setup:

   • Via Terminal/Console (Recommended):
     • cd \text-generation-webui\extensions\
     • git clone https://github.com/erew123/alltalk_tts
   • Via Releases Page (Cannot be automatically updated after install as its not linked to Github):
     • Download the latest alltalk_tts.zip from Releases and extract it to \text-generation-webui\extensions\alltalk_tts\.

2. Start Python Environment:

   • In the text-generation-webui folder, start the environment with the appropriate command:

     • Windows: cmd_windows.bat
     • Linux: ./cmd_linux.sh
       
       

     If you're unfamiliar with Python environments and wish to learn more, consider reviewing Understanding Python Environments Simplified in the Help section.

3. Run AllTalk Setup Script:

   • Navigate to the AllTalk directory and execute the setup script:
     • cd extensions
     • cd alltalk_tts
     • Windows: atsetup.bat
     • Linux: ./atsetup.sh

4. Install Requirements:

   • Follow the on-screen instructions to install the necessary requirements. It's recommended to test AllTalk's functionality before installing DeepSpeed.

Note: Always activate the Text-generation-webui Python environment before making any adjustments or using Fine-tuning. Additional instructions for Fine-tuning and DeepSpeed can be found within the setup utility and on this documentation page.

For a step-by-step video guide, click here.

To perform a Standalone installation of AllTalk:

1. Get AllTalk Setup:

   • Via Terminal/Console (Recommended):
     • Navigate to your preferred directory: cd C:\myfiles\
     • Clone the AllTalk repository: git clone https://github.com/erew123/alltalk_tts
   • Via Releases Page (Cannot be automatically updated after install as its not linked to Github):
     • Download alltalk_tts.zip from Releases and extract it to your chosen directory, for example, C:\myfiles\alltalk_tts\.

2. Start AllTalk Setup:

   • Open a terminal/command prompt, move to the AllTalk directory, and run the setup script:
     • cd alltalk_tts
     • Windows: atsetup.bat
     • Linux: ./atsetup.sh

3. Follow the Setup Prompts:

   • Select Standalone Installation and then Option 1 and follow any on-screen instructions to install the required files. DeepSpeed is automatically installed on Windows based system, but will only work on Nvidia GPU's. Linux based system users will have to follow the DeepSpeed installation instructions.

If you're unfamiliar with Python environments and wish to learn more, consider reviewing Understanding Python Environments Simplified in the Help section.

Important: Do not use spaces in your folder path (e.g. avoid /my folder-is-this/alltalk_tts-main) as this causes issues with Python & Conda.

Manual Installation AllTalk Version 1 for Text Generation Web UI

If you're using a Mac or prefer a manual installation for any other reason, please follow the steps below. This guide is compatible with the current release of Text Generation Web UI as of December 2023. Consider updating your installation if it's been a while, update instructions here.

• For a visual guide on the installation process, watch this video.

1. Navigate to Text Generation Web UI Folder:

   • Open a terminal window and move to your Text Generation Web UI directory with:
     • cd text-generation-webui

2. Activate Text Generation Web UI Python Environment:

   • Start the appropriate Python environment for your OS using one of the following commands:

     • For Windows: cmd_windows.bat
     • For Linux: ./cmd_linux.sh
     • For macOS: cmd_macos.sh
     • For WSL: cmd_wsl.bat

   • Loading the Text Generation Web UI's Python environment is crucial. If unsure about what a loaded Python environment should look like, refer to this image and video guide.

   If you're unfamiliar with Python environments and wish to learn more, consider reviewing Understanding Python Environments Simplified in the Help section.

3. Move to Extensions Folder:

   • cd extensions

4. Clone the AllTalk TTS Repository:

   • git clone https://github.com/erew123/alltalk_tts

5. Navigate to the AllTalk TTS Folder:

   • cd alltalk_tts

6. Install Required Dependencies:

   • Install dependencies for your machine type:
     • For Windows: pip install -r system\requirements\requirements_textgen.txt
     • For Linux/Mac: pip install -r system/requirements/requirements_textgen.txt

7. Optional DeepSpeed Installation:

• If you're using an Nvidia graphics card on Linux or Windows and wish to install DeepSpeed, follow the instructions here.
• Recommendation: Start Text Generation Web UI and ensure AllTalk functions correctly before installing DeepSpeed.

8. Start Text Generation Web UI:

• Return to the main Text Generation Web UI folder using cd .. (repeat as necessary).

  • Start the appropriate Python environment for your OS using one of the following commands:
    • For Windows: start_windows.bat
    • For Linux: ./start_linux.sh
    • For macOS: start_macos.sh
    • For WSL: start_wsl.bat

• Load the AllTalk extension in the Text Generation Web UI session tab.

• For any updates to AllTalk or for tasks like Finetuning, always activate the Text Generation Web UI Python environment first.

Refer to 🟩 Other installation notes for further details, including information on additional voices, changing IP, character card notes etc.

Running AllTalk as a Standalone Application alongside Text Generation Web UI

If you have AllTalk installed as an extension of Text Generation Web UI but wish to run it as a standalone application, follow these steps:

1. Activate Text Generation Web UI Python Environment:

   • Use the appropriate command for your operating system to load the Python environment:
     • Windows: cmd_windows.bat
     • Linux: ./cmd_linux.sh
     • macOS: cmd_macos.sh
     • WSL: cmd_wsl.bat

2. Navigate to the AllTalk Directory:

   • Move to the AllTalk folder with the following commands:
     • cd extensions
     • cd alltalk_tts

3. Start AllTalk:

   • Run AllTalk with the command:
     • python script.py

   There are no additional steps required to run AllTalk as a standalone application from this point.

Custom Installation of AllTalk

Support for custom Python environments is limted. Please read Custom Python environments Limitations Notice below this section.

To run AllTalk as a standalone application with a custom Python environment, ensure you install AllTalk's requirements into the environment of your choice. The instructions provided are generalized due to the variety of potential Python environments.

• Python Compatibility: The TTS engine requires Python 3.9.x to 3.11.x. AllTalk is tested with Python 3.11.x. See TTS Engine details.
• Path Names: Avoid spaces in path names as this can cause issues.
• Custom Python Environments: If encountering issues potentially related to a custom environment, consider testing AllTalk with the quick setup standalone method that builds its own environment.

Quick Overview of Python Environments

If you're unfamiliar with Python environments and wish to learn more, consider reviewing Understanding Python Environments Simplified in the Help section.

Building a Custom Python Environment with Miniconda

1. Initial Setup:

   • Ensure python --version and pip commands work in your terminal. Install Python and Pip if necessary.
     • Python Installation
     • Pip Installation

2. Install Miniconda:

   • Download and install Miniconda for your OS from the Miniconda Website. Use the Anaconda Prompt from the Start Menu or Application Launcher to access the base Conda environment.

3. Clone AllTalk Repository:

   • Navigate to your desired folder (e.g., c:\myfiles\) and clone the AllTalk repository:
     • git clone https://github.com/erew123/alltalk_tts
   • Move to the AllTalk folder with the following commands:
     • cd alltalk_tts

4. Create Conda Environment:

   • Create a Conda environment named alltalkenv with Python 3.11.5:
     • conda create --name alltalkenv python=3.11.5
   • Activate the new environment:
     • conda activate alltalkenv

5. Install Requirements:

   • Install dependencies based on your machine type:
     • For Windows: pip install -r system\requirements\requirements_standalone.txt
     • For Linux/Mac: pip install -r system/requirements/requirements_standalone.txt

6. Start AllTalk:

   • Run AllTalk with the following:
     • python script.py

Note: For updates, DeepSpeed installations, or other modifications, always activate the alltalkenv Conda environment first. Custom scripts or batch files can simplify launching AllTalk.

AllTalk is coded to start on 127.0.0.1, meaning that it will ONLY be accessable to the local computer it is running on. If you want to make AllTalk available to other systems on your network, you will need to change its IP address to match the IP address of your network card/computers current IP address. There are 2x ways to change the IP address:

1. Start AllTalk and within its web interface and you can edit the IP address on the "AllTalk Startup Settings".
2. You can edit the confignew.jsonfile in a text editor and change "ip_address": "127.0.0.1", to the IP address of your choosing.

So, for example, if your computer's network card was on IP address 192.168.0.20, you would change AllTalk's setting to 192.168.1.20 and then restart AllTalk. You will need to ensure your machine stays on this IP address each time it is restarted, by setting your machine to have a static IP address.

The Stable Diffusion plugin for Text-generation-webui strips out some of the text, which is passed to Stable Diffusion for image/scene generation. Because this text is stripped, its important to consider the load order of the plugins to get the desired result you want. Lets assume the AI has just generated the following message *He walks into the room with a smile on his face and says* Hello how are you?. Depending on the load order will change what text reaches AllTalk for generation e.g.

SD Plugin loaded before AllTalk - Only Hi how are you? is sent to AllTalk, with the *He walks into the room with a smile on his face and says* being sent over to SD for image generation. Narration of the scene is not possible.

AllTalk loaded before SD Plugin - *He walks into the room with a smile on his face and says* Hello how are you? is sent to AllTalk with the *He walks into the room with a smile on his face and says* being sent over to SD for image generation.

The load order can be changed within Text-generation-webui's settings.yaml file or cmd_flags.txt (depending on how you are managing your extensions).

Messages intended for the Narrator should be enclosed in asterisks * and those for the character inside quotation marks ". However, AI systems often deviate from these rules, resulting in text that is neither in quotes nor asterisks. Sometimes, text may appear with only a single asterisk, and AI models may vary their formatting mid-conversation. For example, they might use asterisks initially and then switch to unmarked text. A properly formatted line should look like this:

"Hey! I'm so excited to finally meet you. I've heard so many great things about you and I'm eager to pick your brain about computers." *She walked across the room and picked up her cup of coffee*

Most narrator/character systems switch voices upon encountering an asterisk or quotation marks, which is somewhat effective. AllTalk has undergone several revisions in its sentence splitting and identification methods. While some irregularities and AI deviations in message formatting are inevitable, any line beginning or ending with an asterisk should now be recognized as Narrator dialogue. Lines enclosed in double quotes are identified as Character dialogue. For any other text, you can choose how AllTalk handles it: whether it should be interpreted as Character or Narrator dialogue (most AI systems tend to lean more towards one format when generating text not enclosed in quotes or asterisks).

With improvements to the splitter/processor, I'm confident it's functioning well. You can monitor what AllTalk identifies as Narrator lines on the command line and adjust its behavior if needed (Text Not Inside - Function).

Currently the XTTS model is the main model used by AllTalk for TTS generation. If you want to know more details about the XTTS model, its capabilties or its technical features you can look at resources such as:

• https://docs.coqui.ai/en/latest/models/xtts.html
• https://github.com/coqui-ai/TTS
• https://github.com/coqui-ai/TTS/discussions

The update process closely mirrors the installation steps. Follow these to ensure your setup remains current:

1. Open a Command Prompt/Terminal:

   • Navigate to your Text-Generation-webui folder with:
     • cd text-generation-webui

2. Start the Python Environment:

   • Activate the Python environment tailored for your operating system. Use the appropriate command from below based on your OS:
     • Windows: cmd_windows.bat
     • Linux: ./cmd_linux.sh
     • macOS: cmd_macos.sh
     • WSL (Windows Subsystem for Linux): cmd_wsl.bat
       
       

   If you're unfamiliar with Python environments and wish to learn more, consider reviewing Understanding Python Environments Simplified in the Help section.

3. Navigate to the AllTalk TTS Folder:

   • Move into your extensions and then the alltalk_tts directory:
     • cd extensions/alltalk_tts

4. Update the Repository:

   • Fetch the latest updates from the repository with:
     • git pull

5. Install Updated Requirements:

   • Depending on your machine's OS, install the required dependencies using pip:
     • For Windows Machines:
       • pip install -r system\requirements\requirements_textgen.txt
     • For Linux/Mac:
       • pip install -r system/requirements/requirements_textgen.txt

6. DeepSpeed Requirements:

   • If Text-gen-webui is using a new version of PyTorch, you may need to uninstall and update your DeepSpeed version.
   • Use AllTalks diagnostics or start-up menu to identify your version of PyTorch.
     
     

If you installed from a ZIP file, you cannot use a git pull to update, as noted in the Quick Setup instructions.

For Standalone Application users, here's how to update your setup:

1. Open a Command Prompt/Terminal:

   • Navigate to your AllTalk folder with:
     • cd alltalk_tts

2. Access the Python Environment:

   • In a command prompt or terminal window, navigate to your alltalk_tts directory and start the Python environment:
     • Windows:
       • start_environment.bat
     • Linux/macOS:
       • ./start_environment.sh

If you're unfamiliar with Python environments and wish to learn more, consider reviewing Understanding Python Environments Simplified in the Help section.

2. Pull the Latest Updates:

   • Retrieve the latest changes from the repository with:
     • git pull

3. Install Updated Requirements:

   • Depending on your machine's OS, install the required dependencies using pip:
     • For Windows Machines:
       • pip install -r system\requirements\requirements_standalone.txt
     • For Linux/Mac:
       • pip install -r system/requirements/requirements_standalone.txt
         
         

The process involves renaming your existing alltalk_tts directory, setting up a fresh instance, and then migrating your data:

1. Rename Existing Directory:

   • First, rename your current alltalk_tts folder to keep it safe e.g. alltalk_tts.old. This preserves any existing data.

2. Follow the Quick Setup instructions:

   • You will now follow the Quick Setup instructions, performing the git clone https://github.com/erew123/alltalk_tts to pull down a new copy of AllTalk and install the requirements.

     If you're not familiar with Python environments, see Understanding Python Environments Simplified in the Help section for more info.

3. Migrate Your Data:

   • Before starting the AllTalk, transfer the models, voices, outputs folders and also confignew.json from alltalk_tts.old to the new alltalk_tts directory. This action preserves your voice history and prevents the need to re-download the model.

4. Launch AllTalk
   • You're now ready to launch AllTalk and check it works correctly.

6. Final Step:
   • Once you've verified that everything is working as expected and you're satisfied with the setup, feel free to delete the alltalk_tts.old directory to free up space.

DeepSpeed Installation for Text generation webUI

1. Nvidia CUDA Development Toolkit Installation:

   • The toolkit is crucial for DeepSpeed to compile/build for your version of Linux and requires around 3GB's of disk space.
   • Install using your package manager (Recommended) e.g. CUDA Toolkit 11.8 or download directly from Nvidia CUDA Toolkit Archive (choose 11.8 or 12.1 for Linux).

2. Open a Terminal Console:

   • After Nvidia CUDA Development Toolkit installation, access your terminal console.

3. Install libaio-dev:

   • Use your Linux distribution's package manager.

   • • sudo apt install libaio-dev for Debian-based systems
     • sudo yum install libaio-devel for RPM-based systems.

4. Navigate to Text generation webUI Folder:

   • Change directory to your Text generation webUI folder with cd text-generation-webui.

5. Activate Text generation webUI Custom Conda Environment:

   • Run ./cmd_linux.sh to start the environment.
     
     

   If you're unfamiliar with Python environments and wish to learn more, consider reviewing Understanding Python Environments Simplified in the Help section.

6. Set CUDA_HOME Environment Variable:

   • DeepSpeed locates the Nvidia toolkit using the CUDA_HOME environment variable.
   • You will only set this temporarily as Text generation webUI sets up its own CUDA_HOME environment each time you use ./cmd_linux.sh or ./start_linux.sh

7. Temporarily Configuring CUDA_HOME:

   • When the Text generation webUI Python environment is active (step 5), set CUDA_HOME.

   • • export CUDA_HOME=/usr/local/cuda
     • export PATH=${CUDA_HOME}/bin:${PATH}
     • export LD_LIBRARY_PATH=${CUDA_HOME}/lib64:$LD_LIBRARY_PATH

   • You can confirm the path is set correctly and working by running the command nvcc --version should confirm Cuda compilation tools, release 11.8..

   • Incorrect path settings may lead to errors. If you encounter path issues or receive errors like [Errno 2] No such file or directory when you run the next step, confirm the path correctness or adjust as necessary.

8. DeepSpeed Installation:

   • Install DeepSpeed using pip install deepspeed.

9. Troubleshooting:

   • Troubleshooting steps for DeepSpeed installation can be located down below.
   • NOTE: You DO NOT need to set Text-generation-webUI's --deepspeed setting for AllTalk to be able to use DeepSpeed. These are two completely separate things and incorrectly setting that on Text-generation-webUI may cause other complications.

DeepSpeed Installation for Standalone AllTalk

1. Nvidia CUDA Development Toolkit Installation:

   • The toolkit is crucial for DeepSpeed to compile/build for your version of Linux and requires around 3GB's of disk space.
   • Install using your package manager (Recommended) e.g. CUDA Toolkit 11.8 or download directly from Nvidia CUDA Toolkit Archive (choose 11.8 or 12.1 for Linux).

2. Open a Terminal Console:

   • After Nvidia CUDA Development Toolkit installation, access your terminal console.

3. Install libaio-dev:

   • Use your Linux distribution's package manager.

   • • sudo apt install libaio-dev for Debian-based systems
     • sudo yum install libaio-devel for RPM-based systems.

4. Navigate to AllTalk TTS Folder:

   • Change directory to your AllTalk TTS folder with cd alltalk_tts.

5. Activate AllTalk Custom Conda Environment:

   • Run ./start_environment.sh to start the AllTalk Python environment.
   • This command will start the custom Python environment that was installed with ./atsetup.sh.
     
     

   If you're unfamiliar with Python environments and wish to learn more, consider reviewing Understanding Python Environments Simplified in the Help section.

6. Set CUDA_HOME Environment Variable:

   • The DeepSpeed installation routine locates the Nvidia toolkit using the CUDA_HOME environment variable. This can be set temporarily for a session or permanently, depending on other requirements you may have for other Python/System environments.
   • For temporary use, proceed to step 8. For a permanent solution, see Conda's manual on setting environment variables.

7. (Optional) Permanent CUDA_HOME Setup:

   • If you choose to set CUDA_HOME permanently, follow the instructions in the provided Conda manual link above.

8. Configuring CUDA_HOME:

   • When your Python environment is active (step 5), set CUDA_HOME.

   • • export CUDA_HOME=/usr/local/cuda
     • export PATH=${CUDA_HOME}/bin:${PATH}
     • export LD_LIBRARY_PATH=${CUDA_HOME}/lib64:$LD_LIBRARY_PATH

   • You can confirm the path is set correctly and working by running the command nvcc --version should confirm Cuda compilation tools, release 11.8..

   • Incorrect path settings may lead to errors. If you encounter path issues or receive errors like [Errno 2] No such file or directory when you run the next step, confirm the path correctness or adjust as necessary.

9. DeepSpeed Installation:

   • Install DeepSpeed using pip install deepspeed.

10. Starting AllTalk TTS WebUI:

    • Launch the AllTalk TTS interface with ./start_alltalk.sh and enable DeepSpeed.

Troubleshooting

• If setting CUDA_HOME results in path duplication errors (e.g., .../bin/bin/nvcc), you can correct this by unsetting CUDA_HOME with unset CUDA_HOME and then adding the correct path to your system's PATH variable.
• Always verify paths and compatibility with other CUDA-dependent applications to avoid conflicts.
• If you have multiple versions of the Nvidia CUDA Development Toolkit installed, you will have to specify the version number in step 8 for the CUDA_HOME path.
• If it became necessary to uninstall DeepSpeed, you can do so by start the Python enviroment and then running pip uninstall deepspeed
  
  

DeepSpeed Installation with Pre-compiled Wheels

1. Introduction to Pre-compiled Wheels:

   • The atsetup.bat utility simplifies the installation of DeepSpeed by automatically downloading and installing pre-compiled wheel files. These files are tailored for specific versions of Python, CUDA, and PyTorch, ensuring compatibility with both the Standalone Installation and a standard build of Text-generation-webui.

2. Manual Installation of Pre-compiled Wheels:

   • If needed, pre-compiled DeepSpeed wheel files that I have built are available on the Releases Page. You can manually install or uninstall these wheels using the following commands:
     • Installation: pip install {deep-speed-wheel-file-name-here}
     • Uninstallation: pip uninstall deepspeed

3. Using atsetup.bat for Simplified Management:

   • For those running the Standalone Installation or a standard build of Text-generation-webui, the atsetup.bat utility offers the simplest and most efficient way to manage DeepSpeed installations on Windows.

Manual DeepSpeed Wheel Compilation

1. Preparation for Manual Compilation:

   • Manual compilation of DeepSpeed wheels is an advanced process that requires:
     • 1-2 hours of your time for initial setup and compilation.
     • 6-10GB of disk space on your computer.
     • A solid technical understanding of Windows environments and Python.

2. Understanding Wheel Compatibility:

   • A compiled DeepSpeed wheel is uniquely tied to the specific versions of Python, PyTorch, and CUDA used during its compilation. If any of these versions are changed, you will need to compile a new DeepSpeed wheel to ensure compatibility.

3. Compiling DeepSpee Resources:

   • Myself and @S95Sedan have worked to simplify the compilation process. @S95Sedan has notably improved the process for later versions of DeepSpeed, ensuring ease of build on Windows.
   • Because @S95Sedan is now maintaining the instructions for compiling DeepSpeed on Windows, please visit @S95Sedan's
     DeepSpeed GitHub page.

If you are on a Windows machine or a Linux machine, you should be able to use the atsetup.bat or ./atsetup.sh utility to create a diagnositcs file. If you are unable to use the atsetup utility, please follow the instructions below.

Think of Python environments like different rooms in your house, each designed for a specific purpose. Just as you wouldn't cook in the bathroom or sleep in the kitchen, different Python applications need their own "spaces" or environments because they have unique requirements. Sometimes, these requirements can clash with those of other applications (imagine trying to cook a meal in a bathroom!). To avoid this, you can create separate Python environments.

Why Separate Environments?

Separate environments, like separate rooms, keep everything organized and prevent conflicts. For instance, one Python application might need a specific version of a library or dependency, while another requires a different version. Just as you wouldn't store kitchen utensils in the bathroom, you wouldn't want these conflicting requirements to interfere with each other. Each environment is tailored and customized for its application, ensuring it has everything it needs without disrupting others.

How It Works in Practice:

Standalone AllTalk Installation: When you install AllTalk standalone, it's akin to adding a new room to your house specifically designed for your AllTalk activities. The setup process, using the atsetup utility, constructs this custom "room" (Python environment alltalk_environment) with all the necessary tools and furnishings (libraries and dependencies) that AllTalk needs to function smoothly, without meddling with the rest of your "house" (computer system). The AllTalk environment is started each time you run start_alltalk or start_environment within the AllTalk folder.

Text-generation-webui Installation: Similarly, installing Text-generation-webui is like setting up another specialized room. Upon installation, it automatically creates its own tailored environment, equipped with everything required for text generation, ensuring a seamless and conflict-free operation. The Text-generation-webui environment is started each time you run start_*your-os-version* or cmd_*your-os-version* within the Text-generation-webui folder.

Managing Environments:

Just as you might renovate a room or bring in new furniture, you can also update or modify Python environments as needed. Tools like Conda or venv make it easy to manage these environments, allowing you to create, duplicate, activate, or delete them much like how you might manage different rooms in your house for comfort and functionality.

Once you're in the right environment, by activating it, installing or updating dependencies (the tools and furniture of your Python application) is straightforward. Using pip, a package installer for Python, you can easily add what you need. For example, to install all required dependencies listed in a requirements.txt file, you'd use:

pip install -r requirements.txt

This command tells pip to read the list of required packages and versions from the requirements.txt file and install them in the current environment, ensuring your application has everything it needs to operate. It's like having a shopping list for outfitting a room and ensuring you have all the right items delivered and set up.

Remember, just as it's important to use the right tools for tasks in different rooms of your house, it's crucial to manage your Python environments and dependencies properly to ensure your applications run as intended.

How do I know if I am in a Python environment?:

When a Python environment starts up, it changes the command prompt to show the Python environment that it currently running within that terminal/console.

ERROR: Microsoft Visual C++ 14.0 or greater is required or ERROR: Could not build wheels for TTS. or ModuleNotFoundError: No module named 'TTS

Python requires that you install C++ development tools on Windows. This is detailed on the Python site here. You would need to install MSVCv142 - VS 2019 C++ x64/x86 build tools and Windows 10/11 SDK from the C++ Build tools section.

You can get hold of the Community edition here the during installation, selecting C++ Build tools and then MSVCv142 - VS 2019 C++ x64/x86 build tools and Windows 10/11 SDK.

This is more than likely caused by having a space in your folder path e.g. c:\program files\alltalk_tts. In this circumstance you would be best moving the folder to a path without a space e.g. c:\myfiles\alltalk_tts. You would have to delete the alltalk_environment folder and start_alltalk.bat or start_alltalk.sh and then re-run atsetup to re-create the environment and startup files.

Ive paid very close attention to not impact what Text-generation-webui is requesting on a factory install. This is one of the requirements of submitting an extension to Text-generation-webui. If you want to look at a comparison of a factory fresh text-generation-webui installed packages (with cuda 12.1, though AllTalk's requirements were set on cuda 11.8) you can find that comparison here. This comparison shows that AllTalk is requesting the same package version numbers as Text-generation-webui or even lower version numbers (meaning AllTalk will not update them to a later version). What other extensions do, I cant really account for that.

I will note that the TTS engine downgrades Pandas data validator to 1.5.3 though its unlikely to cause any issues. You can upgrade it back to text-generation-webui default (december 2023) with pip install pandas==2.1.4 when inside of the python environment. I have noticed no ill effects from it being a lower or higher version, as far as AllTalk goes. This is also the same behaviour as the Coqui_tts extension that comes with Text-generation-webui.

Other people are reporting issues with extensions not starting with errors about Pydantic e.g. pydantic.errors.PydanticImportError: BaseSettings` has been moved to the pydantic-settings package. See https://docs.pydantic.dev/2.5/migration/#basesettings-has-moved-to-pydantic-settings for more details.

Im not sure if the Pydantic version has been recently updated by the Text-generation-webui installer, but this is nothing to do with AllTalk. The other extension you are having an issue with, need to be updated to work with Pydantic 2.5.x. AllTalk was updated in mid december to work with 2.5.x. I am not specifically condoning doing this, as it may have other knock on effects, but within the text-gen Python environment, you can use pip install pydantic==2.5.0 or pip install pydantic==1.10.13 to change the version of Pydantic installed.

I would suggest following Problems Updating and if you still have issues after that, you can raise an issue here

You will need to change the IP address within AllTalk's settings from being 127.0.0.1, which only allows access from the local machine its installed on. To do this, please see Changing AllTalks IP address & Accessing AllTalk over your Network at the top of this page.

You may also need to allow access through your firewall or Antivirus package to AllTalk.

To do this you can edit the confignew.json file within the alltalk_tts folder. You would look for "ip_address": "127.0.0.1", and change the 127.0.0.1 to your chosen IP address,then save the file and start AllTalk.

When doing this, be careful not to impact the formatting of the JSON file. Worst case, you can re-download a fresh copy of confignew.json from this website and that will put you back to a factory setting.

You can either follow the Problems Updating and fresh install your config. Or you can edit the confignew.json file within the alltalk_tts folder. You would look for '"deepspeed_activate": true,' and change the word true to false `"deepspeed_activate": false,' ,then save the file and try starting again.

If you want to use DeepSpeed, you need an Nvidia Graphics card and to install DeepSpeed on your system. Instructions are here

Please see Problems Updating. If that doesnt help you can raise an ticket here. It would be handy to have any log files from the console where your error is being shown. I can only losely support custom built Python environments and give general pointers. Please create a diagnostics.log report file to submit with a support request.

Also, is your text-generation-webui up to date? instructions here

As far as I am aware, these are to do with the chrome browser the gradio text-generation-webui in some way. I raised an issue about this on the text-generation-webui here where you can see that AllTalk is not loaded and the messages persist. Either way, this is more a warning than an actual issue, so shouldnt affect any functionality of either AllTalk or text-generation-webui, they are more just an annoyance.

The AllTalk start-up screen provides various bits of information about the detected Python environment and errors.

Config file check

• Sometimes I need to add/remove something to your existing configuration file settings. Obviously, I don’t want to impact your existing settings, however any new features may need these settings to be created before AllTalk starts up. Ive added extra code that checks alltalk_tts/system/config/at_configupdate.json and alltalk_tts/system/config/at_configdowngrade.json, either adding or removing items to your configuration as necessary. If a change is made, you will be notified and a backup of the previous configuration file will be created in the alltalk_tts folder.

AllTalk startup Mode

• informational. This will state if AllTalk has detected it is running as part of Text-generation-webui or as a Standalone Application.

WAV file deletion

• If you have set deletion of old generated WAV files, this will state the time frame after which they are purged.

DeepSpeed version

• What version of DeepSpeed is installed/detected. This will not tell you if the version of DeepSpeed is compiled for your Python, PyTorch or CUDA version. Its important to remember that DeepSpeed has to be compiled for the exact version of Python, PyTorch and CUDA that you are using, so please ensure you have the correct DeepSpeed version installed if necessary.

Model is available

• AllTalk is checking if your model files exist. This is not a validity check of the actual model files, they can still be corrupted. If files are missing, AllTalk will attempt to download them from Huggingface, however, if Huggingface has an outage/issue or your internet connection has issues, its possible corrupted or incomplete files will be downloaded. Please read RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory if you need to confirm your model files are ok.

Current Python Version

• Informational. Literally tells you the version of Python running in your Python environment.

Current PyTorch Version

• Informational. Tell tells you the version of PyTorch running in your Python environment, however if you have an Nvidia card, you should be running a CUDA based version of Pytorch. This is indicated with a +cXXX after the PyTorch version e.g. 2.2.2+cu121 would be PyTorch version 2.2.2 with CUDA 12.1 extensions. If you don’t have the PyTorch CUDA extensions installed, but you do have an Nvidia card, you may need to re-install PyTorch.

Current CUDA Version

• Informational. This is linked to the Current PyTorch Version, as detailed above.

Current TTS Version

• Informational. The current version of the TTS engine that is running.

AllTalk Github updated

• As long as you have an internet connection, this will tell you last time AllTalk was updated on Github. It is checking the commit list to see when the last commit was made. As such, this could be simply a documentation update, a bug fix or new features. Its simply there as a guide to let you know the last time something was changed on AllTalk's Github.

TTS Subprocess

• When AllTalk reaches this stage, the subprocess that loads in the AI model is starting. This is most likely where an error could occur with loading the TTS model, just after the documentation message.

AllTalk Settings & Documentation: http ://x.x.x.x

• The link where you can reach AllTalks built in settings and documentation page. The TTS model will be loading immediately after this is displayed.
  
  

This is caused by Pytorch (Torch) not having the CUDA extensions installed (You can check by running the diagnostics). Typically this happens (on Standalone installations) because when the setup routine goes to install Pytorch with CUDA, it looks in the PIP cache and if a previous application has downloaded a version of Pytorch that doesn't have CUDA extensions, the PIP installer doesnt recognise this fact and just uses the cached version for installation. To resolve this:

1. On the atsetup utility, on the Standalone menu select to Purge the PIP cache. This will remove cached packages from the PIP cache, meaning it will have to download fresh copies.
2. As we need to force the upgrade to the Python environment, the easiest way to do this will be to use atsetup to Delete AllTalk's custom Python environment. This means it will have to rebuild the Python environment. Note, you may have to run this step twice, as it has to exit the current Python environment, then you have to re-load atsetup and select Delete AllTalk's custom Python environment again.
3. You can now use atsetup to Install AllTalk as a Standalone Application which will download fresh copies of everything and re-install the Python environment.
4. Once this is done you can check if CUDA is now working with the diagnostics or starting AllTalk and checking the model loads into CUDA.

This error message is caused by the model being corrupted or damaged in some way. This error can occur if Huggingface, where the model is downloaded from, have an error (when the model is downloaded) or potentailly internet issues occuring while the model is downloaded on first start-up.

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory

ERROR: Application startup failed. Exiting.
[AllTalk Startup] Warning TTS Subprocess has NOT started up yet, Will keep trying for 120 seconds maximum. Please wait.

To resolve this, first look in your alltalk_tts/models/xttsv2_2.0.2 (or whichever) model folder and confirm that the file sizes are correct.

You can delete one or more suspect files and a factory fresh copy of that file or files will be downloaded on next start-up of AllTalk.

This error message is caused by DeepSpeed being enabled when you do not have a Nvidia GPU. To resolve this, edit confignew.json and change "deepspeed_activate": true, to "deepspeed_activate": false, then restart AllTalk.

  File "C:\alltalk_tts\alltalk_environment\env\Lib\site-packages\torch\cuda\__init__.py", line 302, in _lazy_init
    torch._C._cuda_init()
RuntimeError: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx

ERROR:    Application startup failed. Exiting.
[AllTalk Startup] Warning TTS Subprocess has NOT started up yet, Will keep trying for 120 seconds maximum. Please wait.

This error message is caused by having DeepSpeed enabled, but you have a version of DeepSpeed installed that was compiled for a different version of Python, PyTorch or CUDA (or any mix of those). You will need to start your Python environment and run pip uninstall deepspeed to remove DeepSpeed from your Python environment and then install the correct version of DeepSpeed.

raise RuntimeError("PyTorch version mismatch! DeepSpeed ops were compiled and installed 
RuntimeError: PyTorch version mismatch! DeepSpeed ops were compiled and installed with a different version than what is being used at runtime. Please re-install DeepSpeed or switch torch versions. Install torch version=2.1, Runtime torch version=2.2

A) Its trying to load the voice model into your graphics card VRAM (assuming you have a Nvidia Graphics card, otherwise its your system RAM)
B) Its trying to start up the mini-webserver and send the "ready" signal back to the main process.

Before giving other possibilities a go, some people with old machines are finding their startup times are very slow 2-3 minutes. Ive extended the allowed time within the script from 1 minute to 2 minutes. If you have an older machine and wish to try extending this further, you can do so by editing script.py and changing startup_wait_time = 120 (120 seconds, aka 2 minutes) at the top of the script.py file, to a larger value e.g startup_wait_time = 240 (240 seconds aka 4 minutes).

Note: If you need to create a support ticket, please create a diagnostics.log report file to submit with a support request. Details on doing this are above.

Other possibilities for this issue are:

1. You are starting AllTalk in both your CMD FLAG.txt and settings.yaml file. The CMD FLAG.txt you would have manually edited and the settings.yaml is the one you change and save in the session tab of text-generation-webui and you can Save UI defaults to settings.yaml. Please only have one of those two starting up AllTalk.

2. You are not starting text-generation-webui with its normal Python environment. Please start it with start_{your OS version} as detailed here (start_windows.bat,./start_linux.sh, start_macos.sh or start_wsl.bat) OR (cmd_windows.bat, ./cmd_linux.sh, cmd_macos.sh or cmd_wsl.bat and then python server.py).

3. You have installed the wrong version of DeepSpeed on your system, for the wrong version of Python/Text-generation-webui. You can go to your text-generation-webui folder in a terminal/command prompt and run the correct cmd version for your OS e.g. (cmd_windows.bat, ./cmd_linux.sh, cmd_macos.sh or cmd_wsl.bat) and then you can type pip uninstall deepspeed then try loading it again. If that works, please see here for the correct instructions for installing DeepSpeed here.

4. You have an old version of text-generation-webui (pre Dec 2023) I have not tested on older versions of text-generation-webui, so cannot confirm viability on older versions. For instructions on updating the text-generation-webui, please look here (update_linux.sh, update_windows.bat, update_macos.sh, or update_wsl.bat).

5. You already have something running on port 7851 on your computer, so the mini-webserver cant start on that port. You can change this port number by editing the confignew.json file and changing "port_number": "7851" to "port_number": "7602" or any port number you wish that isn’t reserved. Only change the number and save the file, do not change the formatting of the document. This will at least discount that you have something else clashing on the same port number.

6. You have antivirus/firewalling that is blocking that port from being accessed. If you had to do something to allow text-generation-webui through your antivirus/firewall, you will have to do that for this too.

7. You have quite old graphics drivers and may need to update them.

8. Something within text-generation-webui is not playing nicely for some reason. You can go to your text-generation-webui folder in a terminal/command prompt and run the correct cmd version for your OS e.g. (cmd_windows.bat, ./cmd_linux.sh, cmd_macos.sh or cmd_wsl.bat) and then you can type python extensions\alltalk_tts\script.py and see if AllTalk starts up correctly. If it does then something else is interfering.

9. Something else is already loaded into your VRAM or there is a crashed python process. Either check your task manager for erroneous Python processes or restart your machine and try again.

10. You are running DeepSpeed on a Linux machine and although you are starting with ./start_linux.sh AllTalk is failing there on starting. This is because text-generation-webui will overwrite some environment variables when it loads its python environment. To see if this is the problem, from a terminal go into your text-generation-webui folder and ./cmd_linux.sh then set your environment variable again e.g. export CUDA_HOME=/usr/local/cuda (this may vary depending on your OS, but this is the standard one for Linux, and assuming you have installed the CUDA toolkit), then python server.py and see if it starts up. If you want to edit the environment permanently you can do so, I have not managed to write full instructions yet, but here is the conda guide here.

11. You have built yourself a custom Python environment and something is funky with it. This is very hard to diagnose as its not a standard environment. You may want to updating text-generation-webui and re installing its requirements file (whichever one you use that comes down with text-generation-webui).

Finetuning pulls in various other scripts and some of those scripts can have issues with multiple Nvidia GPU's being present. Until the people that created those other scripts fix up their code, there is a workaround to temporarily tell your system to only use the 1x of your Nvidia GPU's. To do this:

• Windows - You will start the script with set CUDA_VISIBLE_DEVICES=0 && python finetune.py
  After you have completed training, you can reset back with set CUDA_VISIBLE_DEVICES=
  

• Linux - You will start the script with CUDA_VISIBLE_DEVICES=0 python finetune.py
  After you have completed training, you can reset back with unset CUDA_VISIBLE_DEVICES
  

Rebooting your system will also unset this. The setting is only applied temporarily.

Depending on which of your Nvidia GPU's is the more powerful one, you can change the 0 to 1 or whichever of your GPU's is the most powerful.

This is a long standing issue with Mozilla & Firefox and one I am unable to resolve as Mozilla have not resolved the issue with Firefox. The solution is to use another web browser if you want to use Streaming audio. For details of my prior invesitigation please look at this ticket

Hindi support does not officially exist according to Coqui. Ive added a limited Hindi support at this time, however, It only works with API TTS method and Im sure there will be issues. ticket

To manually change the IP address:

1. Navigate to the SillyTavern Public folder located at /sillytavern/public/.
2. Open the settings.json file.
3. Look for the AllTalk section and find the provider_endpoint entry.
4. Replace localhost with your desired IP address, for example, 192.168.1.64.

No, the XTTS AI model does not currently support direct control over emotions or singing capabilities. While XTTS infuses generated speech with a degree of emotional intonation based on the context of the text, users cannot explicitly control this aspect. It's worth noting that regenerating the same line of TTS may yield slightly different emotional inflections, but there is no way to directly control it with XTTS.

Firstly, it's important to clarify that the development and maintenance of the XTTS AI models and core scripts are handled by Coqui, with additional scripts and libraries from entities like huggingface among many other Python scripts and libraries used by AllTalk.

AllTalk is designed to be a straightforward interface that simplifies setup and interaction with AI TTS models like XTTS. Currently, AllTalk supports the XTTS model, with plans to include more models in the future. Please understand that the deep inner workings of XTTS, including reasons why it may skip, repeat, or mispronounce, along with 3rd party scripts and libraries utilized, are ultimately outside my control.

Although I ensure the text processed through AllTalk is accurately relayed to the XTTS model speech generation process, and I have aimed to mitigate as many issues as much as possible; skips, repeats and bad pronounciation can still occur.

Certain aspects I have not been able to investigate due to my own time limitations, are:

• The impact of DeepSpeed on TTS quality. Is this more likely to cause skips or repetition?
• Comparative performance between different XTTS model versions (e.g., 2.0.3 vs. 2.0.2) regarding audio quality and consistency.

From my experience and anecdotally gained knowledge:

• Lower quality voice samples tend to produce more anomalies in generated speech.
• Model finetuning with high-quality voice samples significantly reduces such issues, enhancing overall speech quality.
• Unused/Excessive punctuation causes issues e.g. asterisks *, hashes #, brackets ( ) etc. Many of these AllTalk will filter out.

So for example, the female_01.wav file that is provided with AllTalk is a studio quality voice sample, which the XTTS model was trained on. Typically you will find it unlikely that anomolies occur with TTS generation when using this voice sample. Hence good quality samples and finetuning, generally improve results with XTTS.

If you wish to try out the XTTS version 2.0.3 model and see if it works better, you can download it from here, replacing all the files within your /alltalk_tts/models/xttsv2_2.0.2 folder. This is on my list to both test version 2.0.3 more, but also build a more flexible TTS models downloader, that will not only accomdating other XTTS models, but also other TTS engines. If you try the XTTS version 2.0.3 model and gleen any insights, please let me know.

Before starting finetuning, run the following in your terminal/command prompt:

• On Windows by typing set TRAINER_TELEMETRY=0
• On Linux & Mac by typing export TRAINER_TELEMETRY=0

Before you start finetune.py. You will now be able to finetune offline and no anonymous training data will be sent.

1. Download the Toolkit:

   • Obtain the network install version of the Nvidia CUDA Development Toolkit 11.8 from Nvidia's Archive.

2. Run the Installer:

   • Choose Custom (Advanced) installation.
   • Deselect all options initially.
   • Select the following components:
     • CUDA > Development > Compiler > nvcc
     • CUDA > Development > Libraries > CUBLAS (both development and runtime)

3. Configure Environment Search Path:

   • It's essential that nvcc and CUDA 11.8 library files are discoverable in your environment's search path. Adjustments can be reverted post-fine-tuning if desired.

     For Windows:

     • Edit the Path environment variable to include C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\bin.
     • Add CUDA_HOME and set its path to C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8.

     For Linux:

     • The path may vary by Linux distribution. Here's a generic setup:

       • export CUDA_HOME=/usr/local/cuda

       • export PATH=${CUDA_HOME}/bin:${PATH}

       • export LD_LIBRARY_PATH=${CUDA_HOME}/lib64:$LD_LIBRARY_PATH

       • Consider adding these to your ~/.bashrc for permanence, or apply temporarily for the current session by running the above commands each time you start your Python environment.

     Note: If using Text-generation-webui, its best to set these temporarily.

4. Verify Installation:

   • Open a new terminal/command prompt to refresh the search paths.
   • In a terminal or command prompt, execute nvcc --version.
   • Success is indicated by a response of Cuda compilation tools, release 11.8. Specifically, ensure it is version 11.8.

5. Troubleshooting:

   • If the correct version isn't reported, recheck your environment path settings for accuracy and potential conflicts with other CUDA versions.