Skip to content

Large Language Model (LLM)

Version badge

Large Language Models (LLMs) are a specific type of Generative AI model that are designed for generating human-like text based on the input and context provided. These models are trained on vast amounts of text data, allowing them to learn patterns, syntax, and semantic relationships between words and phrases.

With LLMs, Cognigy virtual agents can understand and respond to user input in a natural way. These models make conversations more engaging by generating relevant and contextually appropriate responses. LLMs also assist in managing dialogues and providing multilingual support, enhancing the overall conversational experience for users.

Supported Models

The table below lists the LLMs supported by Cognigy.

Models/
Cognigy Features		 Intent Sentence Generation AI Enhanced Outputs Lexicon Generation Flow Generation GPT Conversation Node LLM Prompt Node & Search Extract Output Node Generate Node Output Knowledge Search Sentiment Analysis
Microsoft Azure OpenAI
gpt-3.5-turbo (ChatGPT) + + + + + + + - +
text-davinci-003 (Deprecated) + + + + + + + - -
text-embedding-ada-002 - - - - - - - + -
OpenAI
gpt-3.5-turbo (ChatGPT) + + + + + + + - +
text-davinci-003 (Deprecated) + + + + + + + - -
text-embedding-ada-002 - - - - - - - + -
Anthropic
claude-v1-100k - - - - - + - - -
claude-instant-v1 - - - - - + - - -
Google
text-bison-001 (Bard) - - - - - + - - -

Add a Model

To add a model to Cognigy.AI, follow these steps:

  1. Open the Cognigy.AI interface.
  2. Go to Build > LLM.
  3. Click +New LLM.
  4. In the New LLM window, select a model from the Model Type list.
  5. Add a unique name and description for your model and click Save.
  6. In the LLM Editor window, go to the Generative AI Connection field.
  7. On the right side of the field, click +.

  8. Depends on your model provider, do the following:

    8.1 Fill in the following fields:
    - Connection name — create a unique name for your connection.
    - apiKey — add an Azure API Key. This value can be found in the Keys & Endpoint section when examining your resource from the Azure portal. You can use either KEY1 or KEY2.
    - resourceName — add a resource name. This value can be found under Resource Management > Deployments in the Azure portal or alternatively under Management > Deployments in Azure OpenAI Studio.
    8.2 Click Create.
    8.3 Fill in the remaining fields:
    - deploymentName — add a model name.
    - apiVersion — add an API version. The API version to use for this operation in the YYYY-MM-DD format.

    8.1 Fill in the following fields:
    - Connection name — create a unique name for your connection.
    - apiKey — add an API Key from your OpenAI account. You can find this key in the User settings of your OpenAI account.
    8.2 Click Create.

    8.1 Fill in the following fields:
    - Connection name — create a unique name for your connection.
    - apiKey — add an API Key that you generated via Account Settings in Anthropic.
    8.2 Click Create.

    8.1 Fill in the Connection name field by specifying a unique name for your connection.
    8.2 To upload the JSON file with a key for your model, you need to obtain this key. Go to the Google Vertex AI console.
    8.3 Click the Enable All Recommended APIs button to activate an API connection, if it is not activated. Make sure that the Vertex AI API are enabled.
    8.4 In the left-side menu, go to the IAM & Admin > Service Accounts.
    8.5 Select Actions and click Manage Keys.
    8.6 On the Keys page, select Add Key and click Create new Key.
    8.7 In the appeared window, select the JSON key type and click Create. The file will be downloaded.
    8.8 In Cognigy, in the New Connection window, click Upload JSON file and upload the file.
    8.9 Click Create.
    8.10 Fill in the remaining fields:
    - Location — add a region for the model. For example, us-central1.
    - API Endpoint — add a service endpoint for the model. For example, us-central1-aiplatform.googleapis.com. Note that the endpoint should be specified without https:// or http://.
    - Publisher — add an owner's name of the model. If not specified, Google will be used by default. This parameter is optional.

  9. To apply changes, click Save.

  10. To check if the connection was set up, click Test.

When the model is added, you will see it in the list of models.

To apply this model for Cognigy features, go to the settings by clicking Manage LLM Features.

Apply a Model

To apply a model, follow these steps:

  1. Open the Cognigy.AI interface.
  2. In the left-side menu, click Manage > Settings.
  3. In the Generative AI Settings section, activate Enable Generative AI Features. This setting is toggled on by default if you have previously set up the Generative AI credentials.
  4. Navigate to the desired feature and choose a model from the list. If there are no models available for the selected feature, the system will automatically select None.
  5. Click Save.

Clone a Model

To create a copy of the existing model, follow these steps:

  1. Go to Build > LLM.
  2. Hover over the existing model and click vertical-ellipsis.
  3. Select Clone from the list.

The model will contain the same settings as the initial one.

Set a Model as Default

Setting a default model ensures a smooth transition when a specific model is removed. It guarantees that there is always a model available to handle compatible use cases, even if the assigned model is removed.

To set a model as the default, follow these steps:

  1. Go to Build > LLM.
  2. Hover over the existing model and click vertical-ellipsis.
  3. Select Make Default from the list.

The setting will be applied for the selected model.

Export a Model as Package

To reuse a model in other agents, you can package the model.

To package a model, follow these steps:

  1. Go to Build > LLM.
  2. Hover over the existing model and click vertical-ellipsis.
  3. Select Create a package.
  4. Once the package has created, a new task, titled Create package, will be run. To view the task, click task-menu in the upper-right corner.

When the task is completed, the package will be downloaded.

Delete a Model

Note that a default model cannot be deleted. Before deletion, you need to remove the default tag.

To delete a model, follow these steps:

  1. Go to Build > LLM.
  2. Hover over the existing model and click vertical-ellipsis.
  3. Select Delete.
  4. Confirm the deletion. Features relying on this model will stop working if no default model is configured to support those features.

The model will be deleted from the list.

More information