Bank Request Intent Classification With OpenAI

No time to walk through the tutorial? Test this prompt out here.

Use Case

When someone makes an inquiry to some form of customer service, they are driven by a purpose or intention - for example, they may want to request information, change their existing information with the service, or lodge a complaint. The automated categorization of these queries based on customer intention is known as Intent Classification.

In order for a request to be classified by a language processing tool, existing examples with their corresponding labels must be provided as guidance. If a customer places a request such as "I haven't received my card and am worried it is lost,” the model will search through existing examples to determine which are the most similar and which labels belong to them. If there is a label in the existing dataset that is called “card_arrival” and the model is able to match its corresponding examples as the most similar, this new request will be labeled as “card_arrival” as well.

Intent classification can help create the next step in an advanced workflow and is particularly useful for chatbot workflows. For example, classifying the intent of a customer's request could be the first step in a workflow that sends the classification to a generative model that is trained to interact with the particular request, or to pass the request off to a human customer service representative for a more complex interaction.

File Upload

A file upload is required for this prompt, which is recommended to do before beginning prompt creation.

📘

File Format

For classification, OpenAI requires that files used as the prompt body follow a few guidelines:
- They are in .jsonl format
- Input text is stored as {‘text’ : text, ‘label’ : text_label}

We have prepared a file to follow this format and you can download it here:

bank77.json

For an intent classifier to work, the model must first be provided with examples of text and their corresponding labels which it will use as the baseline for classifying new information. Bank77 is a HuggingFace dataset that is composed of 13,083 online banking customer service queries annotated with 77 corresponding intents.

Below is an example from the dataset:

{"text": "I am still waiting on my card?", "label": "card_arrival"}
{"text": "What can I do if my card still hasn't arrived after 2 weeks?", "label": "card_arrival"}

Upload

To upload, click AI Manager > Files > Add File.
Click + and select the correct file.
Select "Classifications" in the Purpose dropdown menu - this corresponds to the endpoint.
Click Upload - you may need to refresh this page for the file to appear in the list.

Prompt Creation

When you are ready to create your prompt, click AI Manager > Prompts > Add New Prompt.

Then, fill out the following:

• Name of Prompt: Bank Intent Classifier
• Description: OpenAI Classifications

Tags and Intelets can be left blank.

For deploying your prompt publicly at the end of this tutorial, you can add a default security policy configured by Mantium. Click Add Security Policies under Security Policies and drag Default Policies from All Policies to Selected Policies. Click Done to save - you will know the policy has been applied when its name is visible under Security Policies.

Provider Settings

• Provider: OpenAI
• Endpoint: Classifications

Prompt Body

Request Body
For this prompt, select "File" from the Request Body dropdown menu.
You will see a list of available files to select from - select the one you just uploaded!

Model | Search Model

• Base Model: Curie
• Search Model: Babbage

Here, we selected Curie as the base model and Babbage as the document search model - Curie is an engine balanced to be fast at processing but also powerful and good for answering questions, and Babbage is a good choice for semantic search classification.

OpenAI’s Engine Documentation

Prompt Settings

• Temperature: 0
• LogProbs: 1
• Max Examples: 200
• Return Prompts: False
• Return Metadata: False

Temperature controls “creativity” - lower temperatures are associated with less creative and less random responses. In this case, the model’s task is to make factual classifications based on real requests, so the temperature should remain at 0.

Max Examples is 200 by default - this is the maximum number of documents/entries that the model will look through to determine the most similar existing entries in order to classify the new request. OpenAI's current maximum value for using a file is 200.

Test Prompt Text

To test out the prompt's performance, paste the following into the Input box:

How old do I need to be to open an account?

The prompt response should return Age_limit. There is still a small possibility that labels may be returned inconsistently despite sufficient data, and we encourage you to click Test Run several times to see if the same text is classified differently with multiple tests. Reducing temperature is a good way to control consistency but it is not always foolproof.

Once you are satisfied with the results, click Save.

Results & Conclusion

For businesses, utilizing intent classification can help improve customer satisfaction by lowering response time and re-allocating resources that were used for determining customer needs - for instance, the representative who normally routes customer requests could be trained to address unique situations instead of handling generic queries that don't require personal interaction.

This is a particularly useful tool for businesses that handle high-volume queries, such as banking or subscription services, where time and resource management have direct impacts on the customer experience.

One-Click Deploy

Mantium enables sharing your prompt using the One-Click Deploy feature, located in each prompt's drawer view. From your list of prompts, click on the Bank Intent Classification prompt, then click Deploy.

Then, add the following configuration settings:

• Name: Bank Intent Classification
• Author: Your Name
• ✅ Add Input Field
• ✅ Include Examples - here you can provide examples of customer service queries to help future users interact with your prompt.
• Public
• Live
• ✅ I have followed my provider's go live requirements

To test out this prompt, input an example of what a customer may request from their bank’s customer service center.

Click here to test out an existing prompt.

Similar Use Cases

Classification has many potentials for use, including both practical and creative! We encourage you to try out a use case that may be more applicable to your interests. Below are a few examples to which classification can also be applied:

• Classify 3 (out of 5) star reviews to be positive or negative.
• Classify to which species a description of an animal belongs.
• Classify a musical group by their genre.