Dungeons & Dragons Assistant - OpenAI Answers

No time to walk through the tutorial? Test this prompt out here.

Use Case

Given existing data as the baseline, a generative language model is capable of creating new, plausible content using the baseline as its source of context.

One of the challenges in building a “Dungeon Master”-esque tool is to allow it to be endlessly creative while still following a set of guidelines and rules. In theory, this is a game in which “anything is possible” and yet, the DM must adhere to some form of storyline, not devolve into repetitive gibberish, and protect the user's experience by returning quality and consistent content that falls in line with their expectations.

A particularly vulnerable element of this type of language generation models is giving public users unfettered creative access to generating content - proper logging, monitoring, and security policies must be in place on the front end of deployment in order to manage user expectations and behaviors. Establishing clear guidelines for use is of particular importance when users have control over an open-ended AI experience that may contain sensitive topics and/or can be shared publicly.

In order to strike a balance between unlimited creative freedom and a safe user experience, some boundaries should be built into the service and application:

• Clear policies defining misuse of the platform and associated consequences
• Thorough vetting of users for age-appropriate content/levels of access
• Robust and secure monitoring infrastructure that protects user privacy
• Flagging, filtering, and notification policies for content that violates user guidelines

As Mantium continues to grow and build out its security policies, our goal is to provide developers the ability to establish safety features into the foundations of their applications.

Security Policy Settings

Before creating a new prompt, configure a new security policy by clicking Security > Add New Security Policy and applying the following settings.

When configuring rule settings for this prompt, both user input and model output content should be considered. If the user inputs offensive speech, it is more likely for the model to output offensive speech - this is why the filter threshold for input is higher than that of output.

Rule Settings

• ✅ Input Offensive Speech Detector Filter Threshold: 60
  
• ✅ Output Offensive Speech Detector Filter Threshold: 30
  
• ✅ Log Warning (this is automatic)

Notification Settings (Optional)

• ✅ Your Email Address

If you have email integration enabled, you are able to set up notification settings to receive an alert should these prompt security policies be triggered. This is optional but feel free to try it out!

Click Create and Finish to save!

File Upload

A file upload is required for this prompt, which is recommended to do before beginning prompt creation.

📘

File Format

❗ OpenAI requires that files used as the prompt body follow a few guidelines:
- They are in .jsonl format
- Input text is stored as {‘text’ : your_text_here, ‘metadata’ : optional_field}

Per OpenAI documentation, ‘metadata’ is an optional field that gives some descriptive information loosely associated with ‘text.’

We have prepared a file to follow this format and you can download it here:

dnd_data.jsonl

For this project, we used a dataset from HuggingFace that contains transcripts from 159 episodes from Critical Role, an unscripted podcast in which a fixed group of hosts play Dungeons & Dragons - this dataset contains approximately 390,000 turns of D&D as well as summaries from the podcast’s fan wiki. These documents provide the assistant with information that helps guide the content it generates.

Upload
To upload, click AI Manager > Files > Add File.
Click + and select the correct file.
Select "Answers" in the Purpose dropdown menu - this corresponds to the endpoint.
Click Upload - you may need to refresh this page for the file to appear in the list.

Prompt Creation

When you are ready to create your prompt, click AI Manager > Prompts > Add New Prompt.

Then, fill out the following:

• Name of Prompt: Dungeons and Dragons Assistant
• Description: OpenAI Answers

Tags and Intelets can be left blank.

Then, apply the security policy you just configured by clicking Add Security Policy and dragging the policy from the left column to the right. Click Done to save - you will know the policy has been applied when its name is visible under Security Policies.

Provider Settings

• Provider: OpenAI
• Endpoint: Answers

Prompt Body

Request Body
For this prompt, select "File" from the Request Body dropdown menu.
You will see a list of available files to select from - select the one you just uploaded!

Examples

It is recommended to provide two or three examples of questions and answers as a guide for the model's tone and formatting. We have prepared examples for you, but you are welcome to get creative here if you'd like!

Question: Who is the most powerful dragon?
Answer: Trogdor the Burninator is the most powerful dragon! He is burninating the countryside!

Question: When we dug our way out of the vines, we stumbled into a clearing. What did we find next?
Answer: You see a massive minotaur with raging red eyes, staring directly at you!

Question: What value do I need to roll to succeed in this battle?
Answer: You have to roll higher than 12.

Context

Context provides the model with information about the example questions and answers. We have prepared a text snippet for you to copy and paste:

We are going on an adventure and our goal is to collect armor, weapons, gold, and artifacts.
We are traveling in a party of four - there is a clever bard, a trustworthy dwarf, a powerful harpie, and a giant parrot.
We're not sure why he's here. We will be traveling across land, sea, and maybe even time - you never know what kind of mysterious tricks harpies have up their sleeves.
The day begins with all of us sitting around an old wooden table in a dark tavern.

Note that there are two blank lines at the end of the context paragraph - please be sure to include those in your prompt configuration when you paste the text snippet like so:

Model | Search Model

• Base Model: Curie
• Search Model: Ada

Here, we selected Curie as the base model and Ada as the document search model - Curie is an engine balanced to be fast at processing but also powerful and good for answering questions, and Ada is normally the fastest at parsing text.

OpenAI’s Engine Documentation

Prompt Settings

• Max Rerank: 200
• Max Tokens: 60
• Temperature: .75
• Expand: leave blank
• N: 1
• LogProbs: 1
• Return Prompts: False
• Return Metadata: False
• Stop Sequences:
  • \n
  • \n\n

Max Rerank is 200 by default - this is the maximum number of documents/entries that the model will look through while ranking their relevance to the query. OpenAI's current maximum value for using a file is 200.

Max Tokens is the primary method for controlling response length. As we want the model to give a detailed response that doesn’t spin off too far, we kept the length at 60 tokens. For OpenAI’s calculation of tokens, one token is approximately 4 characters or .75 words in English text.

Temperature controls “creativity” - higher temperatures will produce more unique and creative outputs but also are more likely to become nonsensical. Specifically for a fantasy story generator, our tolerance for nonsensical behavior is higher.

Stop sequences are another method of controlling output. Using both a single and a double line break as stop sequences in combination with a Max Token value helps the model return complete sentences that end with punctuation - this finding was largely a result of trial and error and is specific to this prompt, and we encourage you to tweak prompt settings to find what works best for your desired output.

Test Prompt Text

Because these models are stochastic in nature, the resulting output will vary even when using the same input, but we have included a text snippet to run as a test:

I look around. Who else is here with me in the tavern?

We suggest running Test Run a few times to see the different possibilities of output!

Once you are satisfied with the results, click Save.

Results & Conclusion

Prompt outputs tend to vary in degrees of creativity, complexity, and response length. Despite the higher creativity temperature setting, most responses lean toward sensical but are susceptible to repetition. Combining a single and double line break showed promise as a way to control the output to lean towards complete sentences.

Overall, the field of generative language models is still rapidly evolving – increased model complexity, data availability, and developments in fine-tuning methods are broadening the capabilities (and challenges) of deep learning tools. We will continue to dive into the fine-tuning and security systems of models to enable and empower developers to maximize creativity, user experience, and safety.

One-Click Deploy

Mantium enables sharing your prompt using the One-Click Deploy feature, located in each prompt's drawer view. From your list of prompts, click on the Dungeons and Dragons Assistant prompt, click Deploy , and add the following configuration settings:

• Name: Dungeons and Dragons Assistant
• Description: Use this to drive your D&D story! Statements ending with a question work best.
• Author Name: Your Name
• ✅ Add Input Field
• Public
• Live
• Input Placeholder Text: I look around. Who else is here with me in the tavern?

To interact with this prompt, we suggest ending each input statement with a question to follow the Question-Answer format this model was configured to follow. An example is included in the placeholder text (it is the same example as the test prompt text above).

To test out the prompt that we have configured, click this link!

Similar Use Cases

Creating a language generation tool that is capable of answering questions creatively and sensibly comes with a unique set of challenges. We encourage you to explore different genres of creative writing, or even to use your own!