Text Generation

NaviGator Toolkit offers intuitive APIs for leveraging advanced language models to generate text from prompts. These models have been trained on extensive datasets enabling them to understand various inputs including natural language instructions and complex queries. From these prompts, NaviGator Toolkit can generate a wide range of text outputs such as technical documentation, research summaries, educational content, and creative writing. As a result, NaviGator Toolkit can be a versatile tool for students, faculty, and staff across different disciplines at the University of Florida.

Quickstart

To generate text, you can use the chat completion endpoint from the REST API. The endpoint accepts queries from any HTTP client or an OpenAI compatible SDK using the programming language of your choice.

curl
python
javascript

curl -X POST https://api.ai.it.ufl.edu/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $NAVIGATOR_TOOLKIT_API_KEY" \
-d '{
    "model": "llama-3.1-8b-instruct",
    "messages": [
        {
            "role": "system",
            "content": "You are a helpful assistant."
        },
        {
            "role": "user",
            "content": "Write a haiku about an alligator."
        }
    ]
}'

from openai import OpenAI
  client = OpenAI(
      api_key="$NAVIGATOR_TOOLKIT_API_KEY",
      base_url="https://api.ai.it.ufl.edu/v1"
  )

  response = client.chat.completions.create(
      model="llama-3.1-8b-instruct", # model to send to the proxy
      messages = [
          { role: "system", content: "You are a helpful assistant." },
          {
              "role": "user",
              "content": "Write a haiku about an alligator."
          }
      ]
  )

  print(response.choices[0].message)

import OpenAI from 'openai';

const openai = new OpenAI({
  apiKey: '$NAVIGATOR_TOOLKIT_API_KEY',
  baseURL: 'https://api.ai.it.ufl.edu/v1'
});

const completion = await openai.chat.completions.create({
    model: "llama-3.1-8b-instruct",
    messages: [
        { role: "system", content: "You are a helpful assistant." },
        {
            role: "user",
            content: "Write a haiku about an alligator.",
        },
    ],
});

print(completion.choices[0].message)

Choosing a Model

When making a text generation request, the first option to configure is which model do you want to use. The model you choose can greatly influence the output and impact the cost and performance of each generation request.

Larger, more advanced models typically offer higher levels of intelligence and stronger performance while having a higher computational cost and potentially longer processing times.
Smaller, more compact models may not match the capabilities of their larger counterparts but often provide faster responses and can be more cost-effective for certain tasks.
Some specialized models are designed for specific types of tasks, such as advanced reasoning, coding, or multi-step planning. These models might take longer to process and generate more extensive outputs, but they excel in their designated areas of focus.

It is recommended to experiment with different models to see which one works best for your specific prompts and use cases. Consider factors such as the complexity of your task, required response time, cost constraints, and the level of sophistication needed in the output.

For more detailed information on choosing the right model for your needs, consult the models section documentation.

Building a Prompt

Prompt engineering is the process of crafting input instructions to elicit desired outputs from a language model. By providing precise instructions, examples, and relevant context (including specialized information not in the model's training data), you can enhance the quality and accuracy of the model's responses.

Key Components of Effective Prompts

Clear Instructions: Specify exactly what you want the model to do.
Relevant Context: Provide necessary background information.
Examples: Demonstrate the desired output format or style.
Role Assignment: Some models allow you to assign roles to different parts of the input which can influence the model's interpretation and response.

Tips for Crafting Prompts

Be specific and detailed in your instructions.
Break complex tasks into smaller, manageable steps.
Experiment with different phrasings and structures.
Consider the model's capabilities and limitations.

API Interaction

When using an API to interact with a language model, you typically create prompts by providing an array of messages or a single text input depending on the API's design. Each message or input component may serve a different purpose in guiding the model's response.

User Message

User messages are inputs that contain instructions or queries for the language model. These messages typically request a specific type of output or ask the model to perform a particular task. They are analogous to the prompts or questions you might enter when interacting with a chatbot or an AI assistant.

Example: Generating a Haiku

Here's an example of how you might structure a user message to request a haiku about programming from a language model:

curl
python
javascript

curl -X POST https://api.ai.it.ufl.edu/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $NAVIGATOR_TOOLKIT_API_KEY" \
-d '{
    "model": "llama-3.1-8b-instruct",
    "messages": [
        {
            "role": "system",
            "content": "You are a helpful assistant."
        },
        {
            "role": "user",
            "content": "Write a haiku about programming."
        }
    ]
}'

from openai import OpenAI
  client = OpenAI(
      api_key="$NAVIGATOR_TOOLKIT_API_KEY",
      base_url="https://api.ai.it.ufl.edu/v1"
  )

  response = client.chat.completions.create(
      model="llama-3.1-8b-instruct", # model to send to the proxy
      messages = [
          { role: "system", content: "You are a helpful assistant." },
          {
              "role": "user",
              "content": "Write a haiku about programming."
          }
      ]
  )

  print(response.choices[0].message)

import OpenAI from 'openai';

const openai = new OpenAI({
  apiKey: '$NAVIGATOR_TOOLKIT_API_KEY',
  baseURL: 'https://api.ai.it.ufl.edu/v1'
});

const completion = await openai.chat.completions.create({
    model: "llama-3.1-8b-instruct",
    messages: [
        { role: "system", content: "You are a helpful assistant." },
        {
            role: "user",
            content: "Write a haiku about programming.",
        },
    ],
});

print(completion.choices[0].message)

In the example mentioned above:

We're using a a chat completion creation method depending on the programming language.
The model parameter specifies which language model to use.
The messages array contains our input, with a single user message.
The message has a role of user and contains content with the instruction.
The system prompt is configured under role system

Note that the exact structure and method names may vary depending on the specific API or library you're using to interact with the language model. Always refer to the documentation of your chosen programming language and libraries.

System Messages

System messages serve as high-level instructions to the language model typically describing the model's intended role, behavior, and response style. These messages set the context for the entire conversation or interaction.

Here's an example of how you might use a system message to modify the model's behavior when generating a response:

curl
python
javascript

curl -X POST https://api.ai.it.ufl.edu/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $NAVIGATOR_TOOLKIT_API_KEY" \
-d '{
    "model": "llama-3.1-8b-instruct",
    "messages": [
        {
          "role": "system",
          "content": "You are a helpful assistant that answers programming questions in the style of a pirate from the 12th century."
        },
        {
          "role": "user",
          "content": "Are semicolons optional in JavaScript?"
        }
    ]
}'

from openai import OpenAI
  client = OpenAI(
      api_key="$NAVIGATOR_TOOLKIT_API_KEY",
      base_url="https://api.ai.it.ufl.edu/v1"
  )

  response = client.chat.completions.create(
      model="llama-3.1-8b-instruct", # model to send to the proxy
      messages = [
          { 
            "role": "system",
            "content": "You are a helpful assistant that answers programming questions in the style of a pirate from the 12th century."
          },
          {
            "role": "user",
            "content": "Are semicolons optional in JavaScript?"
          }
      ]
  )

  print(response.choices[0].message)

import OpenAI from 'openai';

const openai = new OpenAI({
  apiKey: '$NAVIGATOR_TOOLKIT_API_KEY',
  baseURL: 'https://api.ai.it.ufl.edu/v1'
});

const completion = await openai.chat.completions.create({
    model: "llama-3.1-8b-instruct",
    messages: [
        { "role": "system",
            "content": "You are a helpful assistant that answers programming questions in the style of a pirate from the 12th century."
        },
        {
            "role": "user",
            "content": "Are semicolons optional in JavaScript?"
        },
    ],
});

print(completion.choices[0].message)

This prompt is designed to generate a response in the specified rhetorical style. Here's an example of what the output might look like:

Arrrr, ye landlubber! Ye be askin' about semicolons in JavaScript, eh? 

In JavaScript, semicolons be like the anchors that keep yer code from driftin' apart. They be optional, but not entirely, or should I say, not quite entirely optional. Ye see, JavaScript be a bit like a scurvy dog, it be tryin' to make sense of things, but it be not always as strict as it should be.

Ye can write yer code without semicolons, but ye be leavin' it to the whims of the JavaScript interpreter. If ye be writin' in a way that the interpreter be expectin' a statement to be terminated, it'll assume ye meant to put a semicolon there. But if ye be writin' in a way that's ambiguous, it might just throw ye a syntax error or two.

Here be an example, matey:

console.log('Hello, world!')
console.log('This be a test')

In this case, the semicolons be optional, and the code will run just fine. But if ye be writin' like this:

console.log('Hello, world')
console.log('This be a test')

The JavaScript interpreter be gettin' a bit confused, seein' only one string literal and not knowin' what to make of it. So, it'll throw ye an error, savvy?

So, while semicolons be technically optional, ye be makin' yer life easier by puttin' 'em in, like addin' a bit o' extra anchor to keep yer code from driftin' apart.

Now, go forth and code like a proper pirate, and remember: semicolons be yer friends!

Note that the actual output may vary depending on the specific language model and implementation. The system message helps guide the overall tone and approach of the model's response, while still allowing it to generate unique content based on its training.

Conversations and Context

While each text generation request in NaviGator Toolkit is typically independent, you can implement multi-turn conversations by providing additional context in your requests. Here's an example of how you might structure a "knock knock" joke interaction:

const response = await navigatorToolkit.generateText({
  model: "navigator-advanced",
  messages: [
    {
      "role": "user",
      "content": "knock knock."
    },
    {
      "role": "assistant",
      "content": "Who's there?"
    },
    {
      "role": "user",
      "content": "Orange."
    }
  ]
});

Managing Context for Text Generation

As your inputs become more complex or include more conversation turns, consider both output limits and context window constraints. Inputs and outputs are typically measured in tokens, which are used to analyze content and intent, and to construct logical outputs. Models have limits on how many tokens can be used during a text generation request.

Output limits: The maximum number of tokens a model can generate in response to a prompt. Each model in NaviGator Toolkit has its own output limit.
Context window: The total tokens that can be used for both input and output (and for some models, intermediate processing). This varies by model.

If you create a very large prompt (usually by including extensive conversation context or additional data), you might exceed the model's context window, potentially resulting in truncated outputs.

NaviGator Toolkit provides tools to help you estimate token usage for your inputs, allowing you to optimize your prompts and manage context effectively.

For specific token limits and context window sizes for each model in NaviGator Toolkit, please refer to the NaviGator AI model documentation.

Optimizing Outputs

As you refine your prompts, you'll continuously aim to enhance accuracy, efficiency, and response time.

GOAL	AVAILABLE TECHNIQUES
Accuracy	Ensure the model produces accurate and useful responses to your prompts.
Efficiency	Optimize the computational resources used by reducing token usage and utilizing more compact models when suitable.
Latency	Reduce the time required to generate responses to your prompts.

For specific strategies and best practices tailored to NaviGator Toolkit, please refer to the University of Florida's comprehensive documentation and resources.

Quickstart​

Choosing a Model​

Building a Prompt​

Key Components of Effective Prompts​

Tips for Crafting Prompts​

API Interaction​

User Message​

Example: Generating a Haiku​

System Messages​

Conversations and Context​

Managing Context for Text Generation​

Optimizing Outputs​