Granite 3.3 8B

Approved Data Classifications

Description

Granite-3.3-8B-Instruct is a 8B parameter long-context instruct model finetuned from Granite-3.3-8B-Base using a combination of open source instruction datasets with permissive license and internally collected synthetic datasets tailored for solving long context problems. This model is developed using a diverse set of techniques with a structured chat format, including supervised finetuning, model alignment using reinforcement learning, and model merging.

This model has the following capabilities:

• Thinking
• Summarization
• Text classification
• Text extraction
• Question-answering
• Retrieval Augmented Generation (RAG)
• Code related tasks
• Function-calling tasks
• Multilingual dialog use cases
• Long-context tasks including long document/meeting summarization, long document QA, etc.

Capabilities

Model	Training Data	Input	Output	Context Length	Cost (per 1 million tokens)
granite-3.3-8b-instruct	April 2024	Text	Text	128,000	$0.22/1M input $0.22/1M output

info

• 1M represents 1 Million Tokens
• All prices listed are based on 1 Million Tokens

Availability

Cloud Provider

Usage

curl

curl -X POST https://api.ai.it.ufl.edu/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer <API_TOKEN>" \
-d '{
    "model": "granite-3.3-8b-instruct",
    "messages": [
        {
            "role": "system",
            "content": "You are a helpful assistant."
        },
        {
            "role": "user",
            "content": "Write a haiku about an Alligator."
        }
    ]
}'

python

from openai import OpenAI
  client = OpenAI(
      api_key="your_api_key",
      base_url="https://api.ai.it.ufl.edu/v1"
  )

  response = client.chat.completions.create(
      model="granite-3.3-8b-instruct", # model to send to the proxy
      messages = [
          { role: "system", content: "You are a helpful assistant." },
          {
              "role": "user",
              "content": "Write a haiku about an Alligator."
          }
      ]
  )

  print(response.choices[0].message)

javascript

import OpenAI from 'openai';

const openai = new OpenAI({
  apiKey: 'your_api_key',
  baseURL: 'https://api.ai.it.ufl.edu/v1'
});

const completion = await openai.chat.completions.create({
    model: "granite-3.3-8b-instruct",
    messages: [
        { role: "system", content: "You are a helpful assistant." },
        {
            role: "user",
            content: "Write a haiku about an Alligator.",
        },
    ],
});

print(completion.choices[0].message)

---

References

1. LLM Stats
   https://llm-stats.com
2. Artificial Analysis
   https://artificialanalysis.ai
3. Huggingface
   https://huggingface.co/