Llama 3.1 8B

Approved Data Classifications

Description

Llama-3.1-8B is a compact yet powerful language model developed by Meta, designed to deliver efficient performance in multilingual dialogue applications. With 8 billion parameters, this model is optimized for scenarios where computational resources are limited, making it ideal for startups and small businesses seeking to integrate AI capabilities without incurring significant costs. It features a context length of up to 128,000 tokens, allowing it to process extensive text inputs while maintaining coherence and relevance in its outputs. Llama-3.1-8B excels in tasks such as content generation, summarization, and natural language understanding, leveraging advanced training techniques like supervised fine-tuning and reinforcement learning with human feedback to ensure high-quality responses. Its adaptability and efficiency make it a versatile tool for developers looking to implement AI solutions across various industries while balancing performance with resource constraints.

Capabilities

Model	Training Data	Input	Output	Context Length	Cost (per 1 million tokens)
llama-3.1-8b-instruct	July 2024	Text	Text	128,000	$0.22/1M input $0.22/1M output

info

• 1M represents 1 Million Tokens
• All prices listed are based on 1 Million Tokens

Availability

Cloud Provider

Usage

curl

curl -X POST https://api.ai.it.ufl.edu/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer <API_TOKEN>" \
-d '{
    "model": "llama-3.1-8b-instruct",
    "messages": [
        {
            "role": "system",
            "content": "You are a helpful assistant."
        },
        {
            "role": "user",
            "content": "Write a haiku about an Alligator."
        }
    ]
}'

python

from openai import OpenAI
  client = OpenAI(
      api_key="your_api_key",
      base_url="https://api.ai.it.ufl.edu/v1"
  )

  response = client.chat.completions.create(
      model="llama-3.1-8b-instruct", # model to send to the proxy
      messages = [
          { role: "system", content: "You are a helpful assistant." },
          {
              "role": "user",
              "content": "Write a haiku about an Alligator."
          }
      ]
  )

  print(response.choices[0].message)

javascript

import OpenAI from 'openai';

const openai = new OpenAI({
  apiKey: 'your_api_key',
  baseURL: 'https://api.ai.it.ufl.edu/v1'
});

const completion = await openai.chat.completions.create({
    model: "llama-3.1-8b-instruct",
    messages: [
        { role: "system", content: "You are a helpful assistant." },
        {
            role: "user",
            content: "Write a haiku about an Alligator.",
        },
    ],
});

print(completion.choices[0].message)