Llama 3.3 70B Instruct FP8 Fast (Cloudflare Workers AI)

Llama 3.3 70B quantized to fp8 precision, optimized to be faster.

Provider: All Meta models | Cloudflare Workers AI

API Endpoint

https://api.cloudflare.com/client/v4/accounts/{account_id}/ai/v1/chat/completions

Quick Start (Python)

Install: pip install openai

import os

from openai import OpenAI

account_id = os.environ["CLOUDFLARE_ACCOUNT_ID"]

client = OpenAI(
    api_key=os.environ["CLOUDFLARE_API_TOKEN"],
    base_url=f"https://api.cloudflare.com/client/v4/accounts/{account_id}/ai/v1",
)

response = client.chat.completions.create(
    model="@cf/meta/llama-3.3-70b-instruct-fp8-fast",
    messages=[
        {"role": "user", "content": "Hello, how are you?"},
    ],
    max_tokens=1024,
)

print(response.choices[0].message.content)

Additional examples: Basic invoke, REST API (any task)

Supported Parameters

ParameterTypeDescription
max_tokensintegerMaximum tokens to generate. (≥1)
temperaturefloatControls randomness. Supported range varies by model. (≥0)
streambooleanStream response chunks as they are generated. Default: false.

Feature Guides

OpenAI Compatibility

Call text generation and embedding models with OpenAI-compatible client libraries using the account-scoped Workers AI base URL.

Documentation

REST API

Run any Workers AI model, including image, speech, and embedding models, through the account-scoped /ai/run endpoint.

Documentation

Model Catalog

List available models, tasks, and per-model properties with the Workers AI models search API.

Documentation