Using LLM for Sentiment Analysis: Best Practices and Examples
shashank msWe are building a sentiment analysis pipeline that classifies customer feedback, scores confidence, and extracts key themes in a single LLM call. This
We are building a sentiment analysis pipeline that classifies customer feedback, scores confidence, and extracts key themes in a single LLM call. This replaces brittle keyword-based classifiers and gives product teams structured signal without maintaining custom NLP infrastructure. I have used this exact pattern to triage support tickets and monitor app store reviews.
What you'll need
- Python 3.10 or newer
- The OpenAI SDK:
pip install openai - An Oxlo.ai API key from https://portal.oxlo.ai
Oxlo.ai is a developer-first inference platform with flat per-request pricing, so long customer rants cost the same as short thank-you notes. You can explore plans at https://oxlo.ai/pricing.
Step 1: Configure the Oxlo.ai client
Oxlo.ai exposes a fully OpenAI-compatible API, so the official SDK works with two extra configuration lines. I default to llama-3.3-70b because it handles open-ended text reliably and supports JSON mode.
from openai import OpenAI
client = OpenAI(
base_url="https://api.oxlo.ai/v1",
api_key="YOUR_OXLO_API_KEY" # get yours at https://portal.oxlo.ai
)Step 2: Define the analysis schema
To keep output machine-readable, I lock the model into a strict JSON schema with a system prompt. This prevents extra commentary and guarantees parseable results.
SYSTEM_PROMPT = """You are a sentiment analysis engine. Analyze the user provided text and return a single JSON object with exactly these keys:
- sentiment: one of "positive", "negative", "neutral", or "mixed"
- confidence: a float between 0.0 and 1.0 representing your certainty
- themes: an array of up to 3 short strings describing the main topics or issues mentioned
- summary: one concise sentence summarizing the feedback
Rules:
1. Respond with valid JSON only. No markdown code fences, no explanations outside the JSON.
2. If the text is sarcastic, classify by the underlying intent.
3. If the text is empty or unreadable, return {"sentiment": "neutral", "confidence": 0.0, "themes": [], "summary": "No readable content."}
"""Step 3: Build the analyzer function
This function sends text to Oxlo.ai and parses the response. I set response_format to JSON mode and keep temperature low so the schema stays stable.
import json
def analyze_feedback(text: str, model: str = "llama-3.3-70b"):
if not text or not text.strip():
return {
"sentiment": "neutral",
"confidence": 0.0,
"themes": [],
"summary": "No readable content."
}
try:
response = client.chat.completions.create(
model=model,
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": text},
],
response_format={"type": "json_object"},
temperature=0.1,
)
result = json.loads(response.choices[0].message.content)
return result
except Exception as e:
return {
"sentiment": "error",
"confidence": 0.0,
"themes": [],
"summary": f"Processing failed: {e}"
}Step 4: Process feedback in batches
Production support queues do not arrive one line at a time. This loop handles a realistic batch. Because Oxlo.ai uses request-based pricing rather than token-based pricing, a ten-paragraph complaint costs the same as a one-word reply. That makes unpredictable ticket lengths financially manageable.
feedback_batch = [
"The new dashboard is incredible. I can finally see all my metrics in one place.",
"I waited 20 minutes for a chat response and then the agent closed my ticket without solving anything. Frustrating.",
"It is okay. Nothing special but it works.",
"Love the app! Though I wish the export button was easier to find.",
"",
]
results = []
for item in feedback_batch:
out = analyze_feedback(item)
results.append({"input": item, "analysis": out})
print(json.dumps(results, indent=2))Step 5: Filter by confidence
Not every prediction is actionable. I add a threshold check so low-confidence negative items get flagged for human review instead of auto-routing.
def should_escalate(analysis: dict, threshold: float = 0.75) -> bool:
if analysis.get("sentiment") == "negative":
return analysis.get("confidence", 0.0) < threshold
return False
for r in results:
analysis = r["analysis"]
if should_escalate(analysis):
print(f"ESCALATE: {r['input'][:50]}...")
else:
print(f"[{analysis['sentiment']}] {analysis['summary']}")Run it
Here is the complete standalone script followed by the output it produces against Oxlo.ai.
from openai import OpenAI
import json
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
SYSTEM_PROMPT = """You are a sentiment analysis engine. Analyze the user provided text and return a single JSON object with exactly these keys:
- sentiment: one of "positive", "negative", "neutral", or "mixed"
- confidence: a float between 0.0 and 1.0 representing your certainty
- themes: an array of up to 3 short strings describing the main topics or issues mentioned
- summary: one concise sentence summarizing the feedback
Rules:
1. Respond with valid JSON only. No markdown code fences, no explanations outside the JSON.
2. If the text is sarcastic, classify by the underlying intent.
3. If the text is empty or unreadable, return {"sentiment": "neutral", "confidence": 0.0, "themes": [], "summary": "No readable content."}
"""
def analyze_feedback(text: str, model: str = "llama-3.3-70b"):
if not text or not text.strip():
return {"sentiment": "neutral", "confidence": 0.0, "themes": [], "summary": "No readable content."}
try:
response = client.chat.completions.create(
model=model,
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": text},
],
response_format={"type": "json_object"},
temperature=0.1,
)
return json.loads(response.choices[0].message.content)
except Exception as e:
return {"sentiment": "error", "confidence": 0.0, "themes": [], "summary": f"Processing failed: {e}"}
feedback_batch = [
"The new dashboard is incredible. I can finally see all my metrics in one place.",
"I waited 20 minutes for a chat response and then the agent closed my ticket without solving anything. Frustrating.",
"It is okay. Nothing special but it works.",
"Love the app! Though I wish the export button was easier to find.",
"",
]
for item in feedback_batch:
out = analyze_feedback(item)
print(json.dumps({"input": item, "analysis": out}, indent=2))Typical output:
{
"input": "The new dashboard is incredible. I can finally see all my metrics in one place.",
"analysis": {
"sentiment": "positive",
"confidence": 0.95,
"themes": ["dashboard", "metrics", "usability"],
"summary": "User praises the new dashboard for consolidated metrics."
}
}
{
"input": "I waited 20 minutes for a chat response and then the agent closed my ticket without solving anything. Frustrating.",
"analysis": {
"sentiment": "negative",
"confidence": 0.92,
"themes": ["support wait time", "ticket handling", "frustration"],
"summary": "User is frustrated by long wait times and unresolved ticket closure."
}
}
{
"input": "It is okay. Nothing special but it works.",
"analysis": {
"sentiment": "neutral",
"confidence": 0.78,
"themes": ["general feedback"],
"summary": "User expresses indifferent satisfaction with the product."
}
}
{
"input": "Love the app! Though I wish the export button was easier to find.",
"analysis": {
"sentiment": "mixed",
"confidence": 0.85,
"themes": ["app praise", "export feature", "UI navigation"],
"summary": "User likes the app but finds the export button difficult to locate."
}
}
{
"input": "",
"analysis": {
"sentiment": "neutral",
"confidence": 0.0,
"themes": [],
"summary": "No readable content."
}
}Next steps
Swap in kimi-k2.6 if you need to analyze long conversation threads up to 131K tokens, or test against deepseek-v3.2 on Oxlo.ai's free tier to validate results before scaling. After that, wire the analyzer into a webhook so incoming support tickets are classified and routed automatically.