reducto/RolmOCR
Image-to-Text
•
8B
•
Updated
•
3.47k
•
573
Bitsy Darel PRO
bitsydarel
AI & ML interests
None yet
Recent Activity
liked
a model
3 days ago
reducto/RolmOCR
liked
a model
3 days ago
starvector/starvector-8b-im2svg
liked
a model
14 days ago
openai/gpt-oss-120b
Organizations
None yet
liked
2
models
3 days ago
liked
4
models
14 days ago
liked
a
model
16 days ago
OuroRotaryEmbedding ROPE_INIT_FUNCTIONS does not contains a 'default' key
#2 opened 19 days ago
by
bitsydarel
liked
a
model
25 days ago
liked
a
dataset
25 days ago
liked
5
models
25 days ago
unsloth/MiniMax-M2.1-GGUF
Text Generation
•
229B
•
Updated
•
147k
•
146
MiniMaxAI/MiniMax-M2.1
Text Generation
•
229B
•
Updated
•
246k
•
•
1.11k
unsloth/GLM-4.7-GGUF
Text Generation
•
358B
•
Updated
•
143k
•
174
zai-org/GLM-4.7
Text Generation
•
358B
•
Updated
•
73.4k
•
•
1.73k
google/functiongemma-270m-it
Text Generation
•
0.3B
•
Updated
•
83.7k
•
841
liked
a
model
2 months ago
upvoted
a
paper
6 months ago
upvoted
2
changelogs
6 months ago
Changelog
New Model Filtering Options on the Hub
•
75
Changelog
Add MCP-Compatible Spaces to Your Tools
•
85
reacted to
louisbrulenaudet's
post with 👀
6 months ago
Post
2513
Understanding the json format response with HF's Serverless Inference API 🤗
As it stands, there seems to be an inconsistency with the OpenAI documentation on the question of implementing the JSON response format using the InferenceClient completion API.
After investigating the InferenceClient source code, I share the official solution using a JSON Schema. This consolidates the structure of the response and simplifies parsing as part of an automated process for extracting metadata, information:
As a reminder, json mode is activated with the OpenAI client as follows:
One question remains unanswered, however, and will perhaps be answered by the community: it seems that an incompatibility persists for list of dictionaries generation, and currently, the production of simple dictionaries seems to be the only functional option.
As it stands, there seems to be an inconsistency with the OpenAI documentation on the question of implementing the JSON response format using the InferenceClient completion API.
After investigating the InferenceClient source code, I share the official solution using a JSON Schema. This consolidates the structure of the response and simplifies parsing as part of an automated process for extracting metadata, information:
from huggingface_hub import InferenceClient
client = InferenceClient("meta-llama/Meta-Llama-3-70B-Instruct")
messages = [
{
"role": "user",
"content": "I saw a puppy a cat and a raccoon during my bike ride in the park. What did I saw and when?",
},
]
response_format = {
"type": "json",
"value": {
"properties": {
"location": {"type": "string"},
"activity": {"type": "string"},
"animals_seen": {"type": "integer", "minimum": 1, "maximum": 5},
"animals": {"type": "array", "items": {"type": "string"}},
},
"required": ["location", "activity", "animals_seen", "animals"],
},
}
response = client.chat_completion(
messages=messages,
response_format=response_format,
max_tokens=500,
)
print(response.choices[0].message.content)As a reminder, json mode is activated with the OpenAI client as follows:
response = client.chat.completions.create(
model="gpt-3.5-turbo-0125",
messages=[...],
response_format={"type": "json_object"}
)One question remains unanswered, however, and will perhaps be answered by the community: it seems that an incompatibility persists for list of dictionaries generation, and currently, the production of simple dictionaries seems to be the only functional option.
- 3 replies