Update README.md

README.md

@@ -4,24 +4,21 @@ pipeline_tag: image-text-to-text
 license: other
 ---
 
-# Moondream 3 HF
+# Moondream 3 Preview HF
 
-**Moondream 3 HF** is a reimplementation of the [Moondream 3 (Preview)](https://huggingface.co/moondream/moondream3-preview) model using the standard Hugging Face Transformers architecture conventions.  
+**Moondream 3 Preview HF** is a reimplementation of the [Moondream 3 (Preview)](https://huggingface.co/moondream/moondream3-preview) model using the standard HuggingFace Transformers architecture conventions.  
 
 ## Overview
 
 - Multimodal vision-language model with a mixture-of-experts (MoE) text backbone  
 - Architecture and weights correspond to Moondream 3 (Preview) (approximately 9B parameters, 2B active)  
 - Implemented as standard Transformers components:
-  - `Moondream3ForConditionalGeneration`
-  - `Moondream3Model`, `Moondream3TextModel`, `Moondream3VisionModel`
-  - `Moondream3Processor`, `Moondream3ImageProcessor`, `Moondream3Config`
 
 The purpose of this repository is to make Moondream 3 interoperable with the Hugging Face ecosystem so it can be used directly with the Transformers API, including `generate()`, `Trainer`, and PEFT integrations.
 
 ## Example usage
 
-Example for running multimodal inference with the `moondream3-hf` implementation:
+Example for running multimodal inference with the `moondream3-preview-hf` implementation:
 
 ```python
 import torch
@@ -30,8 +27,8 @@ from transformers import AutoProcessor, AutoModelForCausalLM
 
 DEVICE="cuda:0"
 
-model = AutoModelForCausalLM.from_pretrained("NyxKrage/moondream3-hf", dtype="bfloat16", device_map=DEVICE, trust_remote_code=True)
-processor = AutoProcessor.from_pretrained("NyxKrage/moondream3-hf", use_fast=False, trust_remote_code=True)
+model = AutoModelForCausalLM.from_pretrained("NyxKrage/moondream3-preview-hf", dtype="bfloat16", device_map=DEVICE, trust_remote_code=True)
+processor = AutoProcessor.from_pretrained("NyxKrage/moondream3-preview-hf", use_fast=False, trust_remote_code=True)
 
 
 image1 = Image.open("image1.jpg")
@@ -60,33 +57,45 @@ with torch.no_grad():
 
 The `chat_template` uses Hugging Face’s Jinja format and accepts either a single string or a sequence of messages (`user [, assistant]`).
 
+## Prompting modes
+
+The chat template supports multiple task types via text prefixes:
+
+| Mode    | Template prefix                | Example input                              |
+| ------- | ------------------------------ | ------------------------------------------ |
+| Query   | `query:`                       | `query: What is happening in this image?`  |
+| Reason  | `reason:`                      | `reason: What is happening in this image?` |
+| Caption | `caption: [short/normal/long]` | `caption: long`                            |
+| Detect  | `detect:`                      | `detect: dog`                              |
+| Point   | `point:`                       | `point: red car`                           |
+
+If no prefix is provided, the default mode is `caption:normal`. `reason` is the same as query, but makes the model think/reason before providing a final answer.
+
+## Output Format
+
+For query: and caption: prompts, the model behaves like a standard Hugging Face causal language model and returns token IDs to be decoded normally.
+
+For detect: prompts, the model does not return text. Instead, it produces a floating-point tensor of shape `[batch_size, max_detections, 4]`. Each non-zero entry represents a bounding box in normalized coordinates `[x_min, y_min, x_max, y_max]`. Zero rows indicate padding.
+
+For point: prompts, the model likewise returns structured coordinates rather than text. The output is a floating-point tensor of shape `[batch_size, max_points, 2],` where each non-zero row is a point `[x, y]` in normalized image coordinates. Zero rows again represent padding.
+
 ### Training
 
 The model can be trained using `trl` and supports `peft` and `bitsandbytes` out of the box.  
 Included in the repo is also an implementation which replaces the MoE layers with a grouped_gemm implementation which has been adapted from [github:woct0rdho/transformers-qwen3-moe-fused](https://github.com/woct0rdho/transformers-qwen3-moe-fused), and can be used by importing Moondream3ForConditionalGeneration from `modeling_moondream3_fusedmoe.py` instead.
 
+## Limitations
 
-## Prompting modes
+All images within a batch must yield the same number of vision crops. To ensure this, it is recommended to resize every image to the same resolution before preprocessing; otherwise, differences in aspect ratio or size can cause mismatched crop counts and prevent batching.
 
-The chat template supports multiple task types via text prefixes:
+Only a single action type is supported per batch. Every prompt in the batch must use the same mode prefix (such as query:, caption:, point:, or detect:). Mixing different modes in a single batch is not allowed.
 
-| Mode    | Template prefix               | Example input                             |
-| ------- | ----------------------------- | ----------------------------------------- |
-| Query   | `query:`                      | `query: What is happening in this image?` |
-| Caption | `caption:[short/normal/long]` | `caption: long`                           |
-| Detect  | `detect:`                     | `detect: dog`                             |
-| Point   | `point:`                      | `point: red car`                          |
-
-If no prefix is provided, the default mode is `query:`.
-
-This reimplementation aims to provide interoperability and ease of experimentation within the Hugging Face ecosystem. It is not an official release.
+Because both the visual crops and text inputs must align across examples, batches must be homogeneous in both image structure and task type. Inputs that differ in crop count or prompt mode should be run separately or grouped into compatible batches.
 
 ## License
 
 The model weights remain under the **Business Source License 1.1 with an Additional Use Grant (No Third-Party Service)**, identical to the original [Moondream 3 Preview license](https://huggingface.co/moondream/moondream3-preview/blob/main/LICENSE.md).
 
-This allows research, personal, and most commercial use, but prohibits offering hosted or resold access that competes with M87 Labs’ paid services.
-
 All new implementation code in this repository is released under the **Apache 2.0 License**.
 
 ## Credits