Tongyi-MAI/Z-Image
Text-to-Image
•
Updated
•
11.9k
•
•
848
Jamal Id Aissa
yibba
AI & ML interests
None yet
Recent Activity
liked
a model
2 days ago
Tongyi-MAI/Z-Image
liked
a Space
2 days ago
mrfakename/Z-Image-Turbo
liked
a dataset
2 days ago
Alibaba-Apsara/Superior-Reasoning-SFT-gpt-oss-120b
Organizations
liked
a
model
2 days ago
liked
a
Space
2 days ago
liked
a
dataset
2 days ago
liked
3
models
2 days ago
reacted to
IlyasMoutawwakil's
post with 🔥
9 days ago
Post
2939
Transformers v5 just landed! 🚀
It significantly unifies and reduces modeling code across architectures, while opening the door to a whole new class of performance optimizations.
My favorite new feature? 🤔
The new dynamic weight loader + converter. Here’s why 👇
Over the last few months, the core Transformers maintainers built an incredibly fast weight loader, capable of converting tensors on the fly while loading them in parallel threads. This means we’re no longer constrained by how parameters are laid out inside the safetensors weight files.
In practice, this unlocks two big things:
- Much more modular modeling code. You can now clearly see how architectures build on top of each other (DeepSeek v2 → v3, Qwen v2 → v3 → MoE, etc.). This makes shared bottlenecks obvious and lets us optimize the right building blocks once, for all model families.
- Performance optimizations beyond what torch.compile can do alone. torch.compile operates on the computation graph, but it can’t change parameter layouts. With the new loader, we can restructure weights at load time: fusing MoE expert projections, merging attention QKV projections, and enabling more compute-dense kernels that simply weren’t possible before.
Personally, I'm honored to have contributed in this direction, including the work on optimizing MoE implementations and making modeling code more torch-exportable, so these optimizations can be ported cleanly across runtimes.
Overall, Transformers v5 is a strong signal of where the community and industry are converging: Modularity and Performance, without sacrificing Flexibility.
Transformers v5 makes its signature from_pretrained an entrypoint where you can mix and match:
- Parallelism
- Quantization
- Custom kernels
- Flash/Paged attention
- Continuous batching
- ...
Kudos to everyone involved! I highly recommend the:
Release notes: https://github.com/huggingface/transformers/releases/tag/v5.0.0
Blog post: https://huggingface.co/blog/transformers-v5
It significantly unifies and reduces modeling code across architectures, while opening the door to a whole new class of performance optimizations.
My favorite new feature? 🤔
The new dynamic weight loader + converter. Here’s why 👇
Over the last few months, the core Transformers maintainers built an incredibly fast weight loader, capable of converting tensors on the fly while loading them in parallel threads. This means we’re no longer constrained by how parameters are laid out inside the safetensors weight files.
In practice, this unlocks two big things:
- Much more modular modeling code. You can now clearly see how architectures build on top of each other (DeepSeek v2 → v3, Qwen v2 → v3 → MoE, etc.). This makes shared bottlenecks obvious and lets us optimize the right building blocks once, for all model families.
- Performance optimizations beyond what torch.compile can do alone. torch.compile operates on the computation graph, but it can’t change parameter layouts. With the new loader, we can restructure weights at load time: fusing MoE expert projections, merging attention QKV projections, and enabling more compute-dense kernels that simply weren’t possible before.
Personally, I'm honored to have contributed in this direction, including the work on optimizing MoE implementations and making modeling code more torch-exportable, so these optimizations can be ported cleanly across runtimes.
Overall, Transformers v5 is a strong signal of where the community and industry are converging: Modularity and Performance, without sacrificing Flexibility.
Transformers v5 makes its signature from_pretrained an entrypoint where you can mix and match:
- Parallelism
- Quantization
- Custom kernels
- Flash/Paged attention
- Continuous batching
- ...
Kudos to everyone involved! I highly recommend the:
Release notes: https://github.com/huggingface/transformers/releases/tag/v5.0.0
Blog post: https://huggingface.co/blog/transformers-v5
- 3 replies
liked
a
Space
12 days ago
liked
a
model
16 days ago
updated
a
model
16 days ago
liked
a
model
16 days ago
liked
3
models
20 days ago
reacted to
davidmezzetti's
post with 👍
25 days ago
Post
2187
🥃 Distilling Tiny Embeddings. We're happy to build on the BERT Hash Series of models with this new set of fixed dimensional tiny embeddings models.
Ranging from 244K parameters to 970K and 50 dimensions to 128 dimensions these tiny models pack quite a punch.
Use cases include on-device semantic search, similarity comparisons, LLM chunking and Retrieval Augmented Generation (RAG). The advantage is that data never needs to leave the device while still having solid performance.
https://huggingface.co/blog/NeuML/bert-hash-embeddings
Ranging from 244K parameters to 970K and 50 dimensions to 128 dimensions these tiny models pack quite a punch.
Use cases include on-device semantic search, similarity comparisons, LLM chunking and Retrieval Augmented Generation (RAG). The advantage is that data never needs to leave the device while still having solid performance.
https://huggingface.co/blog/NeuML/bert-hash-embeddings
liked
a
model
27 days ago
updated
a
dataset
about 1 month ago
published
a
dataset
about 1 month ago
published
a
model
about 1 month ago
liked
a
dataset
about 1 month ago