A Survey on Data Selection for LLM Instruction Tuning
Paper
•
2402.05123
•
Published
•
3
LLM Data
-
Data-Juicer Sandbox: A Comprehensive Suite for Multimodal Data-Model Co-development
Paper • 2407.11784 • Published • 4
Data Management For Large Language Models: A Survey
Paper • 2312.01700 • PublishedNote 综述
Datasets for Large Language Models: A Comprehensive Survey
Paper • 2402.18041 • Published • 2Note 其他相关文章： The Art of Data Selection: A Survey on Data Selection for Fine-Tuning Large Language Models https://openreview.net/pdf?id=hTBD3LYoqd A Survey on Data Quality Dimensions and Tools for Machine Learning https://arxiv.org/abs/2406.19614
Beyond Scale: the Diversity Coefficient as a Data Quality Metric Demonstrates LLMs are Pre-trained on Formally Diverse Data
Paper • 2306.13840 • Published • 11Note Stanford. 数据多样性
-
From Decoding to Meta-Generation: Inference-time Algorithms for Large Language Models
Paper • 2406.16838 • Published • 2 -
Understanding and Mitigating the Label Noise in Pre-training on Downstream Tasks
Paper • 2309.17002 • Published • 1
A Survey on Data Selection for Language Models
Paper • 2402.16827 • Published • 4Note **
-
BaichuanSEED: Sharing the Potential of ExtensivE Data Collection and Deduplication by Introducing a Competitive Large Language Model Baseline
Paper • 2408.15079 • Published • 54 -
LLM360/TxT360
Updated • 15.2k • 244
HuggingFaceFW/fineweb-edu
Viewer • Updated • 3.5B • 240k • 832Note https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu-score-2 https://huggingface.co/HuggingFaceFW/fineweb-edu-classifier
-
D4: Improving LLM Pretraining via Document De-Duplication and Diversification
Paper • 2308.12284 • Published