The post discusses the latest open LLM releases including Mixtral 8x22B, Llama 3, Phi-3, and OpenELM. It also compares the performance of Mixtral 8x22B to other LLMs and explores the training data size for Llama 3. Additionally, it provides a comprehensive study on whether DPO is superior to PPO for LLM alignment.
Table of contents
1.1 Mixtral 8x22B: Larger models are better!1.2 Llama 3: Larger data is better!1.3 Phi-3: Higher-quality data is better!1.4 Conclusion30 Impressions