https://jiangyigithub.github.io/ai.github.io/tags/phi/Recent content in Phi on Jiang Yi(姜祎)'s HomepageHugo -- gohugo.ioenWed, 08 Apr 2026 21:44:44 +0800https://jiangyigithub.github.io/ai.github.io/p/notes-on-phi-4/Mon, 16 Dec 2024 17:33:52 +0800https://jiangyigithub.github.io/ai.github.io/p/notes-on-phi-4/<h1 id="introduction"><a href="#introduction" class="header-anchor"></a>Introduction </h1><p>微软在12月12号发布了Phi-4的技术报告,主要关注了数据的构建，特别是。Phi-4是一个14B的大语言模型，主要在STEM相关的QA任务以及推理相关的任务上表现比较好。Phi-4主要在三个方面进行了改进：</p> <ol> <li>在pre-training和mid-training阶段使用了合成数据进行训练</li> <li></li> </ol> <h1 id="data"><a href="#data" class="header-anchor"></a>Data </h1><h1 id="pre-training"><a href="#pre-training" class="header-anchor"></a>Pre-training </h1><h2 id="architecture"><a href="#architecture" class="header-anchor"></a>Architecture </h2><h1 id="post-training"><a href="#post-training" class="header-anchor"></a>Post-training </h1><h2 id="dpo"><a href="#dpo" class="header-anchor"></a>DPO </h2><h2 id="sft"><a href="#sft" class="header-anchor"></a>SFT </h2><h1 id="references"><a href="#references" class="header-anchor"></a>References </h1><ul> <li><a class="link" href="https://techcommunity.microsoft.com/blog/aiplatformblog/introducing-phi-4-microsoft%E2%80%99s-newest-small-language-model-specializing-in-comple/4357090" target="_blank" rel="noopener" >Blog</a></li> <li><a class="link" href="https://arxiv.org/abs/2412.08905" target="_blank" rel="noopener" >Arxiv</a></li> </ul>