<div class="dln" itemprop="articleBody"><div class="shdw top"> </div><p>Direct Preference Optimization
</p><div class="shdw btm"> </div><button type="button" class="mxmz icn_ btn_mxmz"></button></div>

<div class="dln" itemprop="articleBody"><div class="shdw top"> </div><p class="no_idt"><a class="URI" href="https://huggingface.co/docs/trl/index" target="_blank" rel="noopener nofollow ugc">https://huggingface.co/docs/trl/index</a>
<br>Transformer Reinforcement Learning
<br><a class="oln" data-kno="K#/E7F3" href="/E7F3" rel="nofollow">DPO</a>を行うためのライブラリ。
</p><div class="shdw btm"> </div><button type="button" class="mxmz icn_ btn_mxmz"></button></div>

<div class="dln" itemprop="articleBody"><div class="shdw top"> </div><p><a class="URI" href="https://huggingface.co/blog/dpo-trl" target="_blank" rel="noopener nofollow ugc">https://huggingface.co/blog/dpo-trl</a>
</p><div class="shdw btm"> </div><button type="button" class="mxmz icn_ btn_mxmz"></button></div>

『Fine-tune Llama 2 with DPO』

<div class="dln" itemprop="articleBody"><div class="shdw top"> </div><p><a class="URI" href="https://arxiv.org/abs/2305.18290" target="_blank" rel="noopener nofollow ugc">https://arxiv.org/abs/2305.18290</a>
</p><div class="shdw btm"> </div><button type="button" class="mxmz icn_ btn_mxmz"></button></div>

t_wの輪郭

DPO

TRL

『Fine-tune Llama 2 with DPO』

『Direct Preference Optimization: Your Language Model is Secretly a Reward Model』