blog/

Field Notes

Written notes from my attempts to satisfy my curious mind.

Series

Standalone

May 22, 2026Length generalization across positional encoding methods — ALiBi flat, RoPE degrading, absolute methods collapsing
My experiments with RoPE

An experiment that broke my faith in RoPE's length generalization.