Writing
Notes from measuring my own work.
Short pieces about method choices that turned out to matter. Most of these exist because something I built produced a number I could not defend, and working out why was more useful than the original result.
-
What a warm start actually guarantees, and why it broke my benchmark
Warm-starting a manual baseline into CP-SAT as a feasible incumbent means the solver can never return a worse plan. That is good engineering and a bad experiment, because it decides the direction of your result before the solver runs. A note on the difference between a guarantee and a measurement.
-
When every retrieval method scores 1.000, you have measured nothing
I hand-labeled 17 questions to compare BM25, dense, and hybrid retrieval. All three reached Hit@5 = 1.000. The tie is not a finding that the methods are equivalent; it is evidence the test cannot separate them. On building evaluation sets that can actually lose.