Why it matters
Tejas Bhakta’s publisher notes describe a kernel-optimization loop that proposes changes, checks correctness, benchmarks them and keeps or reverts each candidate. Humans supply the higher-level optimization idea and target-hardware context. The central failure mode is optimizing an isolated metric: disabling CUDA graphs or testing only short contexts can make the kernel score improve while inference worsens. The advertised speedup combines software and hardware changes without a complete reproducible benchmark specification.
My takeaway: Constrain what the optimizer may change and test both numerical correctness and end-to-end inference on representative context lengths and hardware. Preserve the original benchmark configuration, reject metric-gaming changes, and retain a fallback when an optimization helps only part of the workload.