Causal Estimation of Tokenisation Bias
3 experts are actively discussing the implications.
“One other interesting line of work is the bias you bake into your model by including a specific token (rather than representing it a compositionally) on the probability of strings that include that surface form.”