Deepseek recently published a paper in which they describe that vision tokens contain more information than text tokens and that this can be used to compress context.
We present DeepSeek-OCR as an initial investigation into the feasibility of compressing long
contexts via optical 2D mapping.
Experiments
show that when the number of text tokens is within 10 times that of vision tokens (i.e., a
compression ratio < 10×), the model can achieve decoding (OCR) precision of 97%. Even at a
compression ratio of 20×, the OCR accuracy still remains at about 60%. This shows considerable
promise for research areas such as historical long-context compression and memory forgetting
mechanisms in LLMs.
It reminds me of LLM caveman speak, it used to have another option to use Chinese instead of English. A language like Chinese is seemingly better at encoding information in fewer tokens and I think this is the same mechanism why OCR tokens work so well.
That said, I also doubt that voice messages are more efficient than text prompts, but it’s best not to waste too much time engaging with these sorts of LinkedIn posts (and LinkedIn in general).
Deepseek recently published a paper in which they describe that vision tokens contain more information than text tokens and that this can be used to compress context.
It reminds me of LLM caveman speak, it used to have another option to use Chinese instead of English. A language like Chinese is seemingly better at encoding information in fewer tokens and I think this is the same mechanism why OCR tokens work so well.
That said, I also doubt that voice messages are more efficient than text prompts, but it’s best not to waste too much time engaging with these sorts of LinkedIn posts (and LinkedIn in general).