Brief
Captions outperform direct video QA on long egocentric video, CapMem benchmark finds
A new human-annotated benchmark, CapMem, tests whether text captions can serve as reusable episodic memory for wearable assistants. The reported gains are the authors' own numbers from a single preprint, not independent replication.
CapMem targets the constraints that shape wearable assistants: vision-language models run on bounded frame budgets, visual tokens grow costly, and long-context retrieval fails. The authors propose reusing textual captions as the memory layer instead of re-reading frames for every question.
On the same long-video subset, a matched-frame control across six Qwen models retained mean accuracy gains of 3.22 and 2.55 points for 30-second and 60-second caption windows. That control is the useful part, because it separates the caption representation from the effect of simply feeding the model more frames. A caption-guided retrieve-and-verify harness adds up to 5.3 points on top.
Our reading
For anyone building memory over hour-scale recordings, this is an argument that captions can act as a compact index that is written once and queried many times, rather than paying visual-token cost per question. The result is confined to the authors' benchmark and their model set, so it is a direction to test rather than a settled property of caption memory. Teams shipping wearable or agentic rec…
What to do or watch
Reproduce the matched-frame comparison on a long slice of your own footage with the model you actually run before changing your pipeline; watch for a released harness, data and results on non-Qwen model families.
Source details and supporting facts
Each line is stated by the page named above it.
Stated by arXiv
- On long videos (>20 min), full-coverage CaptionQA with 30s and 60s caption windows outperforms direct VideoQA for 10/12 and 8/12 models, respectively.
- On the same video subset, a matched-frame control across six Qwen models retains mean accuracy gains of 3.22 and 2.55 points.
- A caption-guided retrieve-and-verify harness further improves accuracy by up to 5.3 points.
Sources
- arXivText stored 17 September 2026
How this story was checked. Written from the 1 page listed above, stored 17 September 2026; claims checked against that stored text on 17 September 2026.
What that means
- 3 of 4 reported statements were confirmed against the page that carries them; the rest were removed rather than published.
- Figures in the text were required to appear in the stored source text: yes. Identifiers: yes.
- The check reads stored text only: no claim rests on a fresh look that did not happen.
- Where the reporting was silent, the text says so instead of filling the gap.