“ContextAgent: Context-Aware Proactive LLM Agents with Open-world Sensory Perceptions” was recently accepted to NeurIPS 2025.
In this paper, we propose ContextAgent, the first context-aware proactive agent that incorporates extensive sensory contexts to enhance the proactive capabilities of LLM agents. ContextAgent first extracts multi-dimensional contexts from massive sensory perceptions on wearables to understand user intentions. It then leverages the sensory contexts and the persona contexts from historical data to predict the necessity for proactive services. When proactive assistance is needed, ContextAgent further automatically calls the necessary tools to assist users unobtrusively. Evaluations on two datasets show that it outperforms baselines by achieving up to 8.5% and 6.0% higher accuracy in proactive predictions and tool calling, respectively.
This work is in collaboration with the AIoT Lab at The Chinese University of Hong Kong.