Researchers at Stony Brook University are working to improve the way artificial intelligence (AI) systems handle complex, multi-step tasks. The project is led by Jiawei “Joe” Zhou, assistant professor in the Department of Applied Mathematics and Statistics and the Department of Computer Science, along with Niranjan Balasubramanian, associate professor in the Department of Computer Science.
The research focuses on helping AI systems reason through long-horizon tasks, which require agents to make decisions over extended periods and across many steps. The team has received $70,000 in funding and $50,000 in AWS promotional credits from an Amazon Research Award to support their work.
“Many real-world tasks require agents to reason over extended time horizons while interacting with complex environments,” Zhou said. “As those reasoning chains grow longer, they tend to become more diffuse and error-prone, which ultimately limits how useful these systems can be in practice.”
Large language models (LLMs) are used by AI agents to generate and reason using human language. For example, a system might be tasked with planning a trip that involves booking flights, reserving hotels, and emailing itineraries—requiring it to remember earlier choices and adapt its plan as needed.
Andrew C. Singer, dean of the College of Engineering and Applied Sciences at Stony Brook University, commented on the significance of this research: “This work exemplifies the kind of high-impact, interdisciplinary research that defines StonyBrook Engineering. By tackling the fundamental challenge of how AI systemsreason over time and complexity, this team is advancing capabilities that are essential not onlyfor next-generation artificial intelligence, but for real-world applications where efficiency,adaptability, and reliability truly matter.”
Current AI models perform well on short tasks but often face difficulties when required to make decisions across many steps. In practical settings, these agents must plan ahead while adapting their actions based on feedback.
Balasubramanian noted that increasing computing power alone will not solve these challenges: “Scaling up models alone isn’t enough,” he said. “What we really need are smarter reasoning strategies that allow agents to focus on what matters and discard unnecessary intermediate steps.”
The researchers are building upon AppWorld—a large-scale interactive environment developed by their team—which simulates digital tasks across nine commonly used applications such as email or payment services through hundreds of APIs (application programming interfaces). These APIs enable software programs to communicate with each other.
Tasks within AppWorld can involve up to 40 steps using tens of millions of tokens—small pieces of text processed by AI models—making them particularly challenging for current technology. Despite recent advances in language models, success rates remain below 50 percent for these demanding scenarios.
“AppWorld lets us study what actually goes wrong when agents try to operate in realistic, interactive environments,” Balasubramanian said. “It exposes the limits of long-range reasoning in a way that static benchmarks simply can’t.”
To address these issues, the team is developing reinforcement learning methods aimed at teaching AI agents how to compress their reasoning processes—retaining only relevant information instead of every step taken during problem-solving.
“Our goal is to teach agents how to think more efficiently, not only longer,” Zhou said. “By compressing reasoning chains and pruning redundant steps, we can reduce computational cost while actually improving decision quality.”
Their approach rewards agents for both completing tasks successfully and doing so efficiently—encouraging a balance between accuracy and resource use. The method also helps agents adjust when conditions change unexpectedly during a task.
“We’re especially interested in how agents recover from errors or unexpected feedback,” Balasubramanian said. “Efficient reasoning allows them to revise decisions without having to replay or store everything they’ve done before.”
In addition to improving performance on complex problems, this project addresses concerns about cost and scalability by aiming to reduce token usage and memory requirements for advanced AI systems.
“All of our tools, datasets, and evaluation frameworks will be released openly,” Zhou said. “We want this work to benefit the broader research community and accelerate progress in agentic AI.”
The researchers believe their emphasis on real-world interaction positions Stony Brook University as a leader in efforts aimed at making advanced AI more capable when facing complex situations.
