Trustworthy Generative AI Copilots for Better Data-Driven Decision-Making in Companies

Generative AI copilots are already being used in many companies. Can they assist employees with data analysis and decision-making? And if not, how can this be changed? These are some of the questions addressed by a team at the University of Passau in a new bidt project led by Professor Ulrich Gnewuch. 

The use of data as a basis for business decisions is common practice in many companies. Traditionally, decision-makers without a technical background had to rely on their tech-savvy colleagues to provide them with reports, dashboards, or other applications.

With the advent of generative AI, such as large language models, this is changing. Generative AI copilots enable users, regardless of their technical skills or prior knowledge, to gain insights from their data using natural language. Many companies are therefore already employing such copilots to support employees in making data-driven decisions. 

"Despite the great potential, there are also numerous concerns - not only regarding the limitations of the technology but also how users engage with the copilot’s responses," says Professor Ulrich Gnewuch, who holds the Chair of Explainable AI-Based Business Information Systems at the University of Passau. "Many users struggle to find the right level of trust in these responses. This often leads to ineffective or inefficient decision-making because they either blindly trust the potentially inaccurate responses or spend a lot of time verifying the results first."

The information systems expert leads a research team addressing these challenges in a new project titled "Trustworthy Generative AI Copilots for Data Analytics in Business Decision-Making". Funded by the Bavarian Research Institute for Digital Transformation (bidt) of the Bavarian Academy of Sciences and Humanities, the researchers are investigating how interaction with copilots affects business decision-making processes and what role users' mental models play in this context. Building on these insights, the team develops design principles for trustworthy copilots and analyzes their impact in empirical studies. Finally, recommendations for the use of copilots in companies will be derived.

The project started in January 2025 and has a duration of four years. It is one of ten projects in the new research focus at bidt on the topic of "Humans and Generative AI: Trust in Human-AI Co-Creation". The spokesperson for the research focus is Professor Hannah Schmid-Petri, who holds the Chair of Science Communication at the University of Passau.

About the project leader

Ulrich Gnewuch is a professor of information systems and holds the Chair of Explainable AI-Based Business Information Systems at the University of Passau, a chair established by the Free State of Bavaria as part of the High-Tech Agenda. His research at the intersection of information systems and human-computer interaction focuses on the design, use, and impact of artificial intelligence in business and society.