What happens when AI starts improving itself?

Yi Fang is a professor in the department of computer science and engineering and director of the Responsible AI program at Santa Clara University. He has worked in the field of AI for more than 20 years, and his research spans information retrieval, natural language processing, and machine learning. Most recently, his work focuses on using AI to improve search engines and information access. Through his research, he aims to ensure that advancements in AI are ethical, equitable, and beneficial to society.

What questions or challenges are at the heart of your current work?

Traditionally, humans design AI algorithms and systems, but we’re increasingly seeing that AI itself is being used to optimize models and push cutting-edge research in the field. When AI starts improving itself, the concern is that the human-in-the-loop can slowly become a human-out-of-the-loop situation. The challenge we face is making sure AI systems are still transparent, safe, accountable, and beneficial to humanity. How can we ensure that these systems are still aligned with the goals and values of human beings? 

My current research explores how AI can help discover better search and ranking algorithms. The idea is inspired by the process of evolution. In nature, organisms evolve through variation, mutation, selection, and repeated adaptation. In our research, algorithms go through a similar process, which we call RankEvolve. 

We first define a search task and choose benchmark datasets where we already know which results are relevant to each query. Then we use AI, especially large language models, to generate new ranking strategies or algorithmic ideas. These ideas are like “mutations” which are new variations that may or may not improve the system.

Next, we turn these ideas into code and test them using standard search metrics, which measure how well the system places useful results near the top. If an AI-generated algorithm performs better than existing methods, it is selected and can be further refined. We can then ask the AI to build on that stronger version, creating another round of variation, testing, and selection.

The key point is that we do not simply trust AI-generated ideas. We treat them as research hypotheses: generate, implement, test, compare, and refine. Much of this process can be automated, allowing the system to explore many possible solutions more quickly than humans could do manually. Sometimes the AI proposes weak ideas, just as many mutations in nature are not useful. But sometimes, it discovers surprisingly effective algorithms that outperform human-designed methods. 

There’s huge potential for AI to advance scientific discoveries, improve healthcare, and transform our education system. With greater power comes greater responsibility, and I am excited to help shape AI in ways that expand opportunity and empower people. 

Why is this issue important for the world to address at this time?

This recursive self-improvement of AI is happening right now. For years, AI has been used as a tool for humans to analyze large amounts of data and assist with decision-making. But more recently, AI is moving from tools we use to systems that can improve themselves. Progress is accelerating, and it could be too late if we don’t intervene now. 

In my research, I’m trying to find ways to advance the capabilities of AI while also ensuring that it is responsible. We need to think about safety, fairness, and transparency before these systems scale. The decisions we make now regarding governance and responsible design will shape how AI impacts society for decades.

Why have you chosen to dedicate your career to this research?

My interest in AI grew from my Ph.D. research on search engines 20 years ago. A key moment for me was realizing that search engines are essentially a form of AI. When someone types a query, the system has to understand what the person is looking for, search through enormous amounts of information, and decide which results are most useful. That is a deeply intelligent task. 

What drew me to this field was the combination of technical challenge and real human impact. A search engine is not just a piece of software. It affects what people read, learn, and trust. That made the research meaningful to me. Over time, as AI became more powerful, I became interested in two connected questions: Can AI help us do research better, and how can we make sure these systems are developed responsibly? That led me to my current work on self-improving AI, where AI helps generate new ideas and discover better algorithms, while humans remain responsible for testing, understanding, and guiding the process. This is also why Responsible AI is central to my work. The goal is not just to build more powerful AI, but to build AI that is fair, reliable, transparent, and beneficial to people and society. 

How have your students impacted your research?

My students have a tremendous impact on my research. One of the most rewarding things about being a professor is hearing students share fresh ideas and innovative approaches to complex problems. In the Responsible AI minor that I co-created with philosophy professor Susan Kennedy, students come from engineering, humanities, and business backgrounds. That diversity is incredibly valuable. Working with students from different disciplines broadens how I think about AI’s impact, and their creativity often leads to unconventional ideas that help advance the field. I deeply appreciate the fresh perspectives brought by my students.

What’s a book in your field that you think everyone should read?

I really like Sapiens: A Brief History of Humankind by Yuval Noah Harari. Although it’s not a book about AI, it offers a broad perspective on how technology has shaped humanity through the evolution of tools, technologies, and shared ideas. It shows how technological innovations such as agriculture, printing, and the internet have reshaped humanity in profound ways. That perspective is essential when we talk about responsible AI. I’m optimistic about the impact of AI and I believe that if we guide it in the right way, it will enhance human flourishing and expand human potential.