Janani Sekar:
Hi, everyone. Welcome back. This is video five. This is all about different types of agents that can help us with research. I'm going to present a perspective on different categories of agents and feel free to think while you're listening to this video about which ones will be more or less helpful to you. Obviously, depending on the nature of your work, you're going to find some of this more interesting than the rest. Okay. So this is a perspective that we already saw, I want to say in module one, but this is from OpenAI and it was a breakdown of some of the different types of agents out there. There were data agents and action agents and orchestration agents or agents that managed other agents. I think this is a reasonable categorization. I'll present a different one based off of a lot of what I'm seeing in the social science literature, as well as doing in my own life and my day-to-day research, starting with deep research agents.

So deep research is this idea that there are agents that can do comprehensive literature reviews and then synthesize all of that information and give it to you in a way that ideally you could paste it into a paper. Are we there yet? I think absolutely not. Could we be there in a few months or years? I think definitely. I don't really know why. These are called deep research as opposed to just research, but I am not the person to ask about why this is the case. These are a few of the different deep research agents available on the market. Pretty much every language model provider's got one these days. This table specifically is from CoreNet 2025, and I think it does a nice job of helping you navigate, depending on your use case, which deep research agent you should use. Notably, there is pretty significant heterogeneity in how quickly they do this querying or searching of the literature.

There is also definitely heterogeneity in price and know that some of the paid deep research models are actually pretty expensive to run. So if you're just trying to get a feel for whether or not this type of agent is even something that you would find useful, I would highly recommend that you do start with one of the free ones. It's going to be a slightly more accessible place to start. Okay. So deep research agents can have pretty high value add, but with some limitations. Know that when you run a deep research query about some topic, you may or may not have information that is entirely credible. So oftentimes I've gotten information from Wikipedia or blog posts and not always from peer-reviewed journal articles. And that's always something that you have to look out for because the information can be presented as legitimate and it's on you to look at what those links are and confirm whether or not it is.

The writing might not be good, so this is why I said we're not quite at copy pastable literature review level yet. Oftentimes I find that the outline and the sources, even if they are of quality when they're synthesized, it's not necessarily the way that I would choose to do it. So be prepared to rewrite or reorganize the way that this information is presented. And going back to a pretty recurring problem from that prompt engineering discussion in module three, know that we do expose ourselves to false claims over generalization, even to the extent of falsely causal statements. So keep in mind all of this when you're using deep research agents, and as long as you know to look out for these problems, you can use them pretty effectively, at least for collecting information. So what are some best practices to mitigate these risks? Definitely always be suspicious.

Try to specify filters for quality. Say that you want information from certain journals or from certain authors. Ask for nuance to maintain some standard of paper or data that's reported to you. So things like, I want effect sizes and confidence intervals. I want papers that report these things in their tables. You can even take a paper that you like on a topic and ask the model to find more literature that's similar to that. That's a way to guarantee some quality or some on-topic-ness. And one thing that's worked really well for me in practice is to start very broad and then slowly narrow things down. So when I say start broad, for example, you might do something like give me an overview of the literature on carbon taxation, see what the model gives you. Take the one or two papers that you really liked from that deep literature query or deep research query, and then pass it back in and say, "Now I want more that is pretty similar to this. Please think about this problem in this direction."

Okay. The next category of agent that we will talk about are, of course, the ever present coding agent. If you are a Python person, you've probably worked or tested out one of these. They're also actually pretty good with R in my experience. I have not messed around with them a whole lot for Stato or MATLAB or any of the other statistical programming languages that we might use, but I would imagine that there is a coding agent out there that is reasonably good at any of these languages. And if you're a Python person, we've got Cursor and GitHub Code Pilot and Claude Code. And of course, this process of giving an agent natural language instructions, so just English instructions on what you want your code to do and then getting back out working code is called vibe coding. The appeal is that this definitely reduces the barriers to writing good code.

You no longer have to be a computer scientist or have extensive programming knowledge or no specific syntax to be able to write code. And it allows us to focus on logic rather than get into the weeds of, oh no, I meant to put a colon here and that's why this code didn't work and that's why I got this error. The downside is that if we just let a coding agent go off and write us 5,000 lines of code without checking intermittently, then when it does inevitably break, when things stop working, we will not know where to start looking to try and fix it. And this is a meme that I saw circulating on Twitter a few weeks ago, and I think it is most definitely true if you let your vibe coding agent just get away from you without supervising it or paying attention to what it's writing.

So here are some examples of where vibe coding can go wrong. I will not read them all out loud, but again, feel free to pause the video here to look at each of these scenarios or examples in a little bit more detail to try and see if you can anticipate why it might present a problem.

Here are some other failure modes that I've noticed. These are worth calling out. I've noticed that language models are typically trained on historical code. So even if they have access to the internet and have agentic capabilities like the latest versions of GPT and Claude, they will still sometimes try to load libraries or packages that are old and outdated, which will cause other parts of your code to break because I think they prefer to rely on their internal knowledge base over doing an internet search to the extent possible. So look out for that. Anticipate that as a type of error that might occur. I definitely see all sorts of inefficient and badly written things. This is what we refer to as AI slop. You might get code that works, but just barely because it runs really slowly. You might get code that's just written in a way that's very difficult to follow that doesn't really adhere to good code writing practices or it's not commented and so you don't know what's going on where.

And then there are user specific things. So getting a language model or agent to work with your compute cluster or with the coding guidelines that your lab uses might require some additional work on your end. Best practices for mitigating these failure modes are, of course, planning. So one big thing to do is to write pseudocode or logic, and that way your agent doesn't have to come up with both the logic and the code from scratch. If you give it the logic in English, it's going to be a lot easier to translate that into code. A similar thing goes for debugging. Ask your model to explain at each step when it generates code what it's doing there. This is also going to limit the burden on your part to have to understand the code line by line if you're getting with every 10 lines of code that get generated, a high level overview of what each section does.

And that way when things break, you can roll back to the most recent working version and then restart from there. Ask the agent to catch mistakes as they happen. So say, when you write code to clean this dataset, also write code to check that there are no duplicates, check that the orders of magnitude are right. So if your column has data that is all decimals and then all of a sudden you're seeing 10 to the four, that's maybe a sign that things have gone wrong. And if your agent has internet search capabilities, ask it to give you documentation about the libraries and packages and syntax that it's using, and also feel free to give it documentation. So this is a link to my compute cluster. This is a link to this library that I'm using. Can you make sure that you're using the most recent version of this and that it's not outdated?

Okay. So we've now talked about coding agents and deep research. The next category of agents, I'll very briefly touch on our creative agentic platforms. We are seeing platforms like Lovable and Gamma and Notebook LM. These are agentic platforms where you can talk to a chatbot in natural language and get the tool to generate for you things like slides, graphics, visuals, even videos. They're helpful for creating a rough outline or diagram. I think we're not yet at the point where you might be comfortable using one of these tools to generate slides for a talk, but we are definitely at the point where we can use these tools to generate the preliminary version of slides for a talk that we might have to go in and fill in ourselves. And the final category will, of course, be vibe-coded custom tools. So remember when we can do things like turn English into code, if there isn't an agent for something that we want to do, so for example, you want to work with Qualtrics or you want to work with Spotify.

There is no Spotify API agent that exists on the market that you can buy a subscription to, but with a Spotify API key and an understanding of how APIs work, you have all of the tools that you need to prompt a coding agent to build for you your own custom agent that can access the Spotify API. So I want to leave this here because I think this is going to be something very important and we will actually discuss this in more detail two videos from now.