Webinars
Building human judgment
51 views
AI will accelerate product development, but human judgment will determine which products succeed
Why the future of product development lies in combining AI capabilities with strong human judgment.
How AI is changing team workflows from faster prototyping to deeper experimentation and testing.
The critical sequence teams must follow: Build understanding first, then use AI to challenge and deepen insights.
View transcript
We believe there's one key thing that we continuously need to work with and to strengthen. And this is relationship between human judgment and AI. At Framna, we've really, really been trying to focus a lot on like, HI plus AI, human intelligence plus AI, and how does this work together. And we think that this is the core and one of the key things in the future of good product teams. So let's start here with addressing a bit like the elephant in the room. AI writes code and engineers prompt it. What is the future of engineering? Well, for us, I mean, this question, it doesn't sit well with us. Engineering has always had a bigger purpose than just writing code. And there are still many tasks that AI can't do, or at least that we shouldn't actually rely on AI to do. And one of them is system design and architecture. Knowing, like, the system, like how to build it, why it was built in a way, why certain trade-offs exist, and what's the best way forward. It remains as a task for engineers to focus on and to sit with. Code reviewing. I think there are very few people who's listening to this webinar that actually will be comfortable with an AI having full control over the code base. So code reviewing is something that we still need to focus on. It is both to make sure there's no issue with the generated code, but also to keep control and keep that sort of like that human judgment of like, are we building the right thing that sits within doing good code reviewing? Then there is last 20%. I mean, this could be edge cases. They could be something like the scale of product. But it can also the things that actually make your products stand out from the competition. If you're only using LLMs to prompt the code, etc. You're not going to get there to the last 20% to make you stand out from the crowd, to make you something special. And maybe the last and most important is actually what not to build. AI is made to respond to you and like, you give AI the command of what it builds and it will find a way to build it. But engineers often sit with the question, what are the things that we shouldn't build? What are the sort of like features and what is the technical aspect? How should we not build this? Like thinking and questioning in like what sort of solution they can use and what sort of like tech that you can use to solve this problem. That is where engineers have a key role to play. So, we see that like, yes, we're going to go to a future where more prompting is done by engineers and more code is produced by AI. But engineering, good engineering work still matters a lot. And what is happening actually when we change this context is pretty interesting. We have a few teams here at Framna where we see this shift is already going on. And some of the things that we see is actually that we need to shift a bit how we work. One of the things that is interesting when it comes to this is that actually now in meetings, when we're so much faster to prototype, then we're also so much better at aligning on what we're actually like the ideas we're coming up with. Before we maybe used to sit with like having idea session, doing low fire, like mock ups. Then a designer had to go back, actually wireframe them, get back to the team, talk about like, what was this, what we thought of? Like, how do we think of this? Now we can come to that alignment within the single meeting because we can prototype in a much faster way. We can also actually move from having more discussion into like looking for more evidence. One of the things that we see is that we're right now trying to prototype more in the source code to actually see how does the solutions and ideas that we have fit in with the entirety of our product. But we can also, of course, ideate and test more and more solutions, A, B, C, D testing and like going further in like what we can gather data on now. What is the right way of solving a problem? And the final thing and the final shift that we see and that we think is kind of important is actually that we now have a little bit more time for everyone in the team to participate in research, to actually be out there to talk to the users, to understand them better. Because if we do this, we believe that we are building the entire like judgment of the team together. When things are going faster, we need like to be better aligned on the problems that we're solving for our users and to actually have the entire team with us more within the research context really helps us to speed up things actually when we're building as well. But we did also talk a little bit about product discovery and AI. Like even if we wouldn't say that we would outsource product discovery to AI, we still believe that AI can transform how we do product discovery. We think that actually looking at the research and like our research plans that might have blind spots that we can look through with AI. It can be that we test what methodology we like we're using. Is this the right methodology? AI has a tremendous knowledge of like how you do research in a good way. That's something that is really interesting. It could simulate users. You can use your personas to simulate how do we do user research. So we can go through basically like if we have some like a questionnaire that we're going to bring out to interviews, we stress test those with users before we actually go out and do the interviews. Because then we can know better that we're aligned on the question that we get better results in like when we're actually doing like asking the questions. Another issue that we sometimes struggle with with like doing good discovery work is like, are we researching things that we already know? Well, I mean, now AI can get access to more and more things like backlog, like customer tickets. We can have like, of course, surveys and previous transcript that it can look through. So we can actually start like matching and see, are we asking for new things here or like this is already existing or data, which is already then making the discovery that we're going to do even more valuable because we make sure that we look for new things. Another place where we can see it work is to work together with a team to formulate and to gather the insight when you've been out and doing the product discovery. This is a place where I can have a big role as well. And the final thing where we see that AI actually can transform how we do product discovery is sharing knowledge. I mean, before, like it used to be a meeting where we presented like insight to different stakeholders, etc. But now we can see that we can create much more engaging content that we can send out to much broader audiences within the organization. So the organization can easier understand why the product team decides to prioritize in the way they do. And this is something that is really changing. But we would caution a little bit for this again. Because everything we said now comes under the condition that it is starting from a human perspective. The human is doing the groundwork and then you leverage the AI. For example, if you do a good discovery round, you talk to your users and learn from them. Then you put in like all the transcripts into the AI and the AI gives you the analysis. I mean, the insights will probably look reasonable. It's very, very good at framing things in a way that looks very reasonable. But if you do this enough times, pretty sure that you will start losing your own understanding of why you made the decision that you made. I think we need to flip the script here and say that like research needs to do that. We need to do the research and we need to sit with the raw material first. That is hard work. But it's actually the work that builds our own view of the problems first. And then we can help leverage AI and let AI challenge this. Because what AI now does is deepens the knowledge instead of replacing it. It's the same tools. It's the same data. But the difference is the order, the sequence that we do this. One sequence helps us build human judgment. The other quietly erodes it. And we believe that this is one thing that every team should sit and think of. In what order do we leverage AI in our work? Because in the end, we do not believe that AI will be the moat. Most teams will use and access the same models. So, and we do not believe that like prompting is going to be some sort of like competitive edge. They're going to be best practices. People are going to learn and you're going to be able to copy what you do pretty easily. The moat will be the team and the human judgment that they've built together. Iteration by iteration. Learning by learning. To gather those insights, test the ideas and together answer the question of like, should we build this? Yes. Yes, yes. But what did you talk to teach that?