Co-Founder and CEO, Baydin(Boomerang)
From Engineering to Product: What PMs Need to Be Better Partners
Alex Moore: I'm the founder and CEO of Boomerang. Um, I come at this off again. No, good. I come at this from an engineering background. I studied computer science at MIT. Um, and so what I want to share some lessons we learned as we grow our engineering and product organization from basically the three cofounders of, um, doing everything up to 25 people on the product and engineering teams. Um, so I guess the, uh, the first step in correcting a problem is realizing you have it. So, um, I went the wrong way. So the first, uh, the first indication we had as we started to scale that, that we needed to put processes in place to make it easier for product and engineering to communicate were what I call the pillboxes from hell. Um, these are, uh, composed window pillboxes that happen in our Ios app. So as you type an email address, it turns into a little pill box that has the, that has the recipient's name in it instead of a big text blob that has their name and their email address associated with it.
Alex Moore: Um, when we started building this, we'd kind of, uh, we kind of decided we'd take kind of a hands off role on it, let a couple of folks who are junior on the team kind of figure out their process for, uh, for building and designing these. And so what we ended up having happened was we had our designer or product manager, um, you know, design the Spec for it and then handed it off to an engineer who didn't have a lot. You know, it didn't have a lot of experience in the technology either. And, uh, it turned out that a six months later we finally fixed the last bug that came out of that process. Um, and they don't look that hard and they're really not that hard. Um, but what we had happened was, um, we didn't have very good communication between the engineer working on it and the product manager and the designer who does inspect it out.
Alex Moore: So we had platform behavior that didn't work on Ios. Nobody had done anything like this before. We had pillboxes that didn't work like anyone else's pillboxes either. They didn't work like the ones in Gmail, they didn't look like the work, like the ones that outlook, um, they weren't built to Spec the first time. So then the engineer went back and the product managers said, hey, this doesn't look right. Let's Redo it. Um, we didn't think about edge cases. Like what happens when you have 50 email addresses in the to field? How do you do, how do you deal with the overflow? And then if you load a draft, uh, the pillbox is dependent on the compose, on the iOS context API being present, and that's loaded asynchronously. So it ended up being probably about eight weeks of work before we had anything really ready to ship and see, like I said, six months until we had the last bug worked out of it for something really, really simple.
Alex Moore: Um, by contrast, um, you know, a year later we, uh, we put out a tool we call Respondable, which has real time AI to tell you whether or not you're likely to get a response to an email as you type it to Ios. Um, we're using core ml. We were one of the largest core ml deployed model, uh, one of the largest core ml custom models that apple has an app store right now. We've got it all hooked up to web views. Um, so that we could reuse some of the technology for explaining it and it actually performs better than our, our web version of this. So we built something much, much more complicated in the same window with the same layouts and everything else in six weeks. The day we started working on it until we shifted to users. So that when you know exactly on time, no surprises, no bugs, nothing basically smooth as it could go.
Alex Moore: So what changed for us? Well, so one of the things that we believe is the real magic happens when you have, can have engineering and product, have a conversation where both folks know enough about what's going on to be able to be a creative to each other. So by talking to the product manager, the engineer has new ideas about the technology that they wouldn't have had. And by talking to the engineer, the product manager has new ideas about the product that they wouldn't have already had already had. And if you can kind of get that feedback cycle going, that's when you really hit things like respond to where it's like, hey, okay, could we do this? Oh, you know, there's this thing called core ml that might let us use better models. Oh that would be really cool. And then you kind of build, amplify each other instead of kind of holding each other back.
Alex Moore: So we started to become a lot more mindful about making sure that we put folks in a position to have those conversations and to those conversations would be meaningful. So I've got my one MBA slide here. How do we do that? Well, we've got the three e's and they kind of build on each other. So what are the expectations, um, for everybody in the conversation, we take some time to make sure that the product manager and the engineer are both educated enough about the other one's responsibilities and challenges that they can have a deep conversation. And then we've set up opportunities on a kind of regular cadence for folks to engage for the product managers and the engineers to engage with each other. So I'll share a couple of things that, that we've done that have worked pretty well as we've kind of pivoted our team from, from small to a little bit larger.
Alex Moore: I will make the caveat, I have not worked in this problem in a large organization, so if you're from salesforce or oracle, this may or may not be valuable at all to you, but um, but maybe there'll be something he can take away from it. So the first thing we did is we wanted to set up clear responsibilities. Who's responsible for making every decision as we go along the way. And for us, you know, how it works under the hood. That's an engineering decision. The product managers should not be deciding whether or not we're using core data or you know, a different iOS, uh, data database layer. Um, how it behaves from a user perspective. Product has the final call on that. Um, so if the product manager says, Hey, this is really important and it's gonna take three extra months to get it right, that's their decision to make.
Alex Moore: Um, the final call and estimates though goes to engineering, right? If you, if you can't, the product manager can't say, oh, well this is really only gonna take four weeks. If it's gonna take six weeks, it's gonna take six weeks. And the final call and prioritization goes to the product manager that helps everyone understand what their goals are, both from like a, you know, core responsibility standpoint and from a consulting standpoint, right? If you're an engineer and you're not responsible for figuring out how it behaves from the user perspective, you understand that your role is to consult and help the product manager make those decisions better, but to not make them yourself. Second thing, um, one of the things that we've kind of instituted is one of the company values as a whole is everyone should understand what's the point of what I'm doing right?
Alex Moore: Is the point of this feature to boost retention for the users is the point of what this engine, what this bug and the engineering organization is, is to make a slow part of the app perform faster. You know, is the A, is the goal of this to try something new that might be able to, you know, excite users in a way that they've never had something like this before. So we ask everyone to have an understanding of why am I working on this? Why is this important to the organization as a whole? And if they don't have that, then you know, ultimately it falls back to, to me being able to communicate that to the team. Everyone should understand why are you doing this? And then when you have that understanding, you can say, okay, cool. The goal for this is to boost retention. So it may not matter about, you know, if we do it x or if we do it y as long as we get this core piece built and everyone understands what those, what those core things are.
Alex Moore: So let's talk a little bit about, uh, how we, how we get folks brought up to speed. Um, and this is important for us because we have folks coming in from different backgrounds. So even if you're a senior engineer at Google, the scale of stuff at Google is so different than the scale of stuff with us, right? So everyone needs to know enough about the priorities and the terminology to communicate effectively specific to our business. So let me give you an example. One of the things we do with every engineer is we teach them the rice framework. So reach, impact, confidence and confidence and effort. This is a common framework for figuring out what the, uh, per for prioritizing decisions for prioritizing what you're going to work on. Um, you know, as you build. So the reach is how many users are going to be, how many users is this actually going to, are going to interact with this impact is how big of an impact is it going to have on them?
Alex Moore: Confidence is a, you know, how confident are you that it's going to have the effect you expect? And then the effort is how hard is it going to be to build. So we teach every engineer this framework so that they can kind of look at it and think, okay, cool. Um, you know, this particular edge case will only affect five users. I don't care. Um, you know, I should probably not build this before I check in with the product manager. And you know, like I mentioned, you know, this is specific to us too. So, um, one of our best engineers came from Google. And so a bug that would impact one in 10,000 users at Google, I mean, they've got a billion and a half users, right? All of a sudden you've got, you know, what would that be like 15,000 support tickets, right? For us, you know, we, on our iOS app, we have a quarter of a million, right?
Alex Moore: So if it affects one in 10,000 users, we've got 25 people. If it crashes once a month for 25 people, it's not the end of the world. And so one of the big adjustments he had to make was figuring out, okay, cool. These things that used to be a really big deal for me at Google aren't really a big deal anymore. I've got to be thinking about, you know, on the other hand, if we could spend that time, you know, bringing Respondable into the app and making it differentiated from other email apps, that would have a much bigger impact on the overall user base. Um, likewise for product managers, you know, it really helps for them to have a sense of, um, what's hard, what's easy, what does the framework make hard, what does the framework make easy, um, and have enough of a sense to know when you're kind of fighting the tools you have and when you're going along with them.
Alex Moore: Um, so the technique we've used for that is we've actually have a, anytime a PM joins a project, we have them sit down with the lead engineer and just go through the, the bug trackers, closed bug history and talk about every single one of them and say, okay, cool. What were the surprises here? What was different? What was the same? What was, as you expected, what was easier than you expected? I'm going to have that context. It makes it a lot easier for them to kind of say, you know, develop kind of a spider senses and say, okay, cool, this seems it should be easy, but maybe it won't be. I should probably check in with engineering before I keep going on this spec because, you know, this seems like there might be something, there may be something I'm not understanding here. Um, and when you kind of get folks at that level where they have that basic understanding of, uh, you know, how the other person thinks and what are the things that are kind of going on, that's when things started to become more creative.
Alex Moore: That's when a product manager can engage in a way that helps the engineer be more effective. And that's when an engineer can engage in a way that helps the product manager be more effective. So, um, let me talk about a couple of tactical things we've done for, uh, for getting those conversations to happen once folks are up to speed. So we really wanted to set up a way for a cadence to, to kind of almost forced these conversations to happen more frequently than a bunch of introverts or comp are comfortable with. So y'all probably all do this already. Um, sprint planning meetings. We always have an engineering lead and a product lead sit down together to plan out the sprint and that gets us every single bug, you know, gets a con, a small conversation between the two of them. So if anyone kind of feels like, well this might be a little bit weird or there might be an edge case with this or some complexity or maybe this will impact a different feature that always gets talked about before anyone actually starts doing any work on the feature.
Alex Moore: Um, the main thing that we kind of learned from, from doing these, I mean, we've been doing them forever, is that sometimes it's time as your engineering teams starts to grow to get out of the way. So I used to sit in on all of these, um, and as we scaled, like I couldn't anymore. And so I've kind of taken a, you know, at this point, there are a lot of these that happened without me having any knowledge of it at all or even our chief of product necessarily knowing what's going on except at the end of it and looking at it and saying, okay, cool, this looks reasonable. Um, and we've let some folks make some, some decisions that, you know, we weren't sure if they were mistakes we thought they might be, but not real big ones. And then they've learned from them.
Alex Moore: And it's really kind of helped us, uh, get everyone a little bit more independent to just kind of be like, okay, cool. It's their call. They get to make the decision, let this go. Um, and that's, that's really helped us. The other thing that we started implementing is we do a before. So the way our engineering team works is they make a, they, they write a bunch of code, then they make a pull request that gets merged into the main master branch, um, before it goes live to, uh, to customers. So we asked all of the engineers to, uh, before you actually merge a pull request into a, into the master branch of the code, um, show it to product and talk with them about it and make sure that the behaviors is expected. And um, we did this actually because a bunch of folks who were committing things that weren't according to Spec and we kind of wanted to get ahead of it, but the, the value was actually a lot more than we expected because it kind of set up an expectation that product and engineering, we're going to be talking on an almost daily basis.
Alex Moore: So it basically pulled the engineer out from behind their computer and made them go, go stand in person with the product manager and say, Hey, let's talk about this. And once they, um, once they started having that level of like basically daily interaction, a lot of the problems kind of went away because they talked about, hey, this is why the Spec is written this way and this is why it doesn't match. And it really helped get everyone on the same page repeatedly. And sometimes the answer would be, oh, well, you know, it's really hard to build this way. Okay, cool. Well let's do a compromise and let's figure out how to do it. So we kind of got a side benefit that was even more valuable to us than the, uh, than what we expected to get out of it. Just by making sure those conversations were happening on a one to one basis, not a prescheduled meeting, but something where it's like, you know, ad hoc, but basically every day finally.
Alex Moore: And if there's one thing you take away from this talk, um, this getting support office hours has been huge for us. So, um, what we do in support office hours happens is we get the, uh, the support lead for a product, the product, the product manager for the product and the engineering lead for the product all in a room together every week. And they go over the weird tickets that the customer support team doesn't know what to do with bug reports that are coming in and really high frequency, basically anything that the support team thinks, um, thinks is worth bringing to their attention. And we have it happen every single week. So there's a cadence around it. Um, we did that mostly as a way to make sure that we were responding to customers and realizing that things were going on. But again, it had huge benefits for us. And the reason was because when you have the engineering lead and the product manager for the product sitting in a room together, going over a customer customer feedback together, you kind of get that education that I was talking about again on an every single week basis, right?
Alex Moore: When the customers write in and say, "hey, there's this bug", and the engineer says, "oh, I know why that's going on". It's happening because of x, I think we could fix that. The product manager learns from that and says, okay, cool, x is a source of bugs for this. And kind of has an understanding of why, why it would be going on. And at the same time, the engineer, you know, a lot of the, a, the default reaction as an engineer to a customer support bug is, "oh, I know how to fix that". I'll go fix it right now. And you know, getting the feedback from the product manager. Okay, cool. We've only had one bug report for that, but we've had 16 of this other thing. Let's fix the other thing and we'll get to that a month later. Um, you know, really helps the engineering team understand, okay, this is why they're thinking this way and kind of gets that same kind of a, that same kind of continuing education and reinforcement of, hey, here's how to think about the product management side as an engineer and as a bonus.
Alex Moore: Uh, you know, the, the support team learns how to, how to do this too. So, you know, the first week we have a new support person joined one of these meetings. Um, you know, they bring a bunch of stuff that you know, that they probably could have figured out on their own. Um, but a few months later, all of a sudden you kind got this first pass filter where they're thinking about it and understanding a little bit about the engineering side and a little bit about the product priorities at the same time. So the quality of feedback you get from your support team goes up and their ability to, you know, engage with the customer about a specific issue gets higher too. So you also get really, really competent support people, um, you know, from, from doing this. And finally, um, it's, it's kind of a, an opportunity for everybody to engage with customer feedback on a weekly basis also.
Alex Moore: Cause it's really, really easy to just kind of get s, you know, sucked into, you know, your own cadence and not talk to the customers. Um, and this kind of forces you to, to every single time interacting with them. So there's one thing you take away from the talk, um, do support office hours. It's made an enormous difference for us. Cool. Okay. Um, I probably missed some stuff. The speaker notes weren't showing up, but, um, I think I'm gonna wrap up and let you guys answer, ask them questions and then we'll get to the bar. Thank you. Cool. Any questions? Yeah.
Speaker 1: [inaudible]
Alex Moore: Teah. Right now we're one to four for well, product team members to engineers. I think we actually probably want that to be, um, a little bit lower. So I think we want more folks on the product team cause there's a lot of stuff we're not doing as much as I'd like to write. Um, surveying users. We don't do a lot of that. Um, you know, we, we ended up, we ended up having opportunities where we do interact with customers pretty regularly cause it's like, okay, we're going to go do a field study or something like that. But like the really rigorous UX testing, you know, it's, it's hard to do that stuff, um, on as consistent of a basis as you'd like to. Cause there are always opportunities to make the product just a little bit better that you, that come out of that, you know, but at the same time, if you know, you're, you're dealing with a new Gmail interface, um, you know, you've got to get ready for it. That's, that's sorta gotta be your focus. So I think we'd actually probably like it to be about one to three on the product team to be able to get, make sure we're hitting that bar.
Speaker 1: Yeah. [inaudible]
Alex Moore: Teah. Um, oh yeah, sure. Um, yeah, so the question is, uh, what happens if the PM is missing? How do you, uh, how do you empower the engineers to understand what the priorities should be when there's multiple things in the sprint that they could all work on? Um, that's a great question and, uh, I think we've kind of in maybe a harder problem for us and we're at a hundred people than it is right now. Right now we have rarely had a case where the product manager and the engineering leader both out for an extended period. Um, so usually both of them have been there at the sprint planning meeting and have kind of talked through the prioritization of the bugs. Um, and usually, I mean we're still small enough that, that, you know, my cofounders and I have one of us, we'll have some finger on the polls too, so we can kind of be like, okay, probably this, I hope, um, one of the things that we are trying to do a little bit more of is to try to loop in more of the, uh, the more junior engineers in some of the sprint planning meetings also.
Alex Moore: So they can, you know, with, with kind of an a, an encouragement to, you know, be more of a fly on the wall and a participant, but kind of see what's going on. And then, you know, as you've been to a few of them, you can participate a little bit more. So we do try to get folks even even pretty early on some exposure to that thinking so that they can Kinda, you know, maybe they wouldn't necessarily, I guess the, one of the challenges is your, I don't know how to, I don't know that there's a way to accelerate like that. Hey, I've been doing product stuff for 10 years, so I kind of have a sense of how this is going to go, that I can't explain to you. Um, or the same thing for engineering, right? There's a certain amount of time of things where you just kind of like, this is gonna be hard. And I don't know why, but, and I don't think you can. I don't think you can kind of force distill that down. I think that's a, I think that's what comes with the experience and the suffering and the, the mistakes. Um, but, uh, but I think, you know, just a little bit of a, a little bit of exposure goes a really long way to help folks make decisions that, you know, w be changed by that, you know, 10 years of experience working in the technology, but we'll generally be sensible, um, and won't be crazy. Cool. Anything else?
Alex Moore: Alright, awesome. Thanks y'all.
Keep me posted on Empower 2019.