Suhail Doshi: These are nice chairs.
Sachin Agarwal: You know. High-backed. It's a little Game of Thrones-y, I think. I don't know.
SD: Yeah. Very Game of Thrones. [laughter]
SA: Yeah. Or Dr. Evil. One of the two.
SA: Anyway, so welcome to Empower, our [00:42] ____ inaugural event. It's really sweet of you to give us the time and do this with everyone. If we could just start off... The history of Mixpanel is actually really interesting.
SA: Because it came from, essentially the internship that you had way back when.
SD: Yeah. How many people have heard of Mixpanel? Just quick, so I don't have to re-explain what it is. Cool. Just really simply, Mixpanel does... We do really advanced mobile and web analytics. And we help people understand how people use your app. That's it. So if you were Yelp, we would help you understand how many people are searching or something like that. Where we came up with Mixpanel... It's kind of an interesting story in that, sometimes I think you can get great ideas from almost anywhere. One area that you can get really great ideas about... Or a source of inspiration to help you come up with a company, is just working at a company. And we've all worked at companies and just sat around and said, "Oh my god. This thing is incredibly inefficient in the company." And that's exactly what happened. That's exactly how Mixpanel came about. I was working at a company called, Sly, and they were doing a bunch of data analysis internally. And it was just this atrocious thing that would happen. People would be writing SQL queries, the data would show, the information would show up in that terminal window, and then people would export it to an Excel spreadsheet and then they would copy the data, make a chart, put in a deck in order to please some executive.
SD: And it just was this week-long process that was just incredibly inefficient. And that was the source of inspiration for where Mixpanel came about. But the second part of the story is actually that there are lots of analytics companies. And the first analytics company that probably came about was just like that StatCounter that showed up on all those old websites... Like on eBay, where you could see how many people saw some product listing. StatCounters were everywhere. That evolved into something called Urchin, which then got acquired by Google and turned into Google Analytics.
SD: But where it all started was, it was all very page view-based. And there just seemed like something was very broken. And so when I was working at this company, I noticed that we weren't tracking page views anymore, and that a lot of the Internet was changing from tracking, from being a page view-centric model to being much more rich, interactive and engaging. And so two things happened. One was this huge horrible inefficiency in the company that I was at. But the second really important thing happened, which was the Internet changed in a way that made page views less relevant. And we could just see that that future was so obvious. And we look back now in 2017, and it's so apparent. It's not very inspirational to talk about tracking actions instead of page views. But at the time, that was kind of a risk because nobody was really doing that, and everybody wanted us to track page views. And we had to take that leap of faith to say, "We're gonna try to do something different."
SD: And so early on, I actually had this litmus test where I told myself that we would just not track page views and in fact if... Because I didn't feel like we could do anything that would be a leap over the competition like Google Analytics, I felt it could just be incremental. And I told myself that if people didn't care then the company, or the project at the time, deserved to die. We deserved to die as a company because we weren't doing anything of value.
SA: It's super, super cool. And thinking about like, "Oh. '09 to now." Things that were totally new back then like cohort analysis and looking at funnel analysis and really doing it on event basis. And then that was cool, but we were just talking in the green room, in 2017...
SA: So what's cool in 2017?
SD: Yeah. I think that there is this really interesting phenomenon that happens with using data, which is that... We found at Mixpanel, that we often had to evolve the product because we found that questions beget more questions. An example of that is you go... If I were to tell you, "Oh, look. Look at how many people over time have been searching on Yelp." Your first question might be, "Well, which country is searching the most? Where are all my users?" And then... Or you might say, "But what kind?" What you're trying to understand is you're trying to understand who your customers, who your users actually are. And so if you're building a B2B company, for example, one of the first things you want to understand is, how are people coming to your product? Which features are they using? What's creating the most value?
SD: And so questions often beget more and deeper questions. So what we found is that, we had to start evolving the product to being more sophisticated. And as a result of that, what we find, even in our own product, is this terrible new inefficiency, which is that there's this new inefficiency around people wanting to explore the data to understand something that they didn't understand previously. And that's tedious for our customers. That's tedious for everybody. And so when you think about building a product, it's really important to think about that next evolution.
SD: And so, what's next for a company like Mixpanel, is trying to remove that tediousness. How can we help you explore the data without you having to do that. And one thing that we have now that we didn't have before, that makes this possible is that we have a lot of customers and we have a lot of data. And so that allows us to do things like automatically detect things that are already happening in your business. One feature that we recently released to start evolving that was something simple, like anomaly detection, and really doing a really good job with that, not just making it, "We saw a spike, we saw a dip," but really helping you see that way in advance, proactively, so that you don't have to find that. Often we were surprised at Mixpanel when we see spikes or dips. How can we enable people to see that sooner and faster so you don't have to explore that data anymore?
SA: Yeah. So you're doing that hard work because you have access to this unique and large data set, and you're able to... I mean, would you call it at this point machine learning, is that fair?
SD: Yeah, it is fair. I hesitate to always call everything machine learning 'cause it's so buzzy right now. Sometimes the way you solve a problem certainly is with machine learning. Sometimes its not machine learning, it's simpler than that. But yeah, it's like, how can we utilize the notion that infrastructure and storage are becoming commoditized more and more in the world, which enable us to do more sophisticated analysis like machine learning, exactly to automate you finding something that would have been hard for you as a human to actually find yourself?
SA: Yeah. So we talked in the back about the scale at which you guys operate. Can we say how many data points you guys could do in a year?
SD: Yeah, we ingest like five trillion actions every time someone's clicking something or interacting in your app or a website every single year. Of course, that's constantly growing.
SA: Of data points.
SA: How do you handle that?
SD: You have to hire a really great engineering team that can build infrastructure to do that. But there are a couple ways that we do that. Without going too deep in the technical way of how we achieve that, the biggest thing is just that we have had to build a lot of infrastructure to A] ingest the information, and then really split things apart so that different things are doing exact... They're doing their own specific task. Really partitioning all of that infrastructure across all of our customers. And then finding a way to really be reliable, scalable, secure, and then the hardest part honestly is just the planning for the reality that you will be down. That's honestly the hardest part of figuring out how to scale. It's not the act of scaling but in fact what you do when you know you're going to have an outage. That's often hard. But of course we try to utilize as much modern software as we can. For example, we just recently moved away from a soft layer to Google Cloud because Google Cloud can help us kind of remove the concern and the worry of needing to worry about building every tool in order to scale. So you should try to leverage as... We try to leverage as much as we can to avoid doing as much work as we would ordinarily need to.
SA: Yeah. And that's you guys dealing with it. What about your customers? Your customers are ingesting more and more data they're probably not as sophisticated as you guys are.
SA: So what are you doing to help your customers out?
SD: Yeah. So I like to think about the problem. And I think that's one way to sometimes think about your product is to try to think about the problem that you are solving for your customers. So here's how I think about the problem in at least in 2017. What we find is that we still have customers that use... They still sometimes will build their own analytic system in house. And so... I'm friends with a lot of other entrepreneurs and they run their companies, and so I like to ask them, "Hey, how are you building this if you don't use us? How are you doing it?" And what we find is that people are building the problem we face, or that they face, is that they build this really large sophisticated data pipeline, and what that really includes is that they've gotta... They may use AWS, and they might use some infrastructure that ingests the information. Once they ingest the information they may use... They may build some technology that can kind of ETL that information. Load that into a database so they kinda have to structure it, adjust it, change it, manipulate it in some way so that's more structured.
SD: Then they have to buy something like Redshift or maybe BigQuery, and then they have to kinda set that up in production and put the data in. And then from there, and so on and so forth, until you get to the point where you get to a dashboard where you can see a chart or some information that you can see every day that updates and refreshes and keeps you up to date on what's going on in your business. That's such a huge inefficiency. I talked to a pretty large company, I think there was about 300, 400 employees, kind of a mid market company, fast growing startup, and I asked them how long it took them. They said nine months, just to get that first chart, get the first insight that they needed. And so what we're trying to do is help really collapse that problem so that you can solve that nine month problem as much as we can, and give them the same power, but collapse that problem into something that's a week or a month or a day, depending on the size of the company and the amount of information they have.
SA: Yeah. Well let's pivot a little bit from talking about the data and the scale of the data and what you can do with it and about your business, and how you build Mixpanel as a business.
SA: One of things that I've noticed is having been been a MixPanel user since '09.
SA: So I go way back. Is that it's very clear that you have identified product managers as the person that your company is focused on. I'd love to hear a little bit about what it took to get to that point, and then what you've done now that you know that PMs are who you're going after, to market to them, to sell to them, to do that sort of stuff.
SD: Got it. Cool. So just so I can kind of understand the audience a little bit, how many people really feel like they under... Have clarity and really understand who their buyer is? Their market buyer? Their user? Just out of curiosity. Yeah. That's kind of what I thought the answer would be, which is kind of a mixed answer...
SA: I know.
SA: Which is maybe 30%, 40% of people are are like, "That's really clear to me." And this is actually a really challenging problem early on, but it is really important to figure out and clarify that, especially as you scale. Once you've gone past 40, 30 employees in the company, it's really critical that you figure this out. So there's two parts to the question. One part is, "How do you discover it?" And then there's the second part which is, "What do you do?" So I'll talk about how to discover it. There are two components of it, which is that one part was kinda luck, one thing is we probably didn't even realize that it was an important thing in the beginning, 'cause we started the company at a pretty young age. And then another part was eventually identifying the market that just found our product not very good, where it just wasn't differentiated enough for them, or it would've taken too long to be the best product for.
SD: And so what we found was that back into 2009, although this is less relevant today, that we weren't as useful for let's say a marketing person, but a lot of people in our industry assumed that we were for marketing people. And the way we discovered product managers was that... Well, we were building a product to solve a specific problem. We were very clear about the problem we wanted to solve, which was, "How do we help people make products better?" And then the question really just came down to, "Well, who are the people that make those products better?" And so that typically means that there's an engineer, there's a designer, and then there's a product manager working together in concert to make that product better. And so... And they have different needs and use cases, and they may use our product with differing levels of frequency. So if you can be really crisp about what you're trying to solve, the problem fundamentally that you're seeking to solve, you will naturally find your buyer. And then from there you just have to really clarify that for everybody in the company. And the most obvious way to do that is to... Well, number one, write it down, and then number two, beat a drum in the company about what problem you're trying to solve, and who is it for.
SA: Yeah, and so you've recognized that you are solving problems for PMs who are leading those product decisions and helping the designers and the engineers give them that information. Where do you find PMs? Where do you make sure that they know that like, "Let's not spend nine months building our stuff" like...
SD: Yeah. In the early days.
SD: Yeah. Well we got kind of lucky in that when we joined Y Combinator in 2009, that everybody in our Y Combinator batch, a group of companies, there's about 26. We got lucky in that everybody in Y Combinator was a prospective customer for us. So we had this great community where we just had to convince everybody in that batch to just use our product. And I got this really great piece of advice early on that I still ascribe to today, which is, it was from Max Levchin, he was the co-founder of PayPal, and he's also an investor in our company. And he said that you, "Certainly try to go... " and I know we have a lot of B2B companies in the room today, he said, "certainly try to charge for your product, 'cause you want customers to ascribe value to it. However, if you fail to do that, give it away for free, because the value of the feedback that you get from customers is so much more valuable than the dollars that you would've gotten from them." And so there's a careful balance of that of course when you scale, versus early on.
SD: So when people in our Y Combinator batch were... They didn't have money, and so as a result they couldn't afford it most of the time, we just gave a lot of it away for free, because well, we were gonna get feedback about it. They would complain to me on GChat, and email, and we would just try to solve problems for them, and they were right there, they were so close. So if you have a product like that, where you have a community around you that you're close to, that's a huge advantage, and you should take advantage of that. If you don't, it's always good to be scrappy and go seek that out. I know that the Airbnb founders, a critical part of their story was... I think Brian and Joe would go and sleep in their guest's, in their host's houses in order to just make their product better, and they would come back and so they'd have suitcases going from California to some other place in the US just to figure out how to make their products better. So if you can be somewhat kind of persistent and scrappy about figuring that out, I think that can help really evolve your product, and really help you find those early customers. 'Cause you wanna try to please a very small, narrow set of customers first before you try to go after a bigger swath of the market.
SA: Yeah, and so you've mentioned that you have a community now.
SA: What do you do to cultivate and give back to that community?
SD: Yeah. One thing that we did early on was... Another thing that we did early on was we started to build a huge freemium offering, and that really helped us early on because we could be very generous with a lot of small companies who we knew could probably not always afford the price point of our product. So that's one thing. And then just recently this past 12 months, we have significantly expanded that free option, particularly as we're moving up market, we're going after larger enterprise companies, we didn't want to leave all those small companies behind, so we dramatically increased that free offering. So that's one thing that you can do. But more than that, it's really important to do a lot of the normal... It's really important to take kind of the traditional marketing tactics that you probably hear about in a company, but kind of find the right twist, 'cause if you're doing everything the same in marketing that everybody else is doing, well then you're just part of that noise. So you know, if you're thinking about doing blogging that's a great vehicle, blogging's great it's tried and true, but you have to find your twist, your artistic creative version of that that meets the style of your company, and try to find some differentiated way of blogging.
SD: And you can basically apply that algorithm to almost everything, whether that's social media, whether that's podcasting or videos or conferences for example, just trying to find your own contrarian, own unique differentiated way to apply, to take a traditional tactic and really flip it on its head. So we try to do that at Mixpanel, In a world where people were doing a lot of short form content and just kind of doing quantity, one thing that we did on our content team is we did something similar to first round review, if you've read that, which is like more long form, very long form articles but have a lot of rich information in it because we felt like people were tired of the quantity and wanted more value. So that's an example of that.
SA: Yeah. So you've mentioned that you guys are moving up market.
SA: How has that changed the product? How has that changed the company?
SD: Yeah, it changes things a lot, and it's an evolution. A big portion of your company is totally the right set of people to help you there. But there is a portion of your company that isn't, and so you have to find balance in that and that means you have to go seek people that are really good at solving a set of problems that are up market versus down market, which is your smaller companies, maybe sub 500 employees for example. And you have to figure out how be able to find a way to focus. And so I think that's hard. So one is, you need to find the right people in your company 'cause they're completely different worlds. It's so much more traditional to go after the enterprise than the way you might go after the SMB. You can apply a lot of consumer tactics to get the SMB but those tactics don't work in the same vein for going after enterprise companies. So then the second thing is you just have to acknowledge that what you might do culturally is different. You're gonna have to swing your culture around to be able to go and achieve that, so that's another big, hard step for most people in a company.
SD: And then the third thing is of course you have to see the notion that your product, while it could have been a great product for the SMB, you have to almost allow yourself to put your ego aside for a moment and say, "This product is probably broken in the enterprise." And it's better to assume the worst than it is to... 'Cause it'll make you kind of... It'll create the urgency you need rather than assuming the best. And so you almost have to assume that you don't have all the pieces figured out. And so a lot of the things that I think you do to figure out product market fit, let's say, in the SMB, you kind of have to redo that process again in the enterprise to ensure that you can be pretty confident that you have everything that you need. And so there are really big elements to that. There's some obvious ones like reliability and security in administration, which we may feel like is kind of boring, but maybe necessary to reduce the friction in the sales cycle. But there are other elements, that there are probably really innovative things that you can do for a 3000, 5000, 10000 person organization that you never get to see. There are huge inefficiencies in large companies, we all know that, so we just have to seek that out and that those inefficiencies you have to acknowledge you would have never ever seen in a 100 person company. Because the scale is just so different.
SA: Yeah, that makes sense. So our official time is up, but we have the eight minutes for Q&A so I would love to open up the floor to get some questions from our audience. I have a follow-up question while the mic moves over there. As you're moving up market you chose not to abandon the SMBs.
SD: That's correct.
SA: You actually expanded your freemium offering.
SA: Can you talk a little bit about that decision?
SD: Yeah. I mean, strategically we didn't want, if anyone's ever read Innovator's Dilemma, I did not read Innovator's Dilemma, I read Innovator's Solution, thought I would just jump past the dilemma part...
SD: Of the whole series. But there's this notion that companies typically retreat when they see a threat down market. And we're obviously in a market that's very competitive. So in order to avoid sort of retreating only to have that person come back up into that same market that we would be in eventually, we thought that we would take a little bit more of a scorched-earth policy and try to be generous to our SMB companies because they're invaluable, and they're diamonds in the rough that become humongous companies for us so they're good investments, but still be able to retain that market. And that's a big reason why we increased our overall offering and made it almost more free.
SA: Question in the back?
Audience Q1: I guess, this is a question also about the freemium model. How did you come about with the pricing that you've settled on, and what sort of experimentation did you do to come up with that pricing?
SD: Yeah, it's gone through a whole series of different incarnations. It started from it being... Day one was a guess and all the way to today where now we hire expensive consulting firms, and do an insane amount of customer interviews, and get tons of feedback internally, and we have 100 page decks, just like a big company, to evaluate how to make the perfect pricing offering. So it really depends on your stage. We've even done the thing in the middle which is doing A/B testing. So it kind of depends. I'll pick the thing in the middle just because spending a lot of money on a big consulting firm is sometimes not tenable for the average company. If you don't have that money, and you can't afford that kind of time and the resources, what we did was we did A/B testing of our pricing. And so one thing that we chose to do was we chose to kind of... We wanted to pick something... We wanted to do a big enough change that it would give us information. We find that with A/B testing, if you don't make a big enough adjustment it's hard to really see enough disparity between two groups of users, because it's not just volume, it's disparity of results that really matter and get you to significance.
SD: So we made a big change and went from something that was like free with no feature gates, to more free with feature gates. And we thought that was a pretty big enough sizable change in our product. And then the way we A/B tested that was, we A/B tested it regionally. So we felt like maybe the Canadian market where we had a lot of customers, it was pretty close to our US market, because in your US market it's completely different than like say your market in Asia, in part due to currency and people's standard of living.
SD: And we ran that A/B test to try to find results and then we evaluated those results with great rigor, and then we ship the change 100% and we felt kind of confident. So you just have to kind of pick which approach you want to take, which one is more in depth versus light. But in the earliest of days we just sort of... It was more intuition, more guesswork, more adjustments. Because we just didn't have enough customers to be honest. Yeah.
SA: And, in the early days and now did you guys do tiers, or was it like per usage pricing?
SD: Yeah. I think our first failed pricing experiment was actually like an AWS pricing scheme. And what we found in B2B is that... And even though AWS certainly sells to businesses, we found that people... It's funny, people actually liked, they really care about predictability. And they would rather pay a higher price point, but for predictable pricing than for cheaper pricing where they may not know exactly how much they're gonna spend in that month. And so people really freak out when they don't know what they're gonna pay each month. So it really depends. We're in a volume business so it's a little different. So we failed completely at an AWS price point, like in the first six months of the company. None of our employees were around for that. And then we moved to a tiered model approach where we just had a series of tiers because we wanted to have really transparent pricing for... We felt like, back then more transparent pricing was kind of a novel thing in an industry and B2B where everything was a contact us form.
SD: And now I feel like where we're kind of headed in the industry these days in 2017, is kind of the hybrid approach because people in B2B businesses don't want to leave money on the table, but they want to be somewhat transparent to their customers to allow people to just not have to necessarily talk to someone in sales if they don't want to. That happens. And that certainly happens in industries where product people, engineers, founders typically are more averse. They're trained to kind of avoid that, whereas in other industries that may not be the case.
SA: You guys have salespeople now, right?
SD: We have a lot of salespeople, yeah.
Audience Q2: So you mentioned that you guys were working on automatically detecting things that happen in the business. I was just wondering what sort of things or what sort of trends are you noticing in the ways that people consume information specifically as it relates to analytics?
SD: Yeah. I find that a lot of the analytics tools on the market today are hard to use, and they're hard to use because they're trying to be powerful. And whenever you make something powerful, I mean, most of us have used like Photoshop and then there's like MS Paint. MS paint's not very powerful, so you can't really do much with it, but it's easy to... You know how to draw a circle and like that and/or a line, and so it's straightforward. But Photoshop is crazy, it's hard to get started but it's unbelievably powerful. And so, one thing that I see that's a big problem is finding the balance between those two things. And I think that's where... I think that's where... I think what happens, as a result of that, is that if it's too hard to use, then people won't use it, and if they don't use it, then the thing that you're making is just not valuable anymore. And so one of the trends that I think I see is that what's... Maybe less of a trend and what I think is an opportunity, which is I don't think you can utilize... I don't know that you can utilize machine learning for every possible thing. I think that there are applications where machine learning is great for, and where it's... There's a lot of work to be done.
SD: Or AI for that matter. But where I do see a lot of opportunity is kind of removing the user interface, which is meant to be very powerful to answer the question but not very accessible for most humans to really learn how to use. And if we can utilize... If we can let machines and programming and build great products that can do the work that a human is pretty much pretty bad at, I think that's a huge leap in the world. And so I think we need to simplify to use interface and I think the way we simplify that is just saying, "Here's this really important thing you should know about. Why don't you take a look?" And then you go to the UI, and then you see the thing, and then maybe you adjust and learn a little bit more rather than having to use this complicated UI to find it in the first place. So I don't know if that answers your question, let me know.
SA: One last question for you. Is obviously people will put events into Mixpanel from web applications, they do it from mobile applications. You're probably seeing some internet of things, stuff now. What are the different sources of data that people are putting into Mixpanel now?
SD: Yeah. We have like one crazy example where... I think a company had a treadmill and they hooked it up to the Internet and then every time someone ran on the treadmill they would send it back to Mixpanel. So weird crazy use cases are happening like that all the time. And then there're even more devices like Apple TV and that has a huge app ecosystem. And so we continuously invest in essentially what are the biggest platforms in the world. So anything... Which is pretty much comes down to anything that Android or Apple will basically create. Because I think standardization is happening more than anything because people want distribution. But so I think that... But I think there is a more obvious, overt problem that already exists. People talk a lot about the internet of things, but there's a more obvious problem which is that people just have a lot of data and lots of different business units, and it's super siloed.
SD: You got your sales data and your sales force, and you've got, maybe you use Zendesk and you have support-tickets and stuff in there. And maybe you use Marketo or some marketing automation tool and you've got information in there. And then you have Mixpanel. And then you have your own data in some data base somewhere that's related to just your product. And people don't know how to bring it together. And I think that problem needs to get solved because people have really important, valuable questions they wanna answer across every single one of those data sets. Wouldn't you wanna know how many tickets a certain feature generates? Because if it generates a lot of support tickets in Zendesk, I mean don't you wanna solve that problem? But how do you link that thing together to identify that that's an area that you should prioritize? I think that problem needs to be solved. I think that's a very important problem that we need to solve.
SA: And is that an area that you guys are focused on?
SD: That is an area we're focused on. Yep.
SA: Well I'll leave the future at that. Suhail Doshi, everyone. Thank you so much for your time.
SD: Thank you everyone.