How Do You Know It's A Good Idea?

Yohanes: My name is Yohanes Frezgi, I'm a Director of Product at Audible. I manage our iOS app.

Suzanne: Welcome, Yohanes. You said to me something right before we started rolling the mics that I was excited about, which was you get asked a lot of the questions that I ask on this show just about every day. Can you elaborate on that?

Yohanes: Yeah. Well, I have done a couple of other sort of speaking engagements where I talk to junior product managers, people who are interested in getting into product management, but also at dinners. The conversation kind of goes like this, but people kind of ask the same thing over and over again. They say, "Hey, what do you do?" I say I'm a product manager, they say, "What is a that? I've never heard of it." And I say, "You know I work with engineers and marketers and business people and we build all the things you use on your phone every day. You know, all those apps, we built those."

They say, "Oh, that's really interesting. I have an app idea." And then they tell me their app idea and then it's like, "How did you get into that?" At that point I sort of pause because my background was in business, I had an accounting major and I also did a lot of finance. I interned at Goldman Sachs and did consulting and transitioned into product management, but there's no real easy way to say how you got into product management. There's different paths, but there's no clear way.

So I usually sort of say something to sort of explain my path, but when you emailed me, I thought that these are the questions I get constantly asked.

Suzanne: Right. So I guess a true product manager, in those instances when someone says, "Oh, I've got a great idea for an app," will say, "Well, how do you know it's a great idea?"

Yohanes: Yeah.

Suzanne: Why don't you go and talk to five people and then come back to me and let me know if that's a good idea and we can talk about building it.

Yohanes: Yeah, that's exactly right. That's exactly right. And you know now with some of the books like Sprint, that's the first thing I always try to mention to people. You know, you think you have a great idea, why don't you put up a Facebook Ad and see how many people click on it? There's a lot of really simple ways to test things. InVision is another tool, I'm not sure if you talked about that or not, but it's just a way to prototype things.

But you know, there's so many different ways to test an idea before you actually spend millions of dollars or a lot of time developing it, that there's always a lot of work that goes into the actual development piece that people are always sort of trying to get to so early on.

Suzanne: Yeah, I mean, I'm a big proponent of validated learning as listeners of our shows know. They hear me talk about it a lot. One of the things that I encounter, interestingly, is in a consultative capacity, when I work with clients, especially founder clients, the "I have an idea" folks and one of the first pieces that I put up is how do we know this is a good idea? How do we know this is pursuing? How do you know that writing a check to me and my company for all of this money is going to be money well spent?

Why don't we go out and talk to some customers, why don't we do some of these different tactics, vaporware, right? MVP, just to see. Certainly I think our listeners who are product managers or who are looking to become product managers are familiar with the importance of this validating of ideas. Why do you think entrepreneurs don't want to do it? Like they just want, "No, just build it."

Yohanes: Yeah, well, I mean, I lived in San Francisco for a long time and some of my best friends are incredibly successful entrepreneurs, and a lot of these people are just incredibly driven people, like one of my best friends, Chris, his company is called Life360, where he's done pretty well for himself. When he was like 12 or 13 years old, he basically had his own business where he basically was having people put up satellites, TV satellites. This was in the '90s, right? So he was basically an incredibly driven person and he has a goal, and he wants to make that goal happen and when you tell an incredibly driven person, "Hey, pump the brakes for a second, have you ever thought about does anyone like this idea besides you?"

Then they're like, "Whoa, hold on, I don't think I'm explaining it to you well." But the whole validation piece is actually incredibly useful to an entrepreneur, just because you know, before you go through the blood and sweat and tears of building things from scratch, you want to make sure that people are actually going to use it. So it's this whole idea of if you build it they will come where people just think that "If I just make this product, everyone's going to use it," and that's just not always the case.

Suzanne: Yeah, I think it's almost never the case, actually. I definitely, one of the biggest lessons for me learned through the years is just how important it is to invest in the top of the funnel type of work.

Yohanes: Yeah.

Suzanne: When we talk about growth marketing, which is really just marketing using validated learning concepts in the same way and different tactics to quickly get to the answers or what works, but yeah, just having a product and then expecting that people will start lining up magically overnight is a little bit of wishful thinking and where I see that perpetuated even further is okay, I couldn't convince you to go smaller with your product, or maybe your budget ultimately convinced you that you had to go smaller than you might have initially wanted, but even then the product is live and the instinct is, "We need to get these new features out."

Yohanes: Yes.

Suzanne: It's like, "Well, wait a minute, we still don't even have customers for the first set of features that you insisted had to be there." I think a big part of it is fear.

Yohanes: Yeah.

Suzanne: To be perfectly honest, that there's just a sense of "As long as I'm busying myself with product development, I don't ever have to stare into the belly of the beast of sales." Which is of course the hard part and the critical part and getting to a place where you can consistently bring customers in and then prove value to them. That's the real work of a product manager, especially in an early stage startup venture.

Yohanes: Yeah, that's a great point and at Zynga, we would launch many, many games and we had a practice of soft launching a game. So the US is the big market for any product that I can think of, so we said, what markets are like the US? Canada is an example, Australia's an example and we would soft launch in those markets and this is for mobile games, and we would say, "Okay, how do we know if we can pass GO to go to the next phase?"

We would have a set of metrics. At Zynga it was the three R's, Reach, Revenue, Retention, and we would sort of have to validate ourselves across those three metrics or those sort of categories of metrics before we can say, "You know what? We're ready to go to the next phase." Our lifetime values, our LTVs are here, or our retention is here, our day seven retention, our day one retention. We see people are sharing, which is our virality, or growth and you know, until you validated yourself across those sort of fundamental metrics, then you don't have permission to go forward because if you go forward before then, then what you'll see is that you'll crash and burn.

That's really important, because when you launch a product, you can only launch once, right? A lot of times when you launch, you might want to have a budget, or you might want to ask Apple to feature you or you might want to, you know, you do a whole sort of launch plan and if you burn their your users and they don't like the product or it's not there in terms of your engagement, then the likelihood that you can reactivate them or re-engage them is a lot lower. So you have one big shot and you want to make sure that you're ready for primetime before you turn on your marketing funnel and go live in a big way.

Suzanne: Great. Yeah, let's come back to Zynga, because I have lots of questions for you about your time there. Your path into product management, so you started to talk about you studied business and it was mostly kind of rooted in the accounting and finance side and then you transitioned into product management, but I'm curious to know what catalyzed that transition? Did you just see that it was a cool thing other people were doing?

Yohanes: No, no. So I have a somewhat interesting background, so back when I was in college, I was an intern at Goldman Sachs and I was an accountant on their trading desk for this Block Prop high frequency training which was all the rage back then in 2005, so I'm kind of old.

But anyway, Goldman said that "Hey, Yohanes, you should apply to this program," and they had a relationship with Harvard and it was called SVMP. Summer Venture Management Program. So I did that program. I liked it, they liked me and they suggested that I apply to the business school as an undergrad, so I did that. I was accepted, so that was awesome, so then they said, "Hey, you know, you probably should just come to business school now, why don't you work for a few years?"

And I did one year in consulting, doing management consulting in Chicago with Booz and Company and the second year, I worked at a venture capital fund of an alum who invested in a lot of start ups. And when I was at these start ups, I did all the analysis for it, but I really kind of fell in love with the entrepreneurs and actually building things and launching things and I thought to myself, "I want to be the person who builds these awesome products."

As a business person, it was hard for me to make that transition because the traditional product manager, a lot of times, is somebody who has a computer science background. They're the person who's coding, but then wanted to transition to more of a business side or creative side and that's how they made their way into product management. But I didn't have that sort of luxury because I spent all my time doing finance and accounting, so I thought how could I transition?

We actually had an alum from Harvard whose name was Mark Pincus and he had this cool social gaming startup and I was a very big gamer and I reached out to him saying that, "Hey, I would like to bring some people to your office, just to check them out and maybe we might do a case study on you," and I did not ask for a job. During my conversations with him, he was looking for a particular type of product manager. And this is something I see happening more and more places. Product managers who have some sort of quantitative background who are interested in AB testing, who are interested in being metrics focused and that was the sort of product manager that he was sort of grooming at Zynga and a lot of the people that he got to join that company had background in consulting or investment banking or hedge funds.

And there, I was able to be trained to be a product manager, even though I didn't have the traditional computer science background.

Suzanne: So he saw your proficiency with numbers and said, "I can take that core asset and repackage it into this role that I'm looking for."

Yohanes: Yeah, that's exactly right. So basically I had some deficiencies and I was actually really good at certain things. I was very good at the quantitative piece, the metrics focus. Had not really spoken to many engineers because I was talking to business people at hospitals and oil companies, so that was a deficiency that I had to be trained on. I had a lot of great product manager mentors, a guy named Alex Le is the VP at Reddit now. His wife, who was my manager at the time, Nancy Hang who's the VP at Hipmunk.

And both of them have computer science backgrounds, sort of trained me on the deficiencies and sort of by watching them, I was able to fill in those gaps that I didn't have while also supplementing the things I was already really good and already was very comfortable, which is modeling, AB testing, figuring out how we quantify our performance for a given feature, or determining "Hey, are we ready for primetime?" So yeah, it wasn't the traditional roll into product management, but I think that it's something that is becoming more traditional over time where people are comfortable having product managers who don't have the typical computer science background.

Suzanne: Can you recall the first time you realized, "I don't know how to relate to engineers?"

Yohanes: Yeah. I can. I think I was probably trying to explain something like a flow, but it wasn't a flow. So I was trying to explain a feature to them, and I already knew that I can't talk to these guys, so what I did was I read something about user stories, so I probably wrote at least a 15 page Word document trying to walk a person through this flow and the engineers saw it and they were like, "What is this? I'm not going to read this."

And then Alex, he was basically in charge of the game I was working on. He said, "You know, instead of trying to do it with words, why don't you use pictures? Because pictures are much easier for a person to understand than words," and he just sort of mapped it out using something like Balsamiq or one of those tools and everyone was like, "Oh, that's what you were trying to say."

I was like, "Okay, maybe I should look into using those tools, or maybe I should sort of shadow this guy and see what he can teach me."

Suzanne: Right. Was that the foundation for Words with Friends? So it was like, first we're just doing wireframes and we're like, "This is kind of a fun game actually, maybe we should soft launch this in Canada and see how it goes."

Yohanes: I mean, Words with Friends, we actually acquired Word with Friends, but some of our other games like FarmVille for mobile, pretty much all our other games, once we started to figure out mobile, we always would soft launch and now it's a practice that you always do for any major games or even for a lot of consumer products. You might want to do an enterprise build and you might just have people in New York use it.

Or like HQ Trivia is an example. I'm not sure if you're familiar with those guys, but it's a game that's very big in New York and they probably haven't spent any money on budget yet because they're still trying to figure it out and once you figure it out and you sort of got that flywheel going, then you say, "Okay, let's blow it up. Let's put a million bucks in terms of marketing it or whatever else."

Suzanne: Yeah. I mean, it's equally good advice for raising funds. If you can kind of get your product business to that point, one, raising money is a lot easier because what you're really saying is, "I just need to dump a whole bunch of gasoline on this thing and light a match," versus, "Hey, could you fund me and my pals trying to figure this out for a while?" Which takes a specific type of investor interest and they exist, but that's the distinction too, in that angel money round is, "Sure, you know what, Yohanes, you seem like a good guy, this seems like a decent idea. Take some money and go and try and figure it out for sure.”

Yohanes: Yeah, that's a great point and before I joined Audible, I was at a startup called Fever and we raised our series A through Excel's London office and it was really interesting. You're talking about sort of proving out that model. Talking to VCs in New York, we're very much metrics focused, where they're like, "Okay, what's your LTV? What's your growth numbers? What's your MAU?"

I'm using a lot of acronyms, so MAU is monthly active users, LTV is lifetime value, DAU is daily active users. But they were always very focused in terms of what your metrics are and our West Coast VCs, we pitched were always more about the story and the qualitative and how do you just get really big? So it was interesting seeing both VCs talk about metrics, but I can definitely agree that people were a lot more inclined to listen to us is we had some great numbers to tell them about our engagement or our DAUs or our weekly active users or any of those metrics.

Suzanne: Yeah, and I think the part that gets sometimes overlooked is the numbers don't have to be big. They just have to tell a potentially compelling story. That was true for Facebook, is that what was so interesting for investors was how much time people were spending inside the app, right? So if you've been able to prove out even on a small scale, "This is what my funnel looks like, this is how many people stick around after," and they then ... People can extrapolate and say, "Great. If we put more zeros on that, then there will be more zeros on the output of these numbers as well."

Yohanes: That's exactly right. And you can do it fairly quickly, or just with the ... Like I said, me and a couple of friends of mine, when we were in business school, we would always test different ideas where we would just do some Facebook Ads and have a landing page and then what's the conversion? What's the click through rate? How many people gave us their email address? All those things are incredibly interesting to VCs because they're looking for some sort of way to validate it outside their qualitative feelings of "Do I think this is a good idea?" Right? Do I think this is a good idea? I think this is a good idea.

But they had a click through rate of 39% or 49%. 60% of people who visited the landing page gave them their email address. That could be a business. So that's what they're always looking for.

Suzanne: So you cut your teeth, then, at Zynga.

Yohanes: Yeah.

Suzanne: They basically groomed you as an intern, you become a product manager, you were there for awhile. Is there a difference in your experience of managing games as a product versus a B2B type of product or a B2C product that is worth listening to? Can you share?

Yohanes: So for me, I've always been in consumer, but Zynga was freemium. Audible is a subscription base and there's a very big distinction between those two. So I think with the freemium game, where people aren't putting in any money, then you can try a lot more things, because basically everyone is just using this thing, they haven't put any money down. They're just getting some five minutes out of it that they can enjoy. But with the subscription-

Suzanne: Or more.

Yohanes: Or more. Or more. A lot more for our games, but in five minute increments. But with the subscription, people are putting down a lot of money, they expect a certain level of quality, so in terms of the tests that you do, you just need to make sure that the tests are a lot more well-defined or have a higher bar. Like at Zynga, we would do something called ghetto test and that would basically be a test we could make in two hours. Two or three hours.

We might just put a button somewhere and say, "Do you want to see Cops?" And there would be like a button. You'd say yes, people say green button, right? They would say they want to see Cops, or they want to be able to do a PVP sort of a thing and that was okay, but if you have a subscription or you have paying users who are paying to use something, than you can't just sort of say, "Do you want to do something?" And then don't do it, right?

There's a lot of other ways to validate an idea besides doing an AB test that you put to hundreds of thousands of people that we use at Audible in that we also used at Zynga, but are different than you would do between a freemium and a subscription service.

Suzanne: So tell us about, you said you work on the mobile app here at Audible. First of all, for any of listeners who don't know, what is Audible?

Yohanes: Audible is a digital media service, primarily we are audiobooks, so we're the biggest audiobook company in the world and if a person is interested in listening to audiobooks or increasing the number of books they listen to, then Audible is a great service. I actually used it for three years prior to the recruiter contacting me, which was why I decided to come here.

One thing that I always have done is I will only work on a product that I personally use. And that's why most of my background has been in consumer internet, but for Audible, I used the service for three years prior and with Audible, I listen to it during my daily commute and it basically increased the number of books I could consume in a given year. Like this year, I'll probably listen to 30 to 40 books, a lot of them business books, a lot of them fiction books, just because when I'm jogging or running I can always be engaged with a book that's sort of helping me improve in my craft in terms of product management or a fiction book that I'm just really interested in that I enjoy.

Suzanne: So given that you were already a customer of Audible, did you show up on day one with a long list of grievances, and were like, "Okay, I have been waiting for this opportunity to get these things looked after."

Yohanes: Yeah, yeah, I actually did, and I had a lot of questions and that was sort of where my initial backlog came from for the product. And one of the most important things you realize as a product manager is that a lot of the pain points or people always, "What should I do for my backlog?" Your customers are giving you a lot of your backlog. If you just look at your app reviews, or if you talk to your customer support people, than there are excellent resources for determining what you should do.

So one of the first things we did was I looked through our customer reviews, I talked to our customer support and that helped define some of our backlog.

Suzanne: So do you have a team of product managers that work with you?

Yohanes: Yeah, I have one junior. So our mobile team is only three people. One junior, a person who I work with, and than we have three mobile teams on iOS and then we have three mobile teams on Android as well.

Suzanne: Can you tell us a little bit about how those teams are structured?

Yohanes: We're structured in a typical Agile fashion where we kind of have pods and we have pods of engineering teams and each of those engineering teams may be working on one or two features at a time and we do the typical Scrum where we have a daily standup, we do a demo at the end of that and it's pretty straightforward for the normal Agile process in terms of our Sprintboard and everything else.

Suzanne: You practice SAFE here?

Yohanes: We do practice SAFE. It's a version of SAFE. We basically take what works for us and then we don't use other things that don't work for us. And that's also an important point that I think a lot of times people who are new to Agile don't understand or may not get. They read the Agile manifesto or some of the popular Agile books.

And they say, "Okay, this is what Agile is." And they have a very structured, defined way of what Agile means. But Agile is really, it's what works for your team. Some things may work for your team, some things may not. In terms of SAFE, we use PI Planning and basically, that's just a program, it stands for Program Increment Planning and when you get to a company the size of Audible, there is a lot of different groups in our company and what we sort of say is that, "Here are the things we want to build. We want to build these three things or these five things," and across the entire company, we say, "How do we make this possible? How do we do this?"

And that's really amazing, because it gives everyone visibility in terms of what we're working on, it gives everybody clarity and it gives everyone a sort of roadmap across all our teams for saying we want to build this, so how are you going to help? What do you need to do to make this happen? And we all sort of work together to build our roadmap based off of what's defined in our PI Planning.

Suzanne: Audible's a huge company, as you mentioned. Zynga is a large company. So you've got a lot of experience working in these enterprise scale businesses. One of the things, of course, that we talk about a lot is the importance of involving your key stakeholders throughout the journey. So in your day to day, you're working in these pods, you've got your couple developers, maybe your interaction designer. You're in the granular details, and you have to kind of remember to connect with others along the way.

Does that become more complicated in companies at scale? The exercise of getting buy in? Can you talk a little bit about that?

Yohanes: Well, it depends, right? So for us, we get buy in at the very beginning, which I think is great. Because everyone, at the very beginning of our PI Planning session, we say, "Here's what we're going to build." So from the marketing team to the business team, to our content team, everyone is on board, even beforehand for saying, "These are the things we're going to build."

So in terms of buy in, we have it from the onset. And in terms of making sure everyone stays on the same page, that's really up to the team members. There's no reason why you should only check in with your marketing counterpart during a demo. They should actually be involved throughout the entire process and I think that the more that you bring people in, the more visibility you have. You decrease two things, right? You decrease the chances of you building something that's not going to be useful to one of your key stakeholders and you also decrease the ability of you missing something, some sort of edge case or some sort of a flow that you may not have thought through with your product manager, but someone from customer support may have thought through, or someone from the marketing team may have thought through.

So one of the biggest things I found to be super helpful for me as a product manager is being inclusive. When I was at Zynga, the most successful thing I did, or that our team did, in terms of revenue, was not my idea at all. It was an idea that one of our designers sort of mentioned to me while we were having beers.

Suzanne: Where all great ideas are born.

Yohanes: That's exactly right. So the whole point of thinking that it's all in the product manager to define this backlog I think is not a fair point and it probably puts too much pressure on the product manager. You work with a team of people and great ideas can come from anywhere.

Suzanne: You spoke earlier about your quantitative background, your proficiency with numbers as kind of being that thing that allowed you to transition effectively into product management, and I would imagine there are folks listening in, including myself, who I don't come from numbers, right? I have a long history of numbers boring/scaring me and I made my way into this arena and a lot of people have made their way into this arena.

Yohanes: Yup.

Suzanne: And we also know how important the quantitative aspect of product management is. What advice could you offer to somebody to start gaining proficiency? So kind of the inverse of your scenario where I already have the creative side, or I already understand some of these softer skills, where I'm new is basics around numbers?

Yohanes: Yeah, I think that happens a lot for people with computer science who are transitioning or somebody from design who's transitioning into product management and the great thing is, there's a lot of amazing resources. One thing that I still use to this day because I'm always trying to learn new skills myself is I use Udacity. I'm learning Python] right now, just because I can use SQL, which is a way to search through a database, but I want to become more proficient in terms of machine learning and AI and that's a skill set I never had.

So programs like General Assembly, Udacity, they have some classes on AB testing and being metrics focused. There's also a lot of blog posts in Medium. You have a lot of great content on your side as well and books, the Lean start up, that goes through a high level overview of some of the most important metrics or the ones that people often talk about.

Suzanne: Metrics is a confusing word, 'cause it describes the act of measuring as well as the thing that you're measuring and it sounds scarier than really what it is, right?

Yohanes: Yeah.

Suzanne: Really what it is is saying, "This is the number that we're going to look at," or, "This is the result that we're going to monitor or measure as a way of determining our success." And I think where a lot of folks go wrong isn't specifically in "I wasn't good with numbers or I didn't know how to interpret the data from the Google Analytics," it's actually making decisions without first thinking about how it's going to create impact to begin with.

Yohanes: Yeah,that's right. And I think that if people don't think about it as "What are these numbers," but if they just think about it from their perspective, so if your designer, what does success look like for you? Success is probably going to look like people going through their core loop, or using your product in the way that you defined it to work. So what that actually is is just the funnel, right?

It's that how many people started off at the funnel, then how many people got to the end of the funnel, which is your core loop. And if you take away the acronyms and some of the scary pieces away from it, it's a lot of it is just common sense, right? How many people did you get to visit your page? How long did they stay? How often do they come back? Did they get any of their friends to use it?

These are all common sense questions that you don't really need to have a quantitative background to understand. And when you are building your product, those are the sort of questions you want to ask before you even write your first line of code. Because what you're doing there is you're basically saying, you're determining, "What do I need to do to pass GO?"

Like I said in the very beginning, when our soft launch, how can I say that we're doing well enough for us to feel comfortable with going to our next phase? That next phase can be hiring an engineer, or trying to raise more money or launching more broadly or starting to do some sort of paid advertising. I think that you're right, that a lot of times, people who come from backgrounds that aren't finance or business, may feel scared by the word 'metrics', but if you just think about it from a common sense perspective of "How do I want people to use my product?" Then they can get 80% of what they need from there.

Suzanne: Yeah, I think that's great advice. I talk a lot about, and we spoke at the beginning of this conversation about launching a product initially versus what comes after and in the context of funnels, one of the things that I say is you launch a product initially and your first goal is to build a working funnel, but a lot of the times it's not the best funnel.

Yohanes: Yeah.

Suzanne: It's just working.

Yohanes: Yeah.

Suzanne: And all of the work after that just kind of becomes the optimization, and I use this term 'product management' is a game of inches. What would you say on that topic in terms of how much time should I be spending trying to dial it in and really, really optimize the numbers of each phase, versus just saying, "You know what? This is good enough for now and I'll come back to dial this in at a later point."

Yohanes: Yeah, that's a great question and I don't think that there's a one size fits all answer for that. So if you look at a product, and you could use someone like Facebook for an example, there's phases for what the product should focus on at a given point in a startup. So when you are at the very beginning of your startup, you haven't written any code, you don't have any product, the first thing that you want to do is, "Hey, can I build something that somebody wants to use?"

So basically, you just want to focus on what's your retention? Or can I get people to come back? Can I get people to use a core loop and can I get people to come back to it?" So at the very beginning, and that's just product market fit, is the term. So once you say, "Okay, I've got a couple of people to use it and come back," then the next phase is, is when you're talking to a VC, "Can I get more people to use it?" A lot of times you don't have money, so then you might want to focus more on growth features.

And then you might want to focus on virality or getting people to share or whatever else. And then once you start to grow, then you might want to come back to focus on retention or the next thing you might focus on, "Well, I got people to use it, now I got people sharing it and I now I have a pretty big user base, how do I monetize the thing?" So then you gotta think about what's my LTV or lifetime value or what features do I need to get to make people pay for it, or how do I do that?

There's phases to a product and what you need to focus on always depends on what you're trying to optimize for. Retention, growth, monetization is the traditional route that I see a lot of startups going through.

Suzanne: On this topic of monetization, 'cause you kind of list it as later, and one thing that I see that I think is a mistake is waiting until later to figure out if it can make money. So I agree with you in the notion that first focus on retention because that is a value metric. People don't typically continually use products that they don't like or that aren't making their life better or more interesting in any way, but I do think the distinction is focus on creating value in the form of use and then focus on dialing up the revenue.

Not focus on making a product that people use and then later try to figure out if there's a way to make money off of it. Like Twitter's a great example. We're all seeing where they did a really, really great job with virality and with retention, but retrospectively trying to slap a revenue model onto something doesn't work. I think there does have to be an upfront strategy or at least hypothesis around, and "This is how we will make money."

Yohanes: I think you're right. But it also just depends on the industry. If you looks at WhatsApp, WhatsApp never really had a revenue model, but they were acquired by Facebook for billions of dollars, and one of the reasons is, is because a lot of times when VCs think about an evaluation for a company, they do that based off of their MAU or their DAU, so they say, "If this app has 300, 400, 500 million users, then I can value depending on the type of users they are, are they wealth or lower income, then I know that this user is worth this amount."

This is traditionally for advertising. And they can say, "Okay, based off that, this company's worth $3 billion, $4 billion." Twitter is an interesting example because they got so far in their product life cycle that they kind of waited too long for them to sort of figure out the monetization piece. But I totally agree that you should start to validate early on. The one thing that you want to be careful of is before you have that flywheel going in a way and you really build a deep retention, if you blast people with ads at that point, then maybe you might lose it.

So for example, if Facebook blasted people with a number of ads they have in their feed in 2006, when it was just a college app, it was only college students using it, then maybe they wouldn't have had the same trajectory that they have now. So I think that you're right, you have to think about what your monetization is, but you have to focus on whatever the core metric you have at that time.

And Twitter was probably a little late in doing that.

Suzanne: Well, and just to say that getting acquired is a monetization strategy, but there is merit to identifying that potential strategy up front, because there's ways to be strategic about it and say, "Who would want this? Who would want this customer data or who would want all of this engagement? Who would see value in it?" And growing a user base precisely for the purpose of selling that user base is its own inherent strategy. I think Instagram was 40 million users when they got acquired. Whatsapp was 100 million users, so that's a lot of users actively engaging, and it does inform where you have to put your attention as a product manager.

Yohanes: Mm-hmm (affirmative).

Suzanne: Even earlier, when you were speaking about the differences between a freemium model and how you roll out experiments in something like that versus a subscription, I think it's the same in the context of metrics. On a freemium model, you're going to be very, very interested in what is the right set of features that I can put behind the paywall and how long till somebody is willing to convert to become a premium user. In a subscription model, regarding lifetime value, you gotta show up month on month on month as long as you want to keep that customer force, so how that informs what you do all day as a product manager and where you want to put value, it informs that directly.

Yohanes: Yeah, that's totally right. So it really depends on the sort of business you're building and also the second thing I thought of is it depends on the time, right? So if you think about when Facebook and Twitter were sort of growing up, that was a very heady day in terms of VC dollars. If you could get a big user base, that's all you needed. But now, just to sort of sustain yourself, then you have to some level of revenue nowadays.

So thinking about monetization is more important in 2007 then it was in 2010. So it really depends on the times and I totally agree that monetization has to be in the forethought of any given product in early days.

Suzanne: Alright, Yohanes, we do a segment here called Get the Job, Learn the Job, Love the Job. And you're saying you're getting asked these questions all the time, so you gotta be expert with these answers by now. What do you tell people when they come and solicit you at a dinner, "Hey, how do I get into product management? I want to do what you do, sounds cool."

Yohanes: Yeah, so the first thing is to be an expert at something. That means whatever you want to do. So when I wanted to work at Zynga, I was probably a level 60 or 70 player in Mafia Wars.I knew Mafia Wars up and down. Anybody had questions about Mafia Wars, I could answer them. I would email people about Mafia Wars, I'd email the engineering, so I was an expert about it.

Suzanne: Alright, so be an expert in the product, do like Yohanes, be a customer for years so that you can show up with a long list of grievances. And it is good advice for interviews as well, right? Which is it's impressive I think to a hiring manager to say, "I know your product, I've used it, here's why I love it. Here's where I think you could improve it, here's my version of what an improved product would like," and I think it's a good tell to see how receptive your leadership is, because a lot of the times, we exempt ourselves in the process of looking for work.

We think it's all about "Will somebody choose me?" And forget that we're also choosing people, right? We go to apply for a job, we're choosing people, so if I come into a conversation, I want to know, is this guy Yohanes, my director of product, my potential director of product, is he the kind of guy who's open to new ideas or is he the kind of guy who doesn't like me because I'm showing up to the table with ideas.

So it's equally, I think, an important test for looking for work.

Yohanes: Yeah, and then you made a great point that you choose the company too, right? So if you were to get a job, like I wanted to get a job at Zynga, I knew that they were a company that were looking for product managers like myself. You should go contact these companies. Any startup, or a lot of startups, are all having the same problem where they're trying to move so fast and they don't have enough bodies.

So if you show a big interest in something, then they could be willing to talk to you. There's no barrier between a CEO and some sort of secretary at smaller start ups, so if you have a great idea, if you're an expert at a product, then you should email the CEO and say, "Hey, have you thought about improving this? Have you thought about doing this?" And a lot of times, because you're talking about their product, they're open to listening to you. So I think you be an expert, you have some expertise in whatever you're doing.

So for me, I had an expertise in math and in analytics, so that sort of set me apart. And be interested in learning. Like I said, from people who had experience, like Alex and Nancy, I was willing to learn from them and shadow them and see how can I improve. And learning, if you're already enrolled, you can shadow experts in the field, or if you're not, then you can go to conferences, then you can read blogs, you attend meet ups and doing all those sort of things, will likely introduce you to the right people to help you find that first role.

Suzanne: You can also show up here at Audible headquarters and say, "Hey, Yohanes, I really loved your interview on that podcast and you sound like the kind of guy that I could work for and I took all your advice."

Yohanes: We have open recs , so if you're a product manager in the New York area, reach out. We have open headcount.

Suzanne: What about hard lessons learned? Do you remember any instances maybe early in your career as you were learning the lay of the land and you were like, "Well, I'll try to never do that again."

Yohanes: Yeah. We had this mythical best practice book, I think it was called The Launch Playbook. In our launch playbook, we had everything that was ever successful for us at Zynga and everyone always used The Launch Playbook. That was their go to for their roadmap and we launched a new game that had never been done at Zynga that was like a hidden object game and we had this thing in our launch playbook by adding a little iso character, or they call it a non-playable character, NPC, to your board and everyone will tap on that and everyone will engage with it and it will millions and millions of dollars.

So we did that for Hidden Chronicles and everyone hated it. And our engineering team had been telling me that the whole time, they were like, "This is a dumb idea," but one of my best friends, his name is Samir he was like, "You're going to ruin the game." He always told me everything I was going to do was going to ruin the game. But sometimes it made us millions of bucks.

But in this example it totally did not do well and it really taught me that there is no one size fits all solution, right? When you think about a product, you really have to think about it in the context of what you're building and the context of who your customers are and just because it was effective for X, Y and Z start up, doesn't mean it's going to be effective for you. So you always have to use your brain and be engaged when thinking about how something can work for your users.

If it's that different, then maybe it won't work for your users and you can't put on blinders and just think that it's going to work.

Suzanne: You obviously love products, you love games, you ended up at Zynga, you love books, you're here at Audible. What do you love about being a product manager?

Yohanes: That's a great point. You have to eat your own dog food. To be a product manager, you really have to love to learn about new things, to be creative, to be interested. You have to be willing to talk to people. We haven't really talked about it much in this interview, but product management is an interesting role because sometimes, especially when you're early in your career, you don't really have any power. If you say, "I want you to do this," they don't have to do it.

So you really have to be comfortable talking to people and convincing people that doing what you think is a good idea. So those soft skills, those qualitative skills are something that you have to be at least interested in doing. You could have different strengths, like being quantitative or having a background in programming so you can talk to engineering really well, but you really have to be a maven that can do across all those different groups from the business to engineering to the design team and be comfortable in those spaces.

Suzanne: You talked a little earlier about online learning opportunities, Udacity or places like General Assembly where I teach. Do you have any specific books, blogs, podcasts or other recommended resources that you'd like to mention here for our listeners? We'll put them up on our website at 100productmanagers.com/resources?

Yohanes: So for me it's more just platforms. I mean, I've read a lot of the books like Eric Ries’' Lean Startup. But more just on Medium, there's a lot of great stuff.

One secret that I don't know if a lot of people use is Quora. Quora has a ton of great content, especially if it's a particular problem, like if you're doing something for Material Design, which is Android, their design platform, or you want to figure out how to do ASO, or iOS 11. I'm going to do a blog post on that. Then Quora is a great place, or Medium is a great place.

The books are really good for the framework or the bones, but if you want to be up to date, then you have to read Medium. There's a guy named Nir …

Suzanne: Nir Eyal?

Yohanes: Yeah, Nir Eyal. He used to be a consultant for Zynga and HealthTap, another startup I worked at. But Nir Eyal, he has a great blog post. The guys have 500 startups, they have a lot of great content. They used to do a podcast, I think they stopped doing it now, but Quora and Medium, and some of the books that I've mentioned are sort of my go to resources.

Suzanne: And so do you write on Medium as well? You mentioned you got a blog post.

Yohanes: I do. I've written on Medium. Product School is another program. I've done a talk at Product School. I started to write at Medium years ago and then I didn't do it when it first came out. You try something and then you do and you're like, "Oh, I'm past it." But now that I've sort of become more senior, I've been thinking about writing more, just to sort of crystallize some of my thoughts, like we talked about before in terms of how to get into product management or how do you do ASO apps for optimization. iOS 11, which is an interesting problem that a lot of people are tackling now where I think that I might have something to say.

Suzanne: Alright. Well, just putting this out there, many of listeners don't know that we actually have a fast growing library of written content on the website, 100productmanagers.com, so if you're looking for a home to put some of that great advice, Yohanes, I'll catch up with you after, we'll talk more.

Yohanes: Okay.

Suzanne: Okay, last question for you. Is there a mantra, personal/professional life philosophy that you use to kind of guide you through the world that you want to share with us before we wrap?

Yohanes: I mean, it's kind of passe, or everyone says it, but the most important thing I've seen successful product managers do is eat their own dog food. All my biggest mentors or the most successful people I've ever met in terms of product are avid users of the thing that they're building. And that can be an API or that could be a game, or that could be Audible or whatever it is. So if you're not passionate about what you're building, then you probably shouldn't be working on it.

Suzanne: Beautiful. Yohanes Frezgi, Director of Product at Audible, thank you so much for joining us.

Yohanes: Thank you.