A Product Strategy for Each Swimlane

Gibson: So far, I have focused on defining the overall product strategy for a company. It's essential, however, that each product leader within an organization also articulate their pod's strategy. I'll provide an example from the personalization team at Netflix, circa 2006.

At the time, Todd Yellin was the product leader of a group of engineers, designers, and data analysts focused on personalization. Todd understood that the goal for the product team was to improve retention, and hypothesized that a personalized site experience would help to accomplish this. His top line proxy metric was the percentage of new members who rate at least 50 movies in their first six weeks with the service. The theory was that if members were willing to rate lots of movies, it meant they valued the results of their rating, personalized movie choices.

Here is a simplified version of Todd's strategy, along with proxy metrics and projects. Todd had three high-level strategies. The first was explicit data. That's encouraging rating. The proxy metric was the percentage of new members who rate at least 50 movies in their first six weeks. In projects or tactics against explicit data strategy, lots of ways to get ratings as well as other data like demographics.

Implicit data was the second strategy in this area. The proxy was the percentage of members who add at least six titles to their queue each month. The theory was if we looked carefully at what titles they add to the queue, that would give a signal of the kinds of movies that they like. This implicit strategy in an era of streaming, it's easier to understand if you watch a movie for 10 minutes and then quit, there's an implicit signal that you don't enjoy that kind of movie. An example project for implicit data would be to inform the personalization efforts with knowledge of what titles you've chosen to add to your queue or some of this behavior around quitting as you begin to stream.

The third strategy in the personalization area, once you understood the movie taste of your customers and the data around movies, the job was matching algorithms that magically connect customers with the movies that they'll love. Our proxy metric for this is called RMSE, or root mean squared error. It's basically the difference between what you expected the member to rate and their actual rating. This is one of those things where the closer you get to zero, the better.

Some projects that we engaged in to improve our matching algorithms, we did collaborative filtering, the most successful early algorithm, but we also launched the Netflix Prize to get from one engineer to essentially 5,000 engineers. Then we created a new algorithm called category interest. These were projects against our matching algorithm strategy.

I'll give you a little bit more sense about the thinking at the time. We believed that Netflix would gather lots of taste data from its members, either through explicit ratings or implicit behavior. Titles they hovered over might be another example. The team would gather lots of data about each movie, genre, actors, directors, whether it was a feel-good, leave your brains at the door comedy, and other detailed attributes of each film. Then given an in-depth knowledge of members' movie tastes and data about each movie, matching algorithms would connect members with personalized movie choices.

Over time, Netflix moved the proxy metrics, and in the long-term proved that a highly personalized experience improved retention. Today, I describe the personalization effort at Netflix as a 10 year leap of faith. It took more than a decade to prove that personalization improved retention. A steady improvement in the proxy metrics, however, provided a strong signal that we would eventually succeed.

In 2005, I had a product leader focused on each of the following areas: personalization, new member acquisition, social (our friends experience), DVD merchandising, help and account, used DVD sales, and advertising. Each product leader assigned to one of these seven different areas worked with a dedicated team of engineers, designers, and data analysts, and each could articulate their own high-level metric, along with the key strategies, proxy metrics, and projects for their area.

Here's product strategy exercise number nine. For each swim lane in your product organization, identify the proxy metric each product leader will move, their North Star metric for their pod, along with strategies, proxy metrics, and projects. Ideally, this work is done by the product leader for each swim lane.

In the next essay, we'll input your high-level hypotheses and projects into a four-quarter, rolling map ... rolling roadmap to demonstrate how all this stuff fits together. Coming next, essay number seven: The Product Roadmap.

Suzanne: I'm following along. I'm doing my homework, doing the assignment. Then you throw me this curve ball in this essay which is how the proxy metric essentially splinters apart across the squad themselves.

If I understand correctly what it reminds me of a little it is OKRs and this idea that at the top you've got the biggest objective and the two or three key results that give that dimensionality. Then each sort of executive or department leader below that grabs a key result and that becomes their objective and so on. Is this really what's happening here, that the product strategy if it touches multiple teams or multiple squads as it might in a company like Netflix, or as it did, does that then mean that each of those product leaders needs to take that proxy metric and smash it apart with a sledge hammer, and come up with yet another smaller but more appropriately measurable-

Gibson: Sometimes. That's what I'm trying to say. Imagine you are the product leader at a company. You got a lot of work to do. I say, "Hey, create your overall product strategy with your high level metrics proxies, the projects, the tactics." But recognize that each of the folks that works for you owns a swim lane and you could ask the to do all that same work for their swim lane. So it's a level down.

What I'm trying to do is give a ton of ownership to those product leaders in the swim lane and get them to think like the overall product leader. They love it. I'm encouraging them to think strategically. I'm encouraging them to think long-term, and then I'm hoping that they will move their high level metric. If they do we'll give them more resources. I'm trying to also de-politicize things.

Frankly, these are the kinds of assessments that I make at these quarterly product strategy meetings that I'll talk about a little bit later. The tricky part here is precise people, they want this all to fit into tight puzzle pieces. I never can quite do that. One thing that people notice is I tend to organize a swim lane against each strategy. It's true. I never noticed that but that seemed like a reasonable way to organize people. Of course there's tons of anomalies.

But that was helpful to me. There's no right answer in organizations. Like anything you have to experiment, and whatever the weakness of it you have to make up somewhere else.

Suzanne: Well, it sounds like the key takeaway on this is ownership. That's great. One way that you can do it is as you suggest, is to say, "Look. Let's not have competing product strategies. Let's have our different squads have their own strategies within their areas of influence."

Gibson: Yes.

Suzanne: But more than anything else, or if you have a strategy that cuts across, just it's that boulder, to rock, to pebble approach to say somebody bites a piece off at every step of the way.

Gibson: Yep. Yep. Very consistent with that model. Yeah.

Suzanne: Then what is it, you'll kill them if they're wrong?

Gibson: Boy.

Suzanne: He's going to regret telling me all his secrets.

Gibson: I now think of it as sort of radical job clarity. If you have a metric that defines your job, you know how to evaluate your work. Then you're just asking the question, okay how do I move that metric? You can be maniacally focused on that thing. That's kind of how I operated, but honestly I am not a hard ass.

The teams were supportive of each other et cetera. I was just trying to make sure that everybody had a perspective and understood what their job was. But it's like a joke that hasn't died for me because I keep telling it. What am I thinking? Okay, go ahead.

Suzanne: This is musings on episode six.