Product Management

N Order Consequence Modeling with Autonomous Vehicles

Summary

  • N Order Thinking is simply the ability to think beyond the superficial and consider not just an initial action and potential results, but the actions/potential results N Orders after your initial action/result pair.

  • N Order Thinking is important in product management because it allows a deeper understanding of the potential problem/solution space that can be interacted with and that you as a product manager must understand in order to reach a global maximum, not just a local maximum.

  • In my experience, N Order Thinking is present to varying degrees based on PM skill levels. N Order Thinking is an incredible differentiator in more junior product managers.

  • The below charts for both PM Skill Level and N Order Thinking with Autonomous Vehicles can also be downloaded as the original XLS here.

Detailed Discussion

  • What is N Order Thinking?

    • N Order Thinking is often explained in the context of Second Order or Third Order decision making. While that is useful when introducing the concept, as it is initially difficult to think more than 2 or 3 layers deep, this post is designed to broaden the concept so as to encourage product teams continue beyond the 2 or 3 derivatives this concept typically encapsulates. Hence the term “N” Order Thinking. 

    • At its simplest, an order of thinking is defined as a layer of observation, question, or statement.

    • Example of 1st Order

      •  Social media advertising is effective and efficient because that is where our demographics spend their time.

    • Example of 2nd Order

      • Why is social media advertising effective/efficient just because that is where our demographics spend their time? Is that also where/how they plan to purchase our good?

      • Social media advertising is effective and efficient because that is where our demographics are most receptive to our advertising and digital creative because we are a digital first brand targeting Gen X in coastal cities.

    • The differentiating factor between 1st and 2nd order thinking is going one layer deeper than is typical with three key questions:

      • Is the reality that has been presented in the initial statement broadly true (generalizable) or only in specific scenarios (not generalizable)?

      • Is the observation properly addressing correlation and causation issues?

        • Aside – This is staggeringly overrepresented in product related decisions unfortunately.

      • Is the question attempting to identify the root cause of a certain result or simply the one most readily and easily identifiable?

    • These three questions can be extended all the way down to N orders of thinking until the product manager feels that they have reached a point at which action can be taken.

  • Why is N Order Thinking important and relevant for product teams?

    • N Order Thinking is a necessity for continued and sustainable product innovation as it requires the thinker to identify root causes or customer problems that will move the needle, not just those that are easy to solve or will check off a KPI on number of JIRA tickets resolved that week.

    • It is also a useful tool in working through complex scenarios in which there are a multitude of stakeholders, potential customers, barriers to entry, and strategies for success. Not unlike a decision tree, N Order Thinking attempts to ferret out the most fundamental attributes of what on first approach seem to be intractably large problem spaces.

    • If your team, or at the very least your product leadership team, does not employ N Order Thinking it will be difficult to maintain product leadership in your category over time because you will remain focused on solving problems your customers know they have, not the ones they don’t realize could be solved. Solving problems customers don’t realize they have is the essence of disruption.

  • What is the cost of N Order Thinking? When should it be deployed? How can product teams utilize the concept?

    • N Order Thinking is useful, but shouldn’t be deployed in every scenario or by every product team member due to the time that it takes to think through each question to the Nth degree.

    • It should most often be deployed when entering new markets or creating new products to identify potential problem/solution pairs that may have been overlooked in initial product requirement documents.

    • It can also be deployed for products that have stagnated in their market and the product team is attempting to understand whether that product should be deprecated or can be applied to a new use case.

  • How can product teams learn to be better N Order Thinkers?

    • N Order Thinking is difficult and takes significant effort to understand and incorporate into the way in which your products are built.

    • Before going 10 layers deep in a brand new product, attempt the exercise in an existing product or feature that is stagnating and where the problem sample space is relatively narrow.

    • It can also be helpful to utilize diagrams, such as the Excel file example of Autonomous Vehicles at the beginning of this post, but which can be as simple as a decision tree drawn on a whiteboard.

Examples

Let’s apply N Order Thinking to what is currently an entirely open ended market: autonomous vehicles. While applying N Order Thinking is typically faster than in this example because your sample space will be constrained by your product, you will also encounter scenarios, similar to the one below, in which you are developing a new product or service with a wide-open set of possibilities. These are necessarily more complicated, and N Order Thinking helps to explore, document, and understand those possibilities.

 The following discussion will walk through one branch of the flowchart embedded below, which can also be found in the Excel file linked at the top. This diagram is by no means complete, but is designed to demonstrate how quickly and extensively N Order Thinking can explore new products and business lines.

  • On “zero” glance, or zero order, one might state that autonomous vehicles will change the way we drive and transport goods because there will no longer be a human driver.

    • This is true, but it doesn’t tell us why it will change the way we drive/transport other than the obvious implication that a human will no longer be driving those vehicles.

  • Moving to first order, we might state that there will two types of vehicles that full vehicle autonomy can be applied to: cars and trucks.

    • This is also true, but it doesn’t tell us how those autonomous vehicles will interact with their now non-drivers or the service providers that previously supported non-autonomous vehicles.

    • This is of course not comprehensive of the vehicle types that autonomous technology could be applied to, but is a good example of how to reduce the scope of an N Order analysis.

  • Moving to second order, examining just the car branch, we might now state that autonomous cars will impact the world in two ways: trip types and vehicle service providers.

    • Again, true, but have we delineated those trip types or what kind of support vehicle service providers currently provide and how those might be altered with autonomous vehicles?

  • Moving to third order, examining just the Trip Types, we might now state there are two trip types, personal and commercial.

    • Yet again, a true statement, but we haven’t yet answered one of the three questions above, which is whether we have identified the root cause / problem / potential business.

  • Moving to fourth order, examining just the Commercial Trips, we can identify that there are potentially three stakeholder groups impacted: Deliveries, Emergency Services, and Education.

    • We are getting closer as we are starting to identify individual business segments, but we aren’t yet at a level of specificity where we can identify problems to solve because we haven’t identified potential stakeholders.

  • Moving to fifth order, examining just the Deliveries branch, we can identify a specific enough industry that we can start asking intelligent questions.

    • In this case, if we look at delivering Mail, we can start to ask whether those types of Car -> Trip Type -> Commercial Trips -> Deliveries -> Mail will make sense if the underlying vehicle will be autonomous.

    • Getting to this level allows us to see the entire picture in which those decisions will be made (the other segments such as Food and Packages – E-Commerce) and whether those segments might be more appealing because they have higher/lower barriers to entry, lower / higher CLTVs, more / fewer competitors, etc.

    • In other words, your product team is unlikely to do well building a product that caters to solving the problems of a 4th order segment and all of its corresponding branches (IE – Deliveries -> Food & Packages – E-Commerce & Mail) because the use cases will be so different.

  • N Order thinking allows a bottom up approach to identifying potential business segments and their potential problems that can be turned into business / product lines.

 

Closing Commentary

  • As a product team, your highest priority should be on identifying the highest ROI problem/solution pairs in markets you currently address or could feasibly address.

  • N Order Thinking allows you to model those possibilities in a more quantitative way and hopefully address the three questions that can reduce product team decision quality explored in the Detailed Discussion section above.

  • The next step after getting to what you believe is the final order of thinking for your particular problem is asking the best questions possible, some of which will lead to their own N Order exercise, but will hopefully begin to provide metrics around the business problem with which your team started.

Product Management’s Hippocratic Oath

Summary & Rationale

  • The Hippocratic Oath is a series of ethical and moral principles that most physicians promise to uphold prior to beginning their professional career.

    • The principles generally require respect for prior knowledge, that medicine is both an art and a science, that not knowing is nothing to be ashamed of, that the privacy of patients is sacred, that doctors treat human beings and not statistics.

    • While the phrase “do no harm” is commonly associated with the Hippocratic Oath, it does not appear in the earliest versions, despite its continued popularity.

  • Technology has, and will likely continue to, advance faster than our society’s legislative or ethical capabilities. With this assumption, technologists must understand their role as vanguard in preventing technological harm and creating products that promote the common welfare.

  • The entire industry’s marketing is focused on how technology is a force for good while its actions are difficult to reconcile: dark patterns that trick users into sub-optimal choices, data mining that encourages unsustainable consumption, algorithms that create reinforcing echo chambers, and reward systems modeled after casino games.

  • Product Manager’s Oath in PDF form here.

The Rationale for a Product Management Oath

As both a enthusiastic builder and user of technology products, I have seen the below principles violated and abused on a regular basis, some of which I catalogue. I hope that the below axioms can help guide our industry towards a more responsible and sustainable future.

Many of the mainstream processes currently adopted by what are considered the best technology businesses in the world are not simply doing harm currently, but steal the trust of future users by increasing the chance of burdensome regulation or setting antagonistic precedents that become standard.

If we do not self-regulate, and build products as if our families were the ultimate users, society will rightfully constrain the operational latitude we currently enjoy.

The Product Manager’s Hippocratic Oath

  • Do No Harm - I will do no harm: to users, to the environment, to employees, to customers, to non-users, to future users, to technology, and to society as a whole.

  • Product

    • I will build, design, and construct each product as though I were the ultimate user and would endure all of its challenges as well as enjoy all of its benefits.

    • I will not design, either purposefully or inadvertently, product mechanics or gamification elements that encourage addictive or harmful degrees of product use.

    • I will ascertain whether creating a new product/feature, decommissioning a product/feature, or doing nothing at all provides users with the greatest value.

      • More on Empire Building and its consequences here.

    • I will remember that while some users choose to use my application, some have no choice or rely on it to the degree that they are unable to switch, and I will empathize with the user’s choice and/or lack of mobility, not exploit it.

    • I understand the power of defaults, both psychologically and emotionally, and I will not abuse human nature’s tendency to remain with the application’s suggestions.

  • Data Management

    • I will collect the minimum data required to maximize the value provided to users.

    • Of the data utilized, I will use it with respect and as if it were my own.

    • Of the data collected, I will properly secure it against misappropriation and malfeasance.

    • If in the course of business, data is misappropriated whether due to internal or external actors, I will do everything possible to make amends and notify users as soon as possible.

  • UI/UX

    • I will attempt to make the product as accessible to different users’ abilities, equipment, bandwidth and other circumstances as is possible.

    • I will not construct, either purposefully or inadvertently, pathways that guide the user to making choices that they would not make of their own accord or that are not for their betterment.

    • I will not create, either purposefully or inadvertently, user interfaces that guide individuals towards making individually optimal solutions at the expense of their neighbor.

    • I will endeavor to remember user preferences and incorporate them to the extent possible in the application’s design.

    • I will attempt to understand and plan for not just a product’s 1st order consequences, but any 2nd and 3rd order effects as best as I can.

Improving Uber Express Pool [Product Case Study]

Product Background

  • Uber Express Pool is a recent addition to the Uber ride hailing application that allows the user to pay less for their ride if they agree to:

    • Walk several blocks from their exact pick up location for pick up.

    • Allow drop off that is several blocks from their exact drop destination.

    • Wait up to 5 minutes for a driver to be assigned to their ride. I’ve also seen this as low as 2 minutes as Uber experiments with different wait / matching periods.

  • Rides can often be significantly cheaper than even Uber Pool if the user has a wider tolerance for walking and arrival time.

  • The ability for a rider to optimize according to time (Uber X) or money (Uber Pool -> Uber Express Pool) is incredible from a utility maximization standpoint and allows Uber to more optimally match individuals based on their money/time preferences for greater utility for all parties (driver included).

Potential Improvements

Uber Express Pool (UEP) isn’t perfect however, and could be improved in several ways, some of which are relatively simple and provide immediate value to the user, while others will require significant engineering work to solve. My comments below stem only from my experience as a user and do not take into account potential constraints that the Uber team is operating within.

  • Optimizing Pick Up & Drop Off Zones for Challenging Terrain

    • As anyone who has walked up San Francisco’s steep hilly streets, a pickup and drop off location even one block North or South can present quite a physical challenge.

    • Currently, based on my use of the product, Express Pool assumes that a pick up / drop off location is similarly/equally accessible to the user regardless of which direction / distance it is from the user’s current location. In other words, the pick up zone is calculated based on the user’s Current Location + Some (X) Distance from that location, combined with the most optimal pick up points that Uber has determined over time.

    • Unfortunately, this does not take into account the terrain/geography of the Pick Up Zone.

    • For example, for those of you familiar with San Francisco, the Intercontinental Mark Hopkins (Aka Top of the Mark) is located at the top of very steep hills in nearly all directions. However, as can be seen in the screenshots below, Uber Express Pool doesn’t take this into account.

    • It’s pickup zone for Express Pool extends two blocks N/S/E/W even though the 2 blocks south are incredibly steep descents. This can be seen in the Google Street View Images below.

    • A potentially readily available solution that would improve this aspect of UEP would be to calculate the estimate elevation change from one point to another, or at the least, use calculated walking time or Walk Scores. This would prevent UEP rides where the rider is going up or down steep terrain.

Google Street View facing South on Mason Street (Red Dot)

Google Street View facing North on Mason Street (Green Dot)

  • Optimizing Pick Up & Drop Off Zones w/Additional User Information

    • While I understand the objective of requiring the user to provide as little information as possible, UEP could improve the pick up and drop off experience by being more transparent regarding the locations of other passengers or attempting to better group passenger pick up points, particularly in busy areas.

    • Example 1 - Airport Pickups - Group by Zone

      • While there are now usually multiple airport pick up zones that the user can specify, on my previous UEP ride from the airport, I selected Section 1-2 and my matched Pooler selected zone 3-4, and we were both picked up seperately.

      • While I understand that Uber is trying to minimize walking distance with bags at the airport, it is actually far more convenient to group pickups at the same zone due to traffic and loading times.

    • Example 2 - Drop Offs - Allow User Input / Select Drop Offs

      • While this would likely only apply in the scenario in which a rider is being dropped off and the car is full (I.e. there is no possibility of the car being rerouted to pick up another passenger), it would be a net positive user experience to reveal the drop off location to the user as soon as possible.

      • There have been a number of occasions where first hand knowledge has led me to exit the ride earlier than the Drop Off Point because the drop off point is actually sub-optimal to where I ultimately want to go, and it is a better outcome if I exit somewhere else on the route.

      • Per the two photos below, it would have been faster for me to exit right as my UEP ride right before turning North on to Columbus, and walked the remaining distance, rather than wait in traffic for the right turn, and left turn.

      • Revealing the drop off point sooner, and allowing me to make a choice of a drop off point that is already on the route, would result in a shorter ride for me, shorter ride for my fellow Poolers and more revenue for the driver and Uber.

      • This logic could also be applied to user selectable Pick Up points.

Oakland Airport (OAK) PIckup Zones

UEP Drop Off Zone in San Francisco

UEP Drop Off Point Reveal

  • Optimizing Route Pricing by Allowing Uber X / Non Shared Rides to Add Riders During Ride

    • Another potential route to increase the number of pool rides would be allowing riders to select non-shared rides initially (Uber X / Black / Etc.) and if an individual is an optimal match for the route that the original rider is traveling, allow that rider to be added to the Uber X ride and give the original rider a discount just as they would have had if they had selected Uber Pool to begin with.

    • This would add liquidity to the pool of potential drivers for Uber Pool, as well as add certainty to the route when matching Uber Pool riders with potential drivers.

  • Optimizing Route Pricing by Allowing Shared Rides to “Upgrade” To Non-Shared Rides During Ride

    • Another potential route for increasing revenue would be to allow Pool/Shared rides to “Upgrade” to a non-shared ride after being picked up.

    • It could be that after pick up, those users realize they are later than they’d like, and would be prime targets for upselling to a non-shared ride that gets them there faster.

The Value of Product Management

Summary

  • At a certain point, every Product team will need to justify its existence, whether there is a new CEO that views Product as a cost center or product lines are deprioritized, it is important to both understand and be able to articulate the value product management provides.

  • At its simplest, a Product team (inclusive of Product Management, User Research, UI/UX, Design, Product Marketing) ensures:

    • Product / User Research - That the right problems are identified

    • Product - The right solutions are built to solve those problems

    • Design & UI/UX - That those solutions are designed in such a way that they are readily accessible

    • Product Marketing - That those solutions are marketed in such a way as to be perceived to solve the problems they are designed to solve

  • While we will explore each of the above separately, the value of product management specifically is in ensuring that the right problems are identified and the right solutions are built to solve those problems.


Detailed Discussion

First, let’s examine the value in identifying the right problem and define what the “right problem” means.

  • Right Problem Definition - Right problem in the context of product management means that the underlying user need, not simply the need the user superficially volunteers, has been identified to the greatest extent possible.

    • A famous, albeit likely falsely attributed, example is Henry Ford’s quote when referring to the creation of the Model T “If I had asked people what they wanted, they would have said faster horses.”

    • Regardless of the accuracy of the quote, it perfectly encapsulates the identification of the user’s superficial problem (that the existing mobility solution could be faster/better/cheaper) versus the identification of the user’s underlying need (improving a users ability to move independent of the method in which they are transported).

    • More simply, users didn’t want a faster horse, they wanted a faster method of transportation. The right solution to each of these problems is quite different.

    • This can also be seen as thinking through the problem from a first principles perspective.

  • Value in Identifying the Right Problem

    • While it might appear obvious why identifying the right problem is valuable, it is also important to be able to articulate the reasoning to stakeholders that may not be aware of product management methodology.

    • If your product team does not identify the right problem, there is a close to zero chance that the product team will derive the right solution by chance. This is the same fundamental reason that existing companies often fail to identify up and coming competitors and disruptors - they are focused on solving the right problem for a prior time period and have not adjusted for new technological/societal/preferential catalysts.

    • Identifying the right problem is even more valuable than identifying the right solution, because if you have identified the right problem, your product team will likely ultimately be able to pivot to the right solution via trial and error. If you have identified the wrong problem, your product team will be unable to create the maximum value possible because it will be attempting to figure out different ways to make a horse faster, not invent an entirely new method.

    • Finally, identifying the right problem can provide opportunities in markets that appear to be saturated with solutions, but upon closer inspection, are saturated with wrong solutions to the wrong problems.

Second, let’s understand what the right solution looks like and what the “right solution” means.

  • Right Solution Definition - Right solution in the context of product management means that the underlying user need, not just the need the user has identified, has been satisfied to the greatest extent possible, inclusive of the tradeoffs that the product team has identified in providing that solution relative to other stakeholders.

  • Value in Identifying the Right Solution

    • At its core, providing the right solution is about allowing your company to capture the greatest amount of value from your customers.

    • Providing the right solution to each customer is the root cause behind increases in customer lifetime value, Net Promoter Score, loyalty/retention and every metric that product teams seek to optimize.

    • In contrast to the criticality in identifying the right problem initially, it isn’t critical to identify the most optimal right solution at first, but only to produce a right solution based on the correct factors that influence the most optimal solution. If those factors are identified, and the cost of iteration can be minimized, iteration/trial and error is likely a more optimal path than upfront effort.

    • More simply, if the right solution to air travel is an airplane versus an airship, the optimal number of seats, their materials, and the ticket prices are all variables that can be iterated upon after the fact, while the choice of utilizing an airplane versus an airship is costly.


Examples

The examples below will explore instances where the right problem / right solution were identified/built (Good PM), as well as cases where the right problem/wrong solution, the wrong problem / right solution, and wrong problem / wrong solution were identified/built (Bad PM).

It is important to note that PM discussion/case studies are always in hindsight - it is much more difficult to identify these issues in real time, but without a Product team, there is a zero percent chance that those issues will ever be identified. When I refer to Bad PM, I specifically mean that these are examples where the right problem or right solution was not identified, not that those Product teams are of poor quality or low skill.

In addition, it is difficult to identify Wrong Problem/Wrong Solution and Wrong Problem/Right Solution examples because those companies have either ceased operations or pivoted to a new business model entirely.

A Note on Pivots (Wrong Problem, Right Solution) - It could be argued that some of the most successful technology companies have been the result of pivots, and that product teams should follow the adage of throwing things at the wall to see what sticks. This approach fails to consider the value that attempting to identify the right problem first can provide in terms of time and cost savings while overweighting a product team’s skill at identifying the right solution they have developed is misapplied. This scenario most likely results in total failure rather than a pivot to a successful product.