N Order Consequence Modeling with Autonomous Vehicles


  • N Order Thinking is simply the ability to think beyond the superficial and consider not just an initial action and potential results, but the actions/potential results N Orders after your initial action/result pair.

  • N Order Thinking is important in product management because it allows a deeper understanding of the potential problem/solution space that can be interacted with and that you as a product manager must understand in order to reach a global maximum, not just a local maximum.

  • In my experience, N Order Thinking is present to varying degrees based on PM skill levels. N Order Thinking is an incredible differentiator in more junior product managers.

  • The below charts for both PM Skill Level and N Order Thinking with Autonomous Vehicles can also be downloaded as the original XLS here.

Detailed Discussion

  • What is N Order Thinking?

    • N Order Thinking is often explained in the context of Second Order or Third Order decision making. While that is useful when introducing the concept, as it is initially difficult to think more than 2 or 3 layers deep, this post is designed to broaden the concept so as to encourage product teams continue beyond the 2 or 3 derivatives this concept typically encapsulates. Hence the term “N” Order Thinking. 

    • At its simplest, an order of thinking is defined as a layer of observation, question, or statement.

    • Example of 1st Order

      •  Social media advertising is effective and efficient because that is where our demographics spend their time.

    • Example of 2nd Order

      • Why is social media advertising effective/efficient just because that is where our demographics spend their time? Is that also where/how they plan to purchase our good?

      • Social media advertising is effective and efficient because that is where our demographics are most receptive to our advertising and digital creative because we are a digital first brand targeting Gen X in coastal cities.

    • The differentiating factor between 1st and 2nd order thinking is going one layer deeper than is typical with three key questions:

      • Is the reality that has been presented in the initial statement broadly true (generalizable) or only in specific scenarios (not generalizable)?

      • Is the observation properly addressing correlation and causation issues?

        • Aside – This is staggeringly overrepresented in product related decisions unfortunately.

      • Is the question attempting to identify the root cause of a certain result or simply the one most readily and easily identifiable?

    • These three questions can be extended all the way down to N orders of thinking until the product manager feels that they have reached a point at which action can be taken.

  • Why is N Order Thinking important and relevant for product teams?

    • N Order Thinking is a necessity for continued and sustainable product innovation as it requires the thinker to identify root causes or customer problems that will move the needle, not just those that are easy to solve or will check off a KPI on number of JIRA tickets resolved that week.

    • It is also a useful tool in working through complex scenarios in which there are a multitude of stakeholders, potential customers, barriers to entry, and strategies for success. Not unlike a decision tree, N Order Thinking attempts to ferret out the most fundamental attributes of what on first approach seem to be intractably large problem spaces.

    • If your team, or at the very least your product leadership team, does not employ N Order Thinking it will be difficult to maintain product leadership in your category over time because you will remain focused on solving problems your customers know they have, not the ones they don’t realize could be solved. Solving problems customers don’t realize they have is the essence of disruption.

  • What is the cost of N Order Thinking? When should it be deployed? How can product teams utilize the concept?

    • N Order Thinking is useful, but shouldn’t be deployed in every scenario or by every product team member due to the time that it takes to think through each question to the Nth degree.

    • It should most often be deployed when entering new markets or creating new products to identify potential problem/solution pairs that may have been overlooked in initial product requirement documents.

    • It can also be deployed for products that have stagnated in their market and the product team is attempting to understand whether that product should be deprecated or can be applied to a new use case.

  • How can product teams learn to be better N Order Thinkers?

    • N Order Thinking is difficult and takes significant effort to understand and incorporate into the way in which your products are built.

    • Before going 10 layers deep in a brand new product, attempt the exercise in an existing product or feature that is stagnating and where the problem sample space is relatively narrow.

    • It can also be helpful to utilize diagrams, such as the Excel file example of Autonomous Vehicles at the beginning of this post, but which can be as simple as a decision tree drawn on a whiteboard.


Let’s apply N Order Thinking to what is currently an entirely open ended market: autonomous vehicles. While applying N Order Thinking is typically faster than in this example because your sample space will be constrained by your product, you will also encounter scenarios, similar to the one below, in which you are developing a new product or service with a wide-open set of possibilities. These are necessarily more complicated, and N Order Thinking helps to explore, document, and understand those possibilities.

 The following discussion will walk through one branch of the flowchart embedded below, which can also be found in the Excel file linked at the top. This diagram is by no means complete, but is designed to demonstrate how quickly and extensively N Order Thinking can explore new products and business lines.

  • On “zero” glance, or zero order, one might state that autonomous vehicles will change the way we drive and transport goods because there will no longer be a human driver.

    • This is true, but it doesn’t tell us why it will change the way we drive/transport other than the obvious implication that a human will no longer be driving those vehicles.

  • Moving to first order, we might state that there will two types of vehicles that full vehicle autonomy can be applied to: cars and trucks.

    • This is also true, but it doesn’t tell us how those autonomous vehicles will interact with their now non-drivers or the service providers that previously supported non-autonomous vehicles.

    • This is of course not comprehensive of the vehicle types that autonomous technology could be applied to, but is a good example of how to reduce the scope of an N Order analysis.

  • Moving to second order, examining just the car branch, we might now state that autonomous cars will impact the world in two ways: trip types and vehicle service providers.

    • Again, true, but have we delineated those trip types or what kind of support vehicle service providers currently provide and how those might be altered with autonomous vehicles?

  • Moving to third order, examining just the Trip Types, we might now state there are two trip types, personal and commercial.

    • Yet again, a true statement, but we haven’t yet answered one of the three questions above, which is whether we have identified the root cause / problem / potential business.

  • Moving to fourth order, examining just the Commercial Trips, we can identify that there are potentially three stakeholder groups impacted: Deliveries, Emergency Services, and Education.

    • We are getting closer as we are starting to identify individual business segments, but we aren’t yet at a level of specificity where we can identify problems to solve because we haven’t identified potential stakeholders.

  • Moving to fifth order, examining just the Deliveries branch, we can identify a specific enough industry that we can start asking intelligent questions.

    • In this case, if we look at delivering Mail, we can start to ask whether those types of Car -> Trip Type -> Commercial Trips -> Deliveries -> Mail will make sense if the underlying vehicle will be autonomous.

    • Getting to this level allows us to see the entire picture in which those decisions will be made (the other segments such as Food and Packages – E-Commerce) and whether those segments might be more appealing because they have higher/lower barriers to entry, lower / higher CLTVs, more / fewer competitors, etc.

    • In other words, your product team is unlikely to do well building a product that caters to solving the problems of a 4th order segment and all of its corresponding branches (IE – Deliveries -> Food & Packages – E-Commerce & Mail) because the use cases will be so different.

  • N Order thinking allows a bottom up approach to identifying potential business segments and their potential problems that can be turned into business / product lines.


Closing Commentary

  • As a product team, your highest priority should be on identifying the highest ROI problem/solution pairs in markets you currently address or could feasibly address.

  • N Order Thinking allows you to model those possibilities in a more quantitative way and hopefully address the three questions that can reduce product team decision quality explored in the Detailed Discussion section above.

  • The next step after getting to what you believe is the final order of thinking for your particular problem is asking the best questions possible, some of which will lead to their own N Order exercise, but will hopefully begin to provide metrics around the business problem with which your team started.

LinkedIn's Utilization of Small Print as a Shield for Bad Behavior [Bad UX]

Product Background

  • While a significant violation of the Product Manager’s Hippocratic Oath, companies regularly shield bad behavior behind small type, Terms of Service updates sent at midnight on Fridays, and unclear or outright false information.

  • It is unfair to the user to both assume that they can completely understand the information presented to them and therefore make an informed choice or even care enough to spend time to do so.

  • While LinkedIn has a history of disregarding user’s privacy and disrespecting data fair use, they have quietly rolled out 3rd party ad retargeting based on LinkedIn contact and profile data without adequately asking for consent or informing users of the ramifications.

    • Last Week - LinkedIn violated data protection by using 18M email addresses of non-members to buy targeted ads on Facebook (Link, PDF)

    • 2015 - History of LinkedIn Dark Patterns (Link, PDF)

    • 2014 - LinkedIn Feature Exposes Email Addresses (Link, PDF)

  • A (poorly structured) argument against this post is whether LinkedIn’s behavior is any worse than any other company. While I would argue they are a significant offender, most other technology companies are similarly bad actors, but that doesn’t support lesser condemnation of LinkedIn, it simply supports condemnation of the behavior and industry as a whole.

UX Experience & Detailed Discussion

  • Upon a recent login to LinkedIn, I received the following notification, alongside what I would assume to be millions of other users.

  • Let’s break this update down piece by piece:

    • As it appeared initially to the user, all of the information from “Information you’ve shared with other companies” down to If you select “Agree” was hidden from the user, and was only revealed after clicking the carrot. This is a dark pattern to start.

    • Allow LinkedIn to keep showing you relevant jobs and ads.

      • This is nefarious because if LinkedIn is already doing it, why do they need my approval again? If they need my approval, then they must be doing something different. This sentence preys on users continuing to agree to what has occurred in the past.

      • In addition, it also implies that if you don’t agree, you may not be shown relevant jobs in the future, which most users wouldn’t like. However, this is not precisely what is occurring, it is just that LinkedIn won’t use additional data to make those recommendations more relevant, not that the recommendations won’t be relevant at all. These are two very different things.

    • Information You’ve Shared With Other Companies

      • “When You show interest (Like…” - The like here implies that there may be other ways in which I may show interest to a company and that I am agreeing to potentially unlimited, undetermined ways by agreeing to this. I could theoretically click on the link provided but there is no guarantee that I’ll find the information there.

      • “When you visit their websites, some companies may use tools (like cookies)…” - Again there is the implication that there may be other ways that companies use and that I am agreeing to whatever those are, which LinkedIn has no control over and how they will use/misuse my information.

      • “Also, if you provided consent directly to a company you trust to use your data for ad relevant, LinkedIn may rely on that consent when showing you ads.” - This translates to “If any company that has a direct incentive to obtain your consent to show you targeted ads is somehow able to obtain that consent, through whatever means or dark pattern they can, then LinkedIn will blindly trust that 1 or 0 signaling your consent and show you ads even if you click decline here that are more heavily personalized and targeted, which will have higher cost structures and will make us additional revenue.

      • Finally, highlighting the Accept & Continue button in blue, which is the default action color for users on the site, abuses user trust in LinkedIn and pushes users towards that selection by default, likely increasing conversion rates.

    • Most users likely will not understand that this one click will allow their data to be used across LinkedIn’s entire retargeting ad network. The gravity of that situation is not proportional to the copy or visuals of the dialogue.

LinkedIn Ad Opt In.png

Detailed Discussion

Upon further digging through their latest privacy policy (Link, PDF in case of Revisions) and LinkedIn’s additional primer on 3rd Party Ad Targeting (Link, PDF in case of Revisions) as of this post, there are a number of additional data points that are collected that users may not realize. While the below are by no means exhaustive of the data items I believe are being misused, they illustrate the complexity and effort required to understand how a users data is used, and how most users have no idea.

LinkedIn (and others) also do a poor job, either intentionally or unintentionally, describing the interactions of data points that they collect. As a result, while they may be following the letter of their privacy policy, they may be doing so either by omitting the scenarios where data is combined, or describing them poorly.

In the below examples, text from the policy is italicized, while my comments are just below:

  • Different Services & Device Tracking - Logging Sites after LinkedIn Visit

    • Section 1.4 - Cookies, Web Beacons & Other Similar Technologies

      • As further described in our Cookie Policy, we use cookies and similar technologies (e.g., web beacons, pixels, ad tags and device identifiers) to recognize you and/or your device(s) on, off and across different Services and devices.

      • Fair enough, but what does across different Services and Devices mean? Let’s go to section 1.5

    • Section 1.5 - Your Device & Location

      • When you visit or leave our Services (including our plugins or cookies or similar technology on the sites of others), we receive the URL of both the site you came from and the one you go to next. We also get information about your IP address, proxy server, operating system, web browser and add-ons, device identifier and features, and/or ISP or your mobile carrier. If you use our Services from a mobile device, that device will send us data about your location based on your phone settings. We will ask you to opt-in before we use GPS or other tools to identify your precise location.

      • I would hazard a guess that most users don’t understand that the site they go to after LinkedIn is also logged by LinkedIn. It is fairly standard practice to log which pages users are on before exiting, but I’ve rarely seen the logging of pages after exiting an application.

  • 3rd Party Advertising

    • Section 2.4 - Advertising

      • We do not share your personal data with any third-party advertisers or ad networks for their advertising except for: (i) hashed or device identifiers (to the extent they are personal data in some countries); (ii) with your separate permission (e.g., lead generation form) or (iii) data already visible to any users of the Services (e.g. profile). However, if you view or click on an ad on or off our site or apps, the ad provider will get a signal that someone visited the page that displayed the ad, and they may through the use of mechanisms such as cookies determine it is you. Advertising partners can associate personal data collected by the advertiser directly from you with our cookies and similar technologies. In such instances, we seek to contractually require such advertising partners to obtain your explicit, opt-in consent before doing so.

      • Potentially my favorite section in here, with key bits underlined.

      • We do not share your personal data with any third-party advertisers or ad networks for their advertising except

        • So LinkedIn doesn’t share my personal information with 3rd party advertisers, except that it does exactly that thing.

      • Advertising partners can associate personal data collected by the advertiser directly from you with our cookies and similar technologies. In such instances, we seek to contractually require such advertising partners to obtain your explicit, opt-in consent before doing so.

        • LinkedIn seeks to obtain that consent, but it is not clear whether they will allow targeting even if they do not obtain proof from the advertiser of explicit, opt-in consent.

    • How Businesses & Websites Can Use Third-Party Data for Advertising on LinkedIn

      • There is an additional and entirely separate page on Third Party data usage (the second link in the intro paragraph above and here), the text of which is not included in the primary privacy policy.

      • This policy essentially allows LinkedIn to serve retargeted ads across its entire network as well as across 3rd party sites with which it has a retargeting partnership.

      • While this is similar to other retargeting relationships/providers, what is not clear is what unique situations LinkedIn may be able to exploit due to the network information effects that LinkedIn is the recipient of.

      • For example, if a colleague uploads my email via the LinkedIn address book feature, is that email used for retargeting even though I never consenting to that being a retargeting data point? I would assume it is likely the case, but this scenario is not detailed implicitly or explicitly in these documents to my knowledge.

Product Management’s Hippocratic Oath

Summary & Rationale

  • The Hippocratic Oath is a series of ethical and moral principles that most physicians promise to uphold prior to beginning their professional career.

    • The principles generally require respect for prior knowledge, that medicine is both an art and a science, that not knowing is nothing to be ashamed of, that the privacy of patients is sacred, that doctors treat human beings and not statistics.

    • While the phrase “do no harm” is commonly associated with the Hippocratic Oath, it does not appear in the earliest versions, despite its continued popularity.

  • Technology has, and will likely continue to, advance faster than our society’s legislative or ethical capabilities. With this assumption, technologists must understand their role as vanguard in preventing technological harm and creating products that promote the common welfare.

  • The entire industry’s marketing is focused on how technology is a force for good while its actions are difficult to reconcile: dark patterns that trick users into sub-optimal choices, data mining that encourages unsustainable consumption, algorithms that create reinforcing echo chambers, and reward systems modeled after casino games.

  • Product Manager’s Oath in PDF form here.

The Rationale for a Product Management Oath

As both a enthusiastic builder and user of technology products, I have seen the below principles violated and abused on a regular basis, some of which I catalogue. I hope that the below axioms can help guide our industry towards a more responsible and sustainable future.

Many of the mainstream processes currently adopted by what are considered the best technology businesses in the world are not simply doing harm currently, but steal the trust of future users by increasing the chance of burdensome regulation or setting antagonistic precedents that become standard.

If we do not self-regulate, and build products as if our families were the ultimate users, society will rightfully constrain the operational latitude we currently enjoy.

The Product Manager’s Hippocratic Oath

  • Do No Harm - I will do no harm: to users, to the environment, to employees, to customers, to non-users, to future users, to technology, and to society as a whole.

  • Product

    • I will build, design, and construct each product as though I were the ultimate user and would endure all of its challenges as well as enjoy all of its benefits.

    • I will not design, either purposefully or inadvertently, product mechanics or gamification elements that encourage addictive or harmful degrees of product use.

    • I will ascertain whether creating a new product/feature, decommissioning a product/feature, or doing nothing at all provides users with the greatest value.

      • More on Empire Building and its consequences here.

    • I will remember that while some users choose to use my application, some have no choice or rely on it to the degree that they are unable to switch, and I will empathize with the user’s choice and/or lack of mobility, not exploit it.

    • I understand the power of defaults, both psychologically and emotionally, and I will not abuse human nature’s tendency to remain with the application’s suggestions.

  • Data Management

    • I will collect the minimum data required to maximize the value provided to users.

    • Of the data utilized, I will use it with respect and as if it were my own.

    • Of the data collected, I will properly secure it against misappropriation and malfeasance.

    • If in the course of business, data is misappropriated whether due to internal or external actors, I will do everything possible to make amends and notify users as soon as possible.

  • UI/UX

    • I will attempt to make the product as accessible to different users’ abilities, equipment, bandwidth and other circumstances as is possible.

    • I will not construct, either purposefully or inadvertently, pathways that guide the user to making choices that they would not make of their own accord or that are not for their betterment.

    • I will not create, either purposefully or inadvertently, user interfaces that guide individuals towards making individually optimal solutions at the expense of their neighbor.

    • I will endeavor to remember user preferences and incorporate them to the extent possible in the application’s design.

    • I will attempt to understand and plan for not just a product’s 1st order consequences, but any 2nd and 3rd order effects as best as I can.

Improving Uber Express Pool [Product Case Study]

Product Background

  • Uber Express Pool is a recent addition to the Uber ride hailing application that allows the user to pay less for their ride if they agree to:

    • Walk several blocks from their exact pick up location for pick up.

    • Allow drop off that is several blocks from their exact drop destination.

    • Wait up to 5 minutes for a driver to be assigned to their ride. I’ve also seen this as low as 2 minutes as Uber experiments with different wait / matching periods.

  • Rides can often be significantly cheaper than even Uber Pool if the user has a wider tolerance for walking and arrival time.

  • The ability for a rider to optimize according to time (Uber X) or money (Uber Pool -> Uber Express Pool) is incredible from a utility maximization standpoint and allows Uber to more optimally match individuals based on their money/time preferences for greater utility for all parties (driver included).

Potential Improvements

Uber Express Pool (UEP) isn’t perfect however, and could be improved in several ways, some of which are relatively simple and provide immediate value to the user, while others will require significant engineering work to solve. My comments below stem only from my experience as a user and do not take into account potential constraints that the Uber team is operating within.

  • Optimizing Pick Up & Drop Off Zones for Challenging Terrain

    • As anyone who has walked up San Francisco’s steep hilly streets, a pickup and drop off location even one block North or South can present quite a physical challenge.

    • Currently, based on my use of the product, Express Pool assumes that a pick up / drop off location is similarly/equally accessible to the user regardless of which direction / distance it is from the user’s current location. In other words, the pick up zone is calculated based on the user’s Current Location + Some (X) Distance from that location, combined with the most optimal pick up points that Uber has determined over time.

    • Unfortunately, this does not take into account the terrain/geography of the Pick Up Zone.

    • For example, for those of you familiar with San Francisco, the Intercontinental Mark Hopkins (Aka Top of the Mark) is located at the top of very steep hills in nearly all directions. However, as can be seen in the screenshots below, Uber Express Pool doesn’t take this into account.

    • It’s pickup zone for Express Pool extends two blocks N/S/E/W even though the 2 blocks south are incredibly steep descents. This can be seen in the Google Street View Images below.

    • A potentially readily available solution that would improve this aspect of UEP would be to calculate the estimate elevation change from one point to another, or at the least, use calculated walking time or Walk Scores. This would prevent UEP rides where the rider is going up or down steep terrain.

Google Street View facing South on Mason Street (Red Dot)

Google Street View facing North on Mason Street (Green Dot)

  • Optimizing Pick Up & Drop Off Zones w/Additional User Information

    • While I understand the objective of requiring the user to provide as little information as possible, UEP could improve the pick up and drop off experience by being more transparent regarding the locations of other passengers or attempting to better group passenger pick up points, particularly in busy areas.

    • Example 1 - Airport Pickups - Group by Zone

      • While there are now usually multiple airport pick up zones that the user can specify, on my previous UEP ride from the airport, I selected Section 1-2 and my matched Pooler selected zone 3-4, and we were both picked up seperately.

      • While I understand that Uber is trying to minimize walking distance with bags at the airport, it is actually far more convenient to group pickups at the same zone due to traffic and loading times.

    • Example 2 - Drop Offs - Allow User Input / Select Drop Offs

      • While this would likely only apply in the scenario in which a rider is being dropped off and the car is full (I.e. there is no possibility of the car being rerouted to pick up another passenger), it would be a net positive user experience to reveal the drop off location to the user as soon as possible.

      • There have been a number of occasions where first hand knowledge has led me to exit the ride earlier than the Drop Off Point because the drop off point is actually sub-optimal to where I ultimately want to go, and it is a better outcome if I exit somewhere else on the route.

      • Per the two photos below, it would have been faster for me to exit right as my UEP ride right before turning North on to Columbus, and walked the remaining distance, rather than wait in traffic for the right turn, and left turn.

      • Revealing the drop off point sooner, and allowing me to make a choice of a drop off point that is already on the route, would result in a shorter ride for me, shorter ride for my fellow Poolers and more revenue for the driver and Uber.

      • This logic could also be applied to user selectable Pick Up points.

Oakland Airport (OAK) PIckup Zones

UEP Drop Off Zone in San Francisco

UEP Drop Off Point Reveal

  • Optimizing Route Pricing by Allowing Uber X / Non Shared Rides to Add Riders During Ride

    • Another potential route to increase the number of pool rides would be allowing riders to select non-shared rides initially (Uber X / Black / Etc.) and if an individual is an optimal match for the route that the original rider is traveling, allow that rider to be added to the Uber X ride and give the original rider a discount just as they would have had if they had selected Uber Pool to begin with.

    • This would add liquidity to the pool of potential drivers for Uber Pool, as well as add certainty to the route when matching Uber Pool riders with potential drivers.

  • Optimizing Route Pricing by Allowing Shared Rides to “Upgrade” To Non-Shared Rides During Ride

    • Another potential route for increasing revenue would be to allow Pool/Shared rides to “Upgrade” to a non-shared ride after being picked up.

    • It could be that after pick up, those users realize they are later than they’d like, and would be prime targets for upselling to a non-shared ride that gets them there faster.