Understanding and onboarding with product team using ML

Recently, I felt a need to write a self bookmark post explaining why and how to understand a product that you are getting introduced to.

From my past experience, I think a great understanding comes from going through the following three sets of activities –

• Onboarding (getting to know your product & users)
• One ship cycle of some new feature
• ..and one critical firefight (with customers / legal escalation OR responding to competition)

In this post, I’m trying to put down a few thoughts on onboarding and customer interaction.

Onboarding is one of the most important aspects of understanding core product value. I’ve personally seen situations where team paid huge penalties for improper or no-onboarding onboarding process. If you are onboarding a PM, sub-optimal onboarding could have a negative impact on short term product roadmap. Cost of hiring is huge, and sucky onboarding compounds that cost for product teams and individuals.

Onboarding to a product that uses machine learning (ML) could add extra twist. Many times it’s not clear if machine learning is a product or a feature.

Few observations:-

For a mature product or product line, there are a couple of interesting scenarios where ML is being introduced:-

• Adding machine learning is an afterthought for old mature product. Value add is usually questionable. Secondly, these features might show some resistance in terms of customer adoption. Designers are making such features optional (not to mess up with core product value) leading to a discover-ability problem.
• In many teams, lack of correct skillset to implement ML creates organizational dependencies, slowing and risking ship cycles.
• Current product success metric is possibly not taking into account if and how these new ML features are helping customers or business.

If you are getting introduced to a V1 product with no PMF (product market fit) yet:

• Understanding if ML is a feature or a product is even more critical. For e.g. Gmail’s smart reply is feature. On the other hand, it’s hard to imagine Google translate wold exist w/o machine learning advances we have today
• Is the job product promising to do feasible without ML? (with similar outcomes to customers). There are many products blotted with ML features adding no value
• Lastly, what is the acceptable and desirable quality of ML features to your product. Important- Sad but true, policy and regulations are far more important than quality of ML system output and what it can do for your users.

---

Here is a general framework that I use myself or onboarding new team members

1. Understand product in isolation –

• Use your product — I can not stress this enough. Just use your product convering all the use cases. Easier said than done, many don’t do it. Observing a user or group of users use your product is valuable but not a substitute for you using it. While you are taking your product for a test drive, the few important things to watch for are –

1. How efficiently product is delivering the value to users and what I like to call is Time to value (TTV)
2. What is onboarding experience for your customers- Are they getting what they bought product for OR they are going through series on tutorials understanding technical stuff? are they able to find features in context, time and flow?
3. Does the product and its features really stand for your company brand and brand promise?

After doing above, stack rank the problems your product solved for you. If you are not sure, continue going through use cases till you get this. This is the key point.

• Understand analytics, social sentiment and media perception about your product. This will help you with 3 things

1. Knowing product’s sweet spot and are customers using it the way it was designed for? This will help prioritize future investments.
2. Analytics will help you understand friction points and low hanging opportunities.
3. Analytics will put light on trends for growth and how user feedback has impacted product direction so far.

• Get to know technical stack

1. This will help most feasibility argument while building new features
2. This will also clarify “why” things are built the way they are. Help expose dependencies on other products and services, licencing costs, time to iterate and organizational dependencies.
3. For machine learning features, understanding where “Data” comes from for your ML features. This is very critical and I think its fun.
4. Understanding technical details will build credibility with technical staff.

2. Understand Internal dynamics

It takes a lot to build a successful product than just an idea, team, timing and market. Understanding internal team, level of trust and dynamic is valuable to know how team and operates internally and responds to external input.

• If you are joining a team where initial team members are still around, that’s a great opportunity to know product genesis. Ask them about pivots, change in customer landscapes, market and priorities, including dead competition in similar space.
• Product importance — as a product leader, understanding importance of product to organization, revenue and to top leadership is super important. This will help clarify leverage you can enjoy and general operating principles followed in the organization.
• Product principles and anti-principles — Understanding and establishing process on how decisions are made will make you a predictable leader. You will also understand why certain decisions are made the way they were. As a side note — Company, team and product principles are always changing as markets and products evolve. In many cases if they don’t, it’s a great recipe for becoming irr-relevant and vulnerable to disruptions.
• If the product is monetized, get to know all the important metrics that company relies on from your product. Get to know which key indicators you can experiment with and which are absolutely un-compromisable. Talk to finance people. There is a lot more they will tell you about your customers that you would know from anyone else. This is a well kept secret 🙂
• Understand you internal competition — yes, internal competition. In big companies, there are high chances that you are competing against something or organizational priorities for investments.
• Talk to other teams, marketing, legal and HR — this is obvious but super helpful.

3. Understand external dynamics –

• Talk to your customers, Talk to your customers, Talk to customers and ask why they decided on your product. Funny it sounds, but many times you will be surprised they answers you get.
• Pricing, positioning and packaging — You are mostly aware of these 3P’s from your research before joining the team. If not, understand how marketing has positioned the product and the story they tell to your customers. How customers perceive and what are they really paying for.
• Understand direct and indirect competition — so far with all the above data points, you will have very different understanding of how product stacks against competition.
• Understanding where / which products your users are churning to and what is causing you +ve churn. This will help you strategize what’s important to your customers.
• Talk to your partners and distributors — There is a very fine balance between a great product and well distributed product. In many B2B segments better distribution wins.
• In theory, most metrics add value to your understanding. Its equally important to know how they are calculated and derived. Without understanding data and context in which your product will be used, precision metrics with 95% value is useless.
• Lastly- understand if you are charging enough to your customers. There is possibly a easy win here. Most importantly, know if your sales team is targeting the right customer segments given the maturity of your product.

I think above should make a good onboarding guide and help get started. As a PM, most important outcome of this exercise should prepare you to strategize long and short term compelling vision for your business.