Balancing Discovery and Rapid Iteration: A Real-World Example

This article is originally written for airfocus blog.

Who hasn’t made mistakes as a product manager? It’s not just that time an “easy fix” exploded into a multi-sprints implementation; or when you had bet on one pain point over the others and didn’t gain enough traction after feature release; or the over-developed feature that should’ve been released months earlier… It’s in many things we do. This story is about mistakes and learning.

Hardcore discovery vs. ship it early

As a rule of thumb, we should mitigate possible risks when defining product features.

• How many customers will pay for it?
• Does it work for our overall business strategy?
• Can our teams build it without stacking nasty technical debt?
• Will end users find it useful? etc.

This can easily lead to analysing forever to find that absolute certainty and perfection, but not developing anything, the so-called analysis paralysis. At the other end of the scale is iteration — you’ll never have all the facts, especially not before a feature sits vividly in front of you and the customers, so ship early and find that balance as you go. Which one is better? I would have said “The truth lies somewhere in between” if not for this failure.

We recently shipped a new feature aimed at protecting the identity of vulnerable users. What it does is it anonymises the involved person’s full name on a frequently visited page, where comprehensive and sensitive information about an occupational incident is listed.

With the help of the new functionality, the identities of those involved in incidents don’t have to be compromised when people inform themselves about the incidents. This is not only the desire of our customers, but also seen as a strong reason to start properly using the software without data privacy concerns. We’ve had an internal release, where I explained and demo-ed in detail. The first round of feedback was positive. Until I was contacted by the Data Security Manager at our company.

It had turned out, ironically, that this data protection feature could pose a potential data privacy issue.

What could happen is, some users could attach files to the page where we anonymised the full name. And in those files, the full name can be compromised. This was an inconvenient fact. No time to dwell. It needed to be fixed. The immediate response was to define a solution, talk to the team, disturb their lunch break, and implement a hotfix, which was released the next day. Since it was an internal release, no customer data was at risk. But while things were in motion, this guilty product manager had only one thing in mind — I had thought of the attachment during the ideation phase, and consciously eliminated it from the first iteration.

My consideration was simple — very few people use the attachments, and there’s no reason for the attached files to be passed on, especially to any external party. So the plan was to focus on the main functionalities, release, and review when real-world feedback come in. Common product management. Apart from the fact that it’s not a good idea to iterate data privacy, which is either watertight or nothing. A valuable lesson that I have learnt.

So should you only develop with perfect certainty, or do something and make it better? I would still say, don’t analyse forever and chase the Holy Grail, but be conscious of the product’s nature. If it’s as serious as privacy, health, finance, etc, rather be a bit conservative.

Iteration for a mature product

There’s one more catch in the story. During the QA phase, we had actually captured the issue with the attachments. Strictly speaking, one of the many ways of attaching files. We fixed exactly that one, and that was it. In hindsight, I should’ve slammed the break and analysed the other ways of attaching files.

This product is a complex one, you may have noticed. It is. The labyrinth in logic makes it incredibly difficult to maintain, let alone innovate. It’s often the case that we have to scope out certain cases to be able to move forward.

Back in when I communicated the concept of this feature internally, I received several 😮and ❤️ emoji reactions. No one had hoped or thought that this concept could be done. In all honesty, perhaps myself too. When I first presented the tickets to the Dev team, I had the plan of pulling them back any second if the team posed doubt on feasibility. But the truth that mattered is, that the main functionality had already taken up too much mind space, and required a whirlwind of brain activities, since I hadn’t specified the behaviours of every attachment in the acceptance criteria, it was difficult for the team to be the expected gatekeeper or the last line of defence.

There won’t be a big talk on not to overcomplicate your product. I believe it can even be an inevitable step for a mature business.

But a reminder to all the Product Managers who manage a complex product, think carefully about what agility means to you, especially considering the level of complexity and interdependency in the product. Moreover, whether shipping the feature is the only way to get user feedback.

Consider tools to aggregate documentation

In the then upcoming sprint review, I shared my personal reflections on this event and had a follow-up chat with our Data Privacy Manager. We immediately came to the same conclusion that we shall identify privacy-related features and enable him to assist early. But only to realise how scattered our documentation is. To follow the discovery and ideation of any feature, he would have to go through four different tools, i.e. Miro, Figma, Confluence, and Jira. To an extent, it is incredibly difficult for a supporting role to stay on top of things.

So once again, it comes down to the product manager to spot early signs and proactively involve others when needed. For anyone who doesn’t have too much time and attention to spare, it’s safe to assume tools that are tailored for PMs (like airfocus) make the product manager’s life easier and make these processes less prone to mistakes.

Conclusion

Mistake happens. When it does, be solution-driven and get it out of the door fast. Afterwards, make time to reflect and share your learnings. Don’t forget to shield the team from the stress, they only need to focus on implementing the solution. Finally, never underestimate the importance of data privacy.