Product Engineering Challenges: How to Embrace Digital Transformation in a Continuously Changing World with Christopher Van Horn, Delivery Manager at Kanda Software
According to a recent study, the market for digital product engineering services is projected to reach a staggering $1.62 trillion by 2031. These figures become even more impressive in the era of digital transformation, where developing a software product requires careful planning and decision-making. To stay ahead, businesses must prioritize meticulous planning, strategic decision-making, and innovation. And as organizations are increasingly asked to do more with less, it becomes crucial to measure, track, assess, and analyze the results of their work with every release.
But it doesn’t stop there. True success lies in pushing the boundaries of innovation, exploring uncharted territories, and consistently exceeding customer expectations. It’s about being at the forefront of technological advancements and ensuring that every product release leaves a lasting impression.
In this article, we will deep dive into the challenges of the digital product design cycle. To provide an expert perspective, we’ll be sharing invaluable insights from Christopher Van Horn, our dedicated expert and Delivery Manager at Kanda.
What is product engineering?
Product engineering encompasses all stages of the product development cycle, including ideation, prototyping, development, testing, and deployment. Typically, the product engineering life cycle consists of the following stages:
Stage 1. Ideation of the product
During this stage, the team, consisting of engineers, design specialists, and a product delivery manager, generates ideas and formulates the concept of the product.
It is worth noting that it is imperative to have a product strategy that encompasses various aspects, including acquisition, improving adoption, and a vision for the product at the onset. This strategy guides the team in aligning their efforts with the overall objectives. With each release, it is essential to ensure that the work being done supports the strategy.
As the team progresses through the product development process, having these objectives in mind helps them make the best possible decisions.
Stage 2. Product architecture development and design
Once the idea is formulated, the product engineering team proceeds to develop the architecture and overall design of the product by defining its structure and features.
During the product design stage, the product delivery manager identifies the features that would help solve the business and product goals, while the design and engineering team focuses on how to design these features to achieve those goals. This involves incorporating user-friendly visual and interactive design elements.
Stage 3: Testing the product
Prior to the product launch, the product engineering team works diligently to ensure the product’s quality, functionality, and reliability. When discussing this stage, Christopher Van Horn emphasizes the importance of quality and reliability during the testing phase.
I strongly believe in integrating quality assurance (QA) within the team itself. This approach involves comprehensive and repeated testing, including unit tests and automation. It is crucial to incorporate automation into each story and ensure its completion before releasing the functionality. Additionally, it is essential to provide the team with knowledge about the personas involved, enabling them to make well-informed decisions regarding which issues to address and where to focus their efforts. Furthermore, it is of utmost importance to include performance testing to guarantee scalability.
Quality should be an integral part of the process since resolving issues after release is significantly more costly and it has a negative impact on both the brand and the product.
Here’s a step-by-step approach, suggested by Christopher, to ensure effective testing and validation throughout the product development process
1. Early stage validation
Validate the value and usability of the problem you are solving. If necessary, be prepared to pivot and adjust your approach accordingly.
2. Design stage
Use varying fidelity mocks to assess whether the product is meeting the desired objectives.
3. Development stage
To find success in this stage, follow these suggestions:
- Empower the team with a strong understanding of the problem space, target personas, and product objectives.
- Embed a Quality Engineer (QE) within the team to design approaches and set standards aligned with the objectives. Begin this work in parallel with development.
- Foster collaboration, allowing the team to make informed decisions about priorities.
- Guide the team with QE expertise in foundational aspects and automation strategies.
- Collaborate with a QE to develop and maintain test automation, APIs, frameworks, and infrastructure. Coach developers to write automated test cases that adhere to guidelines.
- Prevent scope creep and ensure developers are aware of limits, scale, performance, security, environmental constraints, testability, and serviceability criteria with the assistance of a QE.
- Continuously validate and modify testing strategies as the product evolves.
- Adjust tests as new features are added or updated, leveraging real-world usage data for further refinement.
The last thing you want is to build your product and realize that you completely missed the mark on the problem being solved. This structured approach can assist product engineering teams in enhancing product quality by identifying and resolving issues at the earliest stage possible.
4. Migrating and porting the product
When it comes to digital product engineering, the stage of migrating and porting the product refers to the process of adapting the product to different platforms and environments.
This can involve making the product compatible with various operating systems such as iOS and Android, ensuring it can be accessed on different devices. It can also involve integrating the product with different systems or platforms, such as Salesforce (SFDC) or ServiceNow, to expand its capabilities and reach.
Additionally, the migration process may include transitioning the product from one cloud platform, such as AWS, to another, like Google Cloud Platform (GCP), to leverage different features and benefits. The goal is to make the product more accessible and adaptable, reaching a wider audience and maximizing its potential.
5. Product support
During the course of product operation, product support should be involved every step of the way, as this helps the team to stay up to speed on the product deliverables and makes it easier to provide effective support. It helps provide input on what is needed to best support the product, such as data, logs, and other relevant information.
The product support team often serves as the immediate voice of the customer, and they are the ones who will surface repeated challenges that need to be addressed. By fixing these issues, it not only improves the overall product adoption but also leads to lower support costs and fewer support calls. The product support team’s involvement throughout the product life cycle ensures that customer needs are met and any challenges are resolved promptly, resulting in a more successful and satisfying customer experience.
6. Product maintenance
This stage involves constant product improvement and maintenance, as well as update management and feedback review. The team also conducts performance and load testing to ensure that the product scales and can support customer needs.
Achieving these objectives is facilitated through ongoing monitoring of metrics that help assess the effectiveness of the team’s deployments. The team closely monitors whether the deployed solutions are functioning as expected, including tracking adoptions and identifying any increase in support tickets.
This evaluation enables the team to gauge the success of their approach and make necessary adjustments.
Challenges in product engineering innovation
As the field of product engineering continues to evolve, teams engaged in innovation projects may encounter a range of challenges. The following challenges encompass various aspects of the product engineering life cycle:
Keeping pace with rapidly evolving technologies
Staying up-to-date with the swift advancements in technology poses a challenge, especially for companies that prioritize research and development. Incorporating new technologies into products and staying ahead of the competition can be demanding.
Managing complexity and integration
As products become more intricate and interconnected with other systems, effectively managing multiple components and ensuring seamless integration becomes a significant challenge.
Ensuring quality and reliability
With the increasing complexity of products, it becomes crucial to ensure that they are reliable and meet user expectations. This challenge compounds when developing products that incorporate novel technologies or operate in complex environments.
Controlling costs
Product development can be costly, requiring effective cost management strategies while maintaining the delivery of high-quality products. Striking a balance between cost control and product excellence poses a notable challenge.
Safeguarding intellectual property
As products become more complex and reliant on advanced technologies, safeguarding intellectual property becomes a critical concern. Companies must implement robust measures to protect their valuable IP assets.
Managing regulatory compliance
In regulated industries such as healthcare and finance, products must adhere to rigorous regulatory requirements before entering the market. Ensuring compliance with these regulations poses a significant challenge for companies.
Christopher provides valuable insights on this topic:
When it comes to collaborating with the federal government, it is crucial to have a clear understanding of the entities you intend to work with, including state and local agencies, federal agencies, and the Department of Defense (DoD). Each level of government has specific requirements that must be followed, along with certifications that need to be obtained. Achieving compliance demands a dedicated effort and a substantial commitment from your organization. You will need the support of your team members and may also involve Third-Party Assessment Organizations. Furthermore, it requires the establishment of well-defined processes and security policies.
Collaborating with internal teams and external partners
In the context of working with internal teams and external partners, it is important to recognize that product engineering goes beyond its core functions. While product engineering is indeed crucial and requires team buy-in, it is equally essential to have sales teams ready to sell, customer success teams prepared for cross-selling and making recommendations, support teams ready to provide assistance, and adequate resources for training, documentation, and marketing.
Apart from that, product engineering often entails collaborating with various external partners, including suppliers, contractors, and research institutions. Effectively managing these partnerships can be challenging, particularly when working across different time zones and cultural contexts.
These aspects are vital for the success of the organization, as it extends beyond simply releasing the product. The organization must be equipped and empowered to provide comprehensive support.
Here is Christopher’s concise viewpoint regarding the challenges mentioned above:
Throughout my career, I have encountered challenges associated with each of these factors that impact product delivery. Successfully delivering a product requires collaboration between Engineering, Design, QA (often embedded in the team), Performance Testing, and all cross-functional teams.
Christopher gives the advice below to overcome these challenges:
- Equip your team with knowledge and work together to devise the best solutions, allowing the team to be a driver of the process.
- When creating a release plan, collaborate with the teams affected to identify dependencies.
- Be prepared for ambiguities that may arise.
- Address risks and dependencies early in the process by proactively managing them through research, collaboration, alignment, and mitigating dependencies.
- Conduct thorough research to determine the appropriate approach and develop a plan accordingly.
- Embrace the possibility of change and remain adaptable to pivot when necessary.
- Despite thorough preparation and analysis, unexpected events can still occur. Place trust in your team to handle these situations effectively.
Remember, product delivery is a collective effort, and by implementing these strategies, you can overcome challenges and achieve successful outcomes.
Product engineering challenges and practical solutions from Kanda’s Delivery Manager
Christopher Van Horn speaks on his experience with overcoming challenges and finding solutions in the product engineering process below:
One notable instance involved addressing technical debt as an opportunity to rethink challenges and develop innovative solutions. In the past, our focus was primarily on building the next big feature to drive sales and enhance our product’s viability, often neglecting technical debt to a large extent.
The below challenges are two of many that Christopher’s team encountered during the process:
Challenge #1
Trust in product development services is a big part of preserving and maintaining business. While having desirable features is important, ensuring that the services are up and running reliably is equally crucial. Christopher’s team recognized that addressing the behind-the-scenes aspects was super important in overcoming the scalability and stability challenges they encountered.
Solution
When system issues arose, leading to performance degradation, outages, or significant oversights, the team took the following approach:
- Recognized the importance of prioritizing these issues as they directly impacted customer trust and satisfaction.
- Devoted an entire engineering team to address these core concerns, while also allocating a percentage of other teams’ resources to tackle relevant issues within their purview.
- Established weekly meetings to effectively manage and prioritize issues.
- Assigned each issue to a dedicated team.
- Conducted thorough issue identification, root cause analysis, and implementation of proactive monitoring through alerting and metrics.
- Set clear expectations for resolution within defined timeframes, based on the impact of each issue.
- Developed appropriate solutions, prioritizing long-term resolutions while implementing short-term measures to mitigate any immediate gaps.
Building and maintaining trust with customers requires consistent investments in addressing underlying issues and delivering a reliable product.
These efforts ultimately led to improved customer satisfaction, reduced recurring issues, and the integration of continuous improvement as an integral part of our process.
Challenge #2
Along the way, Christopher’s team also ran into the problem below:
We encountered performance challenges related to the traditional file and folder model used for content security within our product’s repository. While it generally performed well, there were cases where it proved to be inefficient for highly complex models and use cases.
Solution
Here’s how Christopher’s team approached this challenge:
- Identified customers who were impacted by the performance issues.
- Conducted interviews with those customers to gain a deeper understanding of their software usage and specific use cases.
- Identified usability, configuration, and setup issues that the customers faced, realizing the need to rethink their approach.
- Leveraged the opportunity to foster stronger relationships with their customers and enhance their understanding of customer needs. Every challenge became an opportunity to grow and gain trust.
- Collaborated with the entire team to communicate the problem and created space for innovative solutions.
- Encouraged the team to approach the problem space with a fresh perspective and provided the freedom to develop new approaches.
- Through this process, Christopher’s team devised a new solution that not only resolved the performance issues but also addressed the real problems their customers encountered.
- The team regularly reviewed their designs with customers to ensure alignment and gain further validation.
This experience taught us the value of leveraging technical debt as an opportunity to gain deeper customer insights, rethink problem spaces, provide autonomy to the team, and foster creative solutions.
Conclusion
The product engineering process is a complicated journey that often includes many setbacks along the way. Christopher Van Horn, Kanda’s Delivery Manager, gave us insight into addressing technical debt, understanding customer needs, and empowering teams to innovate and take responsibility for solving challenges in product engineering.
At Kanda, we’re not afraid to face challenges, and thanks to our 20+ years of experience, we have a solid approach to fielding issues of various complexities.
Still have doubts about your product engineering cycle?