I’ve been teaching businesses about RI3P for the last 4 years. It’s a structure that works to discover machine learning concepts and turn them into revenue generating products. Without a strategy and structure around machine learning, it will not succeed. That’s true in startups or Fortune 100 companies.
For companies lacking structure, it’s impossible to hire qualified employees because there’s no way to tell an individual where he or she fits in the process. Roles become impossible catch all’s with skills scattered across multiple disciplines from architecture to research to development, deployment, and maintenance. Employees burn out in unstructured environments because they’re asked to do too much; setting them up to fail.
Without structure, it’s impossible to manage the process or predict its outcomes. Costs and ROI are uncertain in unstructured environments. This is often how machine learning and data science teams become silos, disconnected from users, business goals, and the rest of the company.
Machine learning isn’t like other products so the development cycle isn’t traditional. What’s out there right now, from Agile to Waterfall, won’t work. I built RI3P as a hybrid to allow for the loosely controlled innovation cycles to coexist with tightly defined costs and revenue projections.
The entrance criteria for the research phase is a presentation. This can be inspired by conference driven enthusiasm or research paper euphoria. We’ve all seen these generate great ideas but without structure, they often bounce as products. The presentation showcases the idea and a path for original or further research.
The research proposal is evaluated like most academic research proposals. Is it a valid, interesting, and potentially fruitful area of study? Stay away from potential applications because it’s too early to speculate. Once approved, the research itself starts. Checkpoints are monthly or quarterly with a presentation at each to showcase progress and any early indicators of a finding. At each checkpoint, the research can be approved to continue or terminated due to lack of progress.
The exit criteria is a final presentation on findings. This should be broadly reviewed both by peers and the larger business. Peer review assesses the validity of the research and findings. The larger business review is the entrance criteria for the next phase.
The entrance criteria for the innovation phase are valid research findings. Once the peer review is over, open the door to a review by the rest of the business. The idea here is to generate as many “Can it do this?” type questions as possible. If the answer is yes, evaluate if using the findings is more efficient than the current methodology. This ties the implementation to a business need, either internal or customer. These are validations for the research’s applied potential.
This is a practical way to go beyond saying the research is valid but that it is also innovative. Being novel isn’t the litmus test for innovative research. The second part is practical applications. These should be patentable which provides an immediate return on investment for the research phase. The best applications are the entrance criteria for the next phase.
The entrance criteria for the prototype phase is a review of the potential practical applications. The best supported should survive. There’s some market research to be done here as well as alignment with the company’s overall goals. It’s a mini go to market planning session. Applications that get green lit from this review get sent up for prototyping.
The prototype should be no more complex than a proof of concept. It proves the practical application concept can be built. It showcases the most minimum set of features. It should be cheap to build and have minimum expectations of quality. This should never see production deployment except in the rarest of cases. It’s the type of application you’d expect to come out of a hackathon.
The prototype gets shared with early adopters. These are hand selected users who have a strong connection to the business need and a good understanding of how to evaluate pre-alpha release software. Rave reviews are the entrance criteria for the next phase.
I tell clients that the entrance criteria for the planning phase is users demanding the prototype right now. If the feature is valuable enough, even in its roughest form, users will want it. They will be more than happy to deal with its bugs and bare bones functionality. Obviously, don’t sell it at this phase because that’s how companies ruin their reputation for providing quality products. However, it is the right time to start planning for a more structured release.
This is the typical planning phase most software development businesses are used to. The company has an innovative idea and an IP protection strategy in place. The proof of concept has been built and there’s legitimate interest from users. Enter the standard planning process for software design and development. Put it on the roadmap. Build epics. Assign resources. Whatever it is that the business is used to doing in the planning phase happens here.
Machine learning has different productization concerns than traditional software. It’s not a static product. The learning side of it makes features more open ended and flexible. Users will come up with their own implementations and extensions. The product improves with use. Models will need to be updated or even retrained altogether. Platforms are constantly changing and improving.
For these reasons and more, productization is a process of its own in the structure. Understanding how long the product will be in the field becomes critical. For example, if the product is in the field for more than 2 years, it’ll see a major platform change which needs to be designed and planned for. I can list another dozen checkpoints which will require significant planning. Productization is much more complex and critical for machine learning products. Done well, it insures that only sustainable prototypes make it to market.
The acronym’s similarity to RIP (rest in peace), isn’t accidental. The business will be killing research before it yields findings. Findings will die on the vine without leading to anything patentable. Patents will languish. Prototypes will arrive with a thud. Promising prototypes won’t be sustainable or the business won’t be capable of executing. The point of the RI3P is to arrive at that discovery as early in the process as possible.
It is far cheaper to terminate unproductive research than to discover a product was built based on a researcher tilting at windmills. Each phase is monetizable. Research is publishable. Innovation is patentable and so on. The cost of each initiative is controlled by its promise. High and low ROI projections become possible at each gate. Employees know their role and how they’ll be evaluated. Your research yielded 4 papers and 2 patents. Your prototype yielded 2 green lit products. Beyond that, the rest of the business knows what your machine learning and data science team actually does. Many are involved in reviewing or evaluating their outputs. That level of integration is critical to the team’s success in the larger business.
What I’ve described here is the 50K foot view of a structure to make machine learning capabilities work within a business. I’ve implemented RI3P with great success. It’s part of my overall emphasis on strategy for machine learning. Without a machine learning strategy and structure, it’s just a buzz word in the enterprise.