February 24, 2023

How to validate your idea for a machine learning use case?

How to validate your idea for a machine learning use case?

In every innovation and production deployment we need to mitigate risk. Machine Learning projects pose unique challenges here. Unlike classical software engineering, which can be demanding but is generally more predictable and manageable in terms of planning and execution, there are numerous unknown factors involved in machine learning.

Machine learning projects are more complex, involve a higher dose of uncertainty and are generally more risky. However, successful ones can be substantial and drive innovation across a wide range of industries. 

If you're a visionary entrepreneur and planning to launch a startup, that you think would benefit from AI and machine learning technology, you shouldn't let current limitations discourage you. Instead, you can focus on finding ways to make your ideas work. However, machine learning project ideas often fail not because of mistakes in data processing, ML algorithms or model selection, but rather due to a lack of proper resources, expertise, or strategy from the start.

A visionary enterpreneur by Midjourney

Initiating a project with a proper approach is pivotal for its success. Letting machine learning experts help you determine if your project actually can be built using machine learning techniques, gives you the confidence you need to move forward. 

In this blog post we describe how we can work together and validate your machine learning use case using a holistic scientific approach. This way you know you’re making the best use of the available resources and technology.

Why should you validate your ideas?

Around the world, tech startups bring machine learning and artificial intelligence to tackle problems that were previously unsolved. AI projects are desirable, early movers who have high-quality training data to make accurate predictions are likely to succeed, although this is not always clear from the start.

Therefore, the first step is to ask the question: are we applying ML to the right problem?

When developing any kind of software you want to be lean and reduce risks early. You know the best that development costs money and time. You don’t want to find yourself engaged in a full-blown software project and only after it’s done, find out that the investment should have been done differently. 

In classical software engineering, when an idea pops into your head, you are able to estimate its feasibility based on the current state of the art. You analyse it in terms of architecture and system elements, and then verify how it can be implemented. However, in the case of machine learning, the situation is different, because there are many unknowns. The final result depends on various factors, e.g.: the quality of the data and the idea behind the model. There is a possibility that the proposed solution simply will not work, because there are no technical possibilities to obtain data with high predictive power yet, or the problem is so complicated that even a neural network will not help, or it requires an impossible to obtain computational capability.

That’s why we are often approached by business owners to verify their ideas and support them with answers. It is not always possible to answer right away. Sometimes you need to conduct research or conduct an experiment. That is why it’s our goal to connect the scientific world and business world in ML projects to develop a roadmap for project viability. And in order to do so, we developed three concrete milestones:

  1. Machine Learning Workshop
  2. Machine Learning PoC
  3. Machine Learning MVP

Let’s see how each of these steps look and what their goals are.

Machine Learning Workshop

Does your business idea hold promise in the context of AI and machine learning? The truth is ML won't be the optimal solution to any problem out there, and if there is a better, easier way for you to achieve your goals - we'll tell you.

In the world of AI and machine learning, it's easy to get swept up in the excitement of the latest and greatest algorithms and models. However, no matter how impressive the technology, if it's not being applied to the right problem, it's not going to be effective.

During a 3-to-5 day workshop we create space for business and scientific ideas to collide in order to find the most appropriate way to bring you results. The goal is to figure out a problem-solution alignment and decide if your idea should qualify for business PoC, or rather it’s more of something that needs further research or can be done with more conventional technology. 

How does the workshop look like

The workshop is quite intensive and looks like an interview. Our engineers are no strangers to science. All of them are individuals who have received, or are in the process of obtaining, specialized education in a scientific field. 

At the heart of our research-driven approach is the belief that understanding our clients is key to success. Our workshop is designed to help everybody understand the very roots of the problem. We do it by:

  • asking the right questions about the origin of your idea,
  • learning about your company and your place in the market,
  • matching your needs with what’s out there in the field of science and machine learning.

After the first day, you’ll receive a list of research problems defined based on what we've discovered. This will serve as a launching pad for further exploration. From that list we can end up discussing: areas of concern, things to be improved, obstacles to be eliminated, or troubling questions that exist in scientific theory, or in practice.

Once we've got a clear picture of the problems we're facing, we dive headfirst into analysis mode. We take each problem, one after another, and work together on solutions to overcome them. This is also the time where our machine learning engineers share their wealth of expertise with you, providing insights into the machine learning world. Thus both sides better understand each other's perspectives and work out a better solution.

Who conducts the workshop

The workshop is led by a team of experienced machine learning engineers who are well-versed in conducting research, testing hypotheses, and drawing empirical conclusions. Their expertise extends to both academia and industry.

A workshop with one of our clients

We select specialists for the machine learning role based to the domain and technical competence. For example, if as you need something with an image recognition, a computer vision specialist will join you, and if you need text, an NLP specialist.

Our team will introduce you into the basic concepts of machine learning, like e.g.: what is machine learning vs deep learning, what is artificial intelligence, what are machine learning algorithms and machine learning models, what is computer vision or natural language processing, nlp? All these will help translate your idea into the language of process requirements using machine learning.

What is the workshop goal

After 5 days you will know whether machine learning will work for you, or if there is a better, easier way for you to achieve your goals. 

You will receive a Workshop Report that includes:

  • Your project description in the light of what we’ve uncovered
  • A list of engineering competences needed for the project
  • An idea, plan and time estimation for the Proof of Concept stage
  • A draft of a machine learning architecture for your project

Our goal is to achieve full transparency and notification of the results obtained as soon as they appear in order to be able to make various adjustments to the previous approach. While incremental development that determines whether an idea has value is important in each software project, in a machine learning applications it becomes crucial.

If your project is not the case for machine learning - we will tell you.

Machine Learning PoC

If the Workshop proves your idea is a good fit for machine learning implementation, the next step is to verify it by training a model. Successful proof-of-concept stages require a combination of the right skills and data, and meaningful dialogue between business and engineers. Just that, and all that, to ensure that models remain accurate over time, so a PoC becomes real solutions in production.

How does the PoC looks like

Machine Learning PoC stage lasts from three up to six months. During this time we’ll meet every three weeks and our engineers will demonstrate the progress made in that time frame. This approach lets us concentrate on coming to the right conclusions for more accurate predictions and a common understanding of the progress made. By keeping the lines of communication open and regular, we are able to pivot swiftly and decisively whenever necessary. 

To really show you the work we do we use experiment tracking software like Neptune.ai or Weights and Biases. These  awesome tools let everybody be in the loop when conducting ML experiments. Think of it as peering through an oven window to check on a delicious pie baking inside. This is exactly how Neptune works – it lets you stay in the loop and observe what's cooking in our ML model. We want to ensure that you're able to take a peek at our work any time you want and on your terms. 

A view from neptune.ai -> quickstart docs here

What data is being used at the PoC stage

It depends, and is decided upon a workshop stage. We usually know and can recommend existing example data sets that can be used for the purpose of our experiment, however sometimes there’s a need to collect the data. And we add this part to our PoC stage and help you collect them the right way by providing proper gathering methodology, installing supporting software and even establishing the annotation team.

What is the PoC goal

We want to find out the right data – and enough of it. When selecting the data sample for our proof-of-concept (POC), we focus on a rather small, but enough data set, that enables us to encounter and analyse as many problems as possible. 

With this approach you gain confidence in the quality of data, as well as all the other data science and ML work, for your particular use case. You will also gain experience in discussions with ML/AI teams and the ability to define requirements and verify their work. 

After the proof of concept, we will figure out if implementation of machine learning tools and techniques in your case is worthwhile enough to justify the cost of the entire project.

You will receive a Roadmap that includes:

  • An estimate the cost and time required to solve the problem in production
  • A list of engineering competences needed for the project

Machine Learning MVP

If your idea got to this stage with us, it's time to roll up sleeves and build that software. For MVP purposes we use a fixed-price model and stay budget-friendly, so you know the costs of your minimum viable product forehand.

How does the MVP looks like

We define modules of your system and the competences needed for the project, as well as define a roadmap and costs. Our machine learning engineers provide the necessary know-how and seamlessly integrate with your team. Having experience as software engineers too, our parent company is SoftwareMill, we are able to augment the work of a software development team or do the software completely.

Our team of Ops experts can increase developer productivity with hand-picked infrastructure automation tools. With such a multidisciplinary team you are free to focus on marketing your business and welcoming fist clients.

What is the MVP goal

A positive result from MVP allows you confidently move on to the development of a complete product.

Summary

Putting your idea to the test pays off. Especially in the innovative field of machine learning where new algorithms can be overwhelming. Let us help you navigate this landscape and make informed tech choices. When working with our clients we developed an effective path where we create a space for business and scientific ideas to collide in order to define or roll out machine learning technology. 

Interested in exploring our workshops, Proof of Concept and MVP steps in your project? Let us know and we’ll be happy to use science with you!

- Reviewed by Maciej Adamiak