Visual Quality Control - Rule-Based Systems vs. ML-Based Systems

Episode 2

Author: Hanna Nennewitz

Visual Quality Control - Rule-Based Systems vs. ML-Based Systems

Episode 2

Author: Hanna Nennewitz

maddox

I explained how rule-based systems and ML-based systems work in general in the last episode. Today, I turn to the advantages and disadvantages of the two systems and what new opportunities are offered by ML-based systems. For this, I spoke with Dr. Wieland Brendel, Co-Founder and Lead Machine Learning Advisor at Maddox AI as well as a researcher on Robust Machine Learning. In addition, David Rebhan explained rule-based systems in visual quality control in more detail to me. David worked as an application engineer for rule-based systems for two years before joining Maddox AI. At Maddox AI, he creates hardware concepts and performs on-site hardware installation.

Where one has advantages, the other has disadvantages – that’s how it often works in life. This saying also finds its truth in systems for the automation of visual quality control. As a reminder: rule-based systems follow the rules that a human previously set up for them when inspecting. ML-based systems learn with the help of sample images and thus build themselves a catalog of rules for detecting defective parts. However, we do not know exactly how the system learns and what exactly the rules are that it uses to make decisions.

I explained how rule-based systems and ML-based systems work in general in the last episode. Today, I turn to the advantages and disadvantages of the two systems and what new opportunities are offered by ML-based systems. For this, I spoke with Dr. Wieland Brendel, Co-Founder and Lead Machine Learning Advisor at Maddox AI as well as a researcher on Robust Machine Learning. In addition, David Rebhan explained rule-based systems in visual quality control in more detail to me. David worked as an application engineer for rule-based systems for two years before joining Maddox AI. At Maddox AI, he creates hardware concepts and performs on-site hardware installation.

Where one has advantages, the other has disadvantages – that’s how it often works in life. This saying also finds its truth in systems for the automation of visual quality control. As a reminder: rule-based systems follow the rules that a human previously set up for them when inspecting. ML-based systems learn with the help of sample images and thus build themselves a catalog of rules for detecting defective parts. However, we do not know exactly how the system learns and what exactly the rules are that it uses to make decisions.

The possibility of setting up 100% valid rules as a guideline for action in simple use cases results in some advantages for rule-based systems.

If you look at simple use cases, such as measuring lengths, rule-based systems have the edge here. This has several reasons. First, for simple use cases, implementation can be done faster with rule-based systems than with ML-based systems. David explains, “An experienced programmer programs a simple inspection task in a few minutes.” The main reason for this is that programming rule-based systems requires little or no data. For ML-based quality control, on the other hand, you need sample images that the system learns from. Generating this data and training the system therefore naturally takes a little longer than simply writing down a rule for a rule-based system.

In addition, rule-based systems also have an advantage over ML-based systems for simple use cases, since 100% valid rules can usually be established here. If I can write down 100% valid rules for a rule-based system, I can also be sure that it will catch them 100% of the time. According to Wieland, “For example, if the control system needs to correctly measure every inside diameter of a through hole, a rule-based system is probably a more advantageous option. ML-based systems detect 99% of the errors, but just not 99.999%. There may always be some small variability that the system doesn’t catch after all.” Of course, for very complex testing tasks, this strict adherence to rules also quickly becomes a disadvantage, but more on that later.

The possibility of setting up 100% valid rules to guide action in simple use cases results in another advantage for rule-based systems. The actions of the rule-based system and its test decisions can be understood in such a way clearly better than with ML-based systems. According to Wieland, “A rule-based system is completely controllable because a programmer has written down exactly the rules by which it should act. If the system makes a mistake, I can track exactly what caused it.” It’s not as easy to look into an ML-based system because it learns itself and sets its own rules accordingly, but we don’t understand them well. As a result, tracking down wrong decisions in ML-based systems is more complicated than in rule-based systems.

"The quality control problems that are still done manually today are exactly those kind of problems that rule-based systems cannot help with."

However, Wieland explains, “Everything that can be inspected by rules based quality control has been inspected by rules based quality control for 20 to 30 years already. The things that are still done manually today in quality control problems are exactly those kind of problems that rule-based systems cannot help with.” That’s mainly because you can’t write down rules for these use cases that always apply 100% of the time. Or the inspection conditions change from time to time, making it nearly impossible for a rule-based system to work reliably. ML-based systems, however, are up to these challenges and, accordingly, are clearly ahead of the game here.

The advantages of ML-based systems in complex use cases are primarily due to the fact that ML-based systems do not inspect with the help of man-made rules, but instead develop implicit rules themselves within a learning process, on the basis of which they identify good and bad parts. Many complex inspection tasks, such as surface inspection or component verification, can thus be automated in the first place. Rule-based systems quickly reach their limits when it comes to such tasks. If you are interested in more detail on that topic, please have a look at episode 1.

This learning and own creation of rules presents other advantages besides the possibility to automate more complex use cases. Since ML-based systems act on the basis of their own experience, they are much better able to deal with changing conditions. For example, they can be taught in advance that a ray of sunlight coming through the window and changing the appearance of the parts does not mean a defect. The same goes for parts that look slightly different depending on the batch and the condition of the manufacturing tooling, but are still OK parts. Again, ML-based systems like the Maddox AI system can be taught that differences in brightness or gloss need not mean defects. It’s a different story with rule-based systems. David explains, “Rule-based systems are not very flexible; any change in the product or environmental conditions will degrade the performance of these systems. For example, if the sun shines through the window and changes the contrasts, a rule-based system is then likely to stop finding errors or detect false errors.” Thus, rule-based systems can mean a very high pseudo-rejection rate and constant recalibration work, especially in more complex use cases.

In addition to being robust to changing environmental conditions and varying part appearances, ML-based systems can also adapt much more quickly to product changes. “For example, if the bottles being tested become smaller than before, you can teach that change to the AI very quickly. With rule-based systems, on the other hand, all parameters have to be rechecked and redefined if necessary, which can be very time-consuming depending on the use case,” David notes.

"Maddox AI's user interface is designed in such a way that even beginners can use it with confidence after a short training session."

With Maddox AI in particular, there is another advantage that the ML-based system brings. If recalibrations are necessary, they can be easily performed by someone without AI expertise. The user interface of Maddox AI is designed in a way that even beginners can use it after a short training. You can see a view of the user interface in the following image.
maddox

The situation is quite different with rule-based systems. With rule-based systems, David explains to me, a lot more expertise is required: “To be able to work with a rule-based system, you should definitely be trained in programming. The less you know about programming, the harder it will be to get acquainted with the system’s workings in the first place.” If you use a rule-based system in visual quality control, you should have trained programmers in your company who are familiar with rule-based systems. According to David, however, being familiar with one rule-based system does not mean being familiar with all of them, “There are many different manufacturers and many different programs for rule-based systems, all of which function differently. If I know my way around one system, it doesn’t mean I know how to use another.”

 

Here is an exemplary user interface for a rule-based system from MVTec HALCON:

maddox

So you see, ML-based systems offer many advantages over rule-based systems, especially when it comes to complex use cases. In many cases, the automation of visual quality control here only becomes possible through the use of ML-based systems. If you have not automated your quality control yet, there is a very high probability that Maddox AI will enable you to do so. Feel free to contact us and we’ll see if you too can benefit from the advantages of ML-based visual quality control.
In the next episode, I’ll delve a little deeper into the features of ML-based quality control, taking a closer look at anomaly detection, and finding out what’s the meaning of segmentation.

In der nächsten Folge werde ich noch etwas tiefer in die Funktionen ML-basierter Qualitätskontrolle einsteigen, mir Anomaliedetektion genauer anschauen und herausfinden, was sich hinter dem Wort Segmentierung verbirgt.

Article

Anomaly Detection vs. Segmentation

Episode 3
maddox
In this episode, the modes of operation as well as advantages and disadvantages of different ways of error detection by anomaly detection (=unsupervised learning) and segmentation (=supervised learning) are explaned in more detail.Read the Article