The Hidden Problem of Machine Learning
Since the introduction of Logic Theorist, the first Artificial intelligence (AI) program in December of 1955, AI has been further enhanced to address the new challenges of the 21st century. It has proven to be beneficial in many areas from improving the security of critical infrastructures to enhancing the processing power of our devices. With that said, engineers and scientists have identified issues in the fundamentals of how one of the branches of AI operates. More specifically, they can not describe how a machine learning (ML) artificial neural network (ANN) algorithm reaches a conclusion, other than the fact that it takes input data and produces an output. Many experts doubt whether it can be used in more impactful industries such as medicine and law. Unexplainable AI algorithms similar to ones stated above can not be integrated into critical fields unless new technologies and methods are introduced that describe how the program reaches its decision.
Artificial neural networks produce an algorithm for a pattern they identified in the set of data provided as input, however, they do not produce a description of the pattern they have identified. For fairly simple algorithms such as labelling a photo as blue or red, it is easy for humans to also understand the pattern. That is because humans can also recognize that certain pictures will have more blue elements compared to others and vice versa. With that said, for more complex algorithms such as an AI program playing chess against a professional chess player, it is more difficult for humans to understand how the algorithm “thinks.”
Artificial neural networks (Figure 1. Artificial Neural Network) take a set of data as input (yellow layer), search and identify a pattern (blue and green layer), and produce an output (red layer). In short, the blue and green circles transform the input into the output. With that said, the green and blue layers are interpreted as the “black box” or “hidden layers” because it is greatly difficult for one to explain what operations occur in that section. The illustration below is an oversimplified example of an artificial neural network therefore, a more complex ML algorithm will have many more hidden layers with many more components.
Figure 1. Artificial Neural Network
Based on the fact that operations are done in the black box, many people are concerned about the applications of ML and ANN in medicine and other areas. For instance, IBM intended to promote its supercomputer named Watson by helping cancer doctors diagnose cancer, however, it proved to be a “PR disaster,” described by Jason Bloomberg. There were two main reactions to the program. First, if the results of the algorithm agreed with the doctors, it helped confirm cancer. Second, if Watson did not agree, the doctors simply thought it was wrong. If the engineers and scientists behind the supercomputers were able to explain how the computer arrived at a decision, it might have been able to prove to the doctors that the results are correct and valid. The doctors can not trust the computer therefore they can not risk the life of patients. To solve this, companies are exploring a new area in the AI industry.
Modern-day ML models are interpreted as “black-box” however, naturally, certain AI models are classified as “white-box,” with one example of that being explainable artificial intelligence (XAI). Effective solutions of XAI are not developed yet however, many organizations are investigating this kind of technology, as it might be beneficial to many people of many industries. At a minimum, XAI will allow AI to be integrated more in-depth in the industries since humans will have the ability to trust the tools.
There are three principles that the XAI will have to follow for it to be successful in its mission: transparency, interpretability, and explainability. Firstly, transparency is defined as a designer being able to describe how the AI algorithm operates, and how it arrives at conclusions. Secondly, interpretability simply states that the underlying decision-making process and operation and calculation have to be interpreted for humans to comprehend them. Finally, explainability is interpreted as “the collection of features of the interpretable domain, that have contributed for a given example to produce a decision (e.g., classification or regression)” [Montavon]. The principles could be used as requirements that will help teams to create AI algorithms that can justify their decisions and the operations they performed to achieve the results.
To conclude, the development and advancement of explainable artificial intelligence (XAI) will allow humans to have a sense of control over algorithms. Humans naturally thrive to achieve control over any element of life possible. This has been proven in many psychological studies and researches. Humans thrive to have control because it allows us to control the process leading to the outcome, which also allows us to control the results to a degree. This control is also necessary for the AI industry, For instance, certain AI software has been impacted negatively with inappropriate data, which has caused them to develop a bias towards different ideas. By gaining back control, researchers can prevent current and future problems that might arise from the use of AI by being able to understand how the AI arrived at a decision and where problems might have arisen.