Bayesian and Artificial Intelligence
Technology
In many business applications, it is essential for AI to be able to tell whether it is confident of its output, being able to cast questions such as “do I need to use a larger size of data? or change the neural net structure? or perhaps be careful when making a decision?”. The questions are of fundamentals in Bayesian learning, and have been studied extensively in the field.
However, most commercial deep learning models are deterministic functions, as a result, interpret risky business situations in a very different setting from the probabilistic view point which possess uncertainty information.
Among these settings are scenarios in which control is handed-over to AI, in situations which have the possibility to become life-threatening to humans. These include automated decision making or recommendation systems in the medical domain as well as control of critical systems. These can all be considered under the umbrella field of AI safety. The illustrative example can be extended to serious settings, such as MRI scans with structures a diagnostics system has never observed before, or scenes an autonomous car steering systems has never been trained on.
Therefore, demanding desiderata of commercial AI is the ability to return an answer with the added information that the point lies outside of the data distribution. We develop a model that possesses some quantity conveying a high level of uncertainty with such inputs:
-
Our model can sense measure imprecision. False predictions are often generated from noisy input data or labels.
-
Our model can provide uncertainty of its parameters.
-
Our model can provide structure uncertainty. Many practitioners seek a proper neural structure for their business problems. Our model can tell how much uncertainty of their predictions inherit from the chosen structure.
Application Field
In modern AI stage, the control over critical systems is slowly being handed over to machine learning systems. Thus, the tools are required to reason about model confidence.
Medical field – diagnostics
When a physician advises a patient to based on a medical record analysis, the physician would often rely on the confidence of the expert analyzing the medical record. The introduction of systems such as automated cancer detection based on MRI scans through could make this process much more complicated. Even at the hands of an expert, such systems could introduces biases affecting the judgement of the expert. This can lead to over-diagnosis of doctors.
A system encountering test examples which lie outside of its data distribution could easily make unreasonable suggestions, and as a result unjustifiably bias the expert. However, given model confidence an expert could be informed at times when the system is essentially guessing at random.
Medical field – data collection
Collecting a medical history of a patient is demanding task nowadays. Providing appropriate acquisition function, therefore, is an important problem. Which category of patients should be labeled in priority than other groups? The principled answer to this question enables efficient resource allocation in the medical areas.