Multilingual Conversational AI incorporated with Visual Questions Answering and Intelligent Disease Prediction for Healthcare Industry

Abstract

Artificial intelligence (AI) is becoming more active than ever in everyday life and steadily being incorporated to healthcare. AI, with its seemingly limitless power, affirms a promising future to a revolutionized healthcare system. This paper is proposing a conversational AI solution in two different languages, English and Sinhala, to predict diseases through a conversation, a visual question answering solution to generate answers are based on a given question and a medical image and a disease forecasting module. A robust, accurate prediction is a rather difficult task given the availability of data and absence of preprocessed, clean data. With the aid of outlier rejection, data imputation, vectorization, feature selection and data standardization, the proposed framework gets the advantage of latest machine learning advancements such as AI using DIET classifier and NLU pipelines, for the conversational disease diagnosis which uses support vector machine (SVM) achieved an accuracy of 0.93. Moreover, the visual questions answering module with VGG16 preprocessing, GoogleNews vectors, LSTM networks, scores an accuracy of 0.9721. In addition, time series analysis models such as ARIMA and adaptive models using PROPHET library for forecasting diseases, classification using random forest scoring an accuracy of 0.81, logistic regression scoring an accuracy of 0.84 for predicting diseases. The objective of this research is to compare and select the best fitting models to be used for a centralized framework for healthcare industry.