Publications

Argumentative Stance Prediction: An Exploratory Study on Multimodality and Few-Shot Learning Permalink

Published in The 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023

To advance argumentative stance prediction as a multimodal problem, the First Shared Task in Multimodal Argument Mining hosted stance prediction in crucial social topics of gun control and abortion. Our exploratory study attempts to evaluate the necessity of images for stance prediction in tweets and compare outof-the-box text-based large-language model(LLM) in few-shot settings against fine-tuned unimodal and multimodal models. Our work suggests an ensemble of fine-tuned text-based language models (0.817 F1-score) outperforms both the multimodal (0.677 F1-score) and textbased few-shot prediction using a recent stateof-the-art LLM (0.550 F1-score). In addition to the differences in performance, our findings suggest that the multimodal models tend to perform better when image content is summarized as natural language over their native pixel structure and, using in-context examples improves few-shot performance of LLMs.Code Poster

Recommended citation: @inproceedings{sharma-etal-2023-argumentative, title = "Argumentative Stance Prediction: An Exploratory Study on Multimodality and Few-Shot Learning", author = "Sharma, Arushi and Gupta, Abhibha and Bilalpur, Maneesh", editor = "Alshomary, Milad and Chen, Chung-Chi and Muresan, Smaranda and Park, Joonsuk and Romberg, Julia", booktitle = "Proceedings of the 10th Workshop on Argument Mining", month = dec, year = "2023", address = "Singapore", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2023.argmining-1.18", doi = "10.18653/v1/2023.argmining-1.18", pages = "167--174" }

Enhancing Visual Perception in Novel Environments via Incremental Data Augmentation Based on Style Transfer Permalink

Published in IEEE International Conference on Robotics and Automation (ICRA) (Under review), 2023

The deployment of autonomous agents in real-world scenarios is challenged by “unknown unknowns”, i.e. novel unexpected environments not encountered during training, such as degraded signs. While existing research focuses on anomaly detection and class imbalance, it often fails to address truly novel scenarios. Our approach enhances visual perception by leveraging the Variational Prototyping Encoder (VPE) to adeptly identify and handle novel inputs, then incrementally augmenting data using neural style transfer to enrich underrepresented data. By comparing models trained solely on original datasets with those trained on a combination of original and augmented datasets, we observed a notable improvement in the performance of the latter. This underscores the critical role of data augmentation in enhancing model robustness. Our findings suggest the potential benefits of incorporating generative models for domain-specific augmentation strategies.Code

Recommended citation: @misc{gupta2023enhancing, title={Enhancing Visual Perception in Novel Environments via Incremental Data Augmentation Based on Style Transfer}, author={Abhibha Gupta and Rully Agus Hendrawan and Mansur Arief}, year={2023}, eprint={2309.08851}, archivePrefix={arXiv}, primaryClass={cs.CV} }

Towards Accurate and Clinically Meaningful Summarization of Electronic Health Record Notes: A Guided Approach Permalink

Published in IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI’23), 2023

Abstract—Clinicians are often under time pressure when they review patients’ electronic health records (EHR), therefore, there are great benefits to providing clinicians high-quality summarizations of patients’ EHR. However, existing summarization algorithms fall short in certain key aspects, such as focusing on pertinent information that is clinically significant, and adhering to a structured template that aligns with the formats clinicians are accustomed to. In this paper, we present a novel approach to summarize EHR notes using a guided summarization model. Our model integrates a structured template developed with a clinical domain expert, a Named Entity Recognition (NER) model and sentence classification model for guidance extraction, and a fact-checking metric for evaluating the generated summaries. We trained our model on a large de-identified EHR dataset. The results demonstrate that our guidance, which includes Chief Complaint (CC), NER, guidance from the History of Present Illness (HPI) section, and guidance from the Medical Decision Making (MDM) section, can significantly improve the performance of the models in generating accurate and clinically meaningful summaries. The Gsum (CNN) model with all the guidance aforementioned achieved the highest F1 score of 46.4, demonstrating the effectiveness of introducing precise and in- formative guidance to models from the general domain when the training data on the clinical domain is prohibitively sensitive and expensive. This work contributes to the ongoing efforts to automate the summarization of EHR notes, with the ultimate goal of improving healthcare delivery and patient outcomes.Poster

Recommended citation: @inproceedings{luo2023towards, title={Towards Accurate and Clinically Meaningful Summarization of Electronic Health Record Notes: A Guided Approach}, author={Luo, Zhimeng and Ji, Yuelyu and Gupta, Abhibha and Li, Zhuochun and Frisch, Adam and He, Daqing}, booktitle={2023 IEEE EMBS International Conference on Biomedical and Health Informatics (BHI)}, pages={1--5}, year={2023}, organization={IEEE} }

Disambiguating Spatial Prepositions: The Case of Geo-spatial Sense Detection Permalink

Published in Transactions in GIS, 2022

Spatial relations in natural language are frequently expressed through prepositions. Thus, in the locative expressions “New York in the United States” and “the house on the river” the prepositions “in” and “on,” respectively, serve to communicate the relationships in space between the subject and object of the preposition. Automatic detection of the use of prepositions in a spatial and in particular a geo-spatial sense that refers to geographic context is of interest in supporting automated methods for determining the actual geographic location referred to by locative expressions. This work focuses on disambiguation of prepositions in natural language, with the goal of distinguishing whether a preposition is used in a specifically geo-spatial sense. We conduct machine learning experiments that demonstrate the clear benefit for geo-spatial sense detection of using transformer model deep learning methods when compared with a variety of methods, that include Naive Bayes, support vector machine, and random forest classifiers with handcrafted linguistic features, and a bag of words approach with a meta-classifier that adds geo-spatial features. The best performance was obtained with the Bidirectional Encoder Representation from Transformer-based XLNet transformer model, with a best precision of 0.96 and an F1 score of 0.94 when evaluated on a corpus of natural language expressions that were annotated for this task. We also conducted experiments to detect generic spatial sense, in which the best F1 score, of 0.95, was again obtained with XLNet. Code

Recommended citation: Radke, M. A., Gupta, A., Stock, K., & Jones, C. B. (2022). Disambiguating spatial prepositions: The case of geo-spatial sense detection. Transactions in GIS, 00, 1–31. https://doi.org/10.1111/tgis.12976

Neural Architecture Search for Pneumonia Diagnosis from Chest X-rays Permalink

Published in Nature Scientific Reports, 2022

Pneumonia is one of the diseases that causes the most fatalities worldwide, especially in children. Recently, pneumonia-caused deaths have increased dramatically due to the novel Coronavirus global pandemic. Chest X-ray (CXR) images are one of the most readily available and common imaging modality for the detection and identification of pneumonia. However, the detection of pneumonia from chest radiography is a difficult task even for experienced radiologists. Artificial Intelligence (AI) based systems have great potential in assisting in quick and accurate diagnosis of pneumonia from chest X-rays. The aim of this study is to develop a Neural Architecture Search (NAS) method to find the best convolutional architecture capable of detecting pneumonia from chest X-rays. We propose a Learning by Teaching framework inspired by the teaching-driven learning methodology from humans, and conduct experiments on a pneumonia chest X-ray dataset with over 5000 images. Our proposed method yields an area under ROC curve (AUC) of 97.6% for pneumonia detection, which improves upon previous NAS methods by 5.1% (absolute). Press Coverage Code

Recommended citation: Gupta, A., Sheth, P. & Xie, P. Neural architecture search for pneumonia diagnosis from chest X-rays. Sci Rep 12, 11309 (2022). https://doi.org/10.1038/s41598-022-15341-0.