Articles
A conversation with Danish Contractor
advancing chatbots and intelligent question-answering systems to support complex human querying
Ubiquity, Volume 2019 Issue September, September 2019 | BY Bushra Anjum 
Ubiquity
Volume 2019, Number September (2019), Pages 1-5
Innovation Leaders: A conversation with Danish Contractor: advancing chatbots and intelligent question-answering systems to support complex human querying
Bushra Anjum
DOI: 10.1145/3363787
In this series of interviews with innovation leaders, Ubiquity Associate Editor and software engineer, Dr. Bushra Anjum sits down with Danish Contractor, a senior artificial intelligence researcher at IBM Research India. Contractor discusses his work advancing question-answering systems and chatbots for complex human interaction.
Danish Contractor is a senior artificial intelligence (AI) researcher at IBM Research India (New Delhi) where he works on problems in multi-sentence question-answering and dialog systems. Over the last nine years, he has been working on machine learning (ML) and natural language processing (NLP) applications in the areas of education, tourism, social media, customer relationship management, and information management. In 2018, Contractor was named one of the "Top Innovators Under 35 in India" by MIT Technology Review and Mint. He holds a master's degree from the University of Cambridge and a bachelor's degree in computer engineering from the University of Delhi. Contractor is a member of the ACM Future of Computing Academy and can be reached at [email protected], and via Twitter @danish_c.
What is your big concern about the future of computing to which you are dedicating yourself?
The internet has changed the way we store, organize, share, and search for information. With improvements in the field of AI, question-answering systems and dialog agents (chatbots) are making their way into our lives in the form of digital personal assistants, self-help bots, etc. For instance, AI applications in education require interacting with students. Students may ask questions, seek clarification, or need tutoring for problem-solving. Similarly, problems in business, such as those of customer support, are centered on question-answering and conversations.
In 2017, it was estimated that 265 billion customer support requests are made every year, and that accounts for USD 1.3 trillion of spending by corporations. According to Chatbots Magazine, 30 percent of service costs could be reduced by deploying virtual agents and chatbots [1].
Despite breakthroughs in specialized AI tasks, user engagement, and experience with such real-world systems is far from satisfactory. Most question-answering systems today can only answer relatively simple questions. The questions should be no more than one to three sentences long, have a lucid expression of language, should be answerable by searching for text mentions or by formulating structured queries, and most importantly, involve minimal reasoning. However, human language is complicated, and it evolves. There are multiple ways of stating the same thing; there are metaphors, humor, and sarcasm; all of which are complex constructs for a machine to understand. Similarly, for chatbots, including those built using IBM's Watson Assistant, anticipating how users might communicate with a bot as well as designing conversation flows are hard problems.
How can we make question-answering systems better? How can we support complex real-world questions using chatbots? Addressing some of these challenges form the basis of my current research.
How did you become interested in advancing question-answering systems and chatbots for complex human interaction?
I am currently working at IBM Research India, New Delhi, as a senior research scientist in AI. In the last nine years, I have worked on research problems in NLP and ML, with a focus on real-world applications, including education, tourism, and social media. During this time, I realized how conversing and question-answering are at the core of human-computer interaction.
Since I was a kid, conversational agents and machine intelligence have been a hallmark of sci-fiction movies. NLP program ELIZA, created at MIT in the '60s, was one of the more famous attempts at building one. While going through grad school, I studied the advancements in NLP and ML techniques. When combined with the large volumes of data available today, they can enable building statistical models that can infer patterns in the expression of language like never before. Further, specialized datasets [2] and international collaborations in NLP research is helping us develop reusable components, such as pre-trained word and sentence embeddings [3]. I am excited about the future as we combine these learnings into developing better bots.
Conversing meaningfully and being able to answer questions is critical for intelligent systems. We are far from building a machine that can converse as intelligently as a human being. However, small yet significant breakthroughs in language understanding keep me hopeful that one day soon, we will get there.
What project are you currently leading that have the potential to improve the applicability and understanding of chatbots and question-answering systems?
To improve the design of chatbots, we have been exploring intent modeling and dialog flow, based on human-to-human interactions. Some of this work has been incorporated in IBM Watson Assistant in the form of intent recommendations that are surfaced by analyzing conversation logs [4]. These recommendations help the designer identify and prioritize the high impact intents to start modeling first. For example, if the logs contain conversations in which users reported problems with printers, then the system might identify the types of problems reported along with examples of how users expressed them in language. These groups of examples create intents that are made available to a chatbot designer as intent recommendations.
I am also working on building systems that can answer questions, users post on forums. I have been working on travel queries where users seek recommendations for places to visit, stay, and dining, etc. Answers to these questions are usually names of entities such as the name of a restaurant. We are building methods that utilize large text collections of entity reviews to answer such questions automatically [5]. The work involves being able to understand a user's preference or constraints as well as distill knowledge from the vast collection of reviews describing each of those entities [6].
I am always on the lookout for interesting real-world problems and avenues for collaboration. Please feel free to reach out to me if you are interested in knowing more about these or related areas.
References
[1] Reddy, T. How chatbots can help reduce customer service costs by 30%. IBM. (Oct. 17, 2017)
[2] SemEval-2019. International Workshop on Semantic Evaluation. 2019.
[3] Wolf, T. The current best of universal word embeddings and sentence embeddings. Medium (May 14, 2018).
[4] Desmarais, C. Building intents from human to human logs - Watson assistant. Medium (Nov. 27, 2018).
[5] Contractor, D., Patra, B., Mausam, M., and Singla, P. Understanding complex multi-sentence entity seeking questions. AAAI 2019 Reasoning for Complex Question-Answering Workshop. AAAI, Menlo Park, 2019.
[6] Contractor, D., Shah, K., Partap, A., Mausam, M., and Singla, P. Large scale question answering using tourism data. 2019. arXiv:1909.03527v1 [cs.CL]
Author
Bushra Anjum is a software technical lead at Amazon in San Luis Obispo, CA. She has expertise in Agile Software Development for large scale distributed services with special emphasis on scalability and fault tolerance. Originally a Fulbright scholar from Pakistan, Dr. Anjum has international teaching and mentoring experience and has served in academia for over five years before joining the industry. In 2016, she has been selected as an inaugural member of the ACM Future of Computing Academy, a new initiative created by ACM to support and foster the next generation of computing professionals. Dr. Anjum is a keen enthusiast of promoting diversity in the STEM fields and is a mentor and a regular speaker for such. She received her Ph.D. in computer science at the North Carolina State University (NCSU) in 2012 for her doctoral thesis on Bandwidth Allocation under End-to-End Percentile Delay Bounds. She can be found on Twitter @DrBushraAnjum.
©2019 ACM $15.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2019 ACM, Inc.
COMMENTS