Source-linked AI summary
Understanding Chatbot-mediated Task Management
Carlos Toxtli, Andrés Monroy-Hernández, Justin Cranshaw
TL;DR
Teams discuss and manage collaborative tasks in communication channels but must leave those channels to update tracking tools. The paper deployed TaskBot to mediate task management within chat and reports seven design insights from its use by information-worker teams.
Problem
Teams coordinate tasks in communication channels but have to switch to separate tracking tools, creating friction and inefficiency.
Source: INTRODUCTION · p. 1Method
The authors designed and deployed TaskBot, a bot that lets team members delegate task tracking and reminders within their chat.
Source: INTRODUCTION · p. 1, METHODS · p. 3Results
Eight teams used TaskBot to track 88 tasks, and the authors report findings on task assignment, self-reminders, human-like interactions, and managing multiple tasks.
Source: INTRODUCTION · p. 1Takeaways & Limitations
The authors present seven observed interaction patterns as lessons relevant to designing work-centric chatbots.
Source: DISCUSSION · p. 3TaskBot sometimes failed to understand polite or human-like requests because its language model was not trained for those utterances, or users asked for things beyond its capabilities.
Source: Handling human-like interactions with… · p. 4
Abstract
from arXiv · showhide
Effective task management is essential to successful team collaboration. While the past decade has seen considerable innovation in systems that track and manage group tasks, these innovations have typically been outside of the principal communication channels: email, instant messenger, and group chat. Teams formulate, discuss, refine, assign, and track the progress of their collaborative tasks over electronic communication channels, yet they must leave these channels to update their task-tracking tools, creating a source of friction and inefficiency. To address this problem, we explore how bots might be used to mediate task management for individuals and teams. We deploy a prototype bot to eight different teams of information workers to help them create, assign, and keep track of tasks, all within their main communication channel. We derived seven insights for the design of future bots for coordinating work.
INTRODUCTION
Teams manage collaborative tasks through communication channels but often leave those channels to update tracking tools, adding friction. The paper explores TaskBot as an in-channel mediator and reports findings from its deployment with eight teams.
INTRODUCTION
- Teams discuss and assign work in communication channels but must switch to separate task-tracking tools, while such interruptions can reduce productivity and increase workplace stress.The paper situates in-channel bots as a response to this friction.Source: INTRODUCTION · p. 1
- Chatbots can be summoned from communication channels so people can delegate tasks without leaving the chatroom, messenger app, or email client.The paper notes that bots already support varied work-related tasks in platforms such as Slack and Microsoft Teams.Source: INTRODUCTION · p. 1
- TaskBot lets team members mention it in chat and delegate the tracking and reminders for their tasks.The paper studies how bots can mediate task management through TaskBot’s iterative design, engineering, and deployment.Source: INTRODUCTION · p. 1
- TaskBot was deployed with eight teams of information workers, who used it to track 88 tasks, yielding findings about task assignment, self-reminders, human-like interactions, and managing multiple tasks.The authors present these findings as potentially useful to future designers.Source: INTRODUCTION · p. 1
RELATED WORK
Prior systems connect task tracking with communication through email extensions, proactive task identification, or collaborative task products. TaskBot offers conversational task management across messaging tools, while its experience includes task assignment and follow-up interactions.
RELATED WORK
- Earlier work extended email for task tracking or proactively identified tasks, but TaskBot leaves invocation under user control because precise task intentions are difficult to detect.TaskBot also differs from email-client extensions by not relying on a particular messaging client.Source: RELATED WORK · p. 2
- TaskBot builds on collaborative task trackers’ core functions through a conversational interface, contrasting with bots that use command-driven interactions.It uses Wunderlist as its backend.Source: RELATED WORK · p. 2
TASKBOT USER EXPERIENCE
- For the study, TaskBot focused on Microsoft Teams, although it was available on email, Skype for Business, and Slack.The described workflow has Alice ask the bot to track a task for Bob, after which the bot asks Bob for an ETA.Source: TASKBOT USER EXPERIENCE · p. 2
- TaskBot adapted follow-up timing to a specific time, replacing earlier twice-daily checks that pilot participants found too frequent and annoying.When a message mentioned multiple people, TaskBot assigned the task to all of them rather than using a round-robin approach.Source: TASKBOT USER EXPERIENCE · p. 2
IMPLEMENTATION
TaskBot stored task state in Wunderlist and used an intent model to handle users’ reactive and proactive messages. Figure 2 summarizes task usage by assignment type across study participants.
IMPLEMENTATION
- TaskBot stored tasks in Wunderlist through its API, using a central account owned by the bot for the study.The authors envisioned integration with users’ own accounts on task-tracking platforms.Source: IMPLEMENTATION · p. 2
- Figure 2 groups usage statistics for 19 users by tasks assigned to them, tasks they assigned to others, and personal reminders.Source: Figure 2 · p. 3
- The intent model handled two message categories: replies to TaskBot’s check-in questions and unprompted messages mentioning the bot in group chat or direct messages.These are labeled reactive and proactive messages, respectively.Source: IMPLEMENTATION · p. 3
- The authors first hand-built the intent model from keyword lists, then iteratively improved it using users’ utterances from several rounds of pilot testing.Pilots took place first within the authors’ team and then in their extended research group.Source: IMPLEMENTATION · p. 3
METHODS
The authors evaluated TaskBot in a week-long field deployment with eight information-worker teams, then coded user messages to characterize usage. The study combined team tasks, surveys, and message analysis.
METHODS
- The week-long field deployment involved eight teams of information workers, with two to five people per team and 19 participants overall.Five teams were hierarchical, all were collocated, and participants communicated synchronously, asynchronously, and face-to-face.Source: METHODS · p. 3
- Participants were asked to create at least three TaskBot tasks during one work week, and researchers administered surveys before and after the study.Teams worked closely on the same project and received a tutorial that included creating an example task.Source: METHODS · p. 3
- Two authors independently coded 177 user messages into 27 inductively developed categories, agreeing on 95.6% of messages with Cohen’s kappa = .76.They resolved remaining discrepancies through aggregation or discussion.Source: METHODS · p. 3
RESULTS
Across eight teams, participants created 88 tasks and marked 65 complete; survey responses showed both perceived value and annoyance. Task reminders and progress tracking were commonly rated important.
RESULTS
- Participants created 88 tasks, of which 65 were marked complete; the median was seven tasks per team.Table 3 presents task usage by team, but the supplied passage does not report its individual values.Source: RESULTS · p. 3, Table 3 · p. 3
- TaskBot received 177 messages while tracking the 88 tasks, and 22% of messages were task assignments.On average, users sent 12 messages and received 54 messages and 16 task reminders.Source: RESULTS · p. 3
- Six of 11 survey respondents felt more productive and eight said they would use TaskBot again, while four found it annoying.Ten of 11 considered reminders to others important, and nine of 11 valued task-progress tracking.Source: RESULTS · p. 3
DISCUSSION
The discussion identifies seven patterns from TaskBot use to inform work-centric chatbot design. Users valued in-chat task management but also raised concerns about managing multiple tasks and reminders.
DISCUSSION
- Users valued creating and tracking tasks in conversation, but found handling multiple simultaneous tasks and messages difficult.Participants also requested access through other channels they used, such as Skype for Business.Source: DISCUSSION · p. 3
- Five people found TaskBot reminders invasive or annoying, citing their frequency or lack of context.Some said reminders arrived at random times and interrupted focused work.Source: DISCUSSION · p. 3
Handling human-like interactions with bots
TaskBot use revealed both human-like interaction with the bot and self-reminders, despite training focused on assigning tasks to others. These patterns suggest design considerations for workplace chatbots.
Handling human-like interactions with bots
- Almost all participants (93%) interacted with TaskBot as if it were human at least once, and 20% never treated it as a bot.TaskBot sometimes failed on polite or human-like utterances because its language model was not trained for them, or because requests exceeded its capabilities.Source: Handling human-like interactions with… · pp. 3–4
- Workplace chatbot designers should either signal explicitly that a bot is not personified or handle human-like user responses resiliently.The paper reports that people often spoke naturally and politely to TaskBot.Source: Handling human-like interactions with… · pp. 3–4
Supporting self-communication
- Five users asked TaskBot to create reminders for themselves, sometimes as a practical way to record notes and tasks.The paper recommends that designers expect self-communication, including as a way to test a bot or use it practically.Source: Supporting self-communication · p. 4
Hierarchical task-assignment
TaskBot use varied with workplace hierarchy, and the authors recommend designing for graceful failures when requests exceed a bot’s capabilities. These observations motivate hierarchy-aware design and categorized failure handling.
Hierarchical task-assignment
- 83% of upward requests were reminders, compared with 47% of downward requests; task assignments were distributed nearly evenly across hierarchy directions.35% of tasks were assigned upward, 31% downward, and 34% among people at the same level.Source: Hierarchical task-assignment · p. 4
- Designers should expect different uses of the same social chatbot based on users’ hierarchy.The observed reminder shares differed between subordinate-to-manager and manager-to-subordinate requests.Source: Hierarchical task-assignment · p. 4
Failing gracefully
- TaskBot failed to understand about 10% of directed messages, with failures linked to unknown features and an insufficient training corpus.The passage separately reports that 10.5% of messages could not be understood and answered successfully.Source: Failing gracefully · p. 4
- Designers should categorize failures on novel requests to inform future bot versions, and consider mitigation such as amusing errors or escalation to humans.The recommendation is to fail gracefully when users ask for scenarios the bot cannot handle.Source: Failing gracefully · p. 4
Dealing with human ambiguity
TaskBot encountered ambiguity in both dates and references to people: users sometimes gave unclear times or omitted the @ syntax needed to identify a teammate. The findings point to resolving ambiguity and improving name detection as design concerns for conversational bots.
Dealing with human ambiguity
- 29% of dates and times mentioned in messages were ambiguous, and 60% of users mentioned at least one ambiguous date.The authors note that people may be intentionally ambiguous in conversation.Source: Dealing with human ambiguity · p. 4
- Because people may be intentionally ambiguous, chatbot designers need strategies to resolve ambiguities in line with user expectations.Source: Dealing with human ambiguity · p. 4
Identifying people’s name in conversations
- About 40% of users experienced issues when they omitted the @ symbol before a person’s name, preventing TaskBot from recognizing the intended assignee.The example describes a task addressed to John without the @ prefix.Source: Identifying people’s name in… · p. 4
- Designers should either nudge users to use expected syntax such as @, or improve detection of when a person is mentioned.Source: Identifying people’s name in… · p. 4
Handling multi-threaded conversations
TaskBot struggled to manage multiple active task conversations when a completion message did not identify which task it referred to. The conclusion identifies better thread handling as a future-work priority, alongside support for multiple communication channels and more sophisticated task assignment.
Handling multi-threaded conversations
- When users had multiple assigned tasks, “I’m done with the task” did not tell TaskBot which task they meant.The implemented menu listed all active tasks for cancellation or completion, but the interaction was not considered natural or elegant.Source: Handling multi-threaded conversations · p. 4
- Designers should invest in technology that identifies which active conversation thread a new message belongs to.Source: Handling multi-threaded conversations · p. 4
CONCLUSIONS
- TaskBot is introduced as a bot for team task management, with users delegating task tracking to it.Source: CONCLUSIONS · pp. 4–5
- Future work for TaskBot will explore multiple communication channels, better multi-threaded conversation handling, and more sophisticated task assignment based on task descriptions.Source: CONCLUSIONS · p. 5