Executive Summary of Outcomes: People are increasingly engaged in online dialogue, deliberation, and collaboration. The Internet provides opportunities for increased exchange of ideas, particularly with others who we may not have a chance to engage face-to-face.  There is under-explored opportunity for online systems and tools to directly support participants in having higher quality and more skillful engagements. The overall goal of the project is to support higher quality online deliberation, especially by supporting number of "social deliberative skills" such as perspective taking, empathy, self-reflection, tolerance for uncertainty, listening and question-asking skills, and meta-dialogue—in online contexts.  

            We attempt to do this through software tools and features, some of which directly support participants, and others which support a facilitator or mediator as they engage with participants. To support participants we implemented unobtrusive scaffolding features, and to support facilitators we implemented a "dashboard" visualization tool.  We also investigated using state-of-the-art text analysis and machine learning to measure important properties of deliberative dialogue.  In addition to the development and formative evaluation of these tools, we conducted experimental trials that showed, for a population of college students engaged in online discussion of controversial issues, that our "reflective tools" did indeed lead to deeper, more skilled, and reflective dialogue.  Our work with text analysis had given early indications that automated methods for assessing the quality of online dialogue can be used productively to support higher quality communication (for instance through visualizing this information in the Facilitator's Dashboard).

Wider potential impacts:  Our unobtrusive scaffolding methods and facilitator dashboard concepts will transfer to use in collaborative work, civic engagement, and online dispute resolution.  Discussion forums and commenting features are widely used in educational contexts and social media, but little exists to support higher quality deliberation in these environments. Our methods should also be applicable to "flipped" classrooms and MOOC (Massive Online Open Course) environments, which are in need of better tools for interaction, dialogue, and management.

Work and Results Summary: "The Fourth Party: Improving Computer-Mediated Deliberation through Cognitive, Social and Emotional Support" was funded for three years and was extending into a fourth year (no-cost extension).  The project was interdisciplinary and exploratory, meant to integrate research from several areas and map out future directions in an emerging sub-field.  The primary goal of the project was to study methods to support skills that help individuals successfully negotiate deliberative dialogues in which they are challenged by differences in perspectives, goals, assumptions, etc.—in online contexts specifically. The project produced 17 peer-reviewed research papers and an additional 10 articles, workshop presentations, and radio show interviews. Significant progress was made over the grant period in several areas that can be summarized as follows:

1. Experimental trials of scaffolding features Experimental trials in college classrooms were used to study the effectiveness of passive scaffolding features hypothesized to support social deliberative behavior. Results summary: A combination of the following Reflective Tools were found to have a significant effect (large effect size) on total deliberative skillfulness: meta-dialogue support; personal-stake-and-story support; and productive-reply-reminders.  Intersubjective speech acts (such as questioning, reflecting back, and perspective taking) were particularly affected by these tools.
2. Text Analysis for dialogue quality Research on the use of state-of-the-art text analysis  methods to identify indicators of deliberative dialogue quality, including participant skillfulness. Results summary: the combination of L1 Regularized Logistic Regression with psycholinguistic features (LIWC) showed the best performance on predicting overall deliberative skill and intersubjective speech.
3. Facilitator's Dashboard Prototyping of a Facilitator's Dashboard for assessing and visualizing dialogue quality in online deliberation. Results summary:  The Dashboard was designed over several iterations with input from teachers and professional mediators and online facilitators.  Visualization tools include participation charts and trends, demographic differences, word cloud, and social network diagram.  "Intelligent" indicators using text analysis for several analysis types were implemented.  Evaluation for strictly formative.  Students were interviewed about the possibility of peer-based visualization tools as well.
4. Comparative analysis of diverse dialogue domains We began a comparative analysis of a number of dialogue domains assessing dialogue properties and predictive potential of text analysis. Results summary:  Initial work was done to compare dialogue properties across diverse domains and populations, with pre-existing online discussion data, for these domains: college classes, deliberation within a professional association, a teenage social networking forum, online dispute resolution services, and e-commerce dispute resolution.  Notable patterns were observed regarding these speech act types: intersubjective speech, meta-dialogue, agreement, appreciation, referencing sources, and self-reflection.
5. Theoretical framework Developing a theoretical framework for the analysis and support of social deliberative skills. Results summary: Related literature from cognitive science, communications theory, deliberative theory, conflict and peace studies, and educational pedagogy were synthesized and interrelated.  A novel conceptual framework was developed that sees deliberative skills as the application of cognitively oriented skills to the emotional/social/intersubjective realms.