Collaborative Writing in the Intelligence Community: Investigation of Analytic Collaborative Reporting Processes and Writing Strategies

With new and evolving technology and ever-increasing remote collaboration, intelligence reporting processes are at a crossroads. Key questions about how best to collaboratively produce reports to get the right information to the right decision-makers in the right context are more important than ever. This research project has attempted to address one part of the puzzle: How can reports be produced collaboratively? In this article, throughout two studies, the researchers documented the processes of report generation and consumption as well as assessed three collaborative writing strategies to gain valuable research data and to aid analysts with collaborative reporting, improve collaboration, and advance reporting methodology within the Intelligence Community.


INTRodUCTIoN
Analysts in the intelligence community (IC) gather information from various sources, synthesize the information, and provide their insight through reports.These reports, historically considered standalone work products, cater to diverse audiences (Bruce, 2008).This perception is rooted in the IC's compartmentalized style of working (Hackman & O'Connor, 2004;Treverton, 2016).As technology continues to evolve alongside the surge of remote work, the processes of report generation in the IC are transitioning towards more collaborative and audience-specific approaches.Changes within the IC post 9/11 have underlined the need for increased collaboration, prompting shifts in various workflows, including report writing.
Founded as a partnership between the National Security Agency and North Carolina State University (NC State), the Laboratory for Analytic Sciences (LAS) has been fostering and studying collaborations between the IC, NC State, and industry partners since 2013 (Jameson & Stacy, 2016;

BACKGRoUNd ANd ReLATed WoRKS
There are distinct bodies of literature on analytic team development (see, e.g., Eby et al., 1999;Hackman & O'Connor, 2004), business process improvement (see, e.g., Page, 2015), and CW (see, e.g., Arinze, 2012;Cabanac et al., 2014;Gimenez & Thondhlana, 2012).Each body of literature has its own terms, theories and approaches, which may parallel some of the concepts within this investigation.For example, Hackman and O'Connor (2004) invested in the differences between work teams and co-acting groups, which could explain the variation in CW structures (Hackman & O'Connor, 2004).Others have been investigating collaborative and analytic team development including analytic team workflow (see, e.g., Eby et al., 1999;Hackman & O'Connor, 2004) and trust in team decision making (see, e.g., Pearson et al., 2016).Covering each body of literature with their unique terms and theories leads to confusion.While team development and general business process improvement are important aspects, they fall outside the scope of this investigation.Therefore, in this review, we focus on the literature concerning CW and intelligence reporting.
In the context of the IC, collaboration typically manifests when an analyst solicits help or insights from other individuals such as fellow analysts or subject matter experts.However, Treverton (2016) suggests organizational silos or "stovepipes" (p.19) can often constrain collaboration, creating barriers that impede the free exchange of information and ideas.
Recognizing and embracing collaboration within the IC has significant implications for intelligence reporting, especially for reports that require the integration of diverse knowledge and skills spanning multiple organizational boundaries (Bruce, 2008;Johnston, 2005;Nolan, 2013).The process of collaboration brings together a broad range of perspectives, enhancing the depth and richness of the final product: the report (Ackerman, 2009).
Nevertheless, collaboration is not without its disadvantages.For example, the time and effort invested in coordinating collaborative activities can outweigh the resulting benefits (Treverton, 2016).Strategies for effective collaboration must therefore focus on maximizing gains while minimizing costs.This is achieved through employing appropriate collaborative tools, improving communication channels, and fostering a culture of mutual trust and respect among the collaborators (Eapen, 2007;Schuster & Karis, 1991).

Intelligence Reporting and Collaborative Writing (CW)
We purposely narrowed the scope of this investigation to focus on challenges associated with CW in the IC.The justification for this deliberate emphasis is rooted in the unique symbiotic relationship between CW and intelligence reporting and the transformative potential of improved CW strategies within the IC.
The relationship between CW and intelligence reporting is intertwined.Intelligence reports synthesize multifaceted data from various sources, necessitating collaborative input from multiple analysts.This data collation and synthesis process culminates in CW, making it a vital component of the broader collaborative reporting landscape.
Writing, or reporting, in the IC is often a solo activity (Treverton, 2016).For example, the process of writing in a collaborative manner is lacking in the discussions regarding intelligence writing and training for intelligence analysts' writing skills (Dorn, 2019).Dorn (2019) focuses on assessing the outcome of a training program where intelligence analysts were trained in writing skills.However, the curriculum in the program lacks collaborative perspective.
Similarly, Major (2014) stated that "intelligence is interpretation of fact, not simply recitation of facts" (p.28).But the book positions writing and the development of intelligence from the perspective of a single individual, whereas, based on the framework presented in this research, CW involves intelligence generated and interpreted by multiple individuals, which produces a better product.The book only once mentioned a group exercise in argument development but did not tie it to writing (Major, 2014).The reference noted that IC writers must unlearn years of schooling to learn to produce IC reports; IC reports require a unique approach or mindset.
While CW has been well studied in academic environments and workplaces, it is an emerging area within the IC (Nolan, 2013).In the IC, CW brings together a diverse range of expertise, improving problem-solving as highlighted by National Research Council (2011) in their report.However, the culture and norms in the IC differ from other sectors, making the application of findings from studies outside the IC somewhat challenging (Lefebvre, 2021;Pfaff & Tiel, 2004;Vogel & Dennis, 2018).CW is an interactive and social process that "involves a team focused on a common objective that negotiates, coordinates, and communicates during the creation of a common document" (Nakamura & Csikszentmihalyi, 2002, p. 90).Based on the writing task and the goal of the collaboration, CW includes many different strategies, team roles, and team workflows.Lowry et al. (2004) categorized five typical CW strategies: 1) sequential writing; 2) group single-author writing; 3) parallel writing: stratified-division writing; 4) parallel writing: horizontaldivision writing; and 5) reactive writing.Figure 1 illustrates the five typical CW strategies that were introduced in their work.
Group single-author writing is defined as one person who writes for an entire group.Sequential writing is a variation of group single-author writing.In sequential writing, one person writes at a time.Each person completes their portion and then passes the work to the next person.In contrast, parallel writing occurs when a team divides the CW work into subunits and each author completes their own section or task.Parallel writing can be further divided into two types: Horizontal-division writing and stratified-division writing.In Horizontal-division writing, each person is responsible for a section of the document.Stratified-division writing, on the other hand, is a form of parallel writing in which each person plays a role, such as author, editor, or reviewer, based on their job responsibilities or expertise.Reactive writing occurs when multiple writers work on a document in real time, rearranging or rewriting one another's work without preplanning.
While reactive writing is not inherently cloud-based, it is greatly enhanced by using cloud-based tools.Recent studies have revisited how people write together using modern CW tools like Google Docs, Microsoft Office 365, or other collaborative editing platforms (Ahmad, 2020;D'Angelo et al., 2018;Olson et al., 2017;Wang et al., 2019;Yim et al., 2017).The role of tools designed to support CW has been a focal point of numerous studies.Communication tools, real-time editing platforms, file storage and synchronization services, and knowledge creation and information sharing tools all aim to streamline and expedite collaborative processes (Dishaw et al., 2011;Eapen, 2007;Jones, 1994;Lowry et al., 2003;United States Department of Justice, 2006;Wahl & Kitchel, 2016).These tools offer new avenues to foster collaborative reporting within the IC.A significant issue for IC analysts, however, is the adaptation of tools initially designed for individual report generation to support collaboration.

Challenges and Barriers to Reporting and CW in the IC
Treverton (2016) reported four barriers to collaborative tool use in the IC: 1) organizational culture does not encourage collaboration, 2) lack of trust among users, 3) lack of incentives to collaborate, and 4) practical difficulties in using the tools.
Report generation, or knowledge creation, and consumption, or information sharing, strategies for collaborative reports are more complex than those used to create reports with traditional workflows (Ackerman, 2009;Linden, 2010;Noël & Robert, 2004;Tammaro et al., 1997;Treverton, 2016).Reporting in a collaborative manner, like other types of teamwork, offers benefits and challenges.Authorship, leadership, and group interaction are some of the challenges that have received the most attention (Gooch, 2005;Noël & Robert, 2004;Zutshi et al., 2012).
The barriers to collaborative reporting arise from not knowing its appropriate structures.A study suggested that even for people who frequently generate reports in a collaborative environment, no formal name was given for the writing strategies they used in their workflow (Ede & Lunsford, 1990).Findings from the study demonstrated that people often do not recognize, by name, the CW strategies they have pre-established, even if they have used these strategies for a long time.In addition, strategies for CW are frequently chosen based on an individual's personal preference or experience, or by default based on project organization, instead of being consciously planned to optimize outcomes.Similarly, defining when to use a particular writing strategy based on the pros and cons documented in the literature needs to be contextualized to produce intelligence reports.

Summary
There is a lack of research on CW processes and their relationship with final product quality in the IC.One of the most significant factors is the veil of secrecy and stringent security protocols surrounding the IC, which restricts access to detailed processes of their operations.The IC often employs proprietary systems and tools in their operations, which are inaccessible to external researchers, limiting the scope of study in this area.Additionally, there is a limited number of publication outlets due to the sensitive nature of the IC's work, with rigorous review processes to ensure no inadvertent disclosure of sensitive information.Despite these hurdles, studying CW in the IC remains a valuable and underexplored field.
Since existing CW investigations have focused on student and industry reporting, there remains a gap in the research assessing CW strategies' impact on the quality of reports generated by IC analysts.It is critical to have IC professionals (e.g., reporters and reviewers) participate in the research for results to resonate within the IC.

ReSeARCH QUeSTIoNS, AIMS, ANd oBJeCTIVeS
Initially, in this study, we identified the differences between general collaborative reporting (i.e., report writing processes among analysts who are outside of the IC) and collaboration in the IC (i.e., report writing processes among the IC analysts).This investigation into the contrasting areas will extend the generalizable body of knowledge by addressing the IC's unique collaborative reporting approach.
Next, we explored three existing CW strategies (described in the Related Works section) within the IC workflow to evaluate their applicability and potential for success.More specifically, we accomplished this by addressing the following research questions (RQs): RQ1: What are the needs of the end users (i.e., intelligence report consumers) in consuming an intelligence report?RQ2: What are alternative approaches to collaborative report generation to meet the needs of those consumers?RQ3: How do intelligence analysts' current report generation and consumption strategies differ from those documented in existing literature (e.g., CW, communication, e-research collaboration, etc.)?RQ4: What are the unique needs and approaches of intelligence analysts and report writers compared to analysts in similar fields (e.g., legal, finance, engineering, and law enforcement)?RQ5: To what extent might the shift of reporting strategies change intelligence analysts' workflow?RQ6: How do report writing strategies impact the reports for efficiency, quality, and veracity, and documenting work, where the report is considered the final product?
We addressed these questions in three phases: 1) RQ1 and RQ2 were addressed through literature review, which included intelligence policy and reporting standards reviews, and cross disciplinary team discussions and synthesis, involving intelligence analysts/reporters and report consumers as team members; 2) RQ3 and RQ4 were addressed in Study 1; and 3) RQ5 and RQ6 were addressed in Study 2. The intended outcome from the two studies was to formulate recommendations that improve methods of collaborative report generation in the IC resulting in improvements in report efficiency, quality, and veracity.

STUdy 1: UNdeRSTANdING THe PRoCeSS oF CoLLABoRATIVe RePoRT GeNeRATIoN ANd CoNSUMPTIoN IN MULTIPLe CoMMUNITIeS
The purpose of this study was to answer RQ3 and RQ4 by documenting analysts' current report generation and consumption strategies including methods of collaboration in multiple communities (e.g., IC, law enforcement, legal, financial, and power grid supervisory control and data acquisition [SCADA]).Data was collected using semi-structured interviews and online surveys.We summarized and visualized multiple stakeholders' perceptions of the concept of collaborative report, formal training status on reporting, barriers to report generation and consumption, common tools for report generation, and collaborative reporting workflows.

Participants
The LAS is interested in challenges faced and surmounted by IC and non-IC counterparts.Thus, analysts from IC and non-IC professions were recruited.This study was approved by the NC State's institutional review board (IRB) and National Security Agency (NSA) Human Research Protection Official (HRPO) Review.The criterion for inclusion in this study was that the individual's work required information synthesis and documentation of the interpretation in the form of a report.Participants included eight IC analysts, two financial analysts, two lawyers, two engineers, and one law enforcement officer.This study was conducted in 2018.

Procedure
A set of 50 questions regarding information synthesis and report development were formulated.Two methods of data collection were used.Semi-structured interviews were conducted with non-IC participants (n = 7).Since IC analysts could not participate in interviews (n = 8), they responded to an online survey.The survey was developed on the low side using Qualtrics and distributed on classified systems using a similar proprietary polling program.Informed consent was provided at the beginning of the interview or using click-through electronic consent.

data Analysis
Survey data was printed, reviewed for content, and approved for release before it was provided to the researchers.Printed survey results were digitized and converted to Microsoft Word documents to support consistent analysis with interview results.Audio files recorded during the interview were transcribed into Microsoft Word documents.Transcripts were combined with the notes the researcher took during the interview.Respondents' contents were categorized into five areas: 1) definition of the collaborative report from the participants' point of view, 2) formal report training status, 3) barriers, 4) tools, and 5) workflows in report generation and consumption and collaboration.

Term Definition
Participants were asked to review the provided definition of the term "collaborative report" and answer whether the provided definition was consistent with their own definition.Thirteen out of 15 participants agreed with the provided definition of collaborative report which was "a report generated by two or more people."The main concern for participants who disagreed with the definition was the term "people," especially for participants from the IC.They considered that a collaborative report should span across agency boundaries and be produced by two or more different organizations or agencies.

Formal Report Training Status
During the interviews, three out of seven IC participants commented that there was no formal job training regarding report generation.Analysts in the IC reported more formal and systematic training than participants in non-IC.The training for IC analysts included report formation, compliance and policy, analytic standards and logical argumentation, tools and technology, writing skills, and advanced editing and releasing.

Barriers
The barriers in report generation, consumption, and collaboration were collected from both IC and non-IC participants (see Table 1)."Accessing information" and "time" were the most frequently mentioned barriers among all participants.However, the distribution of these two barriers was different between IC and non-IC sectors.IC analysts indicated that time was the biggest barrier in report generation, report consumption, and collaboration.Whereas accessing information was most often a barrier during the report generation and consumption.Non-IC participants mentioned that it is difficult for them to understand terminologies from other areas during the report consumption process.Also, the quantity and quality of the information provided to them was sometimes insufficient and inadequate."Conflicting analysis" was the most frequently mentioned barrier for all participants when generating a report collaboratively.Egos, opinions and disagreements, lack of clarity for the goal of a report, and different levels of understanding of the information in the report were mentioned to explain the conflicting analysis.

Tools
The most common report generation tools for non-IC participants were Microsoft Excel, Microsoft Word, Google Docs, and Email.They usually used Email, paper-based documents, a PDF viewer, and Microsoft Word for report consumption.Both SCADA participants used Microsoft Cloud for report generation and consumption.Due to the classified nature of their work and processes, however, IC participants were unable to provide details about specific tools.Instead, they focused on the function of the tools such as a database query tool, slide generator, data aggregation tool, or data organization tool.These can be compared with those documents in the analyst toolbox included in United States Department of Justice's Global Justice Information Sharing Initiative Intelligence Working Group (U.S.Department of Justice) ( 2006).The advantages and downsides regarding some of the specific tools collected from the participants in the interviews and surveys are presented in Figure 2.

Workflows
Data collection and report writing, reviewing, and releasing were the common steps in the report generation workflow for the IC participants (see Figure 3).Based on the survey responses, collaboration usually occurred in the outlining (data collection and initial research), report writing, and report reviewing phases.

Figure 2. Summary of reporting tools and users' feedback and comments on tool features
For non-IC participants, data collection, feedback and review, and report generation were the common steps in their report generation workflow (see Figure 4).Collaboration occurred during data collection, data review, report generation, and feedback and review phases of report generation.

discussion
In this study, we addressed RQ3 and RQ4 by providing insights into the distinct practices and unique requirements of intelligence analysts in the context of report generation and consumption.Findings indicate that intelligence analysts apply strategies not extensively documented in existing literature, setting them apart from professionals in other sectors such as legal, finance, engineering, and law enforcement.The specific definition of collaborative report generation, the emphasis on formal report training, the adoption of specialized tools, and the collaborative stages in their workflow, suggest a unique set of needs and approaches among intelligence analysts.This understanding contributes to improving collaborative reporting workflows in the intelligence community, directly addressing RQ4.
The definition and practice of collaborative writing differed significantly among participants from various communities.Unique was the perspective of law enforcement participants, who defined collaborative reports based on the report on consumers' needs, considering reports as collaborative products as they integrated information from multiple sources.In contrast, other participants interpreted the term to signify a product of a cooperative working process.This discrepancy may arise due to the inherent differences in the work culture, regulatory environment, or the nature of tasks in law enforcement compared with other sectors.
During the interview, the law enforcement participant mentioned that they usually do not generate reports collaboratively.They generate reports based on the facts they obtained and observed on site, which is considered to be an individual activity.However, they considered their reports to be the products of collaboration since they required information from multiple sources or organizations (e.g., information from the witness, information from paramedics, etc.).The responses from all other non-IC participants described collaborative activities such as reviewing the report, adding content, providing comments, or working together.
When defining a collaborative report, SCADA participants noted that one person assigns individual names to each section and that they "don't make a big report, just separate reports."These participants were senior engineers with more than eight years of postsecondary education in their field.They mentioned that this writing strategy is the most common one they use in their job and that they also used it while in school.The SCADA participants' strategy for collaborative reporting resembles the horizontal-division writing documented by Lowry et al. (2004).
IC and non-IC participants also had different experiences with formal training in report generation.Although many non-IC participants indicated that they had not received any formal training regarding report generation for their professional work, some of them mentioned that the skillsets they were using for the report generation in their job-including using Excel and PeopleSoft, querying databases, and technical writing-were generally learned during their postsecondary education.Hence, they implied formal report writing training was not necessary considering that the reports they generate do not require a specific format or use any specific software or database.Due to IC-specific reporting workflows and distribution policies, however, IC participants reported more formal training in report generation (e.g., formatting, policy and compliance, and advanced releasing).It is worthwhile to explore how such formal training impacts on the quality and efficiency of the reports generated and whether it provides a significant advantage in the IC.
As for report generation and consumption tools, there are some inconsistencies among the tools listed in the analyst toolbox (U.S.Department of Justice, 2006) and the tools collected from the analysts in the five different communities in this study (see Figure 2).Some of the data collected do not fit into the generic tools listed in the analyst toolbox.One of the reasons is that the it was published in 2006.At that time, real-time document and file-sharing services such as Microsoft Cloud, Google Docs, and Google Drive were not yet on the market.Additionally, the tool list did not specifically mention collaboration and tools for collaborative work.Given that collaboration is one of the most important topics investigated in this study, we specifically asked participants about their collaboration tools.The IC participants mentioned many writing and editing support tools such as a grammar checker, spell checker, thesaurus, dictionary, formatting stylebook, and various usage manuals.However, the analyst toolbox does not cover writing and editing support tools.Future research should delve into this aspect, evaluating the impact of these tools on CW.The inconsistencies may also be attributed to the diversity in resources and target audiences in this study.Since the analyst toolbox tool list was published by the U.S. Department of Justice, the data is assumed to have been collected from people who work in law enforcement, while the tools in this study were identified by participants from the IC, legal, SCADA, finance, and law enforcement sectors.In this case, our results included specific data management and query tools which may not be used in law enforcement such as Oracle database, PeopleSoft, Bank system, and Grant Enterprise Management System (GEMS).
The differences in workflows depicted in Figures 3 and 4 indicate that there were significant differences among the five communities in their report generation workflows, especially at the level of collaborative work.Participants from IC, finance, law enforcement, and SCADA appeared to have lower levels of collaboration as compared with participants from the legal sector.In communities that engaged in collaborative reporting less frequently, collaborative activities typically occurred in the first half of the workflow.When collaborative activities were more frequent, there were iterations between the current phase and the previous process to improve the content of the report.This iterative process was most frequently reported in the legal sector.The differences in the level of collaborative work across sectors could be due to each sector's unique nature.For instance, the legal sector's iterative process is necessary for multiple reviews and revisions to ensure accuracy and compliance.In contrast, sectors dealing with sensitive or classified information, such as IC and law enforcement, may engage less in collaborative work due to confidentiality concerns.Customizing collaborative reporting methods to each sector's specific needs will enhance effectiveness and efficiency.

Limitations
One limitation of the study is that, due to the nature of the IC, workflows and the name of some specific tools are classified and cannot be shared with the researchers during the interview or in the survey response.Another limitation, influenced by the IC's reporting style, is the focus of this investigation on text components of reporting.Additionally, due to the cross-sectional nature of this study, it may not capture the evolution of report generation strategies and tools over time.And the study might not account for specific departmental or organizational policies that could influence the report generation process within each sector.Future research should aim to address these limitations by including a larger, more diverse sample; employing longitudinal designs; and considering organizational nuances.

STUdy 2: A WoRKSHoP-BASed CoLLABoRATIVe RePoRTING INVeSTIGATIoN
With the IC's interest in increasing collaboration in intelligence work, it is necessary to consider whether it is worth creating the tool for the IC focusing on promoting collaborative reporting.A primary insight from Study 1 was that stakeholders choose different CW strategies based on efficiency, job responsibilities, and work environment.For example, the IC analysts reported using a stratifiedstructure strategy, which includes a single-writer along with an editor, a reviewer, and a releaser.Conversely, people from law enforcement, finance, and SCADA are more likely to work separately at the beginning, be responsible for one section of the report, and assemble individual sections afterward when they collaborate.The CW strategy in the IC is a variation of group single-author writing strategy in Lowry et al. (2004) and the latter is more like horizontal-division writing strategy.
The purpose of the study is to examine the impact of alternative CW strategies on report generation by assessing teamed analysts' work product in a workshop.It was conducted to answer RQ5 and RQ6.
The teams of analysts in the workshop collaborated when generating their report.This allowed the researchers to gain a better understanding of how to develop analytic tradecraft to aid analysts with collaborative reporting.The goal was to improve collaboration and advance reporting methodology within the intelligence community.
The three strategies, modified from Lowry et al. (2004), assessed through the workshop were: 1) single-author writing (SAW), a modified version of group single-author writing strategy; 2) horizontaldivision writing (HDW); and 3) reactive writing (RW).In this study, 12 intelligence analysts worked in groups to generate collaborative reports for preselected topics using each CW strategy.Groups used open-source data and software tools to generate reports using various strategies of CW.The implementation of these strategies was examined from the perspective of verbal communication, tool usage, group work engagement, group workflow, and final product assessment.In addition to the 12 intelligence analysts, four senior analysts also engaged with this study by identifying topics of interest and assessing the reports generated during the workshop.

Participants
Two sets of participants were recruited for this research study: intelligence analysts as workshop participants (n = 12) and senior analysts for the workshop topic recommendation and as report reviewers (n = 4).This research study was approved by IRB and NSA HRPO Review.To be eligible to participate in the three-day workshop, an individual had to have been currently or previously been working as an analyst in the IC and have generated reports as a part of the work.To be eligible to be a report reviewer, an individual had to have been currently or previously been working as a senior analyst in the IC and have consumed reports as a part of the work.Potential participants were recruited by an LAS researcher using snowball sampling.This study was conducted in 2019.

Data Collection Tools and Materials
Four data collection tools were used to examine the impact of alternative CW strategies on report generation.Two tools were used as self-report materials for participants: state of flow surveys and a postworkshop questionnaire.Observation sheets were used by the research team during the workshop and the report assessment survey was used by the report reviewers.

State of Flow Survey:
The purpose of this survey was to identify whether an individual was "in flow" during the CW processes by measuring the indices of relative challenge and skill.Nakamura and Csikszentmihalyi (2002) described that "when in flow, the individual operates at full capacity" (p.90).An individual in a state of flow had characteristics such as "intense and focused concentration on what one is doing in the present moment," "merging of action and awareness," and "a sense that one can control one's actions" (Nakamura & Csikszentmihalyi, 2002, p. 90).In the Nakamura and Csikszentmihalyi (2002) model, flow is experienced when perceived challenges and skills are above the actor's average levels.The survey is a modified version of the experience-sampling form based on the concept of flow and the experience-sampling method (ESM) (Csikszentmihalyi & Larson, 2014;Nakamura & Csikszentmihalyi, 2002).In this study, flow was identified when participants rated their perceived challenges and skills (details provided in Procedure section below) about the main activities greater than or equal to five in the 10-point scales.
Postworkshop Questionnaire: The purpose of the questionnaire was to capture participants' descriptions of their group's workflow and insight about that day's writing strategy.In addition to open ended questions capturing descriptions and insights, multiple choice questions were used to capture the writing priorities and criteria for the collaborating authors.Observation Sheets: The observation sheets provided the research team with guidelines and a consistent format for documenting participants' activities.To maintain consistency and improve the efficiency of documentation, researchers used predefined codes to represent some of the activities.A variety of group and individual activities were documented including in-person verbal communication between group members, tool usage, type of group work engagement, main activity of the group, consistency in following the writing strategy, and shift in individuals' roles within the group.For the activities observed on the individual level, researchers used role A, B, C, and D to represent each participant in the writing group (details provided in Procedure section below).

Procedure
We conducted this study in two phases: the workshop phase and the report review phase.During the workshop phase, consisting of three one-day workshops, data were collected by the research team using observation sheets, surveys, and questionnaires.During the review phase, data was collected using surveys.
Workshop: At the beginning of each day-long workshop (8:30 a.m.-4 p.m.), participants were placed into three groups.Each group was assigned a specific CW strategy and each participant was assigned a certain role (see Figure 5).Detailed information and instructions about each role (A, B, C, and D) and CW strategy workflow (SAW, HDW, and RW) were provided to participants.For participants in the groups that were using the SAW strategy, in each group, the participant who was assigned role A wrote for the whole group while B, C, and D collaborated as document contributors.Upon completing the document, those in roles B and C then functioned as editor and reviewer, respectively.For HDW, each participant functioned as contributor or section writer and was only responsible for one section of the document.Upon completing all sections, participant A then functioned as final document assembler who integrated all the sections.For RW, all participants functioned as contributors or cowriters and contributed equally to a document in real time.This workflow required them to react and adjust to others' changes as needed throughout the report generation process.

Figure 5. Structures and roles for three CW strategy groups
To limit the impact of repeated exposure and familiarity, assignments were made such that participants 1) used at least two different writing strategies over the course of three days and 2) each group of four participants was teamed together only once.After forming groups, a written brief on the day's topic, which included background and questions to be answered, was presented and participant questions were answered.
Groups used open-source data and software tools (e.g., Google Suite) to generate their collaborative reports.During the workshop, the research team observed and documented participants' activities in 10-minute increments during the three hours and 40 minutes of their CW activities.Twenty-two data collection points were documented for each group per day.Participants completed the state of flow surveys while generating their report-twice in the morning and once in the afternoon.The survey captured the main activity participants were engaged in, how they felt about the activity, and ratings of the perceived challenge and skill required.These were rated using 10-point scales where 0 = low and 9 = high.After completing the survey, participants continued their CW activities.
At the end of each workshop, groups submitted the final reports in Google Docs and each participant completed the postworkshop survey.
Report Review: After the workshop, four report reviewers received the nine reports generated by the workshop participants.After reading each report, reviewers used an online survey to check whether the report met the Intelligence Community Directive (ICD) 203, Analytic Standards (Office of the Director of National Intelligence, 2015), and provide written feedback.To be in line with the standards a report must meet nine conditions: 1) source reliability, 2) expressing uncertainties, 3) distinguishing face from assessment and assumption, 4) analysis of alternatives, 5) demonstrate relevance, 6) logical argumentation, 7) consistency over time, 8) accuracy, and 9) visualization.The order in which the nine reports were presented to the reviewers was randomized.
Reviewers were blind to participants and CW strategies.

data Analysis
For the individual activities (i.e., participants' in-person verbal communication, tool usage, and role shifting) documented in the observation notes, the number of occurrences of the code was counted for each writing group during the 10-minute period.However, for the group activity, group work engagement and CW strategy followed were noted as either occurring or not.This resulted in 22 data points per group per day; three CW strategy groups and three days produced 198 data points.PROC TTEST (SAS 9.4) was used to compare group means using independent samples t-tests (α = .05).To correct the likelihood of a significant result by chance due to the multiple t-tests, Bonferroni correction was conducted.
Reviewers' comments were analyzed using thematic coding in MAXQDA 2020.The coding system consisted of nine themes (see Table 2).

In-Person Verbal Communication
Four types of in-person verbal communication were observed during the workshop including 1) one-on-one communication with eye contact (1-1EC) (i.e., two group members involved in the conversation); 2) group communication with eye contact (GEC) (i.e., three groups members or the whole group involved in the conversation together); 3) one-on-one communication without eye contact (1-1NEC) (i.e., two group members involved in the conversation with no eye contact or communicated via email or real-time chat window); and 4) group communication without eye contact (GNEC) (i.e., three groups members or the whole group involved in the conversation with no eye contact or communicated via email or real-time chat window).

Tool Usage
Google Docs and Google Search were the most frequently used tools in all three CW strategies.For each tool, the collaboration level was also observed during the three-day workshop.More specifically, whether the participants were observed using a tool during a collaborative activity or a sole activity was documented.The following scenarios considered as working in a collaborative manner I: • Participants communicate while they are writing, editing, or reviewing one document.The document could be a shared document, an individual document, or the group's final report.• Participants communicate while they are searching or consuming information.
• Participants work on a shared document without communication.
The following scenarios were considered as working in a unilateral manner (N): • Participants copy and paste information or organize their thoughts and ideas within their own document without communication.
• Participants search and consume information without communication.
While there was no statistically significant difference among the three CW strategies in overall tool usage or for an individual tool, there are differences in tool use between and with distinct CW strategies.For SAW and HDW, tools were more frequently used during individual activities.However, for RW, the frequency of electronic tool and physical tool usage was similar for both collaborative and unilateral activities.Among the three CW strategies, RW used more Gmail or Chat.Even though in RW participants drafted the report in the same document, which means they consistently worked in a collaborative manner, they still shared information and communicated via various communication platforms during their CW process.

Group Work Engagement
Three types of group work engagement were documented during the workshop.For each data collection point, researchers observed and categorized the group work engagement as engaged €, digitally engaged (DE), or not engaged (NE).Group work was considered to be engaged when everyone in the group equally engaged in discussion, all members shared leadership, or one person took the lead and other members followed the discussion.Group work was identified as digitally engaged when all group members were digitally contributing to the report but there was no communication between the group members.Situations categorized as not engaged included when one person dominated the conversation, someone stopped focusing on work, there was a conflict between group members, or a group member's personal physical condition created frustration.There were no statistically significant differences between the three strategies for each type of group work engagement.Some of the NE observed in RW were due to the personal frustrations (e.g., low back pain) of a few individuals.

Measurement of Flow
Figure 6 is a visualization of participants' perceived challenges and skills regarding the main activities from the first and second state of flow surveys each day.Data from the third state of flow survey were excluded from the results because most of the participants or groups were finishing up their CW process and were no longer in flow by the third survey.Data in the HDW group on day three was incomplete since there were only three participants in the group that day.Based on the current model of the flow state, the upper right corner in each graph represents the flow area.
Over the three days, SAW participants stayed in flow most often.The difference in flow between SAW and the other writing strategies was most pronounced on day one and day three.

Group Workflow
The group workflow of the collaborative reporting process in three CW strategies were synthesized based on the observation notes and participants' self-reported main activities (see Figure 7).Seven main activities and three working patterns were identified in the group workflow.The seven main activities in the group workflow included 1) discussion, 2) information searching, 3) information The three CW strategies had similar main activities for the entire collaborative report generation process except for the report assembling activity.Report assembling occurred in HDW due to the group defined structure, roles, and workflow.HDW groups' workflow was more linear and appeared simpler than the other two groups.SAW and RW groups included overlaps in activities and cyclic behaviors-their workflows appeared to be more iterative.

Strategy Following and Role Shifting
All participants in the SAW group were observed consistently following the strategy during their CW process.And there was no role shifting in this group.HDW also appeared as an easy-to-follow strategy, except one participant shifted from section writer to editor at the end of day one.However, instead of following the structure and workflow of RW, two out of three groups using this strategy were observed following the HDW strategy at the beginning of their CW process.Some participants in the RW group were frequently observed shifting roles from contributor or cowriter to section writer.

Report Assessment
The following section describes the report assessment results provided by the report reviewers in two areas: report rating based on the ICD 203, Analytic Standards, and reviewers' comments in the report assessment survey.
• ICD 203, Analytic Standards: There were no statistically significant differences among the three CW strategies for their report ratings.On average, reports generated under the HDW had a higher rating than the reports generated under the other two strategies, especially in expressing uncertainties, analysis of alternatives, demonstrative relevance, consistency over time, and accuracy.Reports generated under the SAW strategy had a relatively lower rating in almost every criterion except in visualization.Report rating results are listed in Table 4. • Reviewers' Comments: Figure 8 presents the distribution of the aspects and points the report failed to address or insufficiently addressed.In Figure 8, circle size increases with the problems noted by the report reviewers.
The comparison among three strategies indicated that the reports generated by using the SAW strategy had the most problems, especially in argumentation, logic consistency, relevance, and reliability.Reports generated using the HDW strategy had the least problems among the three; however, they had more problems with the language and format consistency.The performance of RW was mediocre compared with the other two strategies in all code themes.
Even though the RW strategy did not stand out compared to the other two groups, the main shortcomings in this CW strategy were lack of alternative analysis, language and format consistency, reliability, and visualization.
The comparison among and within the CW strategies indicated that reliability was the biggest problem.

Writing Priorities
Participants' writing priorities when generating the report were not significantly affected by CW strategies.When participants provided their top three priorities for report generation, relevance, comprehensibility, and readability were most frequently reported.Conclusion and timeliness were least frequently mentioned as top priorities.

Authorship Preferences
Most participants indicated that they would prefer an alphabetical order to present the list of authors for the report they were generating during the workshop, especially for RW group participants.No participants in the HDW group reported a preference for authorship where the person who wrote the most was designated as first author.For participants who selected "other" as their authorship preference, their responses in the follow-up questions indicated that they do not care about the authorship for the report they were generating.

discussion
Addressing RQ5 and RQ6, we investigated the influence of three CW strategies-single-author writing (SAW), horizontal-division writing (HDW), and reactive writing (RW)-on group communication, tool usage, report generation workflow, and final product rating.Each strategy demonstrated unique characteristics and implications for collaborative work.Table 5 depicts the differences among three CW strategies from five main aspects in this study, namely, in-person verbal communication, tool usage, group work engagement, difficulty in CW strategy following, frequency in role shifting, and report rating results.As the tool usage was observed separately when used in collaborative manners and in solo manners, in Table 5, tool usage differences in working collaboratively and not collaboratively were presented as C and N, respectively.SAW facilitated more in-person group communication, pointing to the need for work environments that support such interactions.HDW, on the other hand, showed less reliance on inperson communication and collaborative facilities, suggesting it may be ideal for situations where these resources are limited.RW required continuous collaboration but resulted in more frequent role shifts.Moreover, despite being the most familiar to participants, SAW generated reports with the lowest rating, while HDW led to the highest-rated reports.The following sections delve deeper into the specifics of these findings and their implications.
Participants had more GEC in-person verbal communication when they used SAW.This implied that SAW encouraged group members to have more group communications.Hence, when generating a report using this strategy, it is essential that the working environment provides collaborative facilities which support group in-person communication with eye contact.This is underscored when the report is being generated by a group of people who work remotely.Study 1 results indicate that remote workers in several work communities (e.g., IC, finance, and legal sectors) are highly likely to engage in collaborative reporting.Compared with SAW and RW, participants using HDW had significantly less communication in general.
In terms of overall tool usage regardless of CW, participants spent a similar amount of time using Google Search and Google Docs.This indicates that the writing strategy did not impact the proportion of time spent researching vs. report writing.However, when focusing on usage of Google Search in a collaborative manner, participants engaged with SAW spent the most time researching among the three strategies.For usage of tools in a solo manner, HDW participants spent the most time researching.Google Docs was used differently between CW strategies.RW contributors spent more time collaboratively using Google Docs than those following HDW.Meanwhile, the RW group spent less time using Google Docs in a unilateral manner than in HDW.These findings strongly conformed to the nature of these two writing strategies, considering RW requires team members to work collaboratively from the beginning to the end, while HDW requires that authors write individually and separately in the first place.
In addition, considered as common tools in a collaborative working environment, a projector and whiteboard were both used by participants in SAW and RW but not by the participants in the HDW.
One issue considered during the analysis of tool usage was that participants might have been bored by day 3 and started to explore more tools.This may have skewed data over time.A comparison of total tool usage over the three days, however, confirmed that there was no significant difference in the total tool usage across the three days (day 1 = 302, day 2 = 344, day 3 = 309).
For group work engagement, even though there were no statistically significant differences among the three strategies, there were differences among groups.While using HDW, participants were observed fewer times in engaged group work compared with SAW and RW.Within the HDW strategy, participants spent more time in digitally engaged than verbally engaged group work.
Among the three strategies, HDW had fewer in-person communications, minimum collaborative facility, physical tool usage, and less time verbally engaging in the group work.This suggests that HDW be considered for a group's writing strategy when communication and collaboration mediums are limited in the work environment.Based on the feedback from participants, it was necessary for group members using this strategy to set up requirements regarding subtitles, font size, and citation style for format consistency.A successful use of HDW includes the group's agreement on format considerations (i.e., what would be the separate sections, how many pages each section would be, and what font to use) before writing.By doing so, all sections would merge smoothly during report assembly, minimizing work during report editing.
The natural fit of the strategies varied; some seemed easier to follow while others resulted in frequent role shifting.Participants indicated that SAW was relatively easy to follow and rarely shifted from their assigned roles.This can be explained by Study 1 finding that SAW is regularly used in the IC.In contrast, when using RW some participants mentioned in their postworkshop survey that it was difficult to stick with the strategy, especially in the early stage of their CW process.A few mentioned that it was hard to concentrate in RW because they needed to figure out their location in the document, for it was moving and shifting constantly.Some participants using RW were observed shifting roles from contributor to section writer (e.g., they wrote in an individual document instead of writing in the real-time shared document in Google Docs).The awkwardness of RW is highlighted by one participant's comment that "we need to have some content before we collaborate in the same document."Since little role shifting was observed and few strong comments were provided, HDW appeared easy to follow and naturally supported the IC participants.
Although SAW is most similar to the IC's existing collaborative reporting process and seemed most natural for participants, based on the reviewers' report assessment, reports generated using SAW had the lowest rating.Reports generated using HDW had the highest ratings, followed by the reports generated using RW.
During data analysis, it was noted that concepts such as authorship, attribution of effort, and credits were not foregrounded in the IC in contrast to their prominence in academic writing discussions (Noël & Robert, 2004;Zutshi et al., 2012).Based on the postworkshop survey results, most participants indicated a preference for alphabetical order to list report authors; CW strategy did not affect authorship preference.Only a few participants in SAW and HDW preferred authorship order based on the level of contribution.
It is important to acknowledge that both HDW and RW are quite different than the IC's current reporting workflow.RW was initially rated by the observers as more awkward and the workshop participants as harder in terms of following and sticking to the strategy and roles.Obtaining the same results in the real work environment of IC analysts may be difficult as individuals may resist the change to their workflow as they experience a new way of engaging.HDW and RW may function well in the cloud computing environment on the low side (i.e., unclassified settings) which is where Study 2 was conducted; however, transitioning this approach to the high side may require additional resources or development that is unknown to the authors.

Limitations
This study has several limitations.The focus on IC analysts allows for context-specific findings but restricts wider generalizability.The unique training and knowledge set of IC analysts extends from reporting structures to assessment criteria (i.e., ICD 203, Analytic Standards), distinguishing their behaviors and workflows from other reporting professions, and limiting cross-profession extrapolation.Given the limited number of participants, reports, and assessments (though we took multiple samples throughout the day), data analysis was limited to simple t-tests, appropriate for small sample size assessments, to detect significant differences.
Study 2 also faced limitations in terms of its setting.Conducted in unclassified environments using commercially available tools such as the Google Suite, the ability to accurately gauge the computational complexity and execution times of the proposed collaborative strategies in real-world classified IC settings was restrained.We agreed that having a focused cohort of active IC analysts would produce more meaningful, realistic reports, assessments, and ultimately results, which would be more meaningful to the IC.

CoNCLUSIoN
In this study, we aimed to explore the nuances of collaborative writing strategies in the intelligence community (IC) and understand the impact of these strategies on the quality of intelligence reports.Given the increasing shift towards collaborative work in the IC and the unique challenges posed by this context, our findings offer valuable insights into how collaboration strategies can be optimized to improve report generation.
In Study 1, we focused on delineating the differences between collaborative reporting in the IC and other sectors, revealing distinct preferences for collaboration strategies.IC analysts and reporters predominantly favored a stratified-structure strategy, akin to single-author writing (SAW), while other sectors preferred a more segmented approach similar to horizontal-division writing (HDW).These results suggest that the IC's historically compartmentalized style of working may influence the preferred strategies for collaborative writing, which could have implications for the design of collaborative tools and workflows tailored to this sector.
In Study 2, we further investigated the efficacy of SAW, HDW, and RW strategies within the IC.Interestingly, despite SAW being most familiar to IC participants, reports generated using this strategy were rated lowest for quality.This contrast with the preference for SAW in Study 1 suggests that familiar approaches may not necessarily lead to the best outcomes.Meanwhile, HDW and RW strategies, despite necessitating less in-person communication, led to higher-rated reports.While RW was seen as more challenging due to its demand for continuous collaboration and role shifts, it offers promising avenues for further refinement and optimization in the IC context.
This research focuses on IC analysts and reporters and the use of open-source, commercially available tools and may limit the generalizability of our findings to other sectors and within classified IC settings.Therefore, we recommend future studies broaden their scope to other sectors and consider conditions more closely replicating the IC's work environment.

Figure 3 .
Figure 3. Synthesized report generation workflow in the IC

Figure 4 .
Figure 4. Synthesized report generation workflow in Non-IC sectors

Figure 6 .
Figure 6.Participants' perceived challenges (vertical axis) and perceived skills (horizontal axis) of the main activities

Figure 7 .
Figure 7. Synthesized group workflow of the collaborative reporting process for each CW strategy

Table 1 . Barriers in report generation, report consumption, and the collaboration process
• Conflicting analysis/Interpretation of data • Conflicting analysis/Interpretation of data