Contemporary authorship attribution methods: Background and gaps

The majority of modern authorship attribution methods use statistical analyses that fall under the umbrella of supervised machine learning (SML). SML methods allow a computer system to be “trained” on data where concrete outcomes are known. In practice, trained models can then be used to predict outcomes in new, previously unseen data. For example, if a SML algorithm is trained on a collection of images with known faces, it can be used to accurately identify familiar faces in a novel image as a function of what it has previously learned [13].

The power of SML methods rests in their ability to discriminate between multiple known outcomes with extraordinary accuracy. The appeal of SML in authorship attribution tasks is readily apparent: if a system can be trained on the language patterns to differentiate various known authors, a work of questionable authorship can be statistically assigned to one of those authors with high confidence. These types of problems are known as “multiple-candidate” or “closed-class” problems in authorship attribution–that is, a work is known (or strongly believed) to originate from one of N specific authorial candidates. For additional discussion and clarification on the idea of multiple authorial candidates, please see Section A in S1 File.

A considerably more difficult problem in authorship attribution is one of “single-candidate” attribution tasks. In single-candidate problems, the authorship question boils down to “did Person X write this text?”. Currently, several methods have been proposed to address single-candidate problems, however, these methods typically operate by converting a single-candidate question into one of multiple candidates. This is often done, for example, by introducing “impostors” who are known a priori to have not written the work in question [14]. These methods are useful when comparable data is readily available/accessible in large quantities, however, they are less practical in cases where data is limited or of a unique variety. From a psychological perspective, such methods are also lacking in their ability to furnish deeper insights. Like other authorship attribution methods, most single-candidate attribution methods typically rely on word distributions that provide no further information about an author beyond opaque probability distributions.

To illustrate this last point, consider a hypothetical scenario wherein an unknown author has left behind an unsigned admission of guilt for arson. The police suspect Joseph to be the author of the arson note, and they have obtained several other of Joseph’s “baseline” writings. The question, then, is whether it can be determined that the person who authored the baseline writings (i.e., Joseph) also wrote the admission of guilt. A standard authorship attribution analysis would decay all texts into a series of high-dimensional vectors based on words and word properties (e.g., part of speech, repetition, etc.), then statistically determine the likelihood that they came from the same source using classification algorithms. Particularly when conducting complex tasks such as authorship attribution, advanced statistical and machine learning methods typically preclude any psychological understanding of the person who authored a text [7], which may be of great importance in forensic and courtroom settings.

Now imagine the same scenario described above, yet an analyst employs a psychological approach to authorship attribution rather than a purely statistical approach. Using psychological text analysis procedures, an analyst would be able to extract information about Joseph from his baseline texts (e.g., “Joseph’s baseline texts are indicative of someone who is generally extroverted, honest, and unanalytical.”). Furthermore, Joseph’s friends, neighbors, and family also agree that these traits are an apt description of his general personality. In this scenario, the statistical information of standard authorship attribution analyses can be combined with the additional psychological information (e.g., the observer reports of Joseph, the psychological patterns embedded Joseph’s texts) to form a more robust analysis and interpretation of the findings. If the psychological patterns extracted from the admission of guilt are indicative of someone with the same general psychological traits as Joseph, then the forensic account of the arson admission is strengthened.

The above example is a conceptual demonstration of why it is useful, then, to strike a compromise between computational and psychological methods of authorship attribution that satisfies 2 criteria:

1. Valid authorship attribution frameworks must be underpinned by empirical, reproducible, and quantifiable methods rather than intuition or subjective judgments (computational/statistical perspective).
2. Ideally, an authorship attribution analysis should provide results that can be interpreted in the context of relevant psychological information such observer reports of an individual’s traits, behaviors, and mental profile (psychological perspective).

Such a compromise allows the rigor of advanced authorship attribution methods to be paired with scientifically-validated psychometrics that lend themselves to a deeper, meaningful interpretation from a psychological perspective.

Modern psychological authorship attribution

Recent research has found that individuals can be differentiated based on their unique psychological composition, which can be reliably measured through language use. Boyd and Pennebaker [12] found strong support for this idea by demonstrating that authors could be robustly and reliably differentiated with near-100% accuracy using exclusively psychological measures of language. Unlike most authorship attribution techniques, which are often based on more atomic linguistic measures (e.g., distributions of short phrases, spelling errors, etc.), Boyd and Pennebaker’s methods lent themselves to a scientific, psychological analysis of an author by identifying their unique psychological attributes.

The ability to engage in follow-up interpretations of language patterns stands in stark contrast to most authorship attribution methods. Furthermore, their results were also able to be compared with observer reports of authors’ personalities, dispositions, and even behavioral events. In their work, Boyd and Pennebaker’s language-based analysis of authors’ psychological profiles showed strong convergence with other psychological data, strengthening the forensic account provided by their authorship attribution analysis. Lewis Theobald, for example, exhibited language patterns consistent with a highly analytic yet socially cold and confrontational personality. Indeed, observer reports of Theobald mirrored the insights gained from language analyses, as he was known to be brilliant but quarrelsome with colleagues and had few to no close friends.

Nevertheless, the methods used in Boyd and Pennebaker’s study of psychological authorship attribution possess some drawbacks that are shared with non-psychological methods. For example, the methods used in their work are only useful for multiple-candidate attribution tasks–single-candidate problems are not typically able to be solved by means of classification tasks alone. Therefore, while Boyd and Pennebaker’s analysis represents a promising first step towards the unification of computational and psychological sciences in the domain of forensic analyses, more work is required to extend and expand this new type of hybrid analysis into uncharted territory.

Current study

In the current study, a new method of authorship attribution and, more broadly, psychological profiling is introduced. This new method, named Mental Profile Mapping, aims to address multiple critical gaps in the forensic space of authorship attribution. Namely, there currently exists a marked lack of empirical, theory-based psychological methods for single-candidate authorship questions. Currently, no explicitly quantitative methods for single-candidate authorship attribution exist in the field of psychology. The methods underlying Mental Profile Mapping not only rely exclusively on psychological features that can be measured from language, but the results of this method can be interpreted in a psychological manner through subsequent decomposition and analysis, thereby providing several of the benefits mentioned earlier.

The current study first demonstrates the results of a high-power and validated authorship attribution method hailing from computer science, known as “unmasking”, which is applied to a test case–the works of Aphra Behn. After baseline results are established in the Behn authorship problem space, Mental Profile Mapping methods are introduced. Results from the new analytic approach are then compared to the unmasking results and explored in the context of other psychological information.

Methodological test case: Authorship attribution with the works of Aphra Behn

For the current study, the plays of Aphra Behn were used as the focal data source for analysis. Aphra Behn is regarded as one of England’s first female playwrights and among the 17^th century’s most influential dramatists. Due to the controversial nature of Behn’s writing, as well as gender politics of her own and subsequent periods, much of her work and legacy was suppressed for a considerable stretch of literary history between her death and the latter half of the 20^th century [15]. However, Behn’s legacy has rapidly become the object of much study and interest in modern literary scholarship [16].

Born in 1640, shortly after William Shakespeare’s death, Behn’s childhood is shrouded in mystery, and much of her early life history appears to be obscured either intentionally or due to extraneous factors [17]. Behn served as a spy during the Second Anglo-Dutch War prior to becoming a skilled and well-performed playwright, and she is known to have served several tours during wartime [18]. Much of Behn’s literary work was highly successful during its time, and Behn’s death in 1689 caused an upsurge in public demand for published copies of her work.

Profiteers and publishers of the time met the demand for copies of Behn’s work by compiling her plays and commissioning several rounds of printing. In particular, Charles Gildon and Gerald Langbaine helped to fuel additional demand through their (perhaps sensationalized) biographical accounts, occasionally mixed with praise of her personality and exploits. In recent years, doubts have been cast regarding the true origin of some of Behn’s posthumously-published plays, particularly those in which Gildon was involved. Gildon was an occasional literary forger and writer, and his involvement in the publication and dissemination of Behn’s works has raised suspicions about whether the posthumous publications are authentic, or merely opportunistic forgeries [15].

The works of Aphra Behn are a suitable test case for authorship attribution methodologies for several reasons. First, Behn’s plays are composed of tens of thousands of words, which makes them suitable for extracting stable language patterns within each work. Additionally, lengthy writings such as Behn’s plays allow for highly reliable psychological measurements to be performed via automated text analysis [7,19].

Furthermore, it is helpful that plays by several other playwrights surrounding Behn’s era, both before and after her life, are readily available in machine-readable format and exist in the public domain, providing accessible comparison cases for testing. Lastly, scholars have painstakingly assembled thorough timelines of events, records, and other chronological information for the life and works of Behn [18]. Importantly, this last factor allows the results of a psychological authorship attribution analysis to be tentatively compared to details of Behn’s life, facilitating a convergence of empirical methodology with external psychological information.

Setting an authorship expectation baseline: The “unmasking” analysis

Before introducing the Mental Profile Mapping approach, it is important to first establish basic results within the Aphra Behn authorship problem space using an established, high-powered, and validated technique. By setting baseline expectations, the Mental Profile Mapping analysis can be more thoroughly evaluated for convergent validity. Essentially, if results from the new method are comparable to that of an established computational method, it becomes easier to place faith in the results of both analytic tactics. For the current analysis, a powerful modern authorship attribution technique from the computational sciences was selected to set a baseline for expectations pertaining to authorship results–this method is known as “unmasking” [20].

Data

Aphra Behn corpus.

A collection of works by Aphra Behn was provided by Melanie Evans, Elaine Hobby, and Claire Bowditch [21,22] as part of an upcoming compilation of Behn’s works. All texts were provided with modernized spellings and had extraneous text (e.g., dramatis personae, title pages, prefaces) removed. A complete list of Behn’s canonical plays included in the current study is presented in Table 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Aphra Behn plays included in the current analyses.

https://doi.org/10.1371/journal.pone.0200588.t001

Text analysis method

All plays by all authors were analyzed using the 2015 version of the Linguistic Inquiry and Word Count software (LIWC2015) [24]. The LIWC2015 software codes texts for words belonging to ~80 psychological dimensions of language previously found to be related to emotions, social and cognitive processes, and attentional processes. The LIWC2015 application operates by calculating the frequency of words belonging to each category in a given text, then dividing by the total number of words. For example, if a text has 1 positive emotion word (e.g., “pleasure”) out of 10 total words, the text is scored as 10% for positive emotion words. This method has been extensively validated in research the past 2 decades, and the LIWC categories are often used in authorship attribution studies [25,26]. LIWC features have also been found to be as useful in psychological analyses of authorship attribution as standard n-gram distributions [12]. Additional information on LIWC as a method, as well as how it functions in the context of literary works, is presented in the Sections B and C of S1 File.

Plays by all authors were segmented into chunks of ~250 words and analyzed using LIWC2015 in accordance with unmasking procedures. The unmasking procedure was then applied to the resulting 8,066 play segments in the precise manner outlined in Koppel, Schler, and Bonchek-Dokow [20]. This method was performed using 10 folds in conjunction with a k parameter set to 2 (i.e., 4 features dropped per iteration). Additional prediction degradation curve data was included in the overall feature set, including linear and quadratic polynomial beta weights and i+f_{{1, 2}} degradation Δ scores.

It is important to note that, unlike the majority of other authorship attribution methods, the specific authors selected for unmasking analyses are not of particular importance. Ultimately, the unmasking approach does not cleanly distill into a traditional multiple-candidate authorship problem that must select one of the candidates as the “true” author (i.e., “Text X must have been written by one of these authors–therefore, which author most likely wrote Text X?”). Instead, supplemental authors are primarily used as a sounding board for the purpose of modeling same- versus different-author prediction degradation curves.

Underlying concept

As a way to tackle the single-candidate authorship problem from a psychological perspective, this study introduces the concept of Mental Profile Mapping (hereafter denoted as “MPM”). The underlying concept of MPM is fairly simple: by assuming that several repeated psychological measurements are indicative of the same individual, as does any measure of personality [28], methods can be developed that facilitate the detection of outliers, or observations that fit poorly with the others. Because people show natural variation across time, such a method will want to look for outliers on not just isolated, singular psychological process (e.g., social orientation, emotional state) but instead for outliers across a whole battery of psychological measures. In other words, the MPM is primarily designed to help identify violations in the broad consistency of a person’s traits, including language-based measures of psychological processes [29]. If an observation is different along not just one psychological measure, but instead more generally, suspicion as to the origins of that observation can be reasonably raised.

As an analogy, imagine that you receive an occasional e-mail from a coworker, Margaret, perhaps once or twice a month. Over time, you begin to develop a general sense of Margaret’s personality. In her e-mails, Margaret seems to be a very positive person, keenly tuned in to the “here and now”, and often mentions her personal life (e.g., her leisure activities). While Margaret does not always say the same thing–she sometimes talks about the weather instead of her bowling league, and she seems less positive on some days relative to others–there is an overarching consistency to her personality that emerges in her communications.

One day, your supervisor tells you that she has received an e-mail from an unknown address, but she thinks that she knows who sent it. Your supervisor forwards you the text of the e-mail and asks “Does this e-mail look like it came from Margaret?”. As you read the e-mail, you might check to see if all of the criteria for Margaret’s psychological profile are met. You notice that the e-mail is generally positive and very present-focused. Additionally, there is a comment about recent leisure behaviors–the e-mail’s author states that they went to a concert this past week. Given that this e-mail appears to fall into the general constellation of Margaret’s mental universe, you may tentatively conclude that this e-mail does appear to fit Margaret’s bill for authorship.

The principal concept underlying MPM is fairly simple and parallels the example given above. In order to establish a typical psychological pattern for an individual, several psychological measures must be gathered from different time points. Just like the various qualities of Margaret’s writing (e.g., tone, time-orientation, content), each psychological measure–in this case, LIWC-based measures–will show some variation over time yet still be reflective of the underlying source. Just as you noticed consistencies in Margaret’s style, similar consistencies will emerge for an individual in their psychological processes (e.g., emotions, social processes, attentional processes). Under the assumption that all measurements were generated by the same person, outliers become suspect and merit further investigation.

In the following sections, the underlying statistical approach of the MPM will be laid out in detail. As with virtually all modern authorship attribution methods, the core of the MPM relies on statistical analysis of a person’s words. However, this approach offers several practical advantages over others. First, the underlying statistics upon which MPM works is particularly well-understood; such is not the case for many newer algorithms, as we are only beginning to understand the reasons for and boundary conditions of their efficacy [30]. Additionally, the inclusion of explicitly psychological information affords considerable information gain that is not found in other methods. For example, if a statistical distribution of words is similar to a person’s baseline texts, but the psychological profile is mismatched, we are able to retain a healthy skepticism of our own results that cannot be raised from purely distributional linguistic similarities of a text, again, facilitating a more well-rounded and accurate interpretation of the problem space.

Mental profile mapping: Quantification and statistical methods

By pairing psychological text analysis with a multivariate distance measure, Mahalanobis distance [31,32], it is possible to statistically conduct the type of analysis described above. Ultimately, the goal is to assess the distance of several types of psychological processes from their respective centers for a given individual, then collapse all distance scores into a single metric that can be used to better understand the overarching patterns. For example, if a person’s emotional state is measured 20 times, it is possible to establish the average location, or “center”, of their emotional states. The same could then be done with the same person’s social orientation, decision-making tendencies, and so on, resulting in psychological centers for each of the different types of psychological measurements.

Unlike other distance measures, such as Euclidean distance, the Mahalanobis distance statistic is explicitly designed to handle inter-correlations among multivariate data. From a psychological perspective, the fact that Mahalanobis distance accounts for the covariance structure of the data allows for a more meaningful distance metric to be calculated relative to other distance metrics. In other words, the mathematical underpinnings of the Mahalanobis distance statistic is well-suited to handle the fact that psychological subprocesses are non-independent and may demonstrate overlap. For example, positive and negative emotions are not perfectly orthogonal [33], nor are interpersonal motives and behaviors [34]–interdependencies such as these exist across several of the LIWC psychological measures [19].

For the current MPM analysis, LIWC2015 was used to extract psychological data from all source texts mentioned in the unmasking section above. All LIWC measures were split into respective clusters of psychological processes according to their designation within the LIWC dictionary: style measures, complexity, function words, emotional processes, cognitive processes, perceptual processes, biological processes, motivational processes, temporal processes, spatial-relational processes, personal processes, and utterances (see Table 3).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Psychological process compositions.

https://doi.org/10.1371/journal.pone.0200588.t003

As a point of clarification, it is important to note that the MPM method as described thus far constitutes a considerable departure, both in conceptual and practical terms, from a more traditional LIWC-based analyses of texts. A typical LIWC analysis in psychological research will focuses on isolated psychological markers that can be found in a person’s language. This is often done to understand specific differences between groups, or particular linguistic associations with other psychological constructs, such as self-focus or positive emotionality [35]. Such an approach is incredibly useful for theory-building and hypothesis-testing but is also particularly limited in scope and seldom takes into account the incredible complexity and interplay of psychological processes at the individual level. Conversely, the MPM method described thus far can be seen as an aggregation of lower-level psychometrics into a more unified view of the whole [34], which is inherently more psychological in nature than high-dimensional uses of LIWC measures for the purpose of variance-account maximization [36] or purely computational/ predictive approaches that focus on psychological phenomena [37]. Put simply, the MPM is by its very nature more all-encompassing than standard LIWC-based psychological research, yet is still firmly nested in the psychological sciences rather than being entirely computational and/or black box-esque in nature.

Psychometrics of the mental profile map

Internal consistency of distance metrics.

After calculating the “distance from center” scores for each psychological process for each play, for each author, Cronbach’s alpha was used to determine whether distance scores were consistent with each other. For example, if a play is far from the psychological center of Shakespeare’s collected works on cognitive processes, is the same play generally an outlier across all psychological processes, or are these distance metrics relatively isolated?

Cronbach’s alpha results are presented in Table 4. The internal consistency analysis did, in fact, find that the distances of each psychological process for each play formed a coherent single factor for all authors in the dataset. In other words, when a given play was further away from an author’s center along one set of psychological processes, it was generally further away from the center along all other psychological processes as well. These results were universally true and not isolated to a specific author or subset of authors.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Internal consistency of distance measures for each author.

https://doi.org/10.1371/journal.pone.0200588.t004

The description of the MPM steps performed up to this point is, admittedly, rather intricate. As an illustrative example of the procedures described thus far, Table 5 presents descriptive statistics and inter-item correlations for all distance measures for Behn’s plays. Additionally, a simplified “step-by-step” description of the MPM process is presented in Section E in S1 File to help readers more clearly understand the steps of the process, as well as apply this method to their own work.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Summary statistics and inter-item correlations for all distance measures calculated for the verified plays of Aphra Behn (i.e., excluding questioned plays).

https://doi.org/10.1371/journal.pone.0200588.t005

In effect, for each of the plays of Apha Behn, a separate Mahalanobis distance score was calculated for each of the 13 broad psychological processes captured by LIWC. Put another way, all 16 of Behn’s plays were each assigned 13 separate “distance from center” scores, one for each broad cluster of psychological processes. Across all 13 psychological processes, Behn’s plays were, on average, a modest distance away from the center points, with all scores hovering closely around 50. Furthermore, the distance scores for each of these measures were strongly intercorrelated, meaning that variation across all psychological processes were relatively harmonized with each other. For example, where Behn’s plays showed drastic deviation from (or adherence to) the center in terms of emotional processes, so too did they show considerable deviation in biological and motivational processes, and so on.

The fact that the psychological distance scores all converge on similar information facilitates 2 useful procedures in moving forward. First, given that all distance measures appear to be reflecting a similar phenomenon (distance from the “psychological center” of the map), one is able to collapse all distance scores related to a particular psychological process down to a single score. In this case, the median is preferred over the mean to prevent undue influence of a single psychological process outlier. Much like the unmasking process, drastic variations along one or two psychological processes may be superficial (e.g., different themes between works) and not reflect meaningful psychological differences. Use of the median over the mean helps to reduce excessive influence of superficial extremities. However, in this case, mean and median distance scores showed a strong correlation (R = +0.96) and the choice of aggregation method did not impact results. Therefore, the median of all distance scores was calculated for each play, resulting in a single 0-to-100 score that signified the strength for the case of an individual’s authorship for any given work (e.g., Behn’s M = 48.56, SD = 16.41). This final score is essentially a shorthand metric that tells us the degree to which a play is generally far away from the “psychological center” of the map across a battery of psychological processes, in which case it would merit further investigation/explanation.

Second, one can meaningfully reduce the number of dimensions from 13 psychological distance scores to 2 using multidimensional scaling (e.g., a principal components analysis). Reducing the dimensionality is primarily useful in this case to create a visualization that roughly represents how far away from the center each play rests. This allows for the consumer of these results to more easily and intuitively understand the concept of the MPM method, providing a visual reference for discussion, analysis, and interpretation of results. Given the nature of single-candidate authorship attribution problems and the inclusion of a relatively small sample of works, a mental profile map allows us to get a sense of how far away a questioned work sits from the overall psychological center. Questioned works that show relatively low MPM scores should then be subjected to further scrutiny and questioning.

MPM visualizations for all authors included in MPM analyses are presented in Fig 1. Note that while each author may have 1 or 2 plays that stray from the broader cluster of their psychological maps, all plots show a central “gravity” that visually represents a psychological center across all mental processes. In other words, all written works by all playwrights exhibit a tendency to radiate out from the psychological center of their respective map, demonstrating that each author’s works deviate from their own, internal norm across an entire psychological spectrum rather than single, more atomic psychological processes.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1.

Visualized mental profile maps of 6 playwrights: a) Aphra Behn, b) Thomas Dekker, c) John Fletcher, d) Christopher Marlowe, e) William Shakespeare, and f) Lewis Theobald. Note: The center point (0,0) is the approximate “psychological center” for each playwright when collapsing across all 13 psychological process categories. Note that the projection of distance measures down to 2 dimensions does result in some distortion, and this graph should be interpreted as an approximation of the “true” location of each play. In other words, some plays may actually be somewhat closer to (or farther from) the center of each map than what the image may suggest.

https://doi.org/10.1371/journal.pone.0200588.g001

Testing the mental profile map approach for authorship tasks

In the case of an authorship attribution task, we can adopt the MPM approach by creating a map using all of an author’s known works and, in turn, also including each work of questionable origins. By doing so, one is able to generate scores for each individual questioned work and make judgments regarding how well the questioned works fit into the larger picture. This is the approach that was adopted for all MPM analyses reported below.

As an initial test of the utility of the MPM approach in authorship attribution tasks, it is useful to first examine its performance in cases where all works are of known authorship. This can be done by performing similar procedures to those described above, albeit with some “bogus” insertions of comparable works by other known authors. It is possible, for example, to insert a play by William Shakespeare into Aphra Behn’s map. By inserting bogus plays into the MPM procedures described above, we are essentially operating under the a priori false pretense that Behn actually authored Shakespeare’s play. Operating under this bogus assumption, we should be able to easily spot the false insertion due to its drastic pulling away from the MPM center, both statistically as well as visually.

For this test analysis, 3 plays were chosen by a random number generator from the supplemental playwright corpus. These 3 random plays included The Whore of Babylon by Thomas Dekker, The Fatal Secret by Lewis Theobald, and Julius Caesar by William Shakespeare. Each play was, in turn, inserted into the corpus of verified plays by Aphra Behn–the MPM procedures described in the preceding section were then performed. This resulted in 3 separate “maps”, one for each run of the MPM procedure with the inclusion of each bogus play insertion. Numeric results are presented in Table 6; visualizations are presented in Fig 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Visualized mental profile maps of Aphra Behn when including bogus plays by other playwrights.

Note: The blue diamond denotes the psychological center of each map. Bogus plays are circled in red and stand out considerably from the rest of the map in each case. Bogus plays include Dekker’s The Whore of Babylon (left), Shakespeare’s Julius Caesar (middle), and Theobald’s The Fatal Secret (right).

https://doi.org/10.1371/journal.pone.0200588.g002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Results of the MPM analysis.

https://doi.org/10.1371/journal.pone.0200588.t006

Results from the “bogus play” MPM analysis were extremely promising. All 3 bogus plays that were inserted are markedly distinct from the rest of Behn’s map. Numerically, all 3 insertions scored extremely low for fit in Behn’s corpus, with only one verified Behn play (The Widow Ranter) scoring as a worse fit than all three; the outstanding Behn play is discussed later. Visually, each bogus play was also quite distinct, falling well outside of Behn’s relatively tight ring of verified works. These results strongly suggest that this method is useful for identifying gross psychological departures from an author’s norm or, in this case, works that show a psychological signature of someone other than an author in question.

Mental profile map decomposition

Decomposition of questioned plays.

A primary benefit of the MPM method over more opaque machine learning methods is the ability to decompose the results into interpretable, meaningful units of analysis. Rather than receiving a hard probability score as the final result, it is possible with the MPM method to peer under the hood and look for reasons as to why a given text appears to be a poor fit. This can be particularly useful when looking for clues as to a work’s true author (if identified as a poor fit), or better understanding a drastic variation (e.g., a recent traumatic event or other upheavals) when authorship is certain.

The results presented above in Table 7 include each of the 13 distance metrics for each play, allowing us to manually inspect those psychological processes that appear to be driving the effects above. For all 3 questioned plays that show very low MPM scores (i.e., The Debauchee, The Woman Turned Bully, and The Counterfeit Bridegroom), it is clear that they generally show below-average MPM scores across all measures, suggesting a generally great distance from Behn’s psychological center along most psychological processes. Table 8 highlights those psychological processes along which the 3 low-scoring questioned plays are most discrepant (i.e., MPM scores < = 20).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 8. Decomposed categories of psychological processes that showed particularly great distance from center for the 3 questioned plays with poor support (i.e., MPM scores < = 20) for Behn’s authorship.

https://doi.org/10.1371/journal.pone.0200588.t008

Once broad psychological discrepancies have been identified, it is possible to then return to the raw data to examine the specific psychological constructs that are driving the differences between the questionable plays and Behn’s overarching mental profile. In doing so, one can reference the raw scores underlying the MPM distance scores (available from the corresponding author by request) to look for extremities.

In a manual analysis, it is possible to see that The Counterfeit Bridegroom scores extremely high on stylistic measures such as clout and authenticity, and overall social words, family words and friend words. Additionally, this play showed extremely low past-focus and high future-focus within the dataset, and a generally high score on ‘time’ words from the relational processes cluster. This combination of extremities suggests an author with several discernible psychological characteristics: an extremely socially focused individual with strong social standing, and likely an individual who is also highly goal-directed in their day-to-day behaviors, as evidenced by the high use of future and time concepts.

Similarly, the play The Woman Turned Bully showed a number of extreme psychological differences from the profile extracted from Behn’s other works. In decomposing the psychological processes of this play, several major differences were apparent. The Woman Turned Bully exhibited extremely high authenticity scores, very low affect words (including both positive and negative emotions, and negative emotion subtypes), and low past-focus and high present-focus. Additionally, this play exhibited large differences from Behn’s profile in both reward words (very high) and risk words (very low). Taken together, the psychological profile of this play suggests an author who is extremely impulsive, focused on the “here and now”, low in self-monitoring, and possesses a strong drive for reward at the cost of risk sensitivity.

Of the 3 questioned plays, The Debauchee is perhaps the most generally different from Behn’s mental profile, yet in very few extreme ways. In emotional terms, for example, this play included extremely high use of general negative emotion words (but low use of specific negative emotion words, such as sadness or anger) and low use of positive emotion words. The other extremities for this play occurred in the domain of perceptual processes, with this play exhibiting extremely low use of perceptual words (e.g., “see” words and “feel” words), but high use of words related to sound. This small combination of extremities is generally difficult to interpret; the broader pattern of a more generalized distance from Behn’s profile may instead simply suggest a person whose psychology is fundamentally different from Behn in most ways.

Mental profile mapping method

The current study demonstrated the underlying methods and utility of a new method, MPM. The MPM approach to authorship attribution possesses a number of benefits over other methodologies in the authorship attribution space. Foremost among these benefits is that it is, to the author’s knowledge, the only existing “pure” method for psychological profiling in single-candidate authorship attribution. Unlike other methods that require the introduction of additional data from other sources, either in the form of imposters or comparators for modeling, the MPM method requires “ground truth” text from only a single source. In cases where comparable texts from other authors are unavailable, this feature of the MPM method is particularly valuable.

Results from all analyses in the MPM framework were particularly strong. The psychometric assessment of the MPM as a methodology revealed that its underpinnings do, in fact, form a coherent construct that suggests language samples vary not just in single, isolated ways from a person’s psychological center but, instead, across all included language-based measures of psychology in unison. Additionally, “ground truth” tests that included known bogus insertions performed extremely well–the MPM method was able to capture false Behn plays almost perfectly.

As demonstrated in the current study, the MPM method need not exist or be performed in isolation. In cases where additional “supplemental” texts can be made available for the purposes of establishing baseline functions, several advanced machine learning methods may be used in conjunction with the MPM approach to strengthen an inquirer’s confidence in the results. For example, if multiple, radically different attribution methods converge on similar results, as occurred in the current study, increased confidence can be placed in the outcome. Additionally, features of the MPM method, such as the discrete psychological process distance scores, may be useful for inclusion as features in other authorship attribution frameworks.

Finally, a featured benefit of the MPM approach over other authorship attribution methods is that it is fundamentally a psychological method of authorship attribution. Insofar as psychological information can be extracted from language data, an individual performing the MPM method is able to more deeply and thoroughly examine an attribution problem by combining the results with data from other sources. By decomposing the MPM distance scores, researchers are able to identify cases of questionable origin but, also, extract a psychological profile from questioned texts. This ability may be of particular value in legal and forensic settings, where various forms of evidence and information must be considered together in order to render decisions. For example, if the MPM of an authorial suspect closely aligns with behavioral outcomes (e.g., never late to work, always polite) or personality reports from family and friends (e.g., conscientious, agreeable), a questioned work that shows radically different psychological properties (e.g., hostile and impulsive) is likely to not only show statistical differences from a candidate’s MPM, but can point to the psychological profile of the true author.

The plays of Aphra Behn

In the current study, the MPM procedure was put to the test using the works of Aphra Behn, a prolific female playwright of the 17^th century. Across 2 highly distinct attribution methodologies, Behn’s unique psychological fingerprint was discernible in her work. This remained true with her original works and, additionally, Behn’s adaptations were also clearly imprinted with her unique psychological composition.

Of primary focus in the current test were 5 plays of questioned origins. The unmasking analysis and the MPM analysis converged to identify 2 of the 5 questioned plays, The Revenge and The Younger Brother, as showing a high likelihood of Behn’s authorship. The remaining 3 plays, The Debauchee, The Woman Turned Bully, and The Counterfeit Bridegroom, exhibited an extremely poor fit for Behn’s mental profile across both attribution methodologies. Additionally, the MPM analysis provided results suggesting likely psychological traits of the 3 questioned plays’ authors. The Woman Turned Bully bears the signature of a highly impulsive person with poor self-monitoring abilities. The Counterfeit Bridegroom exhibits language patterns that are commonly associated with individuals of particularly high social standing and a strong, goal-oriented mindset. The Debauchee, unlike the other 2 plays of questionable origin, did not show any particularly outstanding mental profile–rather, the embedded psychological traits appeared to be, quite simply, generally different from those of Behn.

Like all automated authorship attribution studies, care should be taken in interpreting the results of this authorship test. In the world of authorship attribution, particularly with historical data of uncertain origins, results are never a “sure thing” and must be interpreted in the light of converging evidence. In other words, the results of these analyses cannot conclusively prove that Behn did not have an authorial hand in The Debauchee, The Woman Turned Bully, and The Counterfeit Bridegroom. However, the results of both the unmasking and MPM procedures provide a strong impetus for deeper examination from domain experts researching the area of Aphra Behn’s life and work. At the very least, the 3 weak-evidence plays merit some explanation for their divergence from Behn’s verified works on the mental profile map–an explanation is further merited by the results of the unmasking method, which supports the conclusions of the MPM approach.

Limitations

The current study does possess some limitations that should be taken into consideration. Functionally speaking, the current study includes a small sample size; only 6 to 7 authors were included for all of the analyses performed in this work. It is possible that with a greater number of authors included in the current sample, the results of all analyses may shift to favor another conclusion. Importantly, however, the unmasking method has been extensively validated and tested in previous work [20]. In other words, the unmasking method is already proven and established as a valid and powerful form of addressing authorship attribution questions. The convergence of the MPM results with the unmasking method is extremely promising. Nevertheless, future work may benefit from more extensive testing on broader samples.

Additionally, the current study was performed in a rather constrained context. In practical terms, a demonstration of the MPM method on Elizabethan, Stuart period, and Restoration era playwrights may not extend to texts of other eras or genres. Additionally, all texts used in the current analysis were of particularly healthy length. For example, in the Behn/questioned work MPM analyses, the average word count was nearly 26,000 words per play. The degree to which the MPM method would be applicable to shorter texts, such as e-mails, short letters, or social media updates, is unclear.

In spite of the constrained context, there is no reason to suspect that the procedures used within the MPM method would not extend to domains outside of the current tests. Indeed, most authorship attribution tasks are initially tested on long-form texts such as novels, yet are still viewed as applicable to other forms of language samples, assuming that all texts in a given sample are of generally comparable genres. Additionally, the language-based measures used in the current analyses have been extensively validated for the purpose of extracting psychological information from language, often including extremely short texts such as tweets [42,43].

Finally, it is important to note that MPM as it was performed in the current analysis relied heavily on the use of LIWC2015, which was developed in modern times for the explicit purpose of assessing psychological constructs from natural language samples. While the LIWC dictionary has been validated across thousands of studies and in a variety of contexts [6,7,19], the degree to which any specific LIWC—psychology link can be generalized backwards in time is not especially well-known, particularly for texts dating back multiple hundreds of years. However, there is no reason to suspect that fundamental human psychological links between language and cognition have changed substantially in such an evolutionarily brief period of time. Indeed, previous work with texts from the same era explored in this study have demonstrated that language—psychology links obtained from classical authors are not only psychometrically reliable, but conform nicely to observer reports [12]. Moreover, recent analyses of historical texts have demonstrated that psychological processes manifest in modern language can be reliably extended backwards by at least 250 years, and perhaps earlier [44–46]. Nevertheless, the particular historical nature of the current data should be weighed into any consideration of the current results.

Future directions

Future work with the MPM method should focus on an expansion outside of the current context. The most obvious applications for the MPM method are in both legal and forensic contexts wherein a reconstruction of historical events, such as determining a document’s origins, may be absolutely vital for rendering verdicts of guilt or innocence. Further support for the conclusions of the MPM method can be provided in the context of machine learning frameworks as well; the distance metrics generated as a part of the MPM procedures will likely prove useful in the context of other machine learning and authorship attribution frameworks.

Crucially, the underlying concept of the MPM approach may possess extended value outside of forensic applications. In the modern world, technology has opened up diverse and highly complex possibilities for new assessment methods, such as the collection of rich behavioral data from smartphone technologies [47]. As interest grows in the use of rich idiographic data in the fields of mental health and medicine [48], new methods of quantifying an individual’s psychological variations over time will be required. The MPM and methods like it could, for example, be put to meaningful use in clinical settings for patients with psychotic or mood disorders. Such an approach may allow mental health providers to more accurately monitor patients’ day-to-day psychological variations and potentially facilitate faster detection of extreme, generalized psychological variations that could be diagnostic of problematic episodes [49]. Such possibilities currently remain within the scope of future research in psychology and the computational sciences.