By Keir Giles
A friendly-fire incident between counter-disinformation researchers shows that in defending Western democracies against malign influence, no one team or organisation has all the answers.
Malign influence is a complex and multi-dimensional issue, and adversaries that seek to harm Western societies through information operations select from a wide-ranging and constantly evolving toolkit, adapting and developing their tactics, techniques and procedures as they go. It follows that defences against information attack need to cover a similarly diverse range of approaches, spanning different disciplines, theoretical and applied, and from strategic down to tactical level.
In April 2020, two teams from two of these different disciplines came into direct collision, as academic researchers from Reframing Russia took aim at the methodology of counter-disinformation practitioners from EUvsDisinfo. Despite the fact that both groups are notionally on the same side, the incident highlighted a polarisation of opinion on methods to counter disinformation – but also pointed the way to better cooperation between different organisations with widely differing approaches and priorities.
In a short article on their website, Reframing Russia provided a critical assessment of EUvsDisinfo’s methods in selecting and publicising instances of Russian disinformation for their database and analytical products. Highlighting what they described as problems of omission, distortion and terminology, Reframing Russia’s Professors Stephen Hutchings and Vera Tolz suggested that through misrepresenting source material, EUvsDisinfo “is in danger of becoming a source for disinformation itself”.
Reactions from across the spectrum of counter-disinformation activity were swift and uncompromising. Reframing Russia itself came under vehement attack, including by activist group GorseFires Collectif seeking to identify the author of a post uncritically promoting RT on the Reframing Russia website, and a tendentious and personalised attack by “The Insider”, insinuating that Reframing Russia are funded not only by the UK but also by “Russian sources”. Meanwhile Reframing Russia’s article was strongly endorsed not only by RT and pro-Russian activists, but also by RUSI, a think-tank in London, which among valid and constructive points inadvertently repeated inaccurate claims about EUvsDisinfo and its operations. Overall, neither side in the conversation emerged with credit. But a closer look at the nature of the disagreement between the two sides in the conversation – and at how little they have in common despite in theory sharing common aims – shows the way toward a far more effective cooperative relationship between the many organisations working on the problem of disinformation and malign influence.
Where Reframing Russia went into specific detail on its criticism of EUvsDisinfo, that criticism appears valid. Reframing Russia’s article highlighted reports which, it claimed, were misrepresented. Referring back to the original media outputs confirmed that their description by EUvsDisinfo was inexplicable; and Reframing Russia assert that these were just a selection from a substantial number of similarly troubling reports in the EUvsDisinfo database. According to Vera Tolz survey of Covid-19 related entries in EUvsDisinfo’s database showed that “80% contained gross misrepresentations”.
Credibility is critical to EUvsDisinfo’s role. Association with the EU gives the project a mantle of authority, reflected in the widespread quoting of its findings. EUvsDisinfo will inevitably be held to a higher standard of proof and probity than the organisations they report on. It would therefore surely be in their best interest to address doubts in their reporting directly, looking again at the source bulletins and responding frankly and honestly if mistakes have been made. If not, they risk precisely the erosion of trust of which Reframing Russia is warning.
In addition, Reframing Russia highlighted EUvsDisinfo’s “pro-Kremlin” formulation to describe the sectors of the Russian information environment whose disinformation it reports. This too seems to point to a valid problem. Each page of the EUvsDisinfo website carries a caveat that its database collects “messages in the international information space that are identified as providing a partial, distorted, or false depiction of reality and spread key pro-Kremlin messages. This does not necessarily imply, however, that a given outlet is linked to the Kremlin or editorially pro-Kremlin, or that it has intentionally sought to disinform”. The explicit equivocation may well be a necessary substitute for an extended essay on the nebulous and sometimes intangible nature of malign influence, but it presents an obvious hostage to precisely this kind of criticism.
The roots of Reframing Russia’s disquiet about the nature of EUvsDisinfo’s work lies in the latter’s remit. As described by Anneli Ahonen, head of the European External Action Service’s East StratCom Task Force that manages EUvsDisinfo, it “is an awareness raising campaign, with a disclaimer stating that this does not reflect official EU position. This provides us with distance and analytical freedom from political level, and results in more credibility.” The campaigning role drives Reframing Russia’s core concern that in the drive to raise awareness, EUvsDisinfo steps beyond objective appraisal of the material they are handling.
Unfortunately, however, in presenting their concerns over EUvsDisinfo, Reframing Russia convinced some of their readers that they had fallen victim to the same problems of methodology that they themselves were criticising. Highlighting individual examples of miscategorisation by EUvsDisinfo gave the impression of cherry-picking material. Concerned that EUvsDisinfo might be “overgeneralising”, Reframing Russia sampled one news bulletin from one Russian channel on one day (Pervyy kanal on 12 March), and then appeared to draw generalised conclusions from it. Their article linked to contentious sources to bolster its arguments, such as repeated references to an attack on EUvsDisinfo by fringe Dutch activist Arjen Nijeboer in 2018. It included the suggestion that “the East StratCom Task Force relies on as many as 400 volunteers to trace Russian disinformation” (a claim later echoed by RUSI to call EUvsDisinfo “amateurish”), despite this not having been the case for over two years at the time of writing.
Reframing Russia later explained that the challenges to its methods were based on misinterpretations, that its “Two Short Reports” on Russian disinformation on Covid-19 were rapid response pieces produced following requests by a UK newspaper journalist and by a parliamentary committee, and that the short notice at which they were produced explained the strictly limited dataset on which its survey of RT coverage was based.
But in part because none of this was made clear, the response to the reports was savage criticism, extending to questioning the integrity and motives of the project leaders. Reframing Russia’s reference to being “asked to comment” on material produced by EUvsDisinfo sparked particular concern, and speculation over whose initiative it might have been to produce an apparent hit piece on the agency. In posting what they believed was a hasty but reasoned critique of another organisation’s methods, Reframing Russia’s project leaders inadvertently stepped out of academic debate and into a hornets’ nest of accusation and recrimination.
Reframing Russia’s Contribution
Reframing Russia’s contribution to counter-disinformation efforts lies at the academic and theoretical end of the spectrum. Primarily examining RT, the project’s remit is to investigate “the processes by which RT not only reconfigures publics and audiences, but also furnishes the very conditions for unprecedented modes of international mistrust, conflict and insecurity”. Academics friendly to Reframing Russia stress that this is a creditable, serious undertaking, and Professor Hutchings points out that “we have had academic articles deriving from our research accepted for publication in prestigious outlets like European Journal of Cultural Studies” – meaningless to most people, but a significant marker of achievement for academics. Vera Tolz adds that “our project is for academics. We are not political campaigners, practitioners or activists. If our work is useful for policy analysts and practitioners, it is great, but we do not pro-actively seek this impact.”
Theoretical study of disinformation and its effects provide a necessary counterpoint to the practical and tactical focus of organisations like EUvsDisinfo. This is especially important when considering whether disinformation is in fact effective. If Reframing Russia does in the end make a contribution to the vital question of how much impact – if any – malign influence campaigns using mechanisms such as RT do in fact deliver, then their work is crucial.
But the difference of opinion between Reframing Russia and EUvsDisinfo highlights the challenges inherent in crossing the boundaries between theoretical investigation and practical application. As noted above, EUvsDisinfo struggles with the problem of scoping and defining its target material: conversely, a separate attempt by Reframing Russia to deconstruct an individual item of Russian disinformation led to yet another trail of misunderstanding and misrepresentation.
At the time of British Prime Minister Boris Johnson’s admission to hospital with coronavirus in early April, Russian sources reported that he would be put on a ventilator – with the implication both that his condition was more severe than was being made known at the time, and that Russian media had well-placed sources within his health care team. This was widely covered in British media as a prime example of Russian disinformation. Reframing Russia took issue with this characterisation, and set out to prove that it was mistaken. But again, its conclusion that it could find “no evidence of any attempt by Russian news providers to spread disinformation about Prime Minister Johnson’s state of health”, and its means for reaching that conclusion, sparked alarm, concern and fierce criticism.
Reframing Russia pointed out that the original Russian report said Johnson was to be put on a ventilator whereas English translations reported that he already had been, which was true; but then implied that this difference between future and past tense invalidated the charge of disinformation. Vera Tolz’s argument that the Russian phrase for putting someone on a ventilator could also mean simply giving them an oxygen mask was received sceptically by other native Russian speakers. Most strikingly of all, Reframing Russia argued that the Boris Johnson ventilator story could not be called disinformation because it was so easily deniable, and “such blatant falsehoods would seem to serve little purpose anyway”.
This would seem to indicate a gulf between academic and practical approaches to Russian disinformation, since suggesting that Russian disinformation efforts do not include blatant falsehoods runs counter to years of detailed practical research on their methods. (Again, Reframing Russia later clarified that their comment should not be taken at face value, and explained that: “We have never suggested that Russian disinformation does not include blatant falsehoods. Of course, there are numerous examples of those… But in relation to issues where Russia is not directly implicated, the Russian media still can use poor practices, but they are much less likely to disseminate direct, pointless fabrications”).
This gulf between the two approaches to countering disinformation stems not only from mutual incomprehension but also from mutual unawareness. Reframing Russia and EUvsDisinfo were only dimly aware of each other’s existence before colliding, let alone having a clear understanding of each other’s methodology. Reframing Russia could have raised their concerns over EUvsDisinfo output privately before launching a public critique: the EU’s Q&A page explaining what East StratCom Task Force is also explains how to contact it about mistakes spotted in its output. But Reframing Russia felt that the problems they identified were so systematic and significant that this approach was not appropriate; furthermore “there are no specific names on the East StratCom site suggesting who exactly is in charge of the operation, with their personal email addresses”. Therefore, in Reframing Russia’s view, a public commentary was the preferred solution.
Reframing Russia contrasts EUvsDisinfo’s awareness-raising campaign with its own research, which it says aims to be dispassionate and objective, including when this leads to uncomfortable conclusions. Stephen Hutchings suggests that “some of RT’s reporting practices are deficient… but, in the context of the Covid-19 coverage they are undoubtedly superior to EastStartCom’s [sic] own methods”. Vera Tolz adds that “We do, however, believe that EUvsDisinfo’s data on Covid-19 is an exact equivalence [to RT]. In its factual inaccuracy it is considerably worse than what RT International produced. Their database entries include systematic misrepresentations. They can be legitimately called falsifications.”
This, naturally, means Reframing Russia is faced with the need to rebut accusations of drawing false equivalence between Russian malign influence campaigns and those who seek to counter them. A statement that the Institute for Statecraft in London and the Internet Research Agency in St Petersburg “follow not dissimilar tactics” was, Reframing Russia explained, no more than a reference to outsourcing policies by governments. Professor Hutchings adds that “none of us would dream of drawing an equivalence between RT and BBC, there is none. However it seems to us perfectly legitimate to point out instances where the practices are equivalent.”
While EUvsDisinfo may lean too far in flagging content as disinformation, Reframing Russia may risk similar criticism in the opposite direction through presenting research conclusions on RT which inadvertently resemble making excuses for it – particularly when those conclusions echo the talking points of outright apologists. Reframing Russia stresses that “much of RT’s output is purely factual”, which critics say plays down the remainder that is distorted or manipulative.
Where the facts are in question, Reframing Russia is at risk of appearing to give RT the benefit of the doubt. This extends to falling victim to a classic confusion of balance with objectivity. Because the project’s various correspondents unanimously disapprove of RT, in order to address this perceived imbalance the project’s leaders decided to host (and later delete) an article promoting RT on their website, once again generating fierce criticism and deep, yet unfounded, suspicion of their motives.
The manner in which these two organisations work to fundamentally different principles is reflected even in how they present themselves to the world. The purpose of EUvsDisinfo’s public presence is specifically to raise awareness. By contrast, the part of Reframing Russia’s work which it considers most significant is largely hidden from view.
Reframing Russia’s team members have written a number of academic studies, some of which are available for public consumption despite the majority being confined to academic journals. The project has made a significant contribution to understanding of RT’s programming, content, and audiences in multiple languages, as part of its goal “to situate RT in the international media ecology”. But it has also been criticised specifically for this close focus on RT, forming as it does only part – and a relatively insignificant part – of a much wider effort of malign influence using levers way beyond traditional or social media. Practitioners argue that close focus on RT while neglecting the broader disinformation ecosystem – for example social media – means missing essential context, in particular other instruments of subversion or destabilisation like agents of influence, useful idiots, and others who consciously or unconsciously promote and disseminate Russian talking points.
But Reframing Russia points out once again that the impression that its work is limited to RT is a misinterpretation, and further that the project’s apparent emphasis on instances of direct falsification, as opposed to the broader picture of narratives and intent, is also misleading – as is the suggestion that the project does not examine social media. In each case, Vera Tolz points out, visitors to the project’s website have been misguided by the assumption that what is presented there is representative of the project. Instead, she points out, “blog posts” there are “not representative of any research we do”, and in fact “from what is currently on the website it is not easy to get an impression of what project is about”. Instead, Reframing Russia’s primary output is intended to be academic journal articles and contributions to compiled volumes, culminating in a book to be published long after the project ends in August 2020.
The contrast with EUvsDisinfo, whose aim is rapid campaigning on current issues using the website and associated database as a key communication medium, could hardly be greater. But the key point that unites both projects is a need to pre-empt or mitigate censure and denunciation on mistaken grounds by more clearly communicating their aims and objectives, and in particular the principles to which they work, not only with each other but with audiences in general.
The Covid-19 pandemic provides the ideal environment for malign influence to thrive as it fills a vacuum of authoritative information. The perceived lack of reliable data about the virus provides fertile ground for information and disinformation campaigns, especially feeding on fear, uncertainty and doubt. Russian sources, unsurprisingly, are making the most of the opportunities provided by coronavirus to reinforce their long-running narratives. Vigilance against these campaigns, and successful identification of where they do (and do not) have the potential to cause real damage, is essential. But the divergence in approach between Reframing Russia and EUvsDisinfo reflects the fine balance between over-reaction to Russian disinformation campaigns and failing to react at all.
One answer lies in successful collaboration, cooperation or at the very least communication between different groups. For example, flaws in EUvsDisinfo reporting or deeper problems with methodology, when pointed out constructively, should be welcomed and acted on. Meanwhile Reframing Russia could be assisted with the broad context within which their important work sits. The same principles apply to many of the disparate groups, entities and even individuals now active in the counter-disinformation industry.
The organisations working in this space are many and varied, but not all are able to share the results of their endeavours. Professor Tolz, drawing on her substantial experience of monitoring traditional media for Radio Liberty during the 1980s and early 1990s, suggests that “further lessons on how high-quality, dependable media analysis is produced can be learned from the work of BBC Monitoring whose well-trained experts follow media across the world, including Russia.” This may well be true even following BBC Monitoring’s evisceration and repurposing by the BBC in its disapproval of “doing government work”, but at the time of writing BBCM’s contract to provide services to East StratCom has not been renewed, and much of its counter-disinformation work remains accessible by subscription only. Similarly, other centres of expertise appear constitutionally or organisationally constrained from pooling and sharing their knowledge and analysis. The US Global Engagement Center, for example, reports that it has established international partnerships to facilitate information sharing and response coordination, but declines to publicise its methodology on the grounds that this would reveal “tradecraft”.
But in the absence of a broader dialogue across all sectors of the expert community, counter-disinformation efforts will remain hostage to well-intentioned but misguided efforts, such as the detailed and widely-quoted New York Times explainer from mid-April that was roundly condemned by disinformation researchers for being deeply misleading. Those at both extremes of the conversation will continue to argue from entrenched positions rather than considering issues, instances or nuances. In addition some of the most prolific writers on Russia can at times adopt a condescending tone, suggesting that all those who disagree with them do so only because they do not understand Russia as well or are incapable of evidence-based discussion and analysis; and too often this approach is used to attempt to win an argument rather than actually arrive at a sensible conclusion.
That sensible conclusion lies in the middle ground. It is not true that counter-disinformation teams don’t get it wrong and can’t be criticised because they are the good guys. Where they slip up or make mistakes, or even where their core principles of operation may be flawed, this should be highlighted to make their output better and more useful and relevant. But it is also not true that when counter-disinformation teams do get it wrong, that makes them just as bad as the malign influence they are working against. This is a false and damaging equivalence, since aims, transparency, accountability and much more set them poles apart. It is further not true that established academics should not be criticised from outside the profession, or their remit questioned, on the grounds that this constrains free enquiry. There is no reason why universities should not be held to the same standards of relevance and recognition of the real-life impact of their utterances as any other organisations. Mutual support is overdue: by seeking a cooperative, rather than adversarial approach between players on the same side, defenders against malign influence can help mitigate the effects of propaganda outlets rather than providing them with material.
Keir Giles is a Senior Consulting Fellow with the Russia and Eurasia Programme at Chatham House, the Royal Institute of International Affairs.
Read more: Paul Bischoff explores how disinformation is spread on social media in this Comparitech report. “Inside a Facebook bot farm that pumps out 200k+ political posts per month”.