It is Not How Well You Do But How Well Important People Think You Are Doing: Elizabeth Leyne and Yvette Nonté, Is the Intelligence Community Staying Ahead of the Digital Curve?A Survey of its Highest-level Customers and Leaders on the Challenges and Opportunities Ahead

 

 Is the Intelligence Community (IC) staying ahead of the digital curve? Over eight months, the authors conducted in-depth interviews to probe this question with over 45 current and former high-ranking national security professionals, including the leaders of five U.S. intelligence agencies, Cabinet-level officials, two former Chairmen of the Joint Chiefs of Staff, and Congressional leaders. This study finds that while the IC is overwhelmingly critical to U.S. leaders in the Digital Age, it has fallen behind the digital curve. There was unanimity in the imperative for the IC to radically transform many aspects of its business to accelerate through the digital curve and continue to remain relevant.

So begins the Introduction to the Executive Summary of Elizabeth Leyne and Yvette Nonté, Is the Intelligence Community Staying Ahead of the Digital Curve?A Survey of its Highest-level Customers and Leaders on the Challenges and Opportunities Ahead National Security Fellowship Program
Belfer Center for Science and International Affairs Harvard Kennedy School paper (August 2021). 

The paper is a remarkable study, not of the capacities of the American Intelligence Community itself, doubts about which I have long harbored but discretely expressed (eg The Passion of John Brennan), but of the way critical actors who consume the IC's work product think they are doing.  It reminds us that even as intelligence becomes more data driven and metrics oriented--neither metrics nor expectations are as important as the sense of trust one's consumers have in the producers of intelligence, without reference either to performance metrics or to its measurement against an ideal.  In that respect, at least in the United States, the IC appears to have achieved an autonomy from metrics based accountability that is the envy of the rest of the American nomenklatura (public or private).  And that is both a great pity and the largest looming tragedy of the breakdown of the American administrative architecture in the first third of the 21st century. 

One finding of the report perhaps sums it up best:

For the IC customer dataset, there was a stark dichotomy between the highest-level customers at the Cabinet level, who offered glowing praise for the IC, and customers a few echelons below, who gave the IC mixed to poor reviews. Military leaders across the board largely had nothing but high marks for the IC and foot-stomped the criticality of the IC’s analysis to their decisionmaking processes. On the other hand, Congressional customers surprisingly noted that only a small sub-set of Congress values intel products; they claimed that large swaths of Congress do not seek out classified products, preferring the ease of access to unclassified information. Despite these differences in the dataset, all of these customers resoundingly saw the IC as credible and unbiased on the whole. The only major caveat was on Afghanistan, where a fair number of customers perceived some bias in analytic assessments and collection priorities.(Leyne and Nonté, supra. pp. 2-3.

What these customers suggest, for improvement," that all coalesced around strengthening partnerships—with allies, within the US government, and with the private sector" (Ibid., p. 3), sounds suspiciously like the vacuous "Trump lite" political talking points proffered as a return from the precipice of Trump era global irresponsibility (at least the tweeting part). None sound in performance objectives internal to the producers of intelligence, all sound like the sort of political objectives that have yet to be undertaken to any good effect by the consumers of data themselves.  That is the irony, the areas of improvement suggested to the IC are at best a telling reflection of what is wanting in those with the responsibility of consuming and holding to account the producers, of intelligence. That ought to provide fodder for those "at the top." It won't; especially given the penchant for self congratulations among that fairly tight circle of people who inhabit the far reaches of the consuming and producing communities. 

On the other hand, it is worth noting that a goodly part of the failures that might be easily attributed to the producers of intelligence, the IC and its institutions, may actually reflect  related problems that fall squarely on the consumers of intelligence and indict what Tony Blair recently described as their penchant for imbecility.  The first, goes to the pressure from consumers for the IC to adopt the political perspectives and ideological lens and conceits of intelligence consumers and to ensure that intelligence "facts" fit the political picture that these consumers want to paint.  The intelligence community has sometimes (perhaps for its own survival) gotten into the habit of giving intelligence  consumers what they want.  That is, after all, the essence of market culture, and the IC wants to stay in business. Thus it pays for IC to mold its product to fit the bias of those who consume it.  That result, of course, at least from time to time is disastrous. But whose bias?  But sometimes the IC has some discretion in choosing its clients.We have been treated to that for a long time, but has it now become pathological?    During the Trump administration if was never clear who that client was, sometimes but not always the Office of the President, other times who knows--perhaps more autonomous administrative or organs and sometimes (rarely) the press organs and their collaborators who seemed to be more attuned to intelligence that the Commander in Chief (though the fault for that might also be squarely placed where it belongs).

The second, is that the entire structure of metrics bases assessment is at best opaque and at worst absent from the cultures of the IC.  Or perhaps the metrics are a function of norms, ideals and expectations that might come as a surprise. But the problem of measurement is enormous in this context, except perhaps within the IC and those assessments must remain opaque. Part of the problem arises because it is not clear that the IC autonomy, and the power to tell consumer what they do not want to hear--along with emerging cultures of risk aversion--make it possible to develop a responsibility  to provide a range of interpretive assessment some of which may annoy those who consume it.  Part of it involves guidance.  It is one thing to produce intelligence--it is quite another to invest it with meaning, and it is altogether different to use it for exercises in predictive analytics.  How does one develop metrics? Perhaps developing an ethics of responsibility with respect to these functions would be more effective. Bit people have lost their jobs for less.  

The third is relevance. Mr Trump, for example was notorious for treating IC in a publicly humiliating cavalier way--perhaps the mutual hatred necessarily followed, as did the choices on both sides of the consumer-producer divide between 2016 and 2020 (the current Administration presents its own but quite different, challenges). It is hard to tell--that, after all, is fundamental to the nature of the IC after all. So in a sense what we have here is a tease of sorts. But the Report also highlights  that Mr. Trump's approach was hardly aberrational among the key stakeholders with a responsibility to hold IC to account--the political branches. 

All of this is speculation of course. . . .And likely wrong.  But that is also a problem.  The IC cannot at the same time tease  those outside of its inner circles  and then become offended when those teased want more. And it is not enough reassurance to know that those at the top continue to view the IC as trustworthy.  Cultures of accountability and compliance--and the legal standards it has nourished, make clear that it is no longer enough to rely on the opinion of those at the top.  Something more is needed.  And that something more is noticeably absent here. This is not to say that IC is broken, it is not; the assessment of those surveyed rings true ("Unprompted, over 20% of the interviewees emphasized that despite some public and elite opinion, the IC is not broken, and most of these luminaries conveyed a protective attitude to the IC, which they clearly valued" (Ibid., p. 4). Perhaps it is more accurate to say that tools are only as good as those who use (consume) them.  Thus the key insight of the report is not an indictment of the IC but of those who use it, the consumers of intelligence.  

“At times, policy preference can creep into the analytic process. I think it colored analysis on Afghanistan and maybe on Iran over the years. Some maybe had Stockholm syndrome with the Taliban.” —Former National Security Advisor, LTG H.R. McMaster (Ibid., p. 14).

And that leads to the last insight.  Given the intimate connection between producers and consumers, it is clear that issues of bias (on both sides) ought to be at the top of the list of accountability measures and assessments.  But that is easy enough to confront with enough will (mostly the will to control ego and political/institutional ambition) and perhaps autonomous auditing. But the real issue is not that sort of bias. It is instead the effect of bias on the way in which IC chooses to collect intelligence (what it over emphasizes and what it ignores and what is is blind to  because of multi level multi institutional bias), and the way in which it approaches investing that intelligence with meaning in itself and as factors in the emerging reliance on modeling and predictive analytics. Here "quality control" measures, if not systems of accountability, are essential, but will require much more thought. 

It is with that in mind that the eight challenges identified may be ore usefully understood: .

1. Igniting cultural change for the challenges ahead at every level of the Community
2. Striking the right balance between agency autonomy and IC integration
3. Developing and scaling artificial intelligence and machine learning tools
4. Retooling human resources for the future
5. Organizing analytic resources around the national security threats of the Digital Age
6. Integrating Open Source—A necessary but complicated cultural shift
7. Strengthening international sharing constructs with allies
8. Designing new paradigms for sharing with the private sector (Ibid., p. 31; elaborated pp. 34-54).

But the great power of the Report is its semiotic insights.  It starts with the fundamental premise that meaning is a function of collective belief.  It follows that collective belief becomes the "fact" metric" against which something can be measured--in this case aspects of the American IC apparatus.  IC , then, is an "object" the signification of which is created exogenously, by the opinion of the consumers of IC "product." The measure of the value of the work of IC then is a function of consumer satisfaction.  Yet consumer satisfaction is itself a function exogenous to the internal operating principles of IC (at least in theory).  So the process of signification creates a disjunction between how IC ought to be assessed endogenously, as a function of its own organizational goals, mission, and ideology, and how IC must be assessed exogenously, as a function of its ability to meet the needs (including the ideological, political, career, and strategic needs and desires) of its clients.  In that context semiotics also suggests that the determinants of measure are themselves signified by desire, in this case by the desire to keep clients happy.   The difficulty then arises as a function of any need to reconcile endogenous integrity and coherence enhancing internal measures with potentially incompatible  desire meeting client functions. Those, disjunctions,are nicely on display in the 8 challenges. For those involved in the development and deployment of AI, bid data analytics and predictive analytics, including simulations,  these baseline premises for constructing and assessing meaning pose substantial challenges of their own.

In the end, the fundamental ordering question remains unanswered: who (or what) does IC serve first? The Report suggests that it serves first a farm full of consumers and thus its value depends on consumer satisfaction, which then also serves as the basis through which its meaning construction is elaborated.  That has significant consequences--the primary one being the way in which it organizes and analyses , the way it manufactures, it product for "sale" to its consumers.  The dangers then assume the character of the flaws of the desires, or the blindness, of the consumer sector.  Those desires, biases, then inevitably creep into both the technologies and analytics through which the IC engages in its intelligence and other activities. The entire report is worth a careful read.  It may be accessed HERE.