I have written about the upcoming conference: Super-Scoring? Data-driven
societal technologies in
China and Western-style democracies as a new challenge for education, funded and supported by Grimme-Forschungskolleg an der Universität zu Köln and Bundeszentrale für politische Bildung (bpb), which will take place on 11 October 2019 in Cologne, Germany. More information HERE.
I was delighted to have been asked to contribute thoughts on Chinese Social Credit Systems with a group that is producing some very exciting work: Yongxi Chen (Hong Kong University) and Mareike Ohlberg (Mercator Institute for China Studies, MERICS, Berlin); Björn Ahl (Cologne) will chair our engagement.The Conference organizers are circulating the essays of participants. My contribution, "Blacklists and Social Credit Regimes in China" explores the centrality of "lists" to the project of developing systems of credit and rating regimes tied to an equally complex regime of restrictions and privileges that follow from one's place within rating and credit regimes. In the process I suggest how the language of lists (whether lists of ratings or lists of identified persons or businesses that have met some sort of consequential thresh hold for action) is displacing the classical language of law. More important, the sensibilities of ratings and its focus on gathering information and subjecting them to analysis that can be machine learned and administered, shifts the object of law from a normative to a constituting function. The ramifications for education as wel as the construction of rule of law systems remains largely unexplored--if only because the character of the displacing system remains elusive.
The essay first very briefly describes the Chinese Social Credit (CSC) system. It then considers two questions: (1) how does one build a super scoring system through the structures of CSC?; and (2) what role do lists play within that framework. It ends with a short consideration of what may be the principle challenges for political and general education that now arise in the context of these digital regulatory measures.
The essay is available for download on the Super-Scoring? Conference website. It is also available for download on my personal website: Here; and below (pix added).
Blacklists and
Social Credit Regimes in China
By Larry Catá Backer
Prepared for the interdisciplinary symposium
“Super-Scoring? Data-driven societal technologies in China and Western-style
democracies as a new challenge for education.”
Cologne, Germany; October 11, 2019.
Introduction.
Black(white)(red)lists have been an instrument of
regulatory management for a long time.
The city was filled with
murder and there was no counting the executions or setting a limit on them. . .
.Finally one of the younger men, Gaius Metellus, ventured to ask Sulla in the
senate at what point this terrible state of affairs was to end. . . . “We are
not asking you,” he said, “to pardon those whom you have decided to kill; all
we ask is that you should free from suspense those whom you have decided not to
kill.” Sulla replied that he was not sure
yet whom he would spare, and Metellus at once said: “Then let us know
whom you intend to punish.” . . . Then immediately, and without consulting any
magistrate, Sulla published a list of eighty men to be condemned. (Plutarch, The
Fall of the Roman Republic, Sulla, ¶ 31).
In
the West it has long been common for leaders of states (or those whose
leadership guides the state) to establish proscription lists. The most famous are those produced by Marius
and Sulla during the first of the great Roman civil wars in the century before
the collapse of the Republic. The consequences of being listed ranged from loss
of status, to loss of property and life. But it is equally well established to confer
privilege; these (white or red) lists opened opportunity and signaled status.
These lists were administered by any person, public or private entity with the
ability to use the lists as a means of punishing or rewarding those on it.
Chinese history is also full of lists created by officials and others. They are
all were used to similar effect—to identify individuals or societies for the
purpose of reward or punishment. That
was accomplished either by listing or ranking; the former where the list itself
contained the restriction or privilege; the latter where the list permitted
others to use rank to determine consequence.
These lists have changed little in form, or function even by
institutions officially leery of their use. By 2010 even the EU produced its
“Visa Information System” for border management. These lists have always been instruments—these
are the “middlemen” of regulation the operative for of which is expressed in a
designation. That is, these lists merely articulate judgments (placement on the
list) that is the product of applying data to an analytical model useful for
separating those who belong on the list from those who do not. In effect, these
lists are the product of an analytics—certainly crude by contemporary standards
in the era before “Big Data,” but always becoming more potent as technological
levels and the taste for using these structures increased all over the world. Lists
are the way in which ratings are memorialized by reference to a threshold. They are a means of scoring respecting the
aggregation of conditions necessary to produce a judgment of inclusion or
exclusion from a list. Lists, however, in themselves are merely passive
conclusions; they acquire potency only when circulated, and, in the wake of
circulation, when they produce a timely social, political and economic
consequences undertaken through and supported by the state. These consequences,
themselves, must be keyed to societal values that resonate in ways that create
incentives to support or condemn particular practices or behaviors that served
as the basis for placement on the list. In other words, there must be a
connection between that facts that
produce assessments on which list are created, and the consequences, related to
that data for placement (or non-placement) on that list.
The value of lists, then, lies in their utility for managing
effects. Hierarchy, status, and privilege are the key elements around which
black(white)(red) lists are activated. In contemporary societies, color coding
merely makes the ultimate object of the list ore apparent; that is, the color
is meant to match the consequences—black for punitive, white/red for privilege
or reward. But while a single list may be useful means of actualizing a score,
super-scoring becomes possible only by the development and coordination of a
larger aggregation of lists, each potent within its own narrow field, but
together capable of producing a coherent means of managing any aspect of
societal expression (and its underlying beliefs).
It is in that context that it is possible to think about lists,
and a developing list universe system, at the center of China’s Social Credit (CSC)
system. This essay considers the role of
lists in the construction of CSC conceived as a vast super-scoring system that
effectively displaces law and administrative regulation as the engine for
ordering society through government. The essay first very briefly describes the
CSC system. It then considers two
questions: (1) how does one build a super scoring system through the structures
of CSC?; and (2) what role do lists play within that framework. It ends with a
short consideration of what may be the principle challenges for political and
general education that now arise in the context of these digital regulatory
measures.
1. The Chinese social credit system.
The CSC represents a new regulatory methodology which seeks
to displace the traditional system of enforcing law and encouraging approved
behavior, for a complex and interrelated system of rewards and
punishments. That system of rewards and
punishments is based on scoring the way that every social, political, cultural
and economic actor in China conforms to laws, rules, and other expectations,
and on the way that they organize their social, political and economic
relations in conformity to expectations. The CSC system, then, does not
displace the power of law, rules, norms, and the like to describe behavior
expectations, but it transforms implementation by disconnecting the specific
misconduct with a related punishment.
For example, a violation of a court order to pay a fine may place an
individual on a list of untrustworthy persons which then is used to prohibit
the individual from purchasing airplane tickets. In effect, a CSC system is meant to rate
behavior across all aspects of societal life.
Rating every aspect of life is then the means by which behavior can be
regulated (the law becomes the aggregation of actions which affects the
ratings) and in this way to steer the behavior of all social actors—but
principally of individuals, business, and eventually government officials. At
its most developed, CSC as a regulatory system will merge into and become the
way that law itself is expressed.
(Pix credit HERE)
The CSC system has a moral dimension as well, which deeply
informs its regulatory and enforcement dimensions. It was created, in part, to fundamentally
steer the culture and practices of people in virtually every aspect of their
lives. To those ends, the Twelve Cor Socialist Values unveiled in 2012 plays an
important role. At the same time, CSC has as its objective to enhance the
objective of achieving a rule of law society, that is, a compliance culture, by
combining a power to rate compliance with rewards and punishments to touch on
the ability of the individual or firm to function effectively in society. More
importantly, it was meant to delegate that task to the state, under the
guidance of its vanguard. 
To that end, the list represents the end product of a process
of data driven analytics, producing a conclusion related to each list
(compliance with court order lists; student misbehavior lists; subway
misbehavior lists; community service lists; financial responsibility lists;
etc.) based on the factors weighed (e.g., spitting, eating, loud music playing
etc. on the subway for the subway misbehavior list; timely payment of bills,
traffic tickets, utility bills for a financial responsibility list) ) in order
to determine whether a threshold quantum of conduct has occurred that merits
inclusion on the list. The end product
of that analytics—the list—then serves as the signal necessary to either
encourage or compel other social, political, and economic actors to act (reward
or punish) on the basis of inclusion.
That is what gives list its power, something unchanged from the days of
the Roman proscriptions under Sulla, and shifts regulatory power from the rules
(ostensibly the object of all of this fuss) to the decisions about what rules
will be given what weight to produce what sort of lists. One here moves far
from an ordinary conception of ratings as a means to an ends. CSC ratings serve
the critical element of data driven governance founded on analytics that makes
it possible to regulate and steer behavior through data and analytics rather
than through law and traditional enforcement. When produced in multiple ways by multiple
ratings organs and coordinated by and through the state, one has in sight the
possibility of a super-scoring mechanism that itself could point to a new way
of organizing law.
2.
Building a CSC super-scoring system.
The Chinese Social Credit system understood as a complex
network of coordinated scoring (rating) behavior in every aspect of organized
life would be fairly useless if it merely the aggregation of the product of
such scoring by a multitude of actors without an overall design. The Chinese
system, in contrast, has over the course of the last several years,
increasingly evidenced not just its coordination, but also the unity of a very
ambitious conceptualization, an approach of fractured experimentation as the
system moves from conception to implementation, and ultimately the
bureaucratization and loose
centralization of emerging scoring systems providing the platform necessary for
“super” scoring.
A. Conceptualization. The initial conceptualization of
CSC as a super-scoring system was first widely publicized in the now famous
2014 State Council Planning Outline for the Construction of a Social Credit
System (2014-2020). The CSC was founded on the application of emerging
developments in Leninist theory that began to embrace the idea of the need to
overcome the governing methodologies of liberal democratic states which were
inextricable from liberal democratic principles and culture. It followed that an emerging system of
socialist market economy under a socialist political model grounded in
socialist culture required a socialist approach to governance suitable for the
times. This conceptual development ran parallel to the development of what
would become by the time of the 19th Chinese Communist Party
Congress New Era Thought. Its opening
paragraph, in retrospect, provides an excellent summary of the principles
underlying the model:
A social credit system is an important component
part of the Socialist market economy system and the social governance system.
It is founded on laws, regulations, standards and charters, it is based on a
complete network covering the credit records of members of society and credit
infrastructure, it is supported by the lawful application of credit information
and a credit services system, its inherent requirements are establishing the
idea of an sincerity culture, and carrying forward sincerity and traditional
virtues, it uses encouragement to keep trust and constraints against breaking
trust as incentive mechanisms, and its objective is raising the honest
mentality and credit levels of the entire society.
The
2014 Planning Outline outlines an ambitious agenda—to remake society in line
with the new era made possible by the success of the generation long project of
Reform and Opening Up. To that end new
methods are necessary to shape society.
Rules are important (in the form of law, regulation, social norms,
principles and the like), but they remain abstract and remote unless they can
be internalized. But internalization is
possible only by changing cultures of behavior and behavior expectations. As
such, in order to internalize a new weltanschauung , it is necessary to
inject it from the outside. It is here
that super scoring moves from its normative objectives (the new era integrity
society framed by the twelve Core Socialist Values) to its methodologies (data
driven governance, ratings, punishment and reward). Law moves to the
sidelines—it is reduced to the means by which the factors necessary for the
production of scores, or rankings, can be determined.
From this core, the Chinese authorities have sought to
implement Social Credit as a system. In
the usual pattern of substantial reform in China, that required first a space
for experimentation divided among a large class of principal actors (the
scoring part). But it also required a
space for the development of mechanisms for coordination and for the construction
of appropriate punishment and reward systems (the “super” part of
scoring). These are discussed next.
(Pix credit HERE)
B. Fractured
experimentation. Chinese authorities have traditionally embraced a “think
centrally and experiment locally” approach to new initiatives. More importantly, the state authorities have
tended to de-link public authorities from direct control of such
experiments. The effect has been to
produce the great centralizing conceptual vision at the highest level of the
central government, but to implement the vision strategically in stages, first
by delegating initial experiments in
implementation to closely coordinated non-governmental groups and low level
officials, and then by ensuring that those experiments take place either at a
very local level near Beijing, or elsewhere quite far from the seat of the central government—southwest and
southern China have since the last quarter of the 20th century
served those ends, with Shanghai the center of market-finance oriented
experimentation. In this way success could be incorporated into the building of
a comprehensive system, and failures could be blamed on local officials or
private enterprises who could be punished without affecting the “integrity” of
the central government. In effect once
the central authorities produced the premises and principles within which a
system is to be constructed, code writing and application was fractured and
spread out to key experimental centers.
Once useful code was produced and successfully applied, it could be
either expanded or the seeded elsewhere under systems of coordination in which
the central government would again assume a leading role.
Thus, the super-scoring elements of CSC lies in central
control at both the initial stage (conceptualization and vision-parameters; the
normative structures and system objectives) and at the final operational stage
(coordination, management, direction through rules based administrative
discretion systems guided or controlled by a central administrative apparatus).
The success of the super scoring system, however, is also premised on the
fracturing of experimentation at the crucial initial implementation and “code
writing” stage. Central to this effort was the objective of transforming
scoring, or ranking, from an autonomous object to the expression in a simple
and easy to use form of a complex analytics that applied the overarching
conceptual principles of the 2014 State Council Planning Outline
directly onto the bodies of its targeted individuals and enterprises (including
eventually state officials as well).
This is precisely the model followed in the creation of CSC
in the specifically Chinese context. The
initial conceptualization was marked by the 2014 State Council Planning
Outline. The Planning Outline was not the first but rather the last expression
of a long process of working through the core conceptual elements that has been
traced by some to provincial experiments in scoring in the early 20th
century and then through the development of the Twelve Core Socialist Values
and the New Era Theory of Xi Jinping. The
conceptualization document itself then produced a timeline within which, from
2014 through 2020, a large set of more or less free experimentation would be
encouraged, managed, or permitted, operated by both private firms (in strict
coordination with the state) or lower level administrative units, especially
outside of Beijing—plus the Beijing municipal authorities). In the meantime, as
experiments produced successes or (useful) failures, the central authorities
could begin refining (and also experimenting with) approaches necessary to
weave these experiments together into something that could emerge as a
coordinated comprehensive and self-reflexive system (after 2020).
The fractionalization stage is what has tended to fascinate
Western observers. That in part follows
because it has been the most visible element of this long and coordinated (and
unfinished process). But perhaps also
because the fractionalized experimentation phase is the one most accessible to
Western oriented analysts because it can be analogized (and thus misread) along
the lines of Western conceptual frameworks—the market, competition, and
consumer choice incentive models at the heart of recent governance efforts
including regulatory governance, markets based management and international
soft law disclosure frameworks. It was marked by a number of high profile and
quite well publicized efforts chronicled in both Chinese and Western press
outlets. These included a number of initiatives. The most well-known of these include the
licensing of eight companies to develop a mechanics of social credit
scoring—that is to code data that could be usefully converted through analytics
into judgments against which algorithmically produced consequences could be
attached. The most well-known of these are the credit scoring systems developed
or overseen by China Rapid Finance (Tencent) and by Sesame Credit (Alibaba). In simplest terms—they were charged with
developing lists of individuals to be rewarded or punished in accordance with
meta-objectives of the universal principles to be advanced by the
systematization of CSC. Such lists could be single purpose—the lists of
individuals and enterprises that failed to comply with judicial orders could be
transformed into a list (subject to elementary analytics, e.g. threshold
amounts, time lapse between order and payment, etc.).
In the case of what is now known as social
credit scoring, the effectiveness of the lists depends on their
aggregation and weighting. That process of weighted amalgamation (the
incorporation of a moral-normative measurability of data required second order
analytics—a super scoring—that was meant to blend through a process of
weighting the product of multiple lists and to convert the result not into a
binary (on-the-list versus off-the-list) but into a score. For that the West proved useful—it seemed a
simple matter to take financial credit scoring already refined in the West and
used to rate governments, enterprises, and individuals, and deploy it for the
more comprehensive objectives of the 2014 Planning Outline. These scores could
then be used to simultaneously reward and punish.
A central objective of these fractionalized projects was the
development of complex targeted analytics that could be reduced to lists of
individuals reflecting conformity to one or more of the elements of the
trustworthiness principles set out in the conceptual guidance from the central
authorities. While the actual algorithms remain secret, at the initial stage,
these private enterprise experiments in scoring did disclose categories of data
that contributed raw material to the super scoring analytics. These included
(1) credit history; (2) fulfillment capacity (compliance with public and
private obligations); (3) personal characteristics; (4) Behavior and
preferences; and (5) interpersonal relationships. Added together, the categories leave
virtually nothing subject to data harvesting and analysis. For example, the fifth category permits
harvesting data about social networks and rating based on the strength of
interpersonal connections and the ratings of those with whom one has a
relationship (friendship circles can raise or lower scores). Likewise, shopping habits under the fourth
category can be used for a similar purpose (buying diapers may raise scores
while buying too many video games may reduce scores based on an assessment of
commitment to societal objectives). Similarly,
the “quality” of social media postings could also find their way into the
analytics of scoring. The third category also nudges. Scores can depend on
where one lives and the sort of connections one has with society, including
mobile phone and computing. The second category speaks to trustworthiness in commercial
and private relations. But it can also be easily coordinated with public
lists—for example lists of compliance with court orders, or lists of losing
defendants, or lists of the rate of police complaints filed against an
individual. The first category is the simplest. It includes the same utility
and challenges as encountered in the West, but here it is not deployed only
autonomously but is blended with all other activities to produce cross category
behavior nudging effects. For example, if failure to pay bills on time reduce
credit scores, it may impact the ability of the individual to obtain a visa to
travel abroad, rent a car, utilize certain services, or send their children to
a specific school. But the converse is
also true—produce positive contributing data and privileges become
available—cheaper loans, travel, schooling, credit, housing, faster internet
speeds, and the like.
There were a number of other well publicized initiatives. These
included the 2017 announcement by the Supreme People’s Court of the
construction of a blacklist (Supreme People’s Court’s Judgment Defaulter List; 关于印发对失信被执行人实施联合惩戒的合作备忘录的通知)
that started with almost seven million names of people determined to have
engaged in threshold exceeding misdeeds who would be banned from taking
flights. The Supreme People’s Court
Judgement Defaulter List was coordinated through a memorandum of understanding
with several dozen administrative departments that also issued no fly lists.
Inclusion in the various no fly lists was tied either to misconduct related to
air travel or broadly interpreted “untrustworthiness” as determined by the
agency within the scope of their jurisdiction. For the Supreme People’s Curt
that centered on judgement defaults; for the State Taxation Administration
inclusion was triggered by a failure to pay taxes; for the Ministry of Finance
on a finding of financial fraud, or certain overdue debt obligations; for
the Ministry of Human Resources and
Social Security inclusion was triggered
by a finding of fa or failure to cooperate in investigations; and for
the Securities and Futures Commission the trigger was failure to pay fines
or failure of public companies to
perform public commitments.
It was reported that a related effort was undertaken in Henan
Province (Zhengzhou) where a shaming announcement was substituted for a dial
tone on the phones of people who failed to pay debts per judicial order. In Hubei (Wuhan) the targets were students
who could be blacklisted for committing an excessive number of rule infractions
(cheating, unpaid tuition, and the like).
In Shangdung (Rongcheng) the target was public social behavior of
persons (jaywalking, littering) and in Beijing it was behavior on the subway. Not all experiments were targeting
individuals. Business, especially
private business was also an object of scoring.
In Sichuan (Luzhou local authorities sought to implement a social credit
scoring system for liquor businesses focused on regulatory compliance. Business ranking is likely to expand to foreign
enterprises operating in China as well.
C. Bureaucratization, coordination, and loose
centralization. The third stage, and the critical one for the rolling out
of an “all around” CSC system is now currently in its initial phase. It is also among the more difficult (for
Westerners) elements of the move form conceptualization to system to understand
on its own terms rather than through a Western the conceptual lens. Very briefly, the State Council began this 3rd
phase of the construction of the Chinese CSC (super scoring) system as early as
2016 with its “Warning and Punishment Mechanisms for Persons Subject to
Enforcement for Trust Breaking.” The object was to begin the process of
inter-institutional coordination of data and analytics. More importantly, it set the tone for
consequences—while conformity could bring rewards, non-conformity must also
produce punishment. Punishment was not
to be penal or civil (as inevitable within a traditional law-administrative
system) but rather serve a nudging purpose.
The character of punishment then was transformed to a system of
restrictions. The more one failed to
conform, the lower one’s score (or the more likely the placement on a blacklist)
and the more comprehensive and severe the restrictions that followed. The 2016
State Council Warning has a step in the direction of creating an
administrative apparatus for the management of the scoring and rating systems
being developed during the period of experimentation and localized
implementation between 2014 and 2020.
These have been followed by provincial and local regulations
seeking to implement portions of the State Council guidance and from 2019 on
several important new directives from the central government that are meant to
complete a conceptual architecture for the roll out of a nationally coordinated
CSC by the 2020 self-imposed deadline. Among these are the Ministry of
Commerce’s 17 July 2019 Notice on Printing and Distributing “Management
Measures for the List of Business Credit Joint Disciplinary Objects. This one focused on the coordination and
management of credit lists for businesses and sought to implement a portion of
the State Council’s 2016 guidance. It
centered jurisdiction of business social credit within the Commerce Ministry
and its provincial apparatus, and established rules for compiling lists (under
ministry oversight and rules). It specified the scope of data from which lusts
may be created, including data on business legal or rule compliance, judicial
decisions, and an open-ended category of “other laws, regulations, and
regulatory documents.” It specified identifying information to be included on
the lists and the reasons for inclusion. Also included was a specification of a
quite broad range of restrictions that could be imposed on those listed, and
the conditions for removal form the listing.
Also relevant was the 10 July 2019 distribution by the State
Administration of Markets of its “Measures for the Administration of Serious
Illegal and Untrustworthy Lists (Revised Draft for Comments). It specified the competent authorities for
the management of specific categories of lists (and its underlying scoring)
related to violation of the laws and regulations of market supervision and
management including drug supervision and intellectual property
management. It applies to enterprises,
individual industrial and commercial households, other organizations, and
natural persons holding specific positions within these institutions or who
participate in market operations. It invites the establishment of a threshold
based on “the subjective malice, illegal circumstances, and harmful
consequences of the subject” and vests responsibility for guiding and
organizing the lists on the State Administration of Market Supervision. It specifies a procedure for listing those
enterprises and individuals subject to scoring and placement on the list that
includes thirty-six circumstances on which guidance is provided. It also
establishes a process for removal of subjects from the list. Lastly, it establishes
ten categories of restrictions that may be imposed on those who are included in
the lists. Provision is also made for coordination with other government and
private organizations. Governmental organs are encouraged to develop strategies
of joint punishment; relevant industry associations, professional service
organizations, platform-type enterprises, and the like are encouraged to
implement social co-governance. Finally, because ultimately the object of
scoring is to rectify behavior, substantial attention is paid to the mechanics
of rehabilitation.
Perhaps most important was issued 16 July 2019 by the State
Council General Office on Accelerating the Construction of the Social Credit
System—Guiding Opinions on Building a New Credit- Based Regulatory Mechanism.
It speaks to further innovation in the organization of linage of credit
supervision and the expansion of the application of credit reports. It also
directs the strengthening of the chain of credit supervision. These include
enhancing data warehousing as well as better targeting data harvesting, as well
as disclosure system of lists. The
latter is meant to provide enhanced publicity of inclusion on lists for the
imposition of societal repercussions as well as the official restrictions that
may be imposed by law or rule based administrative discretion. Provision for
enhancing self-reporting is encouraged, with the suggestion that voluntary
reporting can itself improve credit scoring. This parallels developments in the
West where, for example, in the United States Justice Department exercises of
prosecutorial discretion may be guided by the extent of the willingness of
subjects to cooperate by complying with DoJ rules for establishing compliance
and reporting systems. It calls for a greater development of a national credit
information sharing platform that is standardized compatible. Related to this
is the encouragement of cooperation among (and thus the approval of the
operation) of cross regional, cross industry and cross disciplinary mechanism. It
also called for a more efficient application of restrictions designed to induce
approved behaviors. The Guiding Opinion also pointed to the need to improve
mechanisms for identifying potential subjects of listing and connecting that to
violation of laws and rules. The State
Council divides this into two tracks—a market track for business and a personal
track for individuals in their social behaviors. The discipline of government personnel is
noticeable by its absence. But measures
for “credit repair” are emphasized.
Importantly, the Guidance speaks to the important role of the
“’Internet +’ and big data on credit supervision . . . [to] effectively
integrate public credit information, market credit information, complaints and
reports, and Internet and third-party related information, and make full use of
next-generation information technologies such as big data and artificial
intelligence to achieve comparable credit supervision data.” Protection of data
integrity is also emphasized. The State Council also emphasized the
construction of supervisory mechanisms and organizational leadership of social
credit mechanisms and workable credit scoring systems. Important among these
system creating measures are control of the narrative of social credit: “so
that operators can fully understand and actively cooperate with new
credit-based regulatory measures. Strengthen guidance and training for
grassroots and frontline supervisors. Organized extensive coverage of news
media, actively promoted credit supervision measures and their effectiveness,
and created a good social atmosphere.”
Coordinating these elements has been a challenge for the
administrative agencies charged with the implementation of social credit
systems by the 2020 deadline. Those difficulties expose the ambitions as well
as the challenges of building a coordinated social credit system on a national
scale. At the same time, it suggests the
relative ease of building less ambitious fractured small social credit systems
within a smaller community of related stakeholders.
3.
At the heart of super-scoring (social credit) systems—The analytics of lists:
pic credit here
The systemic construction of a national, coordinated CSC, then, represents an effort to substitute for law-based systems of behavior management, a system of restrictions and privileges based on a set of behavior models and goals, which is operated through a system of monitoring which is based on conformity to behavior objectives. This is data driven governance articulated through analytics, the consequences of which are established through restriction-reward algorithms. At the center of this system, then, are lists. Lists that follow rating and scoring behaviors (analytics) and provide the basis for the application of judgment (restrictions and privileges). Constructing a list, like the construction of the Social Credit system built around them, then, is the summary expression of the operation of the social credit system itself. To construct these lists requires a tight coordination of at least ten elements. Each of which is briefly considered below.
A. Entity
(Subjects). Here the problem is one of authority and jurisdiction. The State Council has moved to organize the
list of list-producing entities, but it has done little to organize the
jurisdiction of each. Expect much in the
way of overlap, and probably the existence of list “gaps.” Moreover, at least
with respect to private list creators there is the possibility of conflict of
interest or capture. Capture comes where
the list producing entity is also a subject of social credit managed by another
entity. Conflict comes when the list
construction affects the social credit of the listing entity. The issue of
entity touches two significant structuring challenges. The first is to align
data driven analytics targeting national behavior objectives with the division
of jurisdiction which divides authority over persons and activities among a
very large number of governmental organs at the national and provincial levels. To some extent the State Council guidance of
2019 attempts to respond to that challenge.
But it does so without disturbing the administrative structure of the
state apparatus. The suggests the second
challenge—the contradiction between “new era” data driven governance and the
retention of traditional structures of state authority that fundamentally
misaligns the character of regulation with its administrative structure.
B. Class of persons
or institutions that might be included in the (color) list. Every list
includes a universe of “subjects” which may be included on the list. This misalignment inevitably creates
challenges of overlap (multiple administrative units with authority over an
entity or an aspect of entity operations), coherence (multiple state organs
applying different standards on the same entity with worst case irreconcilable
measures, see below), and governance
gaps (entities are not included in the regulatory universe). That suggests two significant
consequences. The first is that every
list necessarily excludes certain actors who exist outside a specific list
system. It also produces a system in
which different subjects will be faced with behavior standards from different
sets of lists. This poses both
coordination problem as well as a transposability problem. As importantly, it
also suggests the likelihood of traps for the unwary (application of lists to
unaware subjects) and the probability of inconsistent application of list
analytics to the same individual data sets by different list creating entities.
While behavior X may get one on List Y, it may be insufficient to get onto List
Z. More interesting is the circumstance
of contradictory analytics—where behavior P will get an individual on List B
but keep that individual off List C.
C. The objective of list production (e.g., promote on of
the 12 Core Socialist Values). The the moment this issue remains largely
undefined. At its best it is meant to
promote the great ideological objectives developed under the guidance of the
CCP and reduced to obtainable specific norms.
At its worst—where it is limited only to the process of law or lawful
decision making, it becomes rudderless. It is clear that legal compliance and
moral adherence to the 12 Core Socialist Values are contemplated. More generally important is the attainment of
the CCP Basic Line as expressed in specific policies and interpreted in
accordance to official approaches to the currently central ideas now organized
as New Era Thought. But the translation of these great principles into
operational objectives—that is into commands that can be coded by behaviors,
becomes a difficult project. At the
moment we have coding but less inclination to match coding to principle.
D. The sector of class of conduct around which data tied
to the objective is limited. The State Council has already suggested an
ecology of data harvesting and list making.
There are divisions between individual and commercial sectors, between
different industrial and commercial sectors, between regions and the like. The
result creates potential incompatibilities among lists and their underlying
analytics. It also produces another area of substantial challenges to
coordination and unified management. This also touches on a related issue—who
is harvesting data. The State Council
has already spoken to the issue by focusing on greater efforts at self-reporting. But self-reporting can produce data bias and
require multiple levels of (self) monitoring.
E. Data—bits of information that connects behaviors or activities. This points to the general problem of the
identification of data that is useful. There are a number of issues; for
purposes of this essay two are worth identifying. The first is the connection between data
choices and rulemaking. That is choosing
data identifies conduct with significance.
Unchosen data suggests the reverse.
People will conform their behaviors to comply with this hierarchy of
importance which may have a perverse effect the consequences of which may not
be apparent until it occurs. That has
the effect of rulemaking. The second is
the conflation of data and ideological perception. Individuals are only capable of recognizing
data that aligns with their cultural conceptions of meaning. One “sees” race because culture has infused
certain characteristics with meaning. That set of cultural constrains will
inevitable corrupt the identification and organization of data. More fundamentally, data is transformed in CSC
systems from an object (a thing one harvests for normative ends) to the way
in which the normative ends themselves are defined. The choice of data
serves as ther definition of the conduct one seeks to regulate and the
behaviors one seeks to manage in a particular direction or with a particular
outcome in mind. But that had been the
traditional role of law and administrative regulation. Those are not necessary where they may be
displaced by decisions about the character of the data to be identified and
collected—and, as discussed in the next subsection, on the analytics applied to
that data.
F. Analytics—the
development of the process of producing meaning from data related to the
objectives. This is fairly self-explanatory. Unless the process for choosing data is
(deliberately or unconsciously) utilized as a form of hidden analytics, the
data that is generated through monitoring carries no inherent meaning. Analytics is the process by which data is
organized and is given meaning. But the coordination of multiple list systems
grounded on multiple (and secret) analytics create challenges. This is especially the case where analytics
permits discretionary choices that may vary among list making entity. The issue of significance looms large. What data or aggregation of data is
significant? How does one measure significance or justify it against objectives
or principles? To what extent is uniformity or predictability sacrificed in the
development of a vocabulary of signification that may vary from list to list?
These are questions that appear only lightly posed and largely unanswered. The
great challenge here (and in the West as well) is secrecy. Societies are moving
toward data transparency—there is no choice especially if data transparency
serves the same function as the publication of law and administrative
regulation (to give notice of expected conduct). But the analytics (usually
termed ambiguously algorithms, a term that includes both the analytics and the
application of consequences depending on the results of analysis) is viewed as
property. Privacy, in this model, is
then transformed from the protection of the subjects generating data (with
respect to the use of that data) to the protection of the analytics (and
algorithms) which then transform data harvested into consequences applied to
the data producing subject. For Chinese CSC, a more significant challenge
arises—the ability of the state or other organs to monitor the system itself;
that is to ensure that it is operating properly and aligned with the normative
objectives for which it was created—becomes far more difficult. Chinese CSC has
yet to deal decisively (or at least publicly) with this issue.
G. Judgment—line drawing; what combination of data in what
manner triggers decision to include or exclude; can be as simple or complex as
the analytics necessary to utilize the data. Analytics gives data
meaning. It provides significance to
data. But it does not produce judgement
or consequence. That is the function of
algorithm or administrative discretion applied to interpret not the meaning
derived from the analytics but rather its consequences.
H. Broadcasting.
Lists lose their power when they remain secret.
But lists that are well distributed also likely reduce the power of the
list creating entity to control its effects.
Sometimes that is desired. When
low personal social credit scores are widely broadcast, it is likely that a
large universe of individuals and entities that come in contact with that
individual will make decisions about their relationships based in part on those
scores. That augments both restrictions (punishments)
and privileges (rewards). But it also reduces the power of the entity to
control those effects (unless those effects are in turn control by the
mechanics of social credit). This becomes more problematic where an agency in
charge of a particular industrial sector publishes lists of businesses over
which it has oversight which empowers other agencies to impose restrictions
without consulting and perhaps in an effort to expand authority at the expense
of the listing entity.
I. Coordination.
A solution to the challenges of broadcasting lies in coordination. Coordination is also a central objective of a
national comprehensive social credit system.
But complexity makes effective coordination difficult. And the realities
of contests for power, influence, money etc. among administrative agencies,
factions, officials and the like will make effective coordination a more
difficult objective to meet. Though here there is a place for law, most likely
it will be filled with contract.
Coordination in China tends to be a function of coordinated Memoranda of
Understanding, rather than of regulatory coordination. This is not unique to CSC; China’s Belt and
Road Initiative is also built on contract.
Nor is it unknown in the West; the administrative state is quite
comfortable with interagency MOUs as a means of recasting webs of regulation
into something that resembles coherence.
Here on encounters a consequence of SCS on administrative practice in
the sense of its incentive to make traditional legal structures more remote. And remoteness here also moves the apparatus
of constitutional norms meant to protect the polity against governmental
excesses or arbitrary conduct more to the margin: in place of rechtsstaat there
is administrative discretion constrained by contract.
J. Consequences—beyond the list. Creating a social
credit system, like the creation of a law-regulatory system in its contemporary
form over the last several centuries, has proven to produce a broad range of
unexpected consequences. It is not clear
that large bureaucracies may be nimble enough to respond effectively when these
consequences emerge. One of the more
interesting emerging elements being developed is predictive blacklisting
(forecasting trustworthy or untrustworthy conduct). Related to this is the
“quality control” issue, and a mechanics for perhaps using super scoring to
score the stakeholders in a social credit system. Chinese authorities have begun to recognize
this problem. In 2019, the National Development and Reform Commission and the
People's Bank of China were tasked with the establishment of a tracking and
evaluation mechanism to assess the construction of social credit systems among
a set of specially designated demonstration cities and to adjust the model as
necessary. Details, of course, are not available.
The entire enterprise of listing (black, white or red) cannot
sit well with academics, government officials, political people, and officials
committed to the principles of liberal democracy and markets as currently
organized around its early 21st century orthodoxies. Some have
argued that in the U.S. context some forms of big data-based triggering or
screening initiatives constitute a liberty-depriving constitutional harm within
the American constitutional order. Many more worry about the privacy
implications and data protection. European might view the entire enterprise
through a several year’s worth of policies at the national and national
level—from Artificial Intelligence and ethical principles to the EU’s General
Data Protection Regulations (GDPR).
Still others worry about bias, implicit or explicit in data driven governance—from
data gathering to analytics to the algorithms, with much made of the ability of
programs to learn biases that reflect inherent in the coding through which
programs are constructed, operated and their products analyzed and interpreted.
Statutory structures in the West, like those in China, have begun to weave in
pace quite distinctive ideological overlays for the management and discipline
(as well as the application) of their distinctive variants of CSC. In the U.S.
a marker of that difference may be found in the courts where, for example, in
August 2019, the California appellate courts permitted the certification of a
class action against Facebook by plaintiffs who allege that Facebook’s facial recognition technology
violates Illinois Biometric Information Privacy Act, with implications for
common law privacy standards and US constitutional protections (Patel v Facebook,
Inc., US 9th Cir. Ct Appeals, 8 Aug. 2019).
But that is the point.
The centrality of lists within CSC nicely spotlights the way that
political ideology drives both the form and function of lists within these
complex systems of restrictions and privilege which are meant to substitute
micro-managerialism for the command enforcement model of law-regulatory
systems. But it does more. It also embeds
the ideology of Chinese Marxist Leninism, including its core legitimating
principles into the regulatory model. These include the prominence of the
leading role of the vanguard party, the centrality of the collective in social
and political life, the prominence of advancing socialist market economy
principles as a central objective of the state, and the imperative of embedding
the twelve Core Socialist Principles into every aspect of life. There is little
space left for the narratives of government function and the rights based
normative principles on which liberal democratic systems are organized within
this construct. That makes Chinese Social Credit systems difficult to transpose
as a normative model to the West—but also easily transposable as a mechanics of
management if undertaken by non-governmental entities, or reframed by the state
to conform to Western normative standards. This, however, will require
conscious effort.
At its root, Chinese CSC and the more privatized and
uncoordinated Western approaches at subject to the same fundamental challenge—to
ensure the integrity of data-driven governance systems as measured against its
conformity to the core values and principles, that is to the “higher law”
values of economic-social-political model.
That means, in some respects, that Western and Chinese CSC are
incomparable because their baseline integrity principles are founded on quite
different ideologies each in its own sphere viewed as legitimate and
authoritative. But it also means that while both systems may share and align
the technical or methodological structures of their systems, the objects for
which they are deployed are unlikely to converge. Therein lies a danger for both systems—to the
extent that methodology is heavily embedded within ideological presumptions,
even the borrowing of technique may pose challenges for the integrity of the system
into which it is imported.
4. The
challenge for political and general education in the context of these digital
transformations.
The thrust of this essay might trouble academics, government
officials, political people, and officials committed to the principles of
liberal democracy and markets as currently organized around its early 21st
century orthodoxies. Yet that troubling is necessary. Chinese Social Credit System construction is
deeply embedded within the Leninist political model of China advancing the
normative principles of Marxism with Chinese characteristics, a model with a
structure of legitimacy and political objectives substantially incompatible
with those of liberal democracy and its normative
constitutional systems. It is to a fundamental understanding of those
differences that any educational reaction ought to be focused. The opposite has
generally been the case. Chinese CSC is usually studied through the lens of
both Western political ambitions for China (e.g., that its system is flawed and
must be nudged (an ironic use of the term here) toward transition toward some
sort of liberal democratic model), and Western ideological premises. Those
premises provide an ecology of verities against which the Chinese effort is
understood as flawed and dangerous (to Western principles) with no effort made
to connect Chinese CSC efforts to Chinese authenticating ideology. That is not
to suggest that either ideological basis for system building is “right” or
“erroneous,” only that systems become comprehensible only within the ecology of
its own ideological framework.
A good example of the difference may be gleaned by a summary
review of the OECD Principles on Artificial Intelligence were adopted on 22 May
2019. It consists of five normative principles (what the OECD terms
"values based") grounded in the sustainability enhancing notion of
responsible stewardship that has gotten much traction in the business context
among influence leaders in recent years. These principles inform AI in the West
but are easily transposable to the Western perspective on the data driven
governance framework of Chinese CSC.
They include five principles. The first is that AI should benefit people
and the planet by driving inclusive growth, sustainable development and
well-being. The second is that AI systems should be designed in a way that
respects the rule of law, human rights, democratic values and diversity, and
they should include appropriate safeguards – for example, enabling human
intervention where necessary – to ensure a fair and just society. The third is
that there should be transparency and responsible disclosure around AI systems
to ensure that people understand AI-based outcomes and can challenge them. The
fourth is that AI systems must function in a robust, secure and safe way
throughout their life cycles and potential risks should be continually assessed
and managed. Fifth, that organizations and individuals developing, deploying or
operating AI systems should be held accountable for their proper functioning in
line with the above principles.
These five principles are then directed to the state, as is
the habit of the OECD regulatory form.
That direction is summarized in five recommended actions that states can
take. The first is that the state should
facilitate public and private investment in research and development to spur
innovation in trustworthy AI. The second is that the state should foster
accessible AI ecosystems with digital infrastructure and technologies and
mechanisms to share data and knowledge. The third is that the state should
ensure a policy environment that will open the way to deployment of trustworthy
AI systems. The fourth is that the state should empower people with the skills
for AI and support workers for a fair transition. And the last is that the
state should co-operate across borders and sectors to progress on responsible
stewardship of trustworthy AI.
While there are points of convergence—for example the focus
on trustworthiness, even those notions are understood in substantially
different ways in Western and Chinese data driven systems. Consider the
differences even within the realm of AI itself.
In its July 2017 New Generation AI Development Plan . (新一代人工智能发展规划的通知, the
Chinese State Council noted “Artificial intelligence brings new opportunities
for social construction. China is now in the final phase of building a well-off
society in an all-round way. Challenges such as population aging and resource
and environmental constraints are still severe. Artificial intelligence is
widely used in education, medical care, pensions, environmental protection,
urban operations, and judicial services, which will greatly increase public
awareness.” (新一代人工智能发展规划的通知
machine translation). The primary challenge
for education, then, is the ability to detach analysis of foreign systems from
the biases inherent in our own. But this should not be taken as a call to
abandon our values; quite the contrary.
Such an exercise makes it possible to be more rigorous in the
understanding and advancement of our values.
But it also makes it possible to understand areas of convergence, areas
of compatibility, and areas of conflict in the construction and operation of
these systems. This is important
especially when Western systems inevitably bump up against Chinese CSC. This is inevitable as global economies become
more closely networked. And it becomes
pressing as Chinese projects its Social Credit System through its Belt and Road
Initiative.
This produces the second suggestion. To think of Social
Credit as a problem of technology is to miss the most important point of the
exercise. System building is not to be
left to engineers. Its normative element
is essential. Thus, CSC and more
generally data driven governance becomes, at heart, a problem for law, for
ethics, for values and for the principles to be advanced in the operation and
delegation of authority to states and governmentalized non state actors. Perhaps
better stated, they become an issue over the language of law and its
mechanisms; where that language and those mechanics migrate to analytics it
raises the further issue of the transposability of rule of law and
constitutional principles to that transformed regulatory enterprise. What impedes this approach, of course, is the
stubborn classical taxonomy of knowledge that continues to plague the
scientific approach to knowledge. This
perhaps may be the greatest challenge of social credit for education—in order
to effectively study and advance a science of social credit, it may be
necessary to rethink the current taxonomy of knowledge so deeply embedded in
the West.
At its core, then, one might consider the following as the
core challenges for education in the wake of these digital transformations. Two
core implications are offered. There are no doubt others.
First, as painful as it may seem, academic culture must
change. To insist on partitioning the
study of data driven governance along the classical taxonomies of academic
scientific organization is to invert the possibilities of knowledge production
and reduce its value. The object of the study of these transformative changes
in collective human behaviors is not to advance the greater glory of classical
academic fields, whose turfs are myopically protected. Rather it is to advance
knowledge in ways that reflect facts. The ideology of education itself, then
may get in the way. For states and other actors, that may pose an issue—if
academic institutions are no longer capable of efficiently advancing knowledge,
then perhaps other means must be developed for those ends. This is particularly true with respect to CSC
which requires a merger social science, law, philosophy and logic, mathematics,
computing, and modeling.
Second, ideology must be exposed as a central element of
scientific study. The object is twofold. The first is to ensure that scientific study
controls for ideology to the extent necessary to advance knowledge in specific
context. The second is to ensure an
alignment of judgements about consequences—interpretation—with reference
explicitly to such ideologies. While “facts” may have no ideology, the choice
of facts, and the interpretation and application of facts is heavily embedded
in ideology. This does not suggest
cultural, scientific or political values relativism. It suggests rather that a
central element of scientific study must be to identify “bias” (values which
shade or nudge interpretation) either to correct for that bias (when dealing
with systems with different bias sets) or to better align the objectives of
that study to the support of that bias (for example in the West by interpreting
scientific study against a baseline of liberal democratic values). The process
must, however, be explicit.
5.
Conclusion.
My object in this essay was to encourage fresh thinking among
Westerners by detaching them from their world view conceits as the lens through
which analysis is undertaken, and to focus others on the challenges for
creating a CSC system that itself mirrors the integrity advancing principles
which is its principal objective. China serves not only as a harbinger of great
transformations in technology, but in a technological revolution that is
changing the relationship of law (and the state) to the advancement of its core
principles. It asks the questions: if
legal codex (the algorithm that is the self-referencing codes of modern states)
is transformed into a “Super” Scoring Codex (an accumulation and combination of
data from a variety of sources that is then actualized through analytics with
nudging consequences) have political communities replaced one set of super
algorithms (law) for another (scoring)?
And if so, what consequence?
The essay deliberately turns the conventional way of thinking
about data-driven governance upside down from the Western perspective but right
side up from the economic political model of the People’s Republic of China as
articulated and guided by its vanguard Party in accordance with its own
normative documents. It starts from a set of premises, first that governance is
driven by data; second that data has displaced the law; third, that the law has
been transformed in function from an ends-in-itself to a means-to-an end. Only from those foundational premises is it
possible to understand how data selection is itself a product of definite
normative premises or structured to lead to identified objectives. From those
premises, however well developed, it is then possible to understand the
inherent and unavoidable bias of these systems however much its minders seek to
present them as objective or "scientific".
This is no indictment of China’s Social Credit system. Instead it suggests that the Chinese have
been far more conscious of these fundamental premises and have sought to use CSC
instrumentally to augment these biases which for them encapsulate the great
political objectives for which they have a responsibility to advance. On the
other hand, it serves as a reminder to Westerners who still believe that
objective systems can be created that the structural foundations of such
systems are meant to advance rather than reduce bias—and that a conscious
articulation of those biases (seen as the positive principles and objectives to
be advanced) is essential to the construction of any CSC whether in China or it
the West.
These points are
factually accurate yet likely will be received differently. Those who build systems, and therefore reason
in terms of technology will agree with it. Those who build models, and reason
in terms of the eternal truths of Mathematics will be disappointed and likely
to be uncomfortable with this perspective. The discomfort emerges form an
unwillingness to entertain the possibility that "objectivity" and
"truth" in a sense have moved from the ideological premises of
governance systems to the very act of modelling something. No longer produced
by ideology, "objectivity" and "truth" are born when the
axioms of a mathematical model are defined. The consequences for education are
clear—as are the challenges for Western constitutional orders. For the Chinese economic-political model, on
the other hand, the development of a comprehensive national social credit
system fits in nicely with evolving principles of Leninism and with the
fundamental principles around which the Chinese political model is organized.
Fr China, Social Credit may well represent a more authentic expression of law
closely tied to (and constrained by) the great political principles the
administrative apparatus has a duty to apply.
For all that, the Chinese Social Credit system still has a
long way to go to achieve its ambitions—the operation of a comprehensive
mechanics for the management of all social actors through a system of
restrictions and privileges based on compliance with legal standards. Those
standards will reach substantially all aspects of organized life deemed
important to the state. CSC are constructed from the inter-relationship of
relevant data applied to analytics that incorporate the standards to be
advanced or the principles to be applied which produce a scoring (as a function
of compliance). Placement along a
hierarchy of scoring then permits the construction of lists from which
restrictions may be imposed or privileges granted. The list, then, reflects the application of
analytics to data that then expresses a judgment in the form of a placement (ranking)
along a continuum of compliance expectations. But the development of lists is
still in a formative stage, and their integrity remains a subject for
study. The coordination of these lists
and their integration into the complex compliance system that is Chinese legal
structures is still far off. Yet it is in the management of layers of lists
that super scoring is possible—though not yet attainable. The resulting
governance gaps remain substantial, while the concerns that might have bedeviled
such a system in the West—privacy and data protection—are entirely absent. Data protection becomes protection of data
integrity under CSC; privacy concerns fail as against the state and in any case
the knowledge of list placement or scoring is an essential feature of the
utility of the lists. Should it ever
mature, it is likely to transform both the character of law and its
relationship to managing behavior. In
the process China will advance a means of governing that is substantially
different from conventional systems identified with the forms and ideologies of
Western liberal democracies.
* * *
References
Ahmed,
Shazeda. “The Messy Truth About Social Credit.” Logic Issue 7 (May 1, 2019).
Available file:///Users/lcb911/Desktop/LogicMapSocialCredit/The%20Messy%20Truth%20About%20Social%20Credit.html.
Backer,
Larry Catá. “China’s Social Credit
System: Data-Driven Governance for a ‘New Era.’” Current History ---(forthcoming, Sept. 2019).
----------.
“Next Generation Law: Data Driven Governance and Accountability Based
Regulatory Systems in the West, and Social Credit Regimes in China.” 28(1) USC Interdisciplinary Law Journal 28(1):123-172
(2018).
Bo,
Xiang. “China Boasts World’s Largest Social Credit System: Official.” Xinhuanet
(14 Hune 2019). Available http://www.xinhuanet.com/english/2019-06/14/c_138143745.htm.
Botsman,
Rachel. “Big Data Meets Big Brother as China Moves to Rate Its Citizens.” Wired
(UK) (25 Oct. 2017). Available http://www.guicolandia.net/files/expansao/Big%20data%20meets%20Big%20Brother%20as%20China%20moves%20to%20rate%20its%20citizens.pdf.
Cheng,
Evelyn. “China Wants to Track and Grade Each Citizens’ Actions: It is in the
Testing Phase.” CNBC (25 July 2019). Available
https://www.cnbc.com/2019/07/26/china-social-credit-system-still-in-testing-phase-amid-trials.html.
China
Law Translate. “Market Regulation Blacklist Overview.” Policy Papers 1 Aug.
2019. Available https://www.chinalawtranslate.com/en/marketplace-regulation-blacklist-overview/.
----------.
Core Structural Documents of the Social Credit System. Available https://www.chinalawtranslate.com/en/social-credit-documents/.
Daum,
Jeremy. “Who did China ban from flying?” China Law Translate 2018/03/21
available https://www.chinalawtranslate.com/en/who-did-china-ban-from-flying/.
Daum,
Jeremy. “The Redlists are Coming! The blacklists are Coming!” China Law
Translate 2018/03/30 https://www.chinalawtranslate.com/the-redlists-are-coming-the-blacklists-are-coming/?lang=en.
China, Ministry of Commerce. Measures on the Management of the List of
Targets for Credit Joint Disciplinary Action in Commercial Affairs 【颁布时间】2019-7-17 【标题】商务部关于印发《商务信用联合惩戒对象.
Available http://www.law-lib.com/law/law_view.asp?id=651440.
China,
Office of the National Development and Reform Commission, Office of the
People’s Bank of China. “Notice on Printing and Distributing the List of Model
Cities (Districts) for the Construction of the Second Batch of Social Credit
Systems” Development and Reform Office Finance [2019] No. 849 关于印发 第二批社会信用体系建设示范城市（区） 名单的通知 发改办财金〔2019〕849号.
Available http://www.ndrc.gov.cn/zcfb/zcfbtz/201908/t20190813_944514.html.
----------. Outline
of the Social Credit System Construction Plan (2014-2020) The division of tasks
and the construction of the social credit system for three years; Notice of key
work tasks (2014-2016). 国家发展改革委 人民银行关于印发
《社会信用体系建设规划纲要（2014—2020年）任务分工》和《社会信用体系建设三年
重点工作任务（2014—2016）》的通知. Available http://www.ndrc.gov.cn/gzdt/201501/t20150105_659408.html.
----------. Division of Tasks for the Outline of the Construction
of Social Credit System (2014-2020) Annex 1 16 Dec. 2014: 《社会信用体系建设规划纲要（2014—2020年）》任务分工 附件1. Available http://www.ndrc.gov.cn/gzdt/201501/W020150105530734502125.pdf.
China,
State Council. New Generation AI Development Plan (新一代人工智能发展规划的通知) ( July 2017) available http://www.gov.cn/zhengce/content/2017-07/20/content_5211996.htm.
---------.
The General Office of the State Council “Guiding Opinions on Accelerating the
Construction of a Social Credit System and Building a New Credit-based
Supervision Mechanism.” 2019-07-16. Xinhuanet. 国务院办公厅印发《关于加快推进社会信用体系建设构建以信用为基础的新型监管机制的指导意见》
----------. The General Office of the State Council on
Accelerating the Construction of Social Credit System: Guiding Opinions on
Building a New Credit-Based Regulatory Mechanism; State Office issued [2019]
No. 35 国务院办公厅关于加快推进社会信用体系建设 构建以信用为基础的新型监管机制的指导意见 国办发〔2019〕35号.
Available http://www.gov.cn/zhengce/content/2019-07/16/content_5410120.htm.
----------.
State Council Guiding Opinions concerning Establishing and Perfecting
Incentives for Promise-keeping and Joint Punishment Systems for Trust-Breaking,
and Accelerating the Construction of Social Sincerity Posted on May 30, 2016
Updated on October 18, 2016 GF No. (2016)33
国务院关于建立完善 守信联合激励和失信联合惩戒制度 加快推进社会诚信建设的指导意见 国发〔2016〕33号. English Translation China Copyright and Media
Website. Available https://chinacopyrightandmedia.wordpress.com/2016/05/30/state-council-guiding-opinions-concerning-establishing-and-perfecting-incentives-for-promise-keeping-and-joint-punishment-systems-for-trust-breaking-and-accelerating-the-construction-of-social-sincer/.
----------.
State Council Notice concerning Issuance of the Planning Outline for the
Construction of a Social Credit System (2014-2020) GF No. (2014)21. 各省、自治区、直辖市人民政府，国务院各部委、各直属机构：现将《社会信用体系建设规划纲要（2014—2020年）》印发给你们，请认真贯彻执行。国务院2014年6月14日. English Translation
China Copyright and Media Website. Available https://chinacopyrightandmedia.wordpress.com/2014/06/14/planning-outline-for-the-construction-of-a-social-credit-system-2014-2020/.
European Commission.
Communication from the Commission to the European Parliament, the European
Council, the Council, the European Economic and Social Committee, and the
Committee of the Regions, Artificial Intelligence for Europe COM(2018) 237;
{SWD(2018) 137 final} (25 April 2018).
----------. Visa Information
System. Available https://ec.europa.eu/home-affairs/what-we-do/policies/borders-and-visas/visa-information-system_en.
European
Union. General Data Protection Regulation (EU) 2016/679 (GDPR).
van
Hotum. “Human Blacklisting: The Global Apartgheid of the EU’s External Border
Regime.” Environemnt and Planning D: Society and Space 28(6):957-76 (2010)
Hu,
Margaret. “Big Data Blacklisting.” Florida
Law Review 67(5):1735-1810 (2015).
Kobie,
Nicole. “The COmpoicated Truth About China’s Social Credit System.” Wired (7
June 2019). Available https://www.wired.co.uk/article/china-social-credit-system-explained.
Larson,
Christina. “Who Needs Democracy When You Have Data?.” MIT Technology Review(20
Aug. 2018). Available https://www.technologyreview.com/s/611815/who-needs-democracy-when-you-have-data/
MacSíthigh,
Daithí and Mathias Siems. “The Chinese Social Credit System: A Model for Other
Countries?. EUI Working Papers: Law 2019/1 (Department of Law). Available https://cadmus.eui.eu/bitstream/handle/1814/60424/LAW_2019_01.pdf?sequence=1&isAllowed=y.
Ohlberg,
Mareike, Shazeda Ahmed, Bertram Lang. Central Planning, Local Experiments:
The complex implementation of China’s Social Credit System. Merics China
Monitor (12 Dec. 2017). Available https://www.merics.org/sites/default/files/2017-12/171212_China_Monitor_43_Social_Credit_System_Implementation.pdf#page=12.
Organization
for Economic Cooperation and Development (OECD). Recommendation of the Council
on Artificial Intelligence (adopted 21 May 2019. Available https://legalinstruments.oecd.org/en/instruments/OECD-LEGAL-0449.
Osaba,
Osonde and William Wesler IV. An Intelligence in Our Image: The Risks of
Bias and Errors in Artificial Intelligence. Washington, D.C., Rand Corp. ,
2017. Available https://www.rand.org/content/dam/rand/pubs/research_reports/RR1700/RR1744/RAND_RR1744.pdf.
Pasquale,
Frank, The Black Box Society: The Secret Algorithms That Control Money and
Information. Boston: Harvard University Press, 2015.
Patel
v Facebook, Inc., No. 18-15982 slip op. (9th Cir) decided 8 Aug.
2019. Available https://www.documentcloud.org/documents/6248797-Patel-Facebook-Opinion.html.
Silberg,
Jake, and James Manyika. “Notes from the
AI frontier: Tackling bias in AI (and in humans).” McKinsey Global Institute
(June 2019). Available https://www.mckinsey.com/~/media/mckinsey/featured%20insights/artificial%20intelligence/tackling%20bias%20in%20artificial%20intelligence%20and%20in%20humans/mgi-tackling-bias-in-ai-june-2019.ashx
Soldo,
Fabio, Anh Le, and Athina Markopoulou. “Predictive Blacklisting as an Implicit
Recommendation System.” 2010 Proceedings IEEE INFOCOM (March 2010).
No comments:
Post a Comment