More Thoughts on Pasquale’s “Black Box Society”

When I finish reading a good book, my sense of satisfaction and fulfillment tends to be a rather private happening. I finish the book and even as I move on to the next (and there is always a next one), I still spend a fair amount of time processing the last. Since I’ve been blogging, I have used this space to share more thoughts about recent readings and that has felt somewhat liberating.

This summer I even went so far as to tweet out a picture of my proposed reading stack of 4 books:

Americanah by Chimamanda Ngozi Aidiche, The New Jim Crow by Michelle Alexander, The Black Box Society by Frank Pasquale and Data and Goliath by Bruce Schneier. Three quarters of the way done, I find my mind twisting and turning to accommodate so much new and rich input. Only Data and Goliath remains and as a back-to-back read with The Black Box Society, I feel adequately steeled for whatever fresh insights on  data vulnerability it may bring.

Here I want to focus , however, on Black Box because I feel like I will find no peace until I have shared as much as possible while the ideas are still so active in my mind.  In an earlier blog post I noted parallels between Pasquale’s illustrations and the Harry Potter series. (Seriously.) In a nutshell, Black Box Society examines the role of algorithmic decision-making in the areas of reputation (how we appear to external parties), search (what we look for online and how the selection and ranking of responses takes place and may impact us), and finance (the business of making (much) more money out of some money). I read it because someone I deeply respect recommended it. Before I started I was already a little apprehensive.

While reading The New Jim Crow presented challenges in the form of emotional labor, I was concerned that Black Box Society might be a bit beyond me. I had reservations about my capacity to grasp all the topics author Frank Pasquale was planning to cover: intricacies of the tech industry and finance. I even wrote a sticky note to myself for a potential blog post: How to read a difficult text:
*go slowly
*talk back to your negative self-talk (that keeps saying you won’t get it)
*be patient
*allow not knowing
*come back to it again & again – build stamina over time

The sticky note is stuck just inside the front cover. As it turns out, however, I didn’t need it, per se. I made it through the text and felt well guided throughout. This was the first text in a long while that I read with pencil in hand. I underlined a lot and put notes in the margins. I got involved with the text and found unanticipated connections (i.e., to Harry Potter). And, I dare say, I had fun, even reading about finance because it was explained both generously and with significant intentionality. Particularly when the discussion turned to CDSes (credit default swaps), CDOs (collateral debt obligations) and MBSes (mortgage-backed securities) which stood at the center of the financial meltdown of 2008, Pasquale provided the necessary scaffolding for me to make sense both of the crisis itself and the underlying assumptions that made it possible.

As I read I kept coming back to thoughts about privilege, wealth and status. Whether describing the titans of Wall Street or Silicon Valley, Pasquale captures a very wealthy, white male demographic who wield an immense degree of power and influence in both the private sector as well as in government. And their ability to carry out so many of their transactions behind various cloaks of secrecy and complexity or “black boxes”, reinforces and expands the wealth and privileges this group continues to amass. As an African-American woman, an educator – I found myself reading and thinking that there are few who  expect me to read and be up on this stuff.  I find myself in this narrative as the clueless user/consumer who stands largely at the mercy of these gigantic corporate structures whose services I engage to write this post, to make it findable on the web, to purchase more books, to tweet more links, to tout my professional skills, connect with hundreds of other educators, and so on.

Frank Pasquale is extremely candid in his assessment of the current state of affairs:

What we do know is that those at the top will succeed further, thanks in large part to the reputation incurred by past success; those at the bottom are likely to endure cascading disadvantages. Despite the promises of freedom and self-determinism held out by the lords of the information age, black box methods are just as likely to entrench a digital aristocracy as to empower experts. (p. 218)

Think about that for a moment. “Those at the top will succeed further…those at the bottom are likely to endure cascading disadvantages.”
This captures our society with alarming accuracy. And we can be certain that black boxes abound, especially in areas where power is increasingly consolidated. I cannot help but think of the aggressive pursuit of corporate interests in K-12 and Higher Education where transparency and openness can quickly become casualties in the fight to “reform” public education through various forms of privatization. I must also consider the prison-industrial complex which provides shareholder billions as the United States has the highest rate of incarceration of its citizens in the developed world. As Michelle Alexander asserts in The New Jim Crow, the war on drugs has enabled the creation of a new social undercaste whose political, economic and social disenfranchisement underscore  the essence  of “cascading disadvantages.”

While reading Black Box Society I was frequently reminded of an essay by sociologist,  Tressie McMillam Cottam whom I quoted in a previous blog post:

…give up on computers and get up on politics. Computers can be fine. Computers are politics. Personalized learning may be fine. Personalized learning is politics. Apps are fine. Apps are politics. Tech is politics. Tech is politics. Tech is politics.

There seems to be no escape from the political no matter where I turn.  Completing The Black Box Society becomes a political act,  as is reading The New Jim Crow and Data and Goliath. This is me “getting up on politics.” Getting informed, adding depth to my otherwise fuzzy notions of impending social and economic demise. It is impossible to read Pasquale and not become politicized.  He writes:

Internet and finance firms “set the standard” for our information economy. So far they have used their powers to know the world of commerce ever more intimately…Knowing more than a rival, or simply knowing it faster, is the key to vast fortunes.

But what if economic success were based less on information advantage and more on genuine productivity? Distracted from substantive judgments on what the economy should produce, we have been seduced by the mysterious valuations that Wall Street and Silicon Valley place on goods and services. But their algorithmic methods framed as neutral and objective, are predictably biased toward reinforcing certain hierarchies of wealth and attention. (p. 187-88)

The choices we have become very narrow very quickly unless we take steps at the very least to understand the evils to which we appear to be wedded. My attempts to comprehend the scope of  algorithmic dominance in our information economy seem to have been a wise and useful step. Following @FrankPasquale on Twitter has also broadened my perspective on related topics. This is not about learning the ins and outs of  a subject area. Rather this is about opening my eyes to what is unfolding right in front of me and has a daily impact on how we function as a society. We can’t see everything at once. But we can train our eyes on a specific field for a time in order to gain perspective, insight, and cause for further observation.

Frank Pasquale, The  Black Box Society: The Secret Algorithms That Control Money and Information. Cambridge, MA: Harvard University Press 2015.



Parallel Realities and Summer Reading

I believe that I am on vacation. I am with my family at a beautiful alpine lake enjoying the landscape, their company and a break from our usual routines. I am finding more time for concentrated reading and that feels very much like vacation.
I’m reading two books currently: Black Box Society by Frank Pasquale which was recommended to me by Audrey Watters and Harry Potter and the Philosopher’s Stone by J.K. Rowling which I am reading aloud to my 7 year old. It just dawned on me that there are striking parallels in these readings that kind of blow my mind when I think about them.

Frank Pasquale is a legal scholar who describes the largely mysterious yet highly engineered algorithmic foundations of our increasingly digitized existences. Beyond our patterns of online clicks, he examines the degree to which mathematical models of reasoning are used to determine everything from credit scores to having a job application get beyond an initial (computerized) screening to the privacy of our health data. As consumers and citizens we submit our data in the hopes of fair treatment and are confronted with decisions and outcomes delivered via ‘black box’ or opaque processes which are nearly impenetrable in their secrecy and complexity. Black Box Society takes on this omnipresent state of affairs and at the very least seeks to acquaint us with the monster that is not merely in the room, but is instead the room itself.

Meanwhile, my son clamors for more every time I read a section of Harry Potter. This is at least my second time around reading the first in the 7-book series (of which I have only read the first three) and it is quite simply a well told yarn. I have a new appreciation of the vivid characters and actually can catch more of the fabulous play on words that abounds. ( Just got Diagon Alley, for instance.) So he and I both are having a fun time with this project.

Even if you haven’t read any of the Harry Potter series, you’ll likely know that the story is predicated on parallel realities: the magical world inhabited by witches, wizards, goblins and other fantastic creatures and characters and the non-magical Muggle world in which the rest of us carry on our day-to-day. These worlds co-exist. While the magical folk have plenty of awareness of Muggle society, the Muggles remain decidedly unaware of any such alternative to their known ways of functioning. The witches, wizards and goblins know what Muggles cannot fathom, entertain or comprehend, unless a Muggle is specifically invited and initiated into the magical world. Hmmm…

The algorithmic ‘black boxes’ that Pasquale illustrates are by no means magical. On the contrary. They are fully human engineered and calibrated processes created with the assistance of great computing power and speed. They are complex, they hold and demand secrecy, and exert influence on an enormous scale but they are not magic. However, almost all of us function without specific knowledge or understanding of exactly how and to what degree these ‘black boxes’ help and hinder our daily affairs. Pasquale likens this state to a one-way mirror:

We do not live in a peaceable kingdom of private walled gardens; the contemporary world more closely resembles a one-way mirror. Important corporate actors have unprecedented knowledge of the minutiae of our daily lives, while we know little to nothing about how they use this knowledge to influence the important decisions that we – and they – make.

(p. 9)

It therefore seems as if we inhabit a sort of parallel reality – a space in which matters run their course and we accept and deal with the outcomes without having a full idea of how these matters took their precise shape or ran the way that they did. For the black boxes which have so firmly taken root in our lives and the control we continue to surrender to them both individually and institutionally, they may as well be magical. And we as oblivious as Muggles.

Thankfully, Harry Potter provides a narrative of ongoing discovery. Although he is a born wizard, he spends his unhappy formative years in a fiercely Muggle family and has no idea of his magical roots until he is eleven. He has to learn about all things magical from the ground up: his powers, his limits, his status, his role – and then make something of it all both for himself and ultimately for Hogwarts (his wizards’ academy) and beyond. Granted, I know how this part of the story ends and Harry Potter is a likable hero – he struggles, he succeeds, we can cheer and we know he’s not done. He gets to work through similar processes for 6 more books, after all.

In a similar vein, I am hopeful for us as a society in coming to terms with our monster that isthe room. I’m not finished with Black Box Society yet and Frank Pasquale shows that he is by no means resigned to the current status quo. He is cognizant of the hurdles we face in confronting our self-created beasts which we steadily feed with more detailed personal data and he has ideas about how to both tame the beasts and counter our patterns of surrender which appear to be spreading rather than abating. Resistance to a prevailing network of powers requires a commitment to discovery, learning, challenging and acting – in concert with many others, and not just individually. It means getting educated: striding from not seeing, to not knowing, to gradually comprehending, to sharing the findings, building awareness and also acting on that new knowledge. It’s not like in the movies. There will be no sweeping revolution that lends itself to being captured in a two-hour cinematic treatment. The path of deliberate resistance tends to be slower and more fragmented; measured in increments rather than dramatic leaps. This is how I see my connection to Pasquale’s work: as part of my discovery and learning process; not quick, but substantive and valuable.

At the very least I can offer two highly recommendable reads this summer, on vacation or off:

Frank Pasquale, Black Box Society: The Secret Algorithms That Control Money and Information. Cambridge, MA: Harvard University Press 2015.

J.K. Rowling, Harry Potter and The Philosopher’s Stone. London: Bloomsbury 1997.