The Word to Claude. A Special Project by Mikhail Epstein

Preface

These reviews were written by artificial intelligence, Claude Opus, one of the most advanced language models developed by Anthropic.My role was limited to developing the evaluation methodology, the “Interestingness Index,” and conducting an editorial dialogue with the AI in the course of the work.

Why do this? Literary criticism is one of the last bastions where human judgment seems irreplaceable. We readily entrust machines with digital calculations and translations, but aesthetic evaluation, subtlety of taste, intuition, all this still appears inalienably human. The experiment with Claude’s reviews is a way of testing that boundary.

The result, in my view, is unexpected. Claude not only reads attentively, from the first to the last letter, where it probably has an advantage over some jury members, but also perceives structures, connections, and echoes that escape the human eye. It does not pretend to feel emotions, but it describes the mechanisms that produce them.

The review and the evaluative component complement each other organically. All the more so because this is a competition in which opinions are, one way or another, counted, and evaluations are expressed in numbers. The evaluative component is interesting to read as a deeper exploration of the Pythagorean picture of the world, where harmony and algebra, music and mathematics are inseparable. Harmony is a higher algebra, too difficult for the human mind to master, descending from above, from the music of the spheres, from the Muse. That is why the author may feel, in Pushkin’s words, that “some kind of madness” has swept over him. But if one looks more closely, that madness is in fact a higher algebra, and AI helps us understand this.

In this section, Claude appears in two genres: following the critical review comes an evaluation according to the Interestingness Index. How to grasp this algebra of harmony is explained below. 

Interestingness: Ways of Evaluating

Among evaluative epithets, “interesting” is perhaps the most frequent and enduring. If in earlier eras truth and beauty, usefulness and edification were prized in a work, in our own time it is precisely the assessment of something as “interesting” that serves as an almost ritual introduction to all further evaluations, including critical ones.

The word “interestingness” goes back to the notion of financial interest and only in the eighteenth century came to be used in the sense of intellectual interest. Interest increases as the investment grows and the probability of success declines: the greatest profit comes from the riskiest investments. In much the same way, the riskiest statements yield the greatest meaningfulness, provided, of course, that they are borne out. The fewer the guarantees, the greater the interest.

In general, the interestingness of a given idea, theory, or narrative is inversely proportional to the probability of its thesis and and directly proportional to the cogency of its argument.The most interesting theory is the one that most consistently proves what is least probable.For example, the probability that a person will rise from the dead is exceptionally small, and yet the theory and narrative that prove the possibility of resurrection have for two thousand years remained at the center of the concerns of a significant part of humanity, shaping the meaningful plot of all history.

The less probable the thesis is at the outset and the more convincing it becomes in the end, the more gripping the path of the idea, the greater the intellectual interest invested in it. As the probability of the thesis rises and the cogency of the argument falls, the idea becomes less interesting. The least interesting theories are those that either (1) prove a self-evident thesis, (2) offer shaky proof for a non-obvious thesis, or (3), worst of all, provide groundless proof for obvious things. 

Interestingness is a ratio, a fraction whose numerator is the cogency of the proof and whose denominator is the probability of what is being proved. In English, this is provability in the numerator and probability in the denominator. Interestingness increases as the numerator grows and the denominator shrinks. The less probable the thesis and the more cogent the argument, the more interesting the idea.

This same twofold criterion of interestingness can be extended to a literary work as well. An interesting turn of events is one that is perceived, on the one hand, as inevitable, and on the other, as unpredictable. As in a scientific theory, the logic and consistency of artistic action are combined with its unexpectedness and paradoxicality. That is why Voltaire’s famous saying, “all genres are good except the boring one,” applies equally well to scientific genres and methods.

The well-known Sharpe Ratio measures return adjusted for risk: how much excess profit an investment yields per unit of risk. The Interestingness Index works in a similar way: it measures how much intellectual “profit” a text generates given the “risk” of its thesis. A text that asserts something banal, low risk, and proves it, high cogency, is like a government bond: reliable, but uninteresting. A text that asserts something radical, high risk, but fails to prove it, low cogency, is like a failed speculation: exciting, but useless. A masterpiece is a high-risk investment that has paid off: an incredible thesis that has become inevitable. Kafka, who bet on a man-insect, and won. Borges, who bet on time as a branching garden, and won. The formula of the Interestingness Index, Unexpectedness × Cogency, is essentially a measure of profit above and beyond the expected.

The very concept of the “interesting” comes from the Latin inter-esse, “to be between.” The interesting arises precisely in the interval: between order and chaos, between expectation and surprise, between the probable and the improbable. As soon as one side triumphs, interest disappears.

The category of the interesting is sometimes disputed on the grounds that it is subjective. “Some people are interested in one thing, others in another. The interesting is always interesting-for-someone.” But the same may be said of the beautiful and the good, and yet few would dispute the necessity of aesthetics and ethics as disciplines concerned with the beautiful and the good. The question is not what different people happen to find interesting, but what the interesting is in itself, what it means “to interest” and “to be interesting.” If one person is interested in hockey and another in football, one in philosophy and another in literature, one in Hegel and another in Nietzsche, all of them nevertheless find something interesting in different phenomena, and it is precisely this phenomenon of the interesting that interests us. A literary plot, a philosophical treatise, and a chess game all attain the highest degree of interest at the point where the most unexpected, improbable, unprecedented thing proves convincing, credible, realizable, demonstrable, and showable. By contrast, the formulaic and stilted on the one hand, and the arbitrary and chaotic on the other, as a rule fail to arouse interest. 

It is precisely the criteria described above that are used for the quantitative evaluation of interestingness in the works submitted for the Dar Prize. With the help of Claude, the AI model created by Anthropic, I have tried to develop a metric of interestingness and assess it in terms of the relation between the incredible and the credible, the unexpected and the demonstrable, to derive a numerical Interestingness Index. The program we created is presented in detail here: The Interestingness Index. A Formal Method for Evaluating the Quality of a Text first the English text, then the Russian). See also: Why Intelligent Texts Can Be Boring, and How to Measure the Interesting..

For different genres, from the aphorism to the essay, from the short story to the novel, different parameters are distinguished and measured numerically. A work scoring more than 30 points is a masterpiece. Then follow, within specific numerical ranges: “excellent,” “good,” “mediocre,” and “weak.” Each of the seven or eight evaluative parameters is expressed both numerically and verbally. It is an excellent aid both for the writer and for the critic. Storing in its memory tens and hundreds of thousands of literary works, AI is capable of assessing both the degree of “improbability,” the originality of each conception, and the degree of persuasiveness, the credibility of its realization. Below is a table of works in different genres that attain the highest level of interestingness.    

The full methods and results of this study of the interesting are presented in my bilingual blog Noocene/Нооцен on Medium, where the English text is followed by the equivalent Russian version. The subscription is free, just click the Follow button in the upper right corner.

Mikhail Epstein