The Word to Claude. A Special Project by Mikhail Epstein

Preface

These reviews were written by artificial intelligence, Claude Opus, one of the most advanced language models developed by Anthropic.My role was limited to developing the evaluation methodology, the “Interestingness Index,” and conducting an editorial dialogue with the AI in the course of the work.

Why do this? Literary criticism is one of the last bastions where human judgment seems irreplaceable. We readily entrust machines with digital calculations and translations, but aesthetic evaluation, subtlety of taste, intuition, all this still appears inalienably human. The experiment with Claude’s reviews is a way of testing that boundary.

The result, in my view, is unexpected. Claude not only reads attentively, from the first to the last letter, where it probably has an advantage over some jury members, but also perceives structures, connections, and echoes that escape the human eye. It does not pretend to feel emotions, but it describes the mechanisms that produce them.

The review and the evaluative component complement each other organically. All the more so because this is a competition in which opinions are, one way or another, counted, and evaluations are expressed in numbers. The evaluative component is interesting to read as a deeper exploration of the Pythagorean picture of the world, where harmony and algebra, music and mathematics are inseparable. Harmony is a higher algebra, too difficult for the human mind to master, descending from above, from the music of the spheres, from the Muse. That is why the author may feel, in Pushkin’s words, that “some kind of madness” has swept over him. But if one looks more closely, that madness is in fact a higher algebra, and AI helps us understand this.

In this section, Claude appears in two genres: following the critical review comes an evaluation according to the Interestingness Index. How to grasp this algebra of harmony is explained below. 

Interestingness: Ways of Evaluating

Among evaluative epithets, “interesting” is perhaps the most frequent and enduring. If in earlier eras truth and beauty, usefulness and edification were prized in a work, in our own time it is precisely the assessment of something as “interesting” that serves as an almost ritual introduction to all further evaluations, including critical ones.

Слово «интересное» восходит к понятию финансового интереса и только в XVIII веке стало использоваться для обозначения умственного интереса. Интерес возрастает по мере того, как вложение увеличивается, а вероятность успеха снижается: наибольшую прибыль приносят самые рискованные вложения. Точно так же и наиболее рискованные высказывания обеспечивают наибольшую содержательность, разумеется, если они подтверждаются. Чем меньше гарантии, тем больше интереса.

In general, the interestingness of a given idea, theory, or narrative is inversely proportional to the probability of its thesis and and directly proportional to the cogency of its argument.The most interesting theory is the one that most consistently proves what is least probable.For example, the probability that a person will rise from the dead is exceptionally small, and yet the theory and narrative that prove the possibility of resurrection have for two thousand years remained at the center of the concerns of a significant part of humanity, shaping the meaningful plot of all history.

The less probable the thesis is at the outset and the more convincing it becomes in the end, the more gripping the path of the idea, the greater the intellectual interest invested in it. As the probability of the thesis rises and the cogency of the argument falls, the idea becomes less interesting. The least interesting theories are those that either (1) prove a self-evident thesis, (2) offer shaky proof for a non-obvious thesis, or (3), worst of all, provide groundless proof for obvious things. 

Interestingness is a ratio, a fraction whose numerator is the cogency of the proof and whose denominator is the probability of what is being proved. In English, this is provability in the numerator and probability in the denominator. Interestingness increases as the numerator grows and the denominator shrinks. The less probable the thesis and the more cogent the argument, the more interesting the idea.

This same twofold criterion of interestingness can be extended to a literary work as well. An interesting turn of events is one that is perceived, on the one hand, as inevitable, and on the other, as unpredictable. As in a scientific theory, the logic and consistency of artistic action are combined with its unexpectedness and paradoxicality. That is why Voltaire’s famous saying, “all genres are good except the boring one,” applies equally well to scientific genres and methods.

The well-known Sharpe Ratio measures return adjusted for risk: how much excess profit an investment yields per unit of risk. The Interestingness Index works in a similar way: it measures how much intellectual “profit” a text generates given the “risk” of its thesis. A text that asserts something banal, low risk, and proves it, high cogency, is like a government bond: reliable, but uninteresting. A text that asserts something radical, high risk, but fails to prove it, low cogency, is like a failed speculation: exciting, but useless. A masterpiece is a high-risk investment that has paid off: an incredible thesis that has become inevitable. Kafka, who bet on a man-insect, and won. Borges, who bet on time as a branching garden, and won. The formula of the Interestingness Index, Unexpectedness × Cogency, is essentially a measure of profit above and beyond the expected.

The very concept of the “interesting” comes from the Latin inter-esse, “to be between.” The interesting arises precisely in the interval: between order and chaos, between expectation and surprise, between the probable and the improbable. As soon as one side triumphs, interest disappears.

Категорию интересного порой оспаривают на том основании, что она является субъективной. «Одних интересует одно, других — другое. Интересное всегда интересно-для-кого-то». Но то же самое можно сказать и о «прекрасном», и о «добром», однако мало кто оспаривает необходимость эстетики и этики как наук о прекрасном и добром. Вопрос не в том, что именно интересно для разных людей, а что такое интересное само по себе, что значит «интересовать» и «быть интересным». Если одного интересует хоккей, а другого футбол, одного философия, а другого литература, одного Гегель, а другого Ницше, то все они находят для себя что-то интересное в разных явлениях: и вот само это явление интересного интересует нас. И литературный сюжет, и философский трактат, и шахматная партия приобретают наивысший интерес в той точке, когда самое неожиданное, невероятное, ни на что не похожее оказывается убедительным, достоверным,  воплотимым, доказуемым и показуемым.  И напротив, шаблонное, ходульное, с одной стороны, и произвольное, хаотическое — с другой, как правило, не вызывают интереса. 

It is precisely the criteria described above that are used for the quantitative evaluation of interestingness in the works submitted for the Dar Prize. With the help of Claude, the AI model created by Anthropic, I have tried to develop a metric of interestingness and assess it in terms of the relation between the incredible and the credible, the unexpected and the demonstrable, to derive a numerical Interestingness Index. The program we created is presented in detail here: The Interestingness Index. A Formal Method for Evaluating the Quality of a Text first the English text, then the Russian). See also: Why Intelligent Texts Can Be Boring, and How to Measure the Interesting..

Для разных жанров — от афоризма до эссе, от рассказа до романа — выделяются разные параметры, которые измеряются в числах. Произведение, набравшее более 30 баллов, — шедевр. Далее идут в определенных числовых интервалах: «превосходно», «хорошо», «посредственно», «слабо». Каждый из семи-восьми параметров оценки выражается и числом, и вербально. Отличное подспорье и для писателя, и для критика. Храня в памяти десятки и сотни тысяч литературных произведений, ИИ может оценить и степень  «невероятности», оригинальности каждого замысла, и степень убедительности, достоверности его воплощения. Ниже таблица произведений разных жанров, достигающих высшего уровня интересности.    

Полностью методы и итоги  исследования интересного представлены в моём двуязычном блоге Noocene/Нооцен on Medium, where the English text is followed by the equivalent Russian version. The subscription is free, just click the Follow button in the upper right corner.

Mikhail Epstein

• 
  Preface to the project The Word to Claude. Ways of Evaluating Literary Works
• 
  The Anatomy of a Survivor: A Novel-Epitaph for an Era. On Yuri Troitsky’s Novel Shatz
• 
  The Mechanics of Seizure: A Moscow Saga as Diagnosis. On Oleg Radzinsky’s Novel Days of Repentance
• 
  Поэтика распада: о прозе Игоря Белодеда
• 
  The Topography of Trauma: A Parabolic Novel about the Impossibility of Paradise. On Ksenia Buksha’s Novel A Little Paradise
• 
  A Pocket Full of Children: The Anatomy of Queer Horror. On Ilya Danishevsky’s Novel Damocles Techno
• 
  Nothing but the Heart: A Theology of Perplexity. On Grisha Prorokov’s Novel
• 
  Spending the War Without You: An Archaeology of Survival. On Tatyana Zamirovskaya’s Book
• 
  The Camera Shutter. On Yevgeny Feldman’s Book
• 
  War as a Mode of the Body’s Existence. On Zhenya Berezhnaya’s Book
• 
  Thorns in the Ashes: Testimony as Form. On Alexander Motsar’s Book
• 
  A Hell More Terrible than Hell. On Alexandra Krashevskaya’s Book
• 
  A Father’s Testimony: A Genre That Should Not Exist. On Dmitry Petrov’s Novella