Judge on a Boat Prologue Falling, falling, falling, that was what was happening, of course falling was what a planetary drop pod was for but this was the bad falling, the kind which could kill you, and her brain - a primitive simian derivative that processed "falling" extremely well, falling being a work-related risk for swinging around on trees - was putting out distress codes, feeding nasty hormones into her bloodstream, which really necessitated triggering her deeper rationality training, but she didn't really remember to do that part. But she survived, and in the end, that was what a brain was for. Rationality was for winning. And perhaps in retrospect the flight-or-fight response had been the right thing to do, at least she survived... Although in retrospect, maybe she should have forked over the added price for the safer space elevator. Chapter 1 Emmanuel Verrens woke with a start when Judge David Adams' receptionist spoke his name, indicating that his meeting with the august Fifth-rank Lord Judge was to start. Emmanuel Verrens was a young Judge, of the interplanetary order of Judges. As yet unranked, his status newly minted with his recent examination. He had been summoned for a meeting, topic unspecified except for a vague "regarding your examination and recent promotion to unranked Judge", and his rationality training suggested that this summons, so recently after his examination and promotion, was not good. It was less good, too, that having arrived at the proper time, he had been asked to wait, and was still waiting almost an hour later. Still, there was insufficient evidence that it was something specifically to fear, and so Verrens had clamped down on his misgivings, invoking some of the standard rationality forms developed against fear. Thus Verrens was quite sane, even as his appointed time came and went, and he was still in the waiting room. He'd even managed to nap a little, out of sheer boredom, that was how little the fear got through his trained defenses against it. Verrens got up and nodded at the receptionist. "Yes, Miss," he said. "I am Judge Emmanuel Verrens." He stood up, waiting to be led into the office. "Excuse me," the receptionist said, "but the Judge would first like you to answer a few questions. I apologize if they are basic, but he does not wish to be pestered by Technological Renouncers." "Ah," said Verrens. "The questions are?" "First, please define 'epistemic rationality.'" "The art of updating your beliefs based on evidence," said Verrens. "It is the art of mapmaking, of creating a mental map for the territory of the real world, and ensuring that the map reflects the territory as closely as possible: to achieve 'truth,' in fewer words. It is the understanding that what is contained in the brain is not reality itself, that what we see and feel and experience is not reality directly, but is in fact a mental map, a map that is not the territory. It answers the question: 'how do I know that what I know is true?'" "Thank you. Now, please define 'instrumental rationality.'" "The art of winning," said Verrens. "It is the art of achieving your goals and keeping your values - values that are shared across standard Human brains. It involves the desire to become stronger, making a supreme effort, and the multitude of techniques to coerce a standard Human brain away from laziness, selfishness, and other weaknesses. It answers the question: 'how do I achieve what I claim to be my values?'" "Good. Please enter the door to your left." Verrens entered the door, vaguely concerned. Chapter 2 Nicole Angel awoke to a blissful, swaying feeling. The sun was just beginning to rise. Then she found that her hand was clutched tight to some rubber, or probably synthetic plastic, and she saw the vast beautiful ocean surrounding her. Memory came flooding back. She had been on the way to her standardized Judgment Acceptance examination, back on Old Earth, from her birthplace at Mars Lagrange 1. The trip was costly, involving an actual drop into and (of course) liftoff from a planetary gravity well, and Earth was still the heaviest gravity well inhabited by humans, and was still the cultural center of Humanity, which meant that the trip was quite expensive. But passing the standard test and becoming a Judge was supposed to be worth that, and more. She'd crammed - her rationality tutor or "Sensei" clucked at students who did so, but that attractor state (defer things later! You Ain't Gonna Need It!) still attracted human beings like flies to honey, or more likely vinegar, but as irrationalities went cramming was a just a slight inefficiency, so Senseis just clucked at it - and of course like many she even dipped a little into using rote memorization instead of actual understanding. Then she kissed her family goodbye, promising to come back a fully-fledged Judge Of No Rank. The actual trip through space was uneventful - her seat companion was a young female Engineer, sponsored by her home company for training at Earth, and somewhere nearby, Nicole had noticed, had been a somewhat attractive young male whose bearing suggested "Executor", with whom Nicole had kept slight eye contact a few seconds more than strictly uninterested strangerness warranted. Then came the failed drop, and now this, floating somehow on an inflatable raft with two strangers and a slightly-more-familiar acquaintance, her Engineer seat companion of the company Earth training trip. No, make that two strangers, her Engineer acquaintance, and that somewhat attractive maybe Executor with whom Nicole had maybe flirted, hanging off the other side of the raft, his chest and head inside, but his lower torso and legs dipped into the ocean. A total of five people. The Engineer tapped her shoulder, and then whispered, "Judge, we have a problem. This raft's capacity is four people." Chapter 3 "Good Afternoon, Judge Adams," he said. "Good Afternoon. Have a seat," answered Adams. It wasn't a personal office, as it turned out. Or if it was, then it was a rather strange one. Had Verrens been asked his opinion, it was actually a meeting room, with a long table, several chairs, a whiteboard, and a screen. Judge David Adams, of Fifth Rank, sat near one end, with several printed papers lying spread out in front of him. Adams was indicating a seat just across his, so Verrens sat into it. There was silence for a moment, and then - "Here," said Adams, handing over a few papers, "is a manuscript - ostensibly of a novel - regarding a Judge, or rather, an aspiring Judge trainee. As you yourself was, just recently, prior to your own examination here on Earth. Think of it as an informal final examination for yourself." Verrens took the papers. Was Adams pulling rank and trying to make him some kind of early proofreader for his own novel? It started, "Falling, falling, falling, that was what was happening, of course falling was what a planetary drop pod was for but this was the bad falling..." Verrens asked, "Judge, what is this for?" "It's being proposed as a form of training, and potentially testing, for student Judges. Informally, you are here since, until recently, you were a student Judge, and thus we would value your input about this manuscript," said Adams. "I see," said Verrens, ruffling some pages. "Out of curiousity, is this based on, say, a true story?" "You could say that," said Adams. "So true, in fact, that we can even trace the thoughts of the main character." "Ah," was all Verrens could say. He read for a moment. "Ah," said Adams, "I see you have had speed reader training, Judge. That is fortunate. Perhaps we can complete our analysis of this novel in one sitting." "Yes, Judge," said Verrens. "I studied it fully. Speed up saccades, eliminate subvocalization, lengthen fixations, and practice, practice, practice -" "That's good," answered Adams. "I myself am just starting speed reading training. And just in time too, since I must read many more of these novels, of various authorships and variations in version. And interview many different new Judges like yourself, in order to gain different views on the novel." Chapter 4 "I'm not a Judge!" Nicole whispered back urgently. "I'm just a candidate for -" "Well, you're the nearest thing to a Judge we have right now," the Engineer answered back just as urgently. "I asked. The other woman is a Medic, the man in the boat is an accountant, and the man hanging onto our boat for dear life is an Executor. You're the nearest to a Judge we have right now. So. I'm asking you, formally, for a Judgment, on a matter of life and death. How should we choose who to kick off this boat, Judge? There's five of us, the boat holds four, and -" "Can't this boat really hold five? Especially with one of those five being technically outside the boat anyway?" There were safety margins, weren't there? Surely an Engineer would know that - "Normally it can," said the Engineer. "That's why I pulled the Executor close. I couldn't pull him all the way in, I'm not physically strong enough, especially last night, and he's a heavy guy. But the boat seems to have a slow leak -" "All right, how about if we all do what the Executor is doing? Hang off the side of the boat?" The Engineer paused. "Plausible," she said. "But Judge, consider - currently with only one extra passenger hanging off the side of the boat, we're already experiencing a slow leak. We could all hang off the side of the boat, but what if that doesn't stop the leaking? That's highly likely, too. If we're not rescued soon, we may have to sacrifice one of us so that the raft doesn't sink completely. It's a metal lined rubber craft, so below a certain air pressure in the bladder it will sink. Now I'm asking you, formally, for a Judgment, on a matter of life and death. In the future, we may need to kick off one of our number. How should we choose who to kick off this boat, Judge? Let this be a precommitment to a method, specified by a Judge, that binds us all." But Nicole wasn't a Judge. It didn't even matter that she hadn't passed, or even taken, the Judge Acceptance Exam. She knew she had the knowledge and the training, and if she had taken it, the chances were good that she would pass and be promoted to an unranked Judge. But passing a test did not make you a Judge, not really. It was rather the community and culture of the Judges that defined your identity as a Judge. Oh, the test was thorough. Written exams, and even scanning of your brain's morality cores, at least those already identified by Neurology Scientists. Practice mock tests, even - the Sociology Scientists were paid handsomely to think up of new tests each year that would look innocuous to even the most high-strung Judge examinee, but which would reveal the inner workings of their morality cores almost as well as, or (it was reputed) even better than the Neurologist's best scans. And once you passed, the eyes of every other living Judge were upon you and every moral decision over which you passed Judgment... Cameras were ubiquitous all over Humanity's inhabited areas. By law, all cameras were to be marked and readily visible as such, and all were accessible to the public. Spy cameras, unmarked and hidden, or accessible only to certain groups or individuals, were highly illegal and always searched for by the Hackers; the Sousveillance Law that banned hidden cameras and required public access was the only existing law that crossed geopolitical boundaries, and the only one allowed to mention the death penalty as a possible punishment. Quis custodiet ipsos custodes? Why, every watcher was watched by fellow watchers, and every person capable of minimum reason - and this category was very broad indeed - was a watcher. And every Judge was judged by every other Judge. A Judge gained rank by a Formal Challenge of another Judge's Judgment. Such a Formal Challenge of Judgment was an assertion that the Judgment had been faulty. Records of the decision, as well as the particular details of the issue to be decided, were often available from the ubiquitous cameras all over inhabited space. And once such a case was brought to light, Judges all over Humanity's domain were invited to vote on whether to strip a rank from challenger, or from the challenged. The stripped rank would be awarded to the other. A Judge of No Rank would lose his or her vocation, if he or she lost a Formal Challenge of Judgment. Slightly more than half the Judges in existence at any one time would be of No Rank. The other remaining half were composed almost entirely of First Rank. A very few achieved higher ranks. Rank was a score, but by law Judges were paid based on age and typical living expenses in their registered living area. But a higher rank still meant that you had a safety margin, in case of an error in Judgment. And without that community of Judges watching over your shoulder, you couldn't be a Judge, not really. With the respect accorded to Judges in Humanity's culture, and the built-in selfishness circuits of your simian-derived brain, you could only be a tyrant. The existence of watching peers balanced out against both forces. There were no cameras on the boat. So, barring illegal unmarked cameras, no one could ever see the details of Nicole's Judgment. Chapter 5 Emmanuel Verrens paused. If the novel was about true events - how had they known about Nicole, without any cameras having seen it? "Judge Adams?" "Yes, Judge Verrens?" Adams asked. "Judge, if there were no cameras on the boat, and this novel is non-fictional -" "Well, I didn't really say the novel is non-fictional, or even say that it was based on true events. I did say, that you could say it was based on true events, but really, the truth of these events is a bit hazy, as you should recognize from the lack of cameras." "Ah," said Verrens, "I see, Judge." "Do you? Good," said Adams. He smiled. "In any case, suffice it to say that nothing illegal was involved in the writing of this novel. Rather, I wish you to focus on the moral dilemmas that the protagonist is facing. What exactly are they, at this point?" Verrens considered for a while, skimming through the earlier pages before he had inserted his finger. "Well, there's the obvious one asked of her - the method by which to select one of their number to kick off the boat. The fact that she is being asked to make a Judgment, even though she is not yet a Judge -" "That one is trivially dismissable, I would say, Judge?" said Adams, interrupting Verrens. "True," Verrens said. "She has the training, and the confidence in her skill to act as a full Judge, despite not having taken the test yet. As I was about to mention, anyway. Not really a moral dilemma, then, although I included it for completeness. She can indeed act as a full Judge - awareness of one's knowledge and level of skill is a requirement in our vocation. And the final dilemma -" Adams looked at him expectantly. "The lack of peer oversight," Verrens said, "set against the necessity of providing Judgment, in a condition where such Judgment is likely to be necessary, and likely to be a matter of life and death." Adams continued to look at him expectantly. "I'm sorry, Judge, those are all the dilemmas I can see right now," Verrens said. Adams nodded. "Judge, both you and Nicole Angel failed to see another moral dilemma, at this point." He paused. "She did not consider investigating for herself whether what the Engineer told her was true, versus her personal embarrasment, should it indeed turn out to be true. A common logical failing that afflicts even the best of us, and as I recall, three of my successful Formal Challenges." Verrens considered this. "But Judge, the exposition of the Engineer -" "- was not sufficient evidence of its truth in this case, even as an expert statement. The veracity of the matter can be checked explicitly and does not require an export's exposition. Of course, I must admit that it would be so, in a typical novel; a writer preserves detail, so as not to overburden the reader with possible red herrings. But in the world, the real world, a Judge in such a position must demand the investigation; must at least do so as background to his or her task." Verrens nodded. "And if this is, as you say, true in a way - then perhaps that is a true moral failing of Nicole Angel." "One she would do well to protect herself against, in her future career. Of course, the novel has yet to reveal whether that will happen, or even possible to happen, or -" Chapter 6 Nicole watched as the Engineer - now known as Julia Montega - called the others - Raphael Yu the accountant, Ekaterina "Katja" Ivanova the Medic, and Michael Thompson the Executor - to a meeting in order for her to pass Judgment. "The standard solution to this," Nicole said once Julia had explained their problem, "is to draw lots. A random choice, which distributes the penalty across all of us." "How about -" Michael started, then swallowed to wet his throat. "How about I just swim off? I'm the one who got added here... it's not fair to the rest of you..." "I'm the Judge," Nicole said. "That's my Judgment." "And you are not likely to survive long without the boat," Katja said. "Unless we all manage to get rescued in an hour or so. Even though the waters are relatively warm - we're probably in tropical waters - treading the water will exhaust you." She paused. "You do have a life vest, but another night of exposure may be too much. So, let me revise my earlier estimate. Unless we manage to get rescued before nightfall, you yourself must be replaced in the water." "Yes," Julia the Engineer said, "we can't really afford to lose any of us here. We're all useful. Oh. I'm sorry, I shouldn't have said that." She was looking at Raphael the accountant. "I'm a demisavant, I'm no good at social relationships..." She trailed off. Nicole was stunned into silence. Trust a demisavant to see that as obvious, and then blow it by making the faux pas of saying it as if it were obvious. A Medic was most obviously useful in their situation. An Engineer would likely be useful, to figure out how to make simple machinery, fix whatever complex machines they might have access to, to help them and the Medic in her work. A demisavant at that; more useful at simple non-social problems (which more likely to occur here, there were only five of them) than complex social exchanges. A Judge to consider policy decisions, an Executor to enforce them - what indeed could Raphael offer them? Nicole shuddered. It was precisely this that the community of Judges was for. No one could be left behind, not even in the name of "the greater good." No matter how "useless" someone might appear to be. Civilization would not allow it. And in practice, in a complex scientific civilization, no one was really useless... A person's vocation only indicated his or her main interest, decided at a time in their life when he or she was just barely mature. Side interests were sometimes much more powerfully attractive to a person, after a while. And those side interests could spell the difference between life and death. Civilization's commitment to helping everyone was based on that observation - that an eager amateur hero already on-site could do as well as, or even better, than a bored expert who arrived too late to do anything. Eleven billion individuals on Earth, and the billion or so on the Lagrange points of various other planets in the solar system, and the half billion on Mars, depended too much on complex machinery - physical machinery, as well as social machinery such as democracy, sousveillance, markets, disputation arenas, the Judgment System, peer review. Complex machines that some individuals found obsessively compelling to study in an amateur fashion. And without the community of Judges looking perenially over your shoulder, it was all too easy to forget that. All too tempting to follow older rules etched into the brain by long millenia of humans living in much more primitive societies. Chapter 7 "So," said Verrens, "she pronounces a formal Judgment." "Wait a moment," said Adams, "I'm not there yet." He read for a short while. "Yes, a standard solution to a standard moral problem. The aside about modern Human civilization is good, and explains well the rationale for our system." Verrens nodded, then, "I somewhat expected her first Judgment would be more, I don't know, maybe more flashy, not just some standard solution." He paused. "Although I admit that I feel relieved that this was her first Judgment. I hope my first Judgment would be as simple and straightforward as that." "I agree," said Adams. "Simplicity, as the Scientists continually say, is the core of scientific beauty. A common mistake that the Reporters commit; they like scientific controversies too much, thinking that controversy is of interest, and simple, straightforward, and classic solutions are boring." He smiled. "I'll give you a hint, if you're shooting for high rank: look for Judges who are too concerned with what the Reporters say; they often commit the mistake of thinking that exceptions to the standard solutions should occur more often than 'almost never'. Simplicity is the core of moral beauty, too. Of course, as simple as possible, but no simpler...." Adams closed his eyes for a moment. "As to amateurs - well, I am an amateur Sensei, and an amateur Scientist; they go well together. Rationality is guided by science, and science is driven by rationality. The world still moves; boats, like the one our heroes are in, still displace an amount of water whose weight equals their own weight; Newton's laws are still accurate, to the extent that General Relativity predicts them to be accurate. Non-controversial issues that I find of great interest and satisfaction, as an amateur." Nobody said anything for a moment. Adams opened his eyes and said, "As to the novel as written - Judge, have you considered if the fallacy of legalism applies to its expression of morality?" Verrens considered this. "I did not, Judge. However, I don't think the charge of legalism applies. The fallacy of legalism is the assumption that law defines morality, rather than the Central Dogma that morality defines law. Our civilization's standard solutions to problems are not law per se, not even unwritten laws, as classically cultural customs have been. The standard solutions are principles that many of our predecessors have evaluated, over and over, and in the absence of strong evidence to the contrary, can generally be safely considered as correct. Democracy may not work for a populace kept ignorant by its media, but beyond that known failure mode, and others I don't remember out of hand, democracy works." Adams smiled, and said, "Shall we continue?" Chapter 8 Julia found a medical kit stashed away in some recess of the lifeboat; aside from the standard first aid equipment, there was a somewhat large bottle of sunblock which Katja insisted everyone apply a fixed amount of, in specific locations of their body. She then poured a measured amount of shark repellent into the ocean; Nicole noted that it was really a combined shark-and-jellyfish repellent, which made sense. There were some energy bars, which Katja declared she would ration out each morning until rescue; everyone had one for breakfast. Julia rooted around the raft for a moment, mumbling something about trying to build a roof when asked, but eventually just sat, looking dissatisfied. Katja recommended that everyone rest, offering some low-grade tranquilizer pills from the medical kit; no one took up her offer, so after taking an expert look over each of them for serious injuries, and bandaging a scratch on Raphael's upper arm, Katja popped one of the pills herself and was soon dozing lightly, unconcerned and relaxed, on her face a small smile. After a while, Julia sat herself comfortably, her face near Michael's, engaged in a whispered conversation; Nicole felt a pang of jealousy, but shrugged it off. Julia had pulled Michael in, anyway; perhaps it was appropriate to give her first chance at him.... Raphael was looking out over the sea; he seemed sad, Nicole noted. Demisavants - in earlier ages known as "shy, but bright", or High IQ Attention Deficit Disorder, or Asperger's Savant Syndrome - were now trained, as early as they could be identified, to explicitly handle social interaction, and encourage it as one of their inevitable hobbies. Many became obessively brilliant Sociology Scientists as a result of that training. Still, the failure of the built-in cognitive empathy cores in their brains sometimes showed; Nicole was looking at one result. Julia was avoiding looking at Raphael, a classic case of an intact affective empathy showing to a demisavant, too late, what her impaired cognitive empathy could not earlier predict; an old, reliable sign of demisavanthood. At least demisavants could operate well in modern society, with training. The opposite psychosis - antisocial personality disorder, with an intact cognitive empathy but impaired affective empathy - was still dangerous. Nicole said to Raphael - - was about to speak to Raphael - Raphael jumped into the ocean, and began swimming away! Some well-trained part of Nicole took over. "Executor! I am Judge Nicole Angel! Formally, I ask you to pursue and subdue that individual!" Michael pushed himself away from the boat; Raphael had jumped off the side opposite to him. Julia covered her mouth in pain; some swift movement of Michael's had hit her accidentally. He swam around the boat, then started in pursuit of Raphael. Raphael was a surprisingly strong swimmer, even with a lifevest encumbering him. Michael could barely keep up; after a minute or so, he paused to pull off his own lifevest and redoubled his efforts. They were almost distant specks when Michael finally reached Raphael. Perhaps a minute had passed; they were strong swimmers. "Engineer. Julia. I am the Judge. Can we navigate the boat towards them?" Michael did not appear to be heading back; perhaps all his effort was in subduing Raphael. "Not very effectively, Judge. The boat is round, not a very hydrodynamic shape. We don't have paddles. A lifeboat isn't exactly meant for travel..." Julia paused. "Take off your lifevests. Their backs are relatively flat and stiff. Not the best paddles, but better than hands." The three women paddled with their improvised paddles. After a while, Katja gave up trying to use her lifevest as a paddle; the lifevest floated and couldn't cut water very well, so she used her hands instead. Julia and Nicole persevered; Nicole could feel her hands grow raw where the lifevest chafed, as she fought to push it under the water, to use it as a larger paddle than her hands. Perhaps an hour or more later they were near enough for Katja to pull the two towards them. Michael had his arms wrapped around Raphael. Both were panting deeply, Raphael still struggling. All of them were exhausted, so they just pulled the two men close to their boat, all three of them holding the pair. They waited, catching their breath. Michael spoke first, a melodious but authoritative voice. "Madam Judge, " he said, "as per your formal request, I have apprehended a certain Raphael Yu. I formally ask proof of vocation as Judge, and formal reason for pursuit and apprehension." He chuckled. "Okay, that's probably not something I'll get right now." The three women laughed a little. Julia said, "Ah, and now you have to formally date me, you stole my first kiss!" Julia spoke giddily. "And I didn't even get the honor of kissing by mouth as a first kiss. All I got was the back of your hand! Give back the kiss you stole from me!" She laughed heartily. The rest were nonplussed for a moment. Katja looked at Julia, then at Michael, who appeared both puzzled and mortified. Then Katja started laughing too, which made Nicole and even Michael laugh. Demisavant, indeed. Then Julia said, "Although somewhat more seriously, Judge, what do we do about Raphael? We lost a lifevest, although we do have one spare here," indicating the stash from which the medical kit had been. Nicole stared at her. Raphael had defied a decision that they had all agreed to be bound to. A precommitment to a method, specified by a Judge. The timing to be determined by an Engineer's expert decision on the continued seaworthiness of their craft. Both had been defied, without starting a disputation arena in which a precommitment could be modified, if new evidence showed that the agreed-upon precommitment would be irrational. A punishment was necessary, but - Chapter 9 Verrens waited as Judge David Adams finished reading the chapter. "A punishment," said Adams at last, "so soon. Would you agree to this, Judge Verrens?" "Yes," Verrens said without hesitation, "As the novel implies that they had all precommitted to the method and timing. As a social species Humanity must punish non-cooperation. Of course, the precommitment was not explicitly shown, but it would be disingenious of the author to expect the reader to assume otherwise." "True," said Adams. He looked at Verrens expectantly. Verrens considered for a moment. "Akrasia, the state of acting against one's better judgment. I wonder if this is what caused non-cooperation from Raphael?" "Probably not," answered Adams. "Strictly speaking, akrasia can only occur due to the overutilization of a human being's limited willpower. If you've already done a good deed, then you are likely not to do a good deed again within a short period of time, as experimenters in the early twenty-first century found. It is even worse if it is a deed that appears good, but is inefficient and does not actually end up giving actual benefit to anyone; then, a person's willpower is consumed needlessly with no lasting effect. Many messages circulated back then, asking readers to pass them on, so that people would know about evil events occurring; but such messages actually prevented people from doing any real good, other than to pass them on. The end effect was letting those evils continue. The wider they spread, the greater the harm they did; hence their widespread illegality in our enlightened era. For Raphael to suffer akrasia, he must have done some earlier good that sapped his will -" "Precisely, Lord Judge," said Verrens. "He was insulted by Julia; inadvertently, as she is characterized as being a demisavant, but the event still occurred. A classic case of a noisy prisoner's dilemma. He had overlooked that insult -" "- and considered it a good deed, which sapped his willpower, setting him up for non-cooperation later. Or in this chapter, rather. Why, that is a good insight, Judge. I would have said that Raphael was never portrayed as doing any good deed; as not having done any deed, as yet." "Thank you, Judge." Adams continued. "So far, the women have been more proactive than the men; Julia is currently the motivating driver of Nicole, who is an active protaganist that has made a Judgment that binds the precommitment of her microcosmic society. Katja has shown expertise in her vocation. In contrast the men have been passive, before Raphael's non-cooperation, with Raphael a patient under Katja's care." Adams paused. "Well, as a Judge myself I would say that Raphael deserves punishment, but -" Chapter 10 - was punishment feasible? Nicole had to think about it. Hard. Raphael could not be left in the water, as he would be in a better position to attempt a foolhardy self-sacrifice again. Restraining him was taking all of Michael's strength, although - "If the Executor lets you go," Nicole said to Raphael, "will you act as a sane and cooperative human?" "Yes," Raphael said. Nicole nodded. "You may release him, Executor. Thank you, Michael." Could social constraint - shame, essentially - bind him strongly enough that physical restraints would be unnecessary? Nicole hoped so; she didn't want any of their physical materials used to build some sort of physical restraint, or take up too much of Julia's time and attention. By the same token, she didn't want Michael spending all his time guarding Raphael. Well, assuming shame would keep him restrained - what punishment could Nicole impose on Raphael? Wait - how did Nicole know what she knew? She had been running on instinct - on an automated System 1 process that had reacted and judged. Now she engaged her deliberative System 2, to work over the decision of her trained System 1; a microcosm of the community of Judges, one brain system overseeing the decisions of another. Was punishment appropriate in this case? Had her instinct failed her? Consider the consequences. Punishment of the non-cooperator would require significant resources of their little group; at the very least, it would require Nicole's willpower to find some punishment that would fit their constrained resources, which could risk akrasia in the near term for her. Non-punishment of the non-cooperator would signal the de facto repudiation of their precommitment, without even any disputation arena to contest it. It would undermine the authority of the Judge to pass Judgment over the moral decisions in their community. She was a Judge; but duty did not - or rather, should not - factor into her decision. Deontology - the ethical theory that right was what was defined as a person's obligations - was considered, by a majority faction of Judges, a failed model for morality. Pragmatic consequentialism - the ethical theory that the consequences of actions, based on a precommitted set of values, subject to scientific research, determined what was right - was, for all its flaws and limitations, considered by the majority of Judges as the best ethical theory; a way to keep growing less and less wrong, but (possibly) never completely right. Would it be right to punish Raphael? How could they trust him again, if he was not punished? At least punishment would, in the minds of the members of their tiny community, balance the books and allow them to treat Raphael as an equal once more, when his punishment was served. It would deter everyone from performing similar, possibly wasteful acts of "heroism" and self-serving "self-sacrifice". At least, that was what it should be, if all of them had at least basic rationality training. How could they trust him again, if he was punished? A person marked as "non-cooperator" may have that mark almost permanently against him or her, in the community; it would stymie his or her future attempts at cooperation. That could happen, especially in a stressful situation, where rationality training could fail, where the built-in animal instincts stood closer than ever. Nicole kept thinking, aware that everyone on the boat was waiting for her decision. "Give me three minutes to think," Nicole said. "Engineer. Julia, I assume you have a timer of some sort?" "Yes, Judge Nicole Angel," Julia said, indicating her wristwatch. She set an alarm. The time was on - Chapter 11 "Ah," Adams said at last, upon finishing the chapter, "luminosity. The act of reflecting, explicitly, one's attention upon one's own thought processes, investigating whether they should be changed." He looked at Verrens expectantly. Verrens was trained in the basic rationality training required of all Full Citizens, and in the intermediate rationality training required of all Judge trainees; he could not remember a topic called 'luminosity' - "You don't know?" said Adams. "Well, I've been advocating its movement into the intermediate stages of rationality training, once I learned about it in my amateur Sensei rationality studies. At least for Judges. Perhaps Nicole is being portrayed as being amateur Sensei. Or it could be an impulsive luminosity, not a deliberately decided one. Don't worry, your lack of knowledge of luminosity is not a point against you; it's not yet a standard topic for the rationality dojos at the intermediate levels that Judges are required to take. Although in any case, some rationality Senseis claim that luminosity should be derivable by any sufficiently bright rationalityka; a view I don't agree with." Adams continued. "These Senseis claim that dual process theory makes luminosity obvious. That the deliberative System 2 processes are quite capable of shining brightly on the automated System 1 processes. Thus, luminosity. But the full luminosity topic is not just about System 2 investigating System 1, but also System 2 investigating itself." Verrens could only nod, taking in the lecture. "But enough of that," said Adams finally. "What do you think about punishing Raphael, now?" "Well, Judge," said Verrens, "I'm not so certain about it now, now that I've thought more deeply about their situation and their current lack of resources to effectively punish Raphael, the current stresses in their situation, et cetera. Perhaps I should also ask the same boon Nicole asked for - three minutes to think." They both chuckled at this. "Well, Judge Verrens," said Adams, "my experience still says that Raphael should be punished, and that punishment is in fact feasible. Although it does require a more complex scheme for punishment, which Nicole must spend her attention for. In real life, she could probably afford to spend that attention, although the situation in this novel suggests that she must ration this resource. In any case, I'll grant you your three minutes -" Chapter 12 Nicole spent the first of her three minutes explicitly assigning whole numbers - utilons - to the expected utility of the outcomes. The utilon technique was a rationality technique by which a person attempted to extract his or her preferences - the utility function - and then use his or her trained mathematics skills rather than the less precise built-in brain circuitry. In this technique, one assigns an arbitrary number to a particular outcome, then compared other outcomes and gave various scores to each outcome, depending on whether the outcomes were judged to be better or worse than the base outcome. It was a refinement of an older technique, where a person might list his or her options, and put the pros and cons of each option, weigh each of them (or essentially, assign utilon scores, positive for pros, negative for cons), and then come up with a decision. With training and practice - which Nicole had much of - the decision-making process could be made speedily, without the crutch of pencil and paper. First arbitrarily assign 10 to the outcome that the community's trust of their shared precommitment would be preserved; this would be a +10 for the punishment option. Maintenance of her authority as Judge was a minor positive for punishment, so give that +2 (punishment 12, non-punishment 0). The resources required for punishment were an issue almost as important as the community's trust in precommitment, and probably a bit more in their situation, so let's call that even and give it a +10 on par with the basis, score for non-punishment (punishment 12, non-punishment 10). Assuming a continued maintenance of sanity, the trust of their tiny community of Raphael would be returned upon punishment. Assuming the community as a whole had a score of 10 - after all, this was what she had assigned as the score for the community's trust and its shared resources - then a fifth of their community, Raphael, would be +2. So give the returned trust after punishment a score of +2 in favor of punishment (punishment 14, non-punishment 10). As for deterrence - assume that the punishment would deter one other member of their community from needlessly sacrificing himself or herself. So that would be another +2 in favor of punishment (punishment 16, non-punishment 10). Suppose that punishment irretrivably marked Raphael as "non-cooperator" in the minds of the community members. Again, give that the individual member score of +2, this time for non-punishment (punishment 16, non-punishment 12). Punishment it was. Nicole then allocated the next minute for considering what punishment to apply. The punishment should, as much as possible, cover the important requirements: (a) deterrence, (b) rehabilitation, (c) incapacitation, (d) retribution. Also, it would have to be as low in the cost to the community as possible, so call that the (e) cost requirement. Before making a decision, the first step was to explore the options available. According to rationality Senseis, one common failure in rationality was to select, consciously or unconsciously, some option too early; this unnecessarily overprivileged that option, biasing it as the de facto status quo that needed to be unseated before another option could take its place. Instead, the Senseis thought, a person or group trying to make a decision should generate as many options as possible, and avoid precommitting to any option. Once a set of options had been selected, only then should they be evaluated; finally, one could be selected. Nicole spent five seconds with her eyes shut, clearing her thoughts of possible options that might have infected her brain before they could properly be captured, restrained, and evaluated. Then she opened her eyes, to see what options she had - (1) she could give Raphael a stern warning, (2) she could ask Julia to jury-rig some kind of restraint, (3) she could ask Michael to always keep an eye on Raphael, (4) she could limit Raphael's right to engage in community benefits for a limited time period, (5) she could obligate Raphael to pay for community costs, also for a limited time period. Now she considered her requirements. The most important was the cost to their tiny five-person community who, for the moment, owned a boat, and such materials that Julia might find or scavenge from it, as communal property; give the (e) cost requirement a base utilon score of 50. Both (a) deterrence and (c) incapacitation would reasonably be considered equal to each other in importance, she thought, and each individually was perhaps almost, but not quite, as important as the cost requirement; give them a base utilon score of 40 each. For the (b) rehabilitation requirement, perhaps it could be considered less immediately vital, at least given their limited situation; give that a base utilon score of 30. For (d) retribution, give that a base score of 10. For the option (1) warning, it could be a reasonable, but not perfect, deterrence, so give that perhaps 30. It might be a good rehabilitation, so maybe 25 (total 55). A good warning might prevent Raphael from trying something similarly foolhardy in the future, but not very well; give that incapacitation score only 20 (total 75). It may satisfy the needs of retribution quite well, so give it the full score (total 85). And of course it was cost-free, so give the full 50 for cost (total 135). - and so on she thought. Option (4) won - prevent Raphael from enjoying community benefits for a time. And for the final minute - How to implement this choice. What benefits could the community have? Nicole was now on a roadblock. Their only benefit was being relatively drier inside the boat. But to have Raphael replace Michael's position hanging on outside the boat would totally eliminate the incapacitation score; being outside the boat meant that Raphael could more easily swim off. Especially now, Raphael knew that Michael could only beat him if Raphael had an encumbering lifevest and Michael did not; the next time, Raphael may very well make good by taking off his own lifevest before attempting to swim off. No, wait, there was another benefit that the community could engage in. Human beings had a desire to give to other people in their community. Now that Nicole had thought about it, it was in fact a highly appropriate punishment. Raphael had attempted to sacrifice his own life in order to give to others in the community, in direct contravention to the agreed precommitment that such a sacrifice would be done by lots. Raphael would be disallowed and exempted from sacrificing himself, for a period of time. This evening, whoever might be selected to replace Michael in the water, it would most definitely not be him. Chapter 13 Judge David Adams was chuckling as he finished reading the chapter. Verrens smiled. "Three minutes, Judge Adams?" "Many things to chuckle about," Adams said. "As you mentioned, three minutes. It took you as long merely to make certain that punishment was appropriate in this case. Although it would be unfair to compare your performance to Nicole's, I think; that would be the fallacy of generalization from fictional evidence. Nicole does not exist, although as I mentioned before, the novel does have a kernel of truth to it - it is not truly perfectly fictional. And further, I myself was able to ascertain that punishment was necessary, in far less time than three minutes, for that matter. So, on reflection, perhaps it would indeed be fair to compare your performance to Nicole's; you can think faster than you did." Verrens bowed his head in assent, accepting his superior's rebuke. "Also," said Adams, "I was amused to see that Nicole was able to derive a punishment that I myself would have passed in this case." "For what it's worth, Judge," answered Verrens, "I did think of such a punishment, but I had not made a complete analysis yet. So I didn't present my thoughts. A poor excuse, of course; I shall try to think faster in the future." He paused. "Aside from the obvious exposition in this chapter on how to make a difficult decision, I also think that this chapter showcases how to actually change one's mind -" "- the third thing to chuckle about in this chapter," said Adams. "Your thought is correct; please continue." "- although it does not actually show a change of mind. The process was indeed the full process, and the fact that the same conclusion was derived - that Nicole did not actually change her mind about imposing punishment - is largely immaterial. The process of changing her mind started in the previous chapter: Nicole actually stopped. Then - clear your mind. Hold off on proposed solutions: first, think of the problem, then propose solutions once the problem and its issuess have been considered. First Nicole thought of the various issues involved, and the different parameters that her decision had to fulfill; she attempted to not propose any actual concrete course of action. Only then did she generate her options. And during the process of selecting among her options, she used a system that could, in fact, allow her to go either way: a utilon scoring system. It is a good example of avoiding the error of privileging a hypothesis." "A complete analysis," said Adams. "I applaud your thoughts over this aspect of the story so far. Now let us discuss Raphael's punishment. Could Raphael have attempted to manipulate Judge Nicole into selecting this particular punishment, specifically to save himself?" "I don't think -" started Verrens. Then he stopped. Adams watched the wall clock, allowing Verrens time to think. A few seconds before the time that Adams had allotted, Verrens said - Chapter 14 "In order for that to be true," Nicole explained to Julia, "we would have to assume that Raphael knew, for a fact, that (1) I would order the Executor to pursue him, (2) that the Executor would successfully catch him, and (3) that this was the punishment I would give him, rather than, for example, just giving him a stern warning. While the actual decisions were rational, he could not be perfectly certain that I would be rational enough to make those decisions, and that Michael would be rational enough to make a fast decision under pressure on whether or not to follow my formal order. Further, he had no evidence that Michael would be able to pursue him. We saw first-hand the evidence that Raphael is a good enough swimmer to elude capture from Michael, and it was only the fact that Michael removed his lifevest that allowed our Executor to catch Raphael." Nicole paused for breath. After passing Judgment and describing the punishment to be meted on Raphael, Julia had objected, pointing out the possibility that Raphael's real goal had been to be specifically exempted from participating in the drawing of lots in case they had to reduce their number. Nicole had asked for another three minutes to think. And after three minutes she gave her true analysis of the situation. Julia seemed unsatisfied, but the rest appeared to accept it. Then Raphael spoke - "Judge," he said, "I ask you, formally, for a Judgment." "What is it?" Nicole asked. Raphael paused for a moment. "Earlier, Julia Montego implied that I was useless to our community. This insult hurt me and was what lead to the current situation. In such a case, should not Julia be punished too?" Nicole paused. The standard explanation accepted by most Judges and taught to trainees filled her mind. But she had to be careful in how to express the argument. In the late twentieth century, and even up to the early twenty-first, when mass interconnected communications became publicly available to a growing percentage of citizens, arguments over difference in opinion flooded the channels, in large-scale events called "flames". The main problem was that the people who had trained themselves to query nature and learn the texture of its internal organization had developed a different method of communicating, compared to the people who trained themselves in actually convincing others. The former, who were more likely to be correct, could only scream in frustration at the latter, who were more likely to spread their - too often flawed - model of reality. The secularization, decentralization, and equalization of society had almost failed to occur at all. Fortunately, the Scientists who studied nature were ultimately correct. As their study of nature spread, they eventually found evidence to support their most basic assumption - that human behavior was, at its source, a completely natural and material phenomenon. And they re-discovered the techniques that their Reporter counterparts had been using all along. "Raphael," Nicole said, "I understand your sentiment -" "No!" interrupted Julia. "I'm a demisavant. It wasn't my fault! I don't do well at social relationships! It was an error that I could have prevented, I'm trained to pause and think to keep my condition from over, over, I don't remember, but I'm extremely stressed..." She broke down in tears. "Julia, Raphael," Nicole said, keeping her voice level, "I understand both your positions." She breathed deeply, a technique that would also encouraged her listeners to imitate her; a good deep breath was a common trigger for rationality techniques, and she hoped that it would bring some sense into the stressed 40% of her community. "Next time, I'll try to do better in my Judgment," she said. Triggering the empathy of those who were irrational was more likely to allow you to lead them back towards some semblance of sanity. "It was my Judge training. That was what my Judge training did. Next time I'll make it up to the both of you, okay?" The Dark Arts. That was the term that the first Scientists re-discovering these techniques used. The name reeked of opprobium. It was intended to do so; some exceedingly wise student of science had thought it appropriate to use one of the techniques itself as a method of discouraging the techniques as a whole. But a rationalist was not allowed to reject the use of a tool, just because of its label... "But please consider," Nicole continued. "In civilization, in Humanity's cities and colonies, people using my Judge training keep peace and order. I'm not saying that my Judgment training is perfect or the ultimate in truth, or better than your past and your experiences. I'm only saying that it's more useful for us all now. That it helps us all keep our sanity and our safety, together." She smiled at them. "We want to be sane and safe together, right?" Focus on what you shared with the people you were trying to convince. Eventually, people will follow what you do, especially if you stopped talking about it and just did it. Herding worked, even among humans. "Now, Julia," she said, "I want you to apologize to Raphael. Okay?" She smiled. Julia bowed her head. "I'm sorry, Raphael. I see you're a good swimmer. Maybe, maybe I think it will be useful. We're in the ocean, after all. So, so, I'm sorry." She looked at Raphael directly, and said, "If I fall into the ocean, save me, okay? I can't swim." Raphael said, "I'm sorry, too. You're right, Judge. It's not fair of me to blame Julia. I'll take my punishment." He looked at all of them. "I'll follow the community's precommitment." A rationalist was not allowed to reject the use of a tool, just because of its label - words were not the same things as what they referred to, just as "control freak" and "leadership instinct" were two labels for the same complex of Human behaviors. Nicole breathed a sigh of relief. The danger was passed. Back when twentieth century science was still discovering the material workings of the human brain, humanity's primitive justice systems began to allow a highly peculiar defense: not guilty by reason of insanity. Because certain behaviors could be tracked to particular reasons - so it was argued then - the behaviors were not, in fact, the fault of the individuals who actually engaged in that behavior. They could not be responsible for those behaviors, and thus it would be unreasonable to punish them. But as the number of anti-social behaviors were traced to their causes, the acceptance of that defense became exceedingly strained. Crimes that in previous eras would be clearly considered as the fault of their perpetrators were being solved, not by punishment, but by halfhearted attempts at "rehabilitation". More and more, it seemed that the study of human behavior would utterly destroy the foundations of morality and society. Some even advocated suppressing the information and research about the material mechanism underlying human behavior, so that human beings could pretend to control their own destiny, and still be "responsible" for their choices. Free will, previously an idle speculation indulged in by well-fed philosophers, became an important policy issue. Science could not be allowed to destroy free will. But finally a revolution occurred, led by Neurological Scientists. Punishment, it was determined, was itself one of the causes of human behavior. Further, it was a social policy that could be controlled independently of human behavior, and thus not be causally entangled with other effects on human behavior. If a particular personality type, or a particular psychosis, was studied and determined to not respond well to punishment as a motivator for better behavior, only then could punishment be foregone, in the interest of reducing the cost of imprisonment and other punitive costs. But such individuals could not be considered as Full Citizens. Full Citizens were those individuals whose decisions could be controlled by the threat of punishment from society. Raphael's denying responsibility for his actions and thus exempting himself from culpability for his punishment, was tantamount to repudiating his status as Full Citizen. Punishment was justified in the typical case where the threat of punishment could be reasonably expected to actually deter the anti-social behavior, by scientific studies of existing data collated from governments implementing various policies. And since a vast majority of humanity were demonstrably more likely to avoid anti-social behavior if such anti-social behavior would lead to punishment, then the issue of the "responsibility" of their choices was considered moot. Free will returned to being a pastime of philosophers. The Judges rejected the doctrine of innocence by reason of insanity. "Come on," said Nicole, "Julia, give the spare lifevest to Michael. Then let's pull in Raphael. We're all in this together, everyone. Ready?" Chapter 15 As Adams finished reading the last page of the chapter, Verrens said, "A well-filled chapter, don't you agree, Judge?" "Yes," said Adams. "I find it quite fascinating that you and Nicole agree quite closely in your conclusions. Although I admit that my question regarding the possibility of Raphael manipulating our main character was largely an idle one." "Really, Judge?" said Verrens. "I had thought it was an attempt at testing whether I would indeed follow-through on my informally stated commitment to improve the speed of my thinking." "There is that," said Adams. "I truly did want to check that, and could think only of my idle thought. Still, I must congratulate you this time; you were able to beat Nicole this time by a full minute. In any case, can you think of anything more we can talk about for this chapter?" Verrens thought for a while. "None, Judge," he said. "I cannot think of anything to disagree about or highlight in this chapter's exposition of our history and our current theory of punishment." "How about its explanation and demonstration of a Dark Art?" asked Adams. Verrens shrugged. "There are Dark Arts and then there are Dark Arts. The very label itself is an application of a Dark technique; by giving a negative label to the set of techniques, we seek to discourage their use and possible misuse." Verrens paused, pensively, then continued, "You mentioned earlier, Judge, that this novel was being proposed as a possible training material for student Judges. If so, then the demonstration of a legitimate use of a Dark Art would be useful in guiding future Judges in determining what would be acceptable situations for using Dark Arts, as well as showing which Dark Arts could be more safely used." "I see," said Adams. "Well, that is feedback that we might give to its author, so that the author will know what to focus on in teaching the next generation of Judges. Thank you for this opinion, Judge Emmanuel Verrens. Let us continue." Chapter 16 After pulling in Raphael, the group settled back again. Katja dropped another dose of the shark-and-jellyfish repellent, citing the fact that they had probably moved beyond the edge of its rated effectiveness. "Julia," said Katja, "can you set an alarm at mid-afternoon? That's probably the best time to swap in Michael for whoever gets off the boat next, so his clothes can get dry before nightfall." "Okay," said Julia. She manipulated her timepiece. "And you can all feel free to douse your head in the water at noon," said Katja. "Or at any time. Or better just keep your heads wet. I don't want heatstroke on any of you. Before we throw in the next person into the water, we all put on a fresh coat of sunblock. And forget lunch, those bars should keep you alive for twelve hours." Nicole settled back to rest, noticed that Michael swam to a position next to Julia, and decided to just close her eyes and doze off - - was about to close her eyes when her averted gaze settled on Raphael's bandage. "Katja," she said, "should we replace Raphael's bandage?" Katja, who had already closed her eyes and covered them with her arm, simply said, "Let him worry about it himself, I'm not wasting more of my bandages on that self-serving jerk -" And Nicole almost wanted to scream at that point, her indignation white-hot, System 1 triggering on a very important moral violation... Fundamental attribution error, it had been called when it was first discovered. Suppose you were to observe another person kicking a vending machine at a point in time A. At a later time B, in a situation with that person and another vending machine, was the probability that that person would kick the vending machine higher than 50%, or lesser? Most people - at least most people back then, before basic rationality training was mandatory - would say that the probability was higher than 50%. They attributed the action observed at point A to be due to a fundamental property of the person, and used that single behavior to predict that person's behavior at a later point in time. Suppose instead that you were to observe yourself kicking a vending machine at a point in time A. At a later time B, in a situation with yourself and another vending machine, was the probability that you would kick that vending machine higher than 50%, or lesser? And there, such as it was, was the kicker. People saw a single memorable action from a particular individual, and assumed that the action would define that individual's personality. And yet a moment of self-reflection on their part would have revealed the utterly laughable stupidity of that error. Never mind that the Medics had some sort of oath to serve all individuals medically, regardless of the situation (or so Nicole knew vaguely as part of civilized culture); never mind that Medic Ekaterina Ivanova had violated that oath (as Nicole understood it, anyway). Medic Ekaterina Ivanova had violated something dearer and nearer to Nicole's vocation. Katja had decided to punish Raphael. Without a Judge's explicit expert Judgment. On reflection, Nicole realized, medical treatment was one of the few benefits their community could offer, and was one benefit that they could have withdrawn from Raphael as punishment. But that was as unthinkable for a civilized person as killing Raphael and using his skin as a roof to protect them from the sun would be. Humanity assumed it as a right to protect, to be withdrawn only with reluctance and the direst of needs. What kind of person was Katja? Did she really have everyone's interest to heart? Was she hoarding the energy bars to herself? Could she have somehow cheated on the basic rationality training? Medics did not need the intermediate course and perhaps some rich parent had decided to have Katja skip the entirety of rationality training. Perhaps it would be wiser to consider Katja as completely irrational. After all, she had not learned to detect and fix the fundamental attribution error that came built-in with her brain. Wait - that was the fundamental attribution error working even on her! The core mistake of fundamental attribution was the tendency to assume that mistakes could only be committed by incarnations of pure evil. But bogeymen did not exist in reality. There was no incarnation of pure evil, no category of existing sapient beings fundamentally composed only of irrationality. Katja was a human being. Nicole could assume this with significantly high prior probability greater than 99%. In addition, she was very likely a modern Human Full Citizen; again, Nicole could assume this with significantly high prior probability (space travel was disallowed for non-Full Citizens; most non-Full Citizens were kept in asylums where Medics and Neurology Scientists did their best to cure them of their disease; and so on). This meant that it was also quite likely that Katja had been given at least the minimum basic rationality training that was a requirement to be declared a Full Citizen. The erroneous conclusion of fundamental attribution was due to not properly considering the effects of situation on an individual's behavior. Nicole consciously observed the situation that Katja was in. In broad strokes, it was basically the same situation that had caused Raphael - who was himself a modern Human Full Citizen, and therefore also trained in at least the minimum basic rationality training required for such a position - to act irrationally and repudiate their communal precommitment, causing him to believe that a senseless self-sacrifice, triggered by an unintended insult from a demisavant who was in essentially the same situation, would be a good deed. The problem was not that they all fundamentally had the attribute "irrational". The problem was that the stress of being in a lifeboat on an unknown sea, with no real knowledge of when they could be rescued, had overpowered their trained rationality responses. Their animal nature, refined and developed by millions of years of evolution, had taken over the domain of their brains. Animal natures fighting against rationality techniques that, for all their sophistication and for all the thought and development that had been put into them, had existed for no more than a couple of hundred years, and were mostly developed in an environment of reasonably peaceful civilization. When their situation put them in an extremely primitive environment that their evolved primitive simian brains had recognized, their built-in circuits had simply put themselves in control. And the final step... Nicole was a human being. She herself could assume this with significantly high prior probability greater than 99%. She was herself a modern Human Full Citizen, and had been given the same basic rationality training (and of course the intermediate rationality training required of Judges). She could very well make that same mistake, in the same situation. All right, she told herself. First, fix the hopefully-temporary irrationality in Katja's brain. Second, fix their situation so that irrationalities were a much less likely occurrence. Attack the problems one at a time. Divide the problems into smaller parts, and attack the sub-problems one at a time. Keep doing this until a sub-problem is trivial. Nicole started to think on how to improve the irrationality in Katja's brain... Chapter 17 "I fear I must protest, Lord Judge," said Verrens, "the pacing of this novel. The design of its plot is..." He glanced at the stack of chapters they had already completed. "Well, it is almost a quarter of the way through. And it is not yet noontime of the first day of the novel! Surely it does not seek to simply describe just a single day of the protagonist's life?" "Well," said Adams, "it is after all first and foremost proposed as training material for student Judges. But in the interest of improving it, would you care to give a diagnosis of its faults?" Verrens considered for a while. "For one, I get this feeling that the characters other than the main are placed simply to be problems for the main character. Although on reflection, it is probably something that I might accuse myself of fundamental attribution." Adams chuckled. "Well, thank you for your opinion," he said. "Although I have heard it expressed that all human fictional stories have an aspect of fundamental attribution. It is all too easy to simply label one side as good and the other evil." Adams continued. "I once read of an author," he said, "who said that the species name of 'homo sapiens' was a too-blatantly self-aggrandizing name for our species. He said that if we were honest, we would instead call ourselves 'pan narrans'. Not the 'wise man', but instead the 'storytelling ape'. I think it fits the description of humanity, at least before the advent of widespread rationality. And one of our fundamental characteristics as storytelling apes is our desire to always consider having two sides to a story, and to consider one side the heroes and the other the villains." He paused. "I think that if we were to make an honest piece of fiction about reality - a true rationality fiction - then it would not be about heroes against villains. Instead, it would at its core be about many different human beings against the inner impulses that make them, well, less than human." "And I suppose," said Verrens, "that this novel seeks to be one of this kind?" "Yes," said Adams. "I can tell you quite well that the designer of this novel's plot structure had good reason for having the moral and rational decisions follow each other so closely in this manner, in a situation where rationality and morality are strained. It is needed as a test of rationality and morality, two things that we as Judges must live by." They were silent. Then, Adams spoke - "Tell me, Judge," he said, "how would you solve Nicole's first problem? Let us consider this before you turn to the next chapter; you should have sufficient training to consider this already -" Chapter 18 Nicole had a diagnosis (of which she was extremely sure) on what was currently Katja's problem: fundamental attribution error. Now she only needed to figure out how to stop it. First, Nicole decided, she needed to have more insight on why Katja had decided to claim that Raphael was irrational. What would cause Katja to effectively say that Raphael was insane? Secondly, Nicole would have to determine what to do based on what she learned from Katja's thought processes. She would have to precommit to that; she could not allow herself to be suddenly stymied by something that she had not considered. So, split the second sub-problem. First, how could Katja respond? The most likely case would be that Katja had committed the error of fundamental attribution. Then the proper response to that would be to guide Katja back to rationality: remind Katja of the map-territory distinction; have Katja explicitly consider the possibility that her map was wrong; and then support Katja in correcting her map. A less likely one was that Katja already knew Raphael beforehand, and that this was not an isolated incident that was more likely caused by situation rather than personality. Just because both were strangers to Nicole did not mean that they were strangers to each other. In such a case, Nicole decided, she would have to inform the others, and revise her previous knowledge regarding Raphael's expected future rationality. For even less likely possibilities that Nicole could not imagine right now, she would ask for further advice from the other members of the community, and very likely ask for more time in order to think. Satisfied that she now had a reasonable plan, Nicole started - "Katja," she said, "why do you think that Raphael is a jerk?" Trivial problem, trivial solution. Katja didn't even bother to uncover her eyes to answer. "Isn't it obvious?" she said. "No," Nicole said, as flatly as possible. "Please explain." Please don't stop being a trivial problem. Katja finally uncovered her eyes then, and said, "Well, mister heroic I'll-sacrifice-myself-for- the-good-of-the-tribe here made us throw away a lifevest. Now we have no spare lifevest. He wasted the efforts of all of us getting him back. Energy and effort that we were better off preserving, this ocean is tropical and who knows when the next major storm will strike? And now the bandage is ruined, it's water resistant but it's not water proof, it wasn't designed to take on both a lot of water and strenuous movement that keeps rubbing on it." She paused, then: "See? It's obvious! You're not a Judge yet, a real Judge would have just let him swim off instead of going all 'Executor, I am Judge' to chase after him!" Ignore insult. Prey has noticed you. Lead off to direction of trap. "Ah," Nicole said, "so on the basis of this data, you conclude that Raphael is an irrational jerk?" "Quite right I do!" Katja said. "So you believe this new data falsifies the reasonable assumption that Raphael is just an ordinary Citizen with basic rationality training?" Nicole said. Jump over trap. Wait for prey to pounce and fall. "Yes!" Katja said. "It's as plain as Science. Take a hypothesis and test it, and if it's falsified, throw it away!" Prey in trap. Let trap close. "Good," said Nicole. "Then I, Nicole Angel, of vocation Judge, solemnly proclaim: I defy the data." And then move in for the kill - In the middle of the twenty-first century, when "Science" was still "science" without the capital letter, and the old habits of hierarchical thought were dying but not yet dead within civilized society, someone had to invent the practice of proclaiming: "I defy the data". Experimental results falsified accepted theories all the time. And some of the old proponents of pre-rational hierarchical thought learned enough of Science to be annoyances, but not enough to be useful members of a Scientific secular civilization. Whenever a pre-rational idol was in danger of being destroyed by the truth wielded by Science, its protectors scoured the journals far and wide for any data - any experiment - that would falsify the truth. And since even the best Scientists were - and continue to be - merely advanced simian descendants, why, certainly any sufficiently motivated search would come up with some experimental result from lazy, corrupt, or just plain erroneous Scientists. Some result that the protectors hoped would destroy the truth. Hence, the development of "I defy the data." "Sure," said the prototypical Scientist who had invented that phrase. "This experimental data you show me would destroy the theory. I won't dismiss your data out of hand. But I defy the data. Here, have my head, I'll put it on the guillotine for you (metaphorically, of course): show me more data. Show me new experimental results, more than this one you have on offer. Replicate the experiment, or else. If you can't, then I hold that this was a fluke, a bit of noise, static from our noisy universe. Then I get to keep my head, and more importantly, my theory. But if you can replicate this result, then by golly let my head be cut clean off (or, in less graphic language, let me withdraw my theory): that which can be destroyed by the truth should be! I, a scientist, solemnly proclaim: I defy the data!" And that simple proclamation (the exact theory and the exact experimental results was just out at the edge of Nicole's memory; she was contemplating being an amateur Historian as well, but in any case, it was that proclamation which had stuck in her mind, and it was that proclamation which was in these days now as much a part of modern cultural consciousness as Hamlet, Tom Sawyer, and Fate/Stay Night) had reverberated throughout the Scientist community. It became a habit, then a tradition; finally, a codified law that was taught not only to Scientists, but to all Free Citizens. A great Scientist who proclaimed "I defy the data" would, indirectly, drive the grants of several experimenters very high indeed. And a few great Scientists did lose their heads, metaphorically, when the data truly went against them. No longer could the pre-rationalist protectors use the language of Science to cut its own tendons. And because of the terms of the wager implied by the words "I defy the data" - why, the pre-rationalists could not use it to defend their idols. For every defiance cost you the possibility of losing your head, metaphorically. Those four words became the shield that was also the sword - - Katja was taken aback. "W-Wait," she stammered, "that doesn't, it's not, it's not applicable here -" "It is," said Nicole, "or otherwise you cannot say 'take a hypothesis and test it'." Katja stopped. Remind her of rationality. "The map is not the territory," Nicole said, "and single actions do not define a person. You saw a tree, assumed a forest, and filled your map of Raphael with it." Katja remained silent, eyes blank. Nicole hoped it was a rationality trance. Have her consider that her map could be wrong. "Yes," Nicole said, "Raphael might be an inconsiderate, irrational jerk who will consistently sacrifice the community's values for his own. But Raphael might also be an ordinary Full Citizen whose basic rationality training was overrun, temporarily, because of the extreme stresses of our current situation. So your map might be wrong. We need more data for better consideration." Have her explicitly mention that possibility. "Do you agree?" "Yes," said Katja. "I'm sorry." Now support her in correcting her map. "So let's give Raphael another chance, all right?" Nicole said. "Let's see if he tries to repudiate the community's precommitment again, before we actually write indelibly in our map, 'Raphael is a jerk'." First problem solved. "Okay," said Nicole. "Now, can we get this bandage replaced?" "No," said Katja, "and this time it's not because he's a jerk. We need to preserve our supply of bandages." "Impossible!" said Raphael. "What kind of great incompetence - I would expect that an emergency medical kit would have good margins for extras." "Yes, but Julia says -" said Katja. Then, "Well, Julia? Shouldn't you be the one to tell them?" "What's this?" asked Michael. "What's happening, Julia? Something wrong with the boat? Other than a slow leak?" Julia was shaking her head. "Oh, blast it," said Katja. "Haven't any of you wondered why we're the only lifeboat around? The planetary drop pod fits how many, 60 or so? Where's everyone else? Julia thinks it's sabotage. Engineer's expert opinion. Why else would an otherwise-safe planetary drop pod fail? And if this boat is the only one that got away - well, it would be highly plausible that it got away because the saboteur specifically chose this boat as an escape vehicle, right? So: one of us here is the saboteur. I wonder who?" Chapter 19 "Whoa," said Verrens as he finished the chapter. "What is it, Judge?" said Adams. "Nothing, Judge," said Verrens quickly. "Judge Emmanuel Verrens," said Adams, "as you know, the most important part of any secret is the fact that the secret exists. You have already revealed the existence of the secret, and from the context of your revelation I conclude the secret is some sufficiently interesting plot twist revealed at the end of this chapter. Suffice it to say that you have already spoiled the ending of the chapter, and the details themselves are now secondary." He sighed audibly. "I shall be quite frustrated if the plot twist is not sufficiently interesting after all, since you have already raised my expectations of it." He turned back to his copy of the manuscript. A few minutes of reading, and then: "Huh." Adams put down his copy. "Well," he said, "the plot advances. Although I honestly cannot say that I am inordinately surprised; I have read various permutations on this kind of novel before, and I know the designer of the plot. It would be necessary, she would say, for this plot twist to occur here, and just so; needed to maintain the interest and focus of the Judge trainee. So perhaps I need not fault you too much; you need not be disappointed in my less-than-amazed reaction at the turnings of this novel's plot." He smiled. Verrens thought for a moment. "Who do you think the saboteur is?" he said. "We have no real evidence yet," said Adams. "It would be premature to say anything. It would even be premature to accept immediately the claim that there is indeed a saboteur; it needs more thought on whether it is true, or a flawed conception of reality. Better to hear the full analysis directly from Engineer Julia Montego. Still, one might consider this possible; but also, one must consider first whether it is more likely that a truly random accident occurred, rather than deliberate sabotage. Modern Humanity can claim to be less wrong than pre-rational humanity; it cannot claim to be perfect." "Yes," said Verrens, "but surely a novel is written about unlikely events; precisely why we have an injunction against generalizing from fictional examples." "A typical novel, yes," said Adams. "But this is not a typical novel. The plot of the novel might be considered fictional, perhaps, but not the knowledge, and most certainly not the decisions made, especially by the main character. Those, as I mentioned before, might be considered to be true, in a way." "Ah," said Verrens, "I understand." "Thus," said Adams, "in considering this novel, we would do best to first keep an open mind; the probability of sabotage is extremely low, and I can assure you that the designer of the plot knew that. We know that there is some issue whose knowledge it would be important to know about; but we also know that we lack crucial evidence to distinguish between a universe where exists-saboteur is true, from a universe where not-exists-saboteur is true. Better to look with fresh eyes and see the evidence. With that, I think it best for us to proceed." Chapter 20 Confusion might have engulfed her. Terror might have overtaken her. They were there, at the edge of rationality; she was not specifically trained to handle that, especially terror, as those were not expected to occur often in her chosen vocation. Still, even basic rationality techniques were general enough to at least serve as speedbumps against confusion and terror. "I don't know," Julia was saying, "I'm not sure..." Katja was still berating Julia, insisting that she tell everyone her expert opinion. Michael was speaking more soothingly, but what he was saying pretty much the same thing that Katja was. The problem with "I don't know" was that it gave too little information to another rational Human being. Consider one person, known to be a perfectly trustworthy perfect observer, visiting another person in a sealed room. The first tells the second that outside there is an apple tree - a true statement, since the person is perfectly trustworthy and a perfect observer - and challenges the second person to give his knowledge on the number of apples on the apple tree. The second person might reasonably say "I don't know," but this was too little information to extract from the given knowledge - the existence of the apple tree. A better rationalist would, more reasonably, say, "I don't know exactly, but I'd assume it to be between 0 apples and 1,000 apples." Obviously, no known apple tree can carry, say, a million apples, much less a billion apples. But a plain "I don't know" did not capture that knowledge that the second person already possessed. Between someone saying "I don't know where the keys are," and someone saying, "I don't know where the keys are, but I last saw them on the dining table," one could say that the second had given more information. And yet both could legitimately say "I don't know." Knowledge was not a binary condition; when you zoomed in on the details, the lines and edges of "known-to-be-true" and "not-known" were actually probability distributions. A rationalist might not bet even odds that the apple tree in question had fewer than 10 apples, but he or she might accept a bet at a hundred to one against. And if Julia had some evidence which increased the probability of sabotage above the base probability of 'almost never', meaning some value infinitesimally lower than 0.01% (given the ubiquity of cameras, the larger reported incidences of accidents versus incidences of deliberate sabotage in the recent years, and so on), then not sharing that evidence would be equivalent to misrepresenting the level of danger they were in. No matter that she did not know for certain: they were all rational, thinking sapient beings. They had the minimum rationality training required of Full Citizens. So: how to extract that evidence from a demisavant? - "Excuse me, may I speak?" someone said. It was Raphael. Good, at least now he was engaging with the rest of the community. Then Nicole realized that if Raphael truly felt like an integrated member of their tiny community, he probably wouldn't be asking permission to speak. Katja glared momentarily at Raphael, then blinked and put her face into a more neutral expression; at least she remembered that she had precommitted to giving Raphael a chance to prove at least basic rationality. Michael stopped speaking, and Julia stopped shaking her head and repeating "I don't know" over and over. There was silence. Raphael still didn't speak. Break the ice, then. "Okay," said Nicole, "please speak, Raphael." Raphael licked his lips, then said, "My current registered vocation is accountant, but some time ago I started to dabble into Administration. My company sent me to Earth for formal Administrator training, so I suppose you could say that they found it more efficient to retrain my hobby rather than hire a registered expert." He paused, looked at Nicole, and said, "Nicole isn't a full Judge yet, but we call her Judge provisionally now because she has the most knowledge of that field, and relative to the rest of us she is an expert. So, may I at least claim to be Administrator in our community?" Nicole nodded. The others seemed less certain, so she said, "Yes, Administrator." She smiled. "Of course," Raphael said, "an Administrator handling a tiny group is not very efficient, so I'm still not really all that useful. However, at least let me point out that we might do better to use some form of disputation arena while deciding on whether or not we are at increased risk because of the existence of a saboteur among us." "Well," said Katja, "how can we hold a disputation arena in this kind of situation? An Arty-Delphi is impossible without a central server, and a Deepy market needs a backing monetary institution, unless Julia can rig a bank and an economy out of metal and rubber." "There are older forms of disputation arenas," said Raphael, "which are much lower tech than a Real Time Delphi Method or a Decision/Prediction Market. We can use a modified form of Nominal Group Technique." "Okay," said Michael, "what's that involve? I've never heard of it." "It's a very simple disputation arena," said Raphael. "Less flexible than any of the Delphi methods, and does not integrate anonymous information at all, unlike most Delphi methods which can integrate minor anonymous information and the prediction markets which are almost completely about anonymous information. However it is simple, low tech, relatively fast, and engenders better feelings of -" "Administrator," said Nicole, "the method, please?" "Sorry," Raphael said. "First, we introduce the problem to be considered. I think Katja's introduction of this issue can serve well enough. Then, each of us thinks, without discussing with anyone else, for a short time - perhaps fifteen minutes will be enough, in order for each of us to review as much information as we can remember of the crash last night. Ideally, we'd write our own notes, but we lack even pen and paper -" "Wait!" said Julia. "I have paper. And a pen. I'm an Engineer, of course I'll have paper and pen!" "Do you have enough paper and pens for all of us?" said Raphael. "No," said Julia, "only one pen, unfortunately. Spares were in the bag, which didn't survive, I suppose. I do have a small notepad. It's not even damp!" "That's fine," said Raphael, "you can be the facilitator later. Anyway, after the fifteen minutes of thinking are up, each of us presents what we thought about during the fifteen minutes, one at a time. I suppose I can start the ball rolling, and the next speaker will be the one to the left of the previous speaker. Now the important bit: during the initial presentation of thoughts, the only questions allowed are questions of clarification, regarding exactly what is meant. No challenges, no counterproposals while someone else is speaking. During this initial presentation, the facilitator jots down notes about what each person said. The main ideas taken by the facilitator from the initial presentation should really be visible to everyone, but I suppose Julia can show her notepad to everyone easily. Any questions yet?" There were no questions, and then Katja said, "Is that all?" "No," said Raphael, "after all members have given the initial presentation of ideas, we then try to merge similar ideas. At this stage we have counterproposals and syntheses of ideas. Then each of us ranks the ideas, and then we have a consensus." "I'm not an Administrator," said Michael, "but is that really a disputation arena?" "Yes," said Nicole, before Raphael could answer. "The requirements of a disputation arena are: the centrifugal phase, where each individual can first decide on their own ideas independently of other individuals; and the centripetal phase, where the ideas are put in competition against other ideas - the actual arena. It's better to have this loop repeatedly, and better to do it in real time, which is why Arty-Delphis and Deepy markets are the preferred ones, but - yes, what Raphael described is a valid disputation arena!" Nicole noted that Raphel looked a lot more pleased and a lot more effectively a valued community member now. "Correct," said Raphael. "Disputation arenas require the centrifugal phase in order to foster independent thought, as otherwise group attempts at aggregating information become subject to groupthink. As we all know, groupthink effectively reverses the smart mob effect; the desire to conform prevents mistakes from being pointed out, and as each new member joins the group, the effect is magnified, increasing the social pressure to conform and further preventing any member from attempting to defect from the accepted, erroneous consensus. End effect: groups in groupthink situations become subject to information cascades that mean that each new member decreases effective rationality, rather than -" "Administrator!" someone said. At this point, it wouldn't have mattered who did. Chapter 21 "Character development, Judge," said Verrens, when he saw Adams put down his copy of the chapter. "Yes," said Adams, "and if I think I know the designer of the novel's plot, well, this is simply a set up for another moral decision soon. Although she has been quite imaginative about such twists recently, and I would rather enjoy what it might end up being, rather than speculating about what it might be." He smiled - no, he grinned - and said, "So I will quite legitimately say 'I don't know', as I prefer to squeeze a bit of fun out of this situation and watch you speculate on what it might be." Verrens sighed in response. "A secret whose existence is known," he said, "and thus may very well haunt me, at least for the next few chapters, until it is revealed. And I suppose you will obstinately say 'I don't know' quite unhelpfully, while you enjoy your more accurate speculations, due to the greater knowledge you have that you will not share with me." Adams laughed. "All right, then," he said, "I will give you my own speculations, but only regarding the topic of sabotage. Regarding the moral decision that I suspect will occur soon, you are on your own. In our modern world, provable sabotage occurs once for perhaps every ten thousand accidents; assume that the true incidence of sabotage is an order of magnitude higher than the ability to show evidence of sabotage - somewhat unlikely, as this means the ubiquitous cameras are not that useful and are probably a waste of energy maintaining. But even so the base probability of sabotage is only about one out of a thousand." Adams cleared his throat. "Now, suppose that our characters reveal additional information. In order to be on par with the case of accidents, that information would have to move up the probability by three orders of magnitude. So, I might bet on sabotage if I would win ten credits for, say, one credit loss if I'm wrong, but for an even bet I'll bet on an accident." Verrens sat silently, considering. "Your analysis seems reasonable, Lord Judge," he said. "However, I'm not certain if you've factored in the growing tide of Technological Renouncers. There is now a growing number of radical members of such groups. Using a similar analysis as yours, I would bet at a rate of five is to one against." Adams frowned. "Is that true?" he said. "Ah, never mind; my dislike of the Reporter vocation has just asked for its cost. Well, then, I should probably update my own estimates, in light of your information. Thank you for showing me an error in my map." "You're welcome, Judge," Verrens said. Chapter 22 Julia's timer chimed the end of the agreed-upon fifteen minute thinking time, and Raphael spoke. "Okay, I'll start," he said. "Last night, as is usual for planetary drops, we were given sedatives in order to relax us during the actual drop; in addition, the given sedatives were intended to synchronize our bodily circadian rhythm to the local time of our drop area. If the accident hadn't occurred, we would be in a much larger boat; basically the drop pod would be the boat itself, and we would be towed by the ships waiting to bring us into port. So three things come to mind at this point." He held up three fingers. "One is that," Raphael said, ticking off the numbers, "we are probably reasonably close to the time zone of our original targeted drop area - we all awoke at about local sunrise, so our circadian rhythms are close to the local time. Although that may not be saying much - a time zone is a rather large section of the Earth, and we might have landed a good distance north or south of our expected drop area. Second, I would expect that currently, a lot of ships - both sea and air - will be searching for any survivors. However, if our planetary drop pod crashed a large distance from here, then searchers are likely to start at the crash site, and search in widening circles around it for drop lifeboats like this one. So we may indeed have to wait quite a while before rescue." Then Raphael looked at Michael. "Which brings me to the third thing, which is quite more interesting. From what Julia said, last night she pulled in Michael close to the boat. Now, we don't see any other lifeboats nearby, and there would be somewhere between fifteen to twenty of those, each capable of carrying four people. It's not possible to land from space without a vehicle with full drop capability, so Michael's original drop lifeboat landed within swimming distance of ours, and then had problems, not during the drop itself, where any problem is likely to be fatal, but rather upon -" "Hey!" said Michael, "let me give my story first before you -" "Point of order," said Nicole, "but I believe the rules we precommitted to require that only one person can speak at this stage, except for questions of clarification." "But he's trying to build a case that I'm the saboteur!" said Michael. "I'm not," said Raphael, "and you can give a full rebuttal of my points during your turn. Which is the point of this stage, anyway. We want to encourage people to fully share all the relevant knowledge they have; thus, anything that could discourage them from sharing is disallowed until the next stage." He smiled sheepishly, mostly at herself, Nicole realized. "Thank you, Judge. As Administrator I should have been the one to actually raise the point of order. I guess I don't have the actual skills yet, just the knowledge from my reading about Administration." "You're welcome," said Nicole. "Michael, you'll get your turn later. Please continue, Raphael." A part of Nicole sank a little as Michael glared at her momentarily, but some things were more important than - "Where was I?" said Raphael. "Ah, yes, the other lifeboats. Now, if the drop lifeboats dropped near each other, then we would expect either to see other lifeboats, or at least their debris - lifeboat parts, dead bodies. But I think that would also mean that the lifeboats themselves would also land relatively near the final crash site of the pod, which would mean that we should be expecting searchers to find us early. It's been what, eight hours, maybe more? Given current technology, that's a long time. Unless, that is, there are other factors slowing them down, such as sheer distance, or - deliberate slowing down by a larger organization than just a single saboteur or group of saboteurs on-board the pod." Raphael dropped his voice, obviously for full effect. "Which, in fact," he said, "makes Michael's being here very cogent. If Michael's lifeboat was indeed nearby 'naturally', meaning that we can expect the other lifeboats to be nearby also, meaning that the delayed rescue is because of an even larger organization preventing proper search. The fact that we see no other lifeboats may mean that the other lifeboats were never successfully ejected from the failing pod - and our survival here could also mean that this lifeboat was specifically chosen as the saboteur's escape." He continued. "But if Michael's lifeboat was a fluke, and just happened to be nearer to our position than on average, then we can relax - we haven't been rescued simply because of the huge distances involved." He paused, apparently thinking. "That's all I thought about." "Is it my turn?" said Nicole. "Wait," said Julia. "So, major ideas: we're in the same, or near the same, time zone as our original drop; we expect many searchers; we expect searchers to start near the actual crash zone; Michael got here, which suggests that sabotage occurred -" "Uhm," said Raphael, "that's not quite what I meant. My main idea is that there are two possibilities: lifeboats landed near each other, which suggests that the lifeboats are near the crash site, which suggests that, since we haven't been found yet, something else is preventing the searchers from searching effectively. The other possibility is that the lifeboats landed far from each other, which suggests that the lifeboats are also far from the crash site, and the reason why rescue is taking so long is because of the sheer distance. Michael being here suggests the 'near' condition, but the fact that we can see no other evidence for other lifeboats suggests the 'far' condition." "So how do I write that down in the notes?" said Julia. "Okay," said Raphael, "write that 'lifeboats near' implies 'lifeboats near crash site' implies 'slow rescue is due to external factors, probably sabotage'. Then write a separate bullet point saying 'lifeboats far' implies 'lifeboats far from crash site' implies 'slow rescue is due simply to distance'. Then write 'Michael is here' implies 'lifeboats near' condition, and then write the separate 'no visible lifeboats' implies 'lifeboats far'." When Julia had written that down, Nicole said, "My turn?" "Yes," said Raphael. "Okay," said Nicole, "I actually thought of the sedatives given to us before the drop, too. And those sedatives got me thinking about just how extensively the saboteur or saboteurs have to operate." She paused. "The main reason that sabotage is unlikely," said Nicole, "is the ubiquituous cameras, which also exist in the cabin, control bridge, and other areas of the drop pod, as per the international Sousveillance Law. The cameras can be viewed by anyone, as per Sousveillance, but I do remember one detail of the pre-drop safety presentation. During the actual drop, at some stage in the drop, the cameras would not be able to transmit to the general public due to the presence of heavily ionized plasma that would form under the pod, or something like that." She breathed in, and said, "So: during the drop, most of the passengers will be under sedation, and I expect the crew would focus more of their attention in technical matters of the drop. Nobody outside the pod can see the cameras in real-time, and a saboteur might arrange to have the camera's recordings get destroyed, or have the pod get destroyed in a manner that would believably cause the camera's recordings to be destroyed. No one knows of the sabotage, everyone thinks it's an accident. A perfect crime." Then Nicole looked at Raphael. "Raphael, however, brought up the possibility that a larger organization is actually pulling the strings of the saboteur. And the main benefit that an organization might get is to be able to claim that they managed to actually sabotage a planetary drop pod. So secrecy should not actually be one of the goals. It wouldn't be necessary to trigger the sabotage at a time when secrecy is possible; the saboteurs only needed to ensure that the passengers were all sedated so that no heroic amateur could have stopped them." She smiled. "So I suppose that I don't really have any good idea, and wasted my 15 minutes." "Oh," said Julia, "so should I strike them out?" "No!" said Raphael. "We need to discuss all the ideas presented first, before we strike any of the out; and we cannot discuss until all members have actually spoken in the first stage." "Okay," said Julia. "So: cameras make sabotage unlikely; cameras cannot transmit during drop; saboteur could commit perfect crime during cameras-off condition; organizatons behind sabotage might not want perfect crime. Is that all, Judge?" "Yes," said Nicole. "So who is next? You or Michael? I'm not sure which of you should go next since both of you are at my left." "Let me," said Michael. "Please." "Okay," said Nicole. "Your turn." "First things first," said Michael, "What happened on my lifeboat was that apparently, the gas injectors failed. During re-entry inside the drop lifeboats, we are protected by simple air bags that are supposed to fill up just prior to ejection of the lifeboats. There were three of us in my lifeboat, but after my air bags filled, theirs only partly filled. And then our lifeboat ejected." He looked at everyone, particularly hard at Raphael. "By the time we splashed down, the necks - and other parts - of my companions were broken. I was lucky; the air bags protecting me had filled enough to just leave me bruised. On splashdown, from what I observe on your lifeboat, the capsule part is supposed to open up and the additional air bladders are supposed to fill up so that the lifeboat floats." He paused. "It didn't happen on my boat. I manually opened the capsule, since the entire lifeboat was sinking. I managed to reach the surface, swam around screaming, until I heard someone shout back and swam towards the sound. It was Julia, of course." And then - "But you don't have to believe that," Michael said. "The problem we're facing now is that we are engaged in a game of Paranoid Debate. We don't know if one of us is a deceiver, and we don't know which one, if there is indeed one. Or maybe more than one, for that matter. And I don't think anyone of you has considered that fact deeply enough..." Chapter 23 "I vaguely remember, Judge," said Verrens, "about Paranoid Debate; isn't it a parlor game of some kind?" "Yes," said Adams. "Although it is one that has been designed specifically as a tool for teaching rationalists how to detect deception. Basically, a game master prepares a special deck of cards, a few of which are red cards and the rest being black. The players are dealt a single card at random, and are instructed not to reveal it. Then a question is asked, usually one with a numerical answer. The players must then debate and the entire group should come up with a single consensus answer." "Ah, yes," said Verrens. "Now I remember. The interesting part of the game isn't the discussion itself, it's how the individual players are scored." "Correct," said Adams. "Players who were dealt black cards are scored higher if the group consensus answer is near the actual answer. However, players who were dealt red cards are scored higher the further away the group consensus answer is from the correct answer -" "- meaning that the players holding red cards should act as deceivers," said Verrens. "But they must also strive not to be detected; if someone suspects them of being deceivers, what they say can be discounted and ignored." "That is not the only twist," said Adams. "In the case where more than one red card is in the deck, then there is the problem of the deceivers trying to cooperate without revealing themselves. Or to make it clearer: consider a Paranoid Debate game where multiple deceivers exist. Focus on one of the deceivers trying to twist the consensus away from accuracy. If another player gives a piece of evidence that the deceiver knows is likely to be false, the deceiver cannot know for certain that the other player is a fellow deceiver or an honest player who had incomplete or erroneous information. If that player subsequently gives information whose accuracy the deceiver truly does not have any knowledge about, the deceiver cannot simply blindly support the other player, since that player might not actually be a fellow deceiver." "I see," said Verrens. "So the deceivers in such a multiple-deceivers game would also experience some of the uncertainty that the honest players would feel when hearing someone else give information that they know is incorrect - uncertainty over whether that person is the deceiver or just an honest player that had an inaccurate map." "Hence the allure of the multiple-deceivers variant," said Adams. "But that is not all. The most complex variant involves not just multiple deceivers, but adds uncertainty over the number of deceivers. The game master prepares a deck whose total cards outnumber the number of total game players by the number of red cards in the deck. Thus there is a small possibility that none of the players actually hold any of the red cards." Verrens was silent; his expression suggested that he was extremely impressed over the variants of the game. "That is the situation, I think," said Adams, "that our characters now find themselves in: a Paranoid Debate game where the number of deceivers itself is unknown. And the question of the game is: how many people are holding red cards?" The two Judges considered this for a while. "Shall we continue?" said Adams. "Wait," said Verrens. "I've thought of it just now, Lord Judge, but this discussion of the Paranoid Debate game makes me more appreciative of something I once read in a general Science interest magazine. Specifically, a theory developed and studied long ago by the Evolution Scientists: the Machiavellian Intelligence Theory." He paused to gather this thoughts. "Of the higher apes, it seems that only homo sapiens was able to achieve a runaway intelligence explosion. Other similar apes - pan, gorilla, and pongo - were not able to achieve the homo level of intelligence." "Ah, the Machiavellian Intelligence Theory," said Adams. "Or Hypothesis when I first heard of it. Homo band in groups for mutual protection, and in the modern world, for trade in an era of specialization. Within the group, selection must occur between individuals, since all members of the group gain the same group benefits. And one mutation in the early homo line allows some individual to gain an advantage, not by being stronger or faster - not by manipulating the low-order ecological environment - but by manipulating the higher-order social environment. And this eventually lead to the spread of genes encouraging the growth of the social cores of the brain - social cores to not only manipulate the social environment, but also to detect attempts at manipulation. And it is the more generalized social cores of the brain that are the basis of modern humanity's intelligence." Verrens chuckled. "I should have remembered that you claim to be amateur Scientist, Lord Judge David Adams," he said. "Anyway, manipulation of the social environment is generally achieved by deception - or in other words, lying. This requires a larger brain, one capable of storing both the true information, and the invented false one; to be able to keep track of which is which; and finally, to extend the false information when more evidence supporting it is demanded. The application of this theory to our character's situation means that they - or rather, those who are honest players in the game - need to exercise their brains fully to root out the red cards - if any." "Yes," said Adams. He appeared to think for a moment. "One of the most powerful techniques for deception is self-deception. Brains are tuned very well for detecting attempts at deliberate lie-telling; a consequence of honest signalling, which prevents too many successful deceivers from imitating signals of honesty, which would deteriorate the signal and cause everyone else to switch to some other signal for honesty. One way of subverting this is for the brain to actually act as if it truly believes the lie, to the point that in all social interactions, the individual acts as honestly as possible. But in private, where no one is looking, the individual - subconsciously, since even the conscious mind is subverted to prevent it from spilling the secret - acts as dishonestly as possible. Thus the abstractions of country that an individual would loudly proclaim their pride in would suffer from those individual's private actions, even as their consciousness congratulated itself over its patriotism." "I'm not sure I follow, Lord Judge," said Verrens. "To give a more concrete example," said Adams, "one of my ancestors grew up in a country that, at that time, was poor, had tremendous corruption, and was under the grip of a powerful memecomplex. In his maturity, he went abroad to work in one of the more affluent countries of the period - in fact, my family's home lies within the geographic borders of his adopted country, which no longer exists now. He constantly proclaimed his original nationality; his internet handles included his nationality; he socialized with other people who grew up in the old country but now lived and worked in his adopted country; and yet, he never actually returned to his birth country." Adams looked grimly, remembering his research into his own past. "He gave gifts to his relatives, but in terms of large-scale economics the value of what he sent back was nowhere near the value of what was spent on him in order to raise him up to adulthood. His children never truly identified with his birth country, even though his primary partner was from the same birth country. His grandchildren never even learned its name. His story was not atypical. When the Helvetian War started, his birth country was one of the first whose financial systems collapsed, due in no small part to people like him, who were numerous." "I see," said Verrens. "Lord Judge, your honorable ancestor -" "- was not honorable," said Adams. "Although I can forgive him. His self-deception blinded him. Today our rationality training prevents such self-deception, which is a good thing. I'm sorry, please continue." "Your ancestor, then," said Verrens. "Your ancestor was not a rationalist, I suppose, as the Helvetian War was triggered by the early rise of the Senseis. Thank you, Lord Judge; it serves as a useful concrete example of self-deception. His social interactions showed him as a patriot, but his actual private actions - the decision to remain in his adopted country and raise his children there, the decision to give only the simplest social gifts that would not have significant economic effect to his birth country - belie his actual self, which was not patriotic." "Yes," said Adams. "A true patriot would hardly need to proclaim aloud his or her patriotism, but simply directly work, for the betterment of his or her country." "Judge," said Verrens, "you are amateur Sensei as well, as you mentioned before. Rationality training would help against self-deception, would it not?" "My young Judge," said Adams, "self-deception was one of the first problems that the early Senseis tackled; they considered it the most important problem of their time. I mentioned that my dishonorable ancestor's birth country was in the grip of a powerful memecomplex. At the time of the rise of the Senseis, that memecomplex, and many other very similar ones, were no longer actually believed by many individuals, but those memecomplexes acquired, as protection, powerful component ideas that were intended to actively trigger self-deception. People professed belief because disbelief was labelled evil, even though a moment's thought and some research would have revealed non-evil disbelievers. Post-modernist ideas were welcomed into the memecomplexes to prevent Science from dissecting them, and to prevent rival memecomplexes from competing against each other, detrimental against their shared struggle for survival against the brightening light of Science. Today's surviving memecomplexes are significantly smaller and much more benign, even beneficial, than the ones that the early Senseis had to destroy. Techniques against self-deception were some of the earliest and most powerful weapons developed, because they were needed in that fight." "Judge Adams," said Verrens, "could you tell me more about the rise of the Senseis?" "Yes," said Adams, "Let me tell you what happened -" Chapter 24 "Question of clarification," said Kate, "I don't see exactly how the, uh, Notional Group Technique is like a game of Paranoid Debate. Can you explain?" "Nominal Group Technique," Raphael muttered. "It's not the technique itself that -" he started to say, and then - "Executor Thompson? If you please?" "As Administrator Yu was about to say," said Michael, "before he remembered the rules he himself told us, it's not the technique itself which is the Paranoid Debate game. It's the fact that one of us may be a deceiver, and thus would deliberately try to lower the group's estimate on the possibility of sabotage." "And," said Nicole, her mind racing, "even if none of us is the saboteur, we would -" "Point of order, Judge Angel," said Raphael - "We should discuss this, Raphael," said Katja. "If Michael is correct, then the disputation arena's decision could be contaminated -" "Wait, wait," said Julia, "I have to tell you all first -" Everybody started talking. "And if we all discuss this now," said Raphael imperiously, "then we might as well freaking, freaking -" his face flushed red as he grew angry - "we might as well just freaking brainstorm!" The rest of their tiny community stammered into silence, Julia last. "Yes, I said it!" Raphael shouted at them. "'Brainstorm', I said!" "There's no need for that sort of language, Administrator," said Nicole - "Yes there is!" said Raphael. He seemed more calm now. "Discussing important matters in an ad-hoc, high-speed, high-pressure manner was precisely the original meaning of that word." He sat straight, leaning forward slightly to radiate dominance - a move that Nicole recognized, from part of her own training. "And the first Sensei-taught Administrators chose to make that word into the pejorative term for forms of discussion that promote groupthink, reduce effective group rationality, and turn into large-scale dominance contests rather than efforts at compromise and truth-finding. Today we no longer use the term for its original meaning, because Administrators simply never allow discussions to devolve into that kind of condition. Only its pejorative meaning remains." He look at each of them. In the early twenty-first century, most proto-rational groups had a problem with a particular facet of Humanity's evolved brain: emotion. Many thought that rationality required the total expunging of emotion from all thought. But the Senseis considered it absurd. If a civilization of barbarians were to attack a civilization of rationalists, the Senseis reasoned, then if the rationalists lost because their people could not defeat the raw savage emotional intensity of the barbarians, they weren't really very rational after all. Rationality - instrumental rationality - was for winning; emotions were a tool that could be wielded to exert rationally-controlled social force (and, thereby, could be used to win). What was needed was not the extirpation of emotion from rational life, but control over its expression. Rationalists should win over barbarians, because they could control their emotions, whip it to just the right level of intensity to maximize the benefits without losing the plusses to the inevitable drawbacks. Nicole was impressed; Raphael had used his outburst almost like a surgically precise scalpel. Nicole spoke. "I'm sorry for triggering the earlier, uh, 'brainstorm', Administrator," she said. "I suppose I can understand why the term was invented." "Point of order, Judge," said Raphael. "Not a question of clarification." Then he closed his eyes momentarily. "Thank you anyway, Judge. I suppose your moral expertise suggests that an apology right now is more important. Now, can we return to the discussion, Michael?" "Not a very good Administrator, are you, Raphael?" said Michael. "Michael -" said Julia - "I'll handle this, Engineer," said Raphael. "Point of order, Executor, no challenges at this stage. Please speak of the ideas you have regarding the problem at hand: whether a saboteur is among the members of our community, how to determine if it is so, and what we can do about it." "Well then," said Michael. "I hold that Raphael attempting to swim away means that he is the saboteur." Many questions and counters to that idea filled Nicole's mind. However, she had precommitted to a method, specified by an Administrator, on how to present counter-ideas. She could understand now why Raphael had commented that ideally, pen and paper for all members of the discussion was really needed. Wait, asking for clarification was allowed at this stage according to the rules she had precommitted to - "Question of clarification," someone said, before Nicole could. It was Raphael. "Can you elaborate more on this thought?" "Certainly, Raphael," said Michael. "Raphael swam away, not as a result of being insulted by Julia, but because he needed to be found by his organization - whatever it is - for rescue. His larger organization, as he himself said, is slowing down rescue so that he can be picked up earlier, and possibly so that the rest of us can be killed too." Michael said nothing more, so - "Is that all, Executor?" said Raphael. "Yes," said Michael. "Okay," said Julia. "Michael's reported observation on his lifeboat: air bags failed, lifeboat floats failed; we are playing Paranoid Debate; Raphael swimming away is evidence for being saboteur." Julia licked her lips. "Is it my turn now?" she said. "Yes," said Michael, "your turn, Julia. Can you handle it?" "Yes," Julia said. She breathed in heavily, eyes shut. And then - "I like the Sousveillance Law," she said, eyes still shut. "Especially when cameras are pointed at machines. Last night, I had my personal computing device tuned to several different cameras that were monitoring the control bridge. I had my persocom magnify the image to focus on the gauges and indicators that the crew was monitoring. Not really good enough to see the values on the indicators, but good enough..." She opened her eyes, then, "The sedatives don't work perfectly with me, you see", she said. "They don't pull me to sleep fast enough before the drop actually starts, and afterwards, when the pod is in port, I'm still very groggy and just half-awake. So I got only the minimum dose that the crew would give me, and set up the persocom in front of me, with the bridge indicators in view, to help me relax. It's happened to me before, so that's what I do to cope." She paused. "Just as I was drifting to sleep, before the start of the drop," she said, "the signal from the cameras failed. That never happened before. Like Nicole said, the plasma ionization during atmospheric re-entry will cut off the camera's transmission to the general public, which is something that is unavoidable and hence exempt from Sousveillance Law, but as a good-faith guarantee the cameras should still transmit locally on board the pod itself." There were nods indicating understanding around the boat. Since Julia had somehow suggested the possibility of sabotage to Katja earlier, her evidence was very interesting to the rest of the community. "That the cameras were shut off was not a likely event. I think it suggests sabotage. However -" she paused - "one detail most people are unaware of is that generally, the cameras transmit all the time, as long as they have power. And I have no reason to expect that the cameras on board the drop pod were an exception. This means that it's likely that the cameras, in a normal drop, would still be transmitting in real-time to the general public even during the drop itself; it's just that we expect the ionization during re-entry to effectively block the transmission. If so, then it might make sense to knock out the sousveillance cameras." She looked at all of them. "I'm an Engineer, and I am an expert - a fangirl, really - of applied Science. One recent demonstration was a receiver that could pick up and reconstruct electromagnetic signals transmitted through shielding. Since the ionization during re-entry works similarly to shielding, it seems likely to me that the saboteurs shut off the sousveillance cameras just in case a specialized reconstructing receiver, like in that demo, could receive the transmitted image. Defense in depth, as we Engineers would call it." She was silent, but still looked as if she had something to say, so Raphael prompted her. "Is that all, Engineer Montega?" he said. "No, no, not at all," she stammered. "It's just that what Raphael said earlier reminded me - the lifeboats are organized by pairs. And, and - I saw Michael enter the lifeboat that was paired with ours." She was blushing at this point, Nicole noticed, and then Nicole realized that her own cheeks, and even her ears, were warm, she hadn't noticed Michael during the emergency evacuation - anyway (she forced herself to pay attention), Julia continued, "The pod has an effectively lower coefficient of air friction during the drop, due to its larger mass versus surface area, than the lifeboats. The effect is that when the lifeboats are ejected, the pod would travel further than the lifeboats, despite inertia and Newtonian physics. So we might expect the lifeboats to be nearby, but the pod itself to be somewhat further away." Raphael shrugged ruefully at this. "A-anyway," Julia said, "I'm not sure, but I would expect that pairs of lifeboats would be ejected simultaneously. So Michael's lifeboat being nearer than the other lifeboats would be plausible. I'm not so certain about how much distance to the other lifeboats we can expect. Depending on the timing, we might expect them to be within swimming distance, or much more distant." Julia scribbled in her notepad; then she said, "I left my persocom behind. Lots of my personal work and hobbies in there. I have backups but it won't have the ideas I thought of during the last trip. And my family will be worried. I think there was sabotage, but I'm not sure. Not really. Because a failure at a critical point can cause cascades of failures. We design machines with defense in depth - multiple layers of protection, redundant systems, etc. - but we know that sometimes, just sometimes, the universe is out to get at a machine. Sometimes the cost to add more protection just can't be justified. So the accountants would say - and they'd be right, Administrator. Life is a risk, and we gamble with it. Most of the time the gamble pays off. Sometimes it doesn't. And I'm babbling, I'm sorry, so I'll just cut it short and point out that a strong-enough power surge can shut off the power supply of the cameras, cause damage to the pod, and possibly cause various failures in other equipment - like the lifeboats. The source of the power surge can be some natural fluke, some accident or oversight. Or it could be sabotage - a very simple sabotage, at that." She raised up her notepad, and said, "Main ideas: Julia's reported observation is that the cameras stopped transmitting locally inside the pod before the accident occurred; cameras shutting down during drop suggests a well-prepared sabotage; Julia's reported observation is that the lifeboats are organized by pairs; Michael's boat and this boat being near to each other is not likely to be a fluke; we can naturally expect the lifeboats to be far from the pod, but near to each other, because of differences in frictional coefficients due to differences in size." Julia sighed loudly. "That's all I've got. I'm really sorry for starting this; right now, I'm not actually sure there was sabotage at all -" "It's my turn now?" said Katja. "Because personally I think there was indeed sabotage." Chapter 25 "Katja seems quite insistent," said Verrens, "even though Julia appears to now express a lower probability for sabotage than she is implied to have expressed to Katja." Adams simply nodded. "Yes," he said, "and, in my considered opinion, this part of the novel continues to be a setup for the actual decision that the main character will have to decide later. Assuming that my friend the plot designer continues to prefer the same pattern, that is. But as for the decision itself - well, as of now only the author would be able to speak of the actual decision that will be taken." That appeared to be all they would discuss for the chapter, so after a short while Verrens said, "Shall we continue, Judge?" "Wait," said Adams, hand raised. "An interesting thought has just occurred to me - if you don't mind a garrulous old Judge delaying you." "Please continue, Judge," said Verrens. "The basic setup of the plot," said Adams, "strikes me as similar to the style of investigative novels. In an investigative novel, after showing a vague, ambiguous view of some crime - in our case, the falling, malfunctioning pod - and introducing the main character, the main character is then somehow forced or hired to investigate who committed the crime, and for what reason. Then the potential perpetrators and witnesses are called upon to present their evidence - the transcript of the Nominal Group Technique disputation arena we are reading through, in this case." Verrens nodded. "I follow your reasoning, Judge," he said. "And there I suppose the similarity ends," said Adams. "In a classic investigative novel, the main character, and the main character alone, is largely responsible for most of the insights that lead to identifying the culprit." "And," said Verrens, "since you bring it up, I think you thus suspect that -" "- in this novel, the main character will not be the only one to have the necessary insights in identifying the culprit," said Adams. "Yes, that is what I think will happen. And in our case, there is also the question of identifying whether or not an actual crime was committed." "Even though an ending revealing a non-crime would not help the flow of the novel itself, being too iconoclastic of the genre," said Verrens. "Again, Judge Verrens," said Adams, "remember that this is first and foremost a novel written about a group of modern rationalists, a novel striving to be as realistic and true as possible. There are no invented Sciences involved, not even the Sociological or Neurological Sciences - which, as an amateur Sensei and an amateur Scientist, and as a Full Citizen with the basic rationality training, and as a Judge with the intermediate rationality training, I can assure you of. And one important aspect of modern rationality is the concept of 'the Craft and the Community'." Adams shifted in his seat, preparing for another lecture. "A rationalist, applying the techniques of rationality as well as he or she can, will be able to bring to bear a large amount of brainpower to any problem. But the central insight that fed the early rise of the Senseis, as I pointed out to you before, is that a motivated group of rationalists, even as they individually strive to become better thinkers and individually retain a sense that more is always possible, can as a group bring to bear even more brainpower to any problem than any single rationalist can. Hence the rise of the Senseis, plural, not the rise of the Sensei, singular. Provided, of course, that the rationalists remember to add the important ingredients: a minimum amount of cooperation, clear communications, a goal, and a structured disputation arena that avoids groupthink, without which the stew of rationality becomes a poisonous, self-defeating mob." Verrens considered this for a moment, then said, "Ingredients which, we hope, the Administrator, Raphael Yu, can provide in the case of our characters." "Yes," said Adams. "And this, I suspect, is the difference between this novel, and similar investigative novels: the tiny community of people, all with at least minimum rationality training, and two - the Judge and presumably, the Executor - with the intermediate level of rationality training. This community of rationalists will, as a group, provide the insight necessary to discover the truth, and not, as in a typical investigative novel, only the privileged main character." "Even if," said Verrens, "it is possible that one of them is a saboteur that is attempting to subvert the thinking of the group?" This brought Adams to a stop. Or rather, it appeared to bring Adams to a stop, because suddenly Adams grinned. "In much the same way," said Adams, "that a rational brain can defeat self-deception, even as one of its modules attempts to subvert its thinking, eh, Judge Verrens?" That brought Verrens to an actual stop, and he simply shrugged ruefully. "The Fallacy of Gray," said Adams. "Shall we review it?" Verrens sighed. "The Fallacy of Gray," he said, "is the error of thinking that, because a thing cannot claim to have absolute certainty, it must be classified with other things of low certainty -" "Not quite," said Adams, "although I suppose that it is the definition given in the intermediate levels. The preferred formulation in the advanced levels of rationality training, as I found in my amateur Sensei studies, is that the Fallacy of Gray is the willful confusion of certainty with truth, such that any hint of uncertainty is considered as direct proof - proof, not even evidence, but incontrovertible proof - of non-truth. But noise exists, and certainty is never possible. There are grays and then there are grays - some grays are so light that in practice they are white, and some grays are so dark that in practice they are black. The Fallacy of Gray is replacing a simplistic black-and-white view of the universe with an even more ridiculously simplistic monocolor view of the universe." "Thank you, Judge," said Verrens, "for pointing out a mistake in my map." "You're welcome," said Adams. "And now let us, for the Community, read on!" Chapter 26 "After Julia told me," said Katja, "or rather, mumbled vaguely about it while flailing around last night in the boat checking for something, I remembered a minor but strange detail about the sedatives we were given, and the wakening gas that brought us up into consciousness prior to the evacuation." Katja paused, gauging the interest of her companions. "Normally," she said, "a specific sedative is paired with a specific wakening gas. During Medic training, I and a group of friends experimented with various combinations of sedatives and wakening gases. It's actually one of the things that was taught to us, but we preferred to confirm it for ourselves. Anyway, basically: using the wrong wakening gas can still stimulate you to wakefulness, but you are, crucially enough, slower, much less alert, and much more likely to make mistakes than if you used the correct gas." The charge was serious; the laws against mental incapacitation were strict in most jurisdictions, Nicole knew. "As I mentioned," Katja said, "I had experimented with such drugs during training - it's not exactly discouraged, most Medic trainers want you to experience them as much as feasible so that you get better empathy for patient experiences - and I can identify the different wakening gases by smell. I didn't note it immediately then - probably because the wakening gas itself didn't wake me up enough - but I remembered it later when Julia reminded me with her mutterings about sabotage." She breathed in, perhaps to clear her mind of the remembered smell, and then she said, "Consider that while we can probably expect the drop pod's bridge crew to not be sedated, the cabin crew also sedate themselves after offering the sedatives to the passengers. And it's the cabin crew that operates the lifeboats, isn't it, Julia?" Julia stammered, looking at Raphael. "It's a question of clarification," said Raphael. "You must speak to answer it, even out of turn, Julia. You may also give a short explanation provided it is solely on topic to the question." "Yes," said Julia, "that's what I would expect. The cabin crew has a different lifeboat that can self-eject, unlike the others that are ejected externally, and is larger than this one." "And as I mentioned," said Katja, "the wrong wakening gas can cause people to make mistakes. Question of clarification, Julia: if the cabin crew makes a mistake in the ejection sequence, could it cause the drop lifeboat to fail to fill the air bags properly, like in Michael's lifeboat?" Julia shook her head. "Not really," said Julia. "The ejection sequence should be entirely automated. Although - if the automation fails, I would expect that it would be possible to perform the sequence manually, and that the cabin crew would be trained to perform the sequence manually if necessary. So yes, it's possible, in my expert opinion. Assuming that the automation fails, and the sequence was done manually. A classic failure cascade." "Good," said Katja. "I rest my case. There was sabotage. And the saboteur is on this boat, at least originally, so Michael can't be the saboteur. It's one of you three - Raphael, Nicole, or Julia." "Question of clarification," said Nicole angrily, "why do you exempt yourself?" "Because I'm not the saboteur!" said Katja. "But you were originally on this boat -" "Point of order," said Raphael. "Judge, no challenges at this point. You can challenge it later during the synthesis stage of the Nominal Group Technique. Let the main idea stand as stated by Katja; the group as a whole can decide to modify it later." "Anyway," said Katja, "I'm done." "Question of clarification," said Raphael. "How did the saboteur ensure that this boat would be safe from a mistake in the ejection sequence?" "Simple," said Katja, "..." She stopped. "Okay," said Katja, "you almost got me there. But I'm on to you, accountant. The sedatives are optional, although practically everyone takes it. The crewperson who operated this lifeboat didn't take the sedative, and his or her thinking was not impaired." "That's speculation," said Raphael, "and -" he waved his hand in front of his face - "I'm sorry, that was a challenge and not allowed at this stage. Sorry. I'll save it for later. Julia, write down the addition as another main idea. Does anyone else have any more questions of clarification?" Nobody spoke up to ask more questions, so Raphael nodded to Julia - "Main ideas," said Julia, "Katja's reported observation is that the wrong wakening gas was used; wrong wakening gas causes mistakes; mistakes might cause failures of lifeboat ejection; one of original members on this boat is saboteur, one of Raphael, Nicole, or Julia; crewperson who ejected this lifeboat is another saboteur. Is that okay?" "Thank you, Engineer," said Raphael. "Now, let us proceed to the next stage: synthesis. It's the centripetal stage of the Nominal Group Technique. In this stage, we can each give our reactions and reflections on each of the main ideas generated and shared in the previous stage. We can also move to have an idea modified, or to be copied and then modified, or to merge two or more ideas together, with the source ideas either retained or removed. Or the reverse operation, split an idea into two or more ideas, with the source idea retained or removed. Ideas cannot be removed unless they are modified, merged, or split. That's important: ideas cannot be deleted. Modification and merging of ideas requires at least three of us five to agree to it without either of the remaining two objecting; or at least four of us to agree if the remaining one objects. I'd like to make the discussion organized, although... give me a moment, okay?" They let Raphael ponder for a while - "Katja," said Raphael, "Medic, can we borrow the bottle of shark repellent?" "Why?" said Katja, genuinely puzzled. "To use as the token," said Raphael. "The token means that whoever holds the token is the only one who can speak, in order to give comments and reflections, and in order to move for ideas to be modified, merged, or split. The bottle will float in the water?" "Yes," said Katja, "I would assume so." She took out the bottle, still puzzled, and experimentally dropped it into the water. "Yes, it floats." "Good," said Raphael, fishing it out of the water, "so in case we mishandle it, I or the Executor can fetch it easily. I was thinking of using Julia's notepad or pen, but we can't get the notepad wet and I doubt the pen will float. Okay, rules for the token: only the current holder of the token can give reflections or reactions to the ideas, and initiate moves for modification or merging. After the holder has run out of things to say for the moment, he or she may offer it to anyone who asks for it. If multiple people ask for it, the holder chooses who to give it to. However, as Administrator, I hold the right to override the choice if I observe that someone is consistently being denied access to the token -" "Wouldn't you be able to abuse that?" said Michael. "You could always override who we choose, to give you an unfair advantage at airing only your own ideas." "- as I was about to say, Executor," said Raphael, "I have the right to override the destination of the token, unless two of you object, or, if one supports me, unless three of you object. In that case, the destination chosen by the current holder is not overridden. Democracy wins, okay?" Michael shrugged, rocking their boat slightly. "Also, when the holder moves to modify, merge, or split ideas, the token rule still holds, but after proposing, the holder should pass the token around so that others can voice arguments for or against, promise to vote particular ways if the exact wording is changed in particular ways, etcetera. However during this time the topic is restricted to the motion being proposed, and after voting the token reverts to the proposer. Got that?" "That," said Julia, "sounds complicated." Raphael shrugged. "True," he said, "but it keeps the discussion from becoming a chaotic shouting match. Is that okay to everyone?" There were shrugs and nods. "Now," said Raphael, "the synthesis stage ends when the holder of the token has nothing further to say, and no one else asks for the token. Then the ranking stage begins. Does everyone agree to the procedure?" There were nods of assent. "Okay," said Raphael, "let me begin -" Nicole raised her hand, and said, "Before the next stage, everyone, first I'd like us all to recite the three litanies. If that is all right, Administrator?" Raphael blinked, and said, "If your moral expertise suggests it, please lead us in the litanies, Madam Judge Angel." She closed her eyes, breathed deeply, to help enter a rationalist trance state. The others would do the same, she knew. "Litany of Tarski," said Nicole, "repeat after me: If there is a saboteur, and he or she is among us, then I desire to believe that there is a saboteur, and he or she is among us. If there is no saboteur, or he or she is not among us, then I desire to believe that there is no saboteur, or he or she is not among us. Let me not become attached to beliefs I do not desire." She intoned, again, hearing the rest of the community repeat the words they knew by heart: "Litany of Gendlin: What is true is already so; owning up to it does not make it worse. Not being open about it does not make it go away; and because it is true, it is there to be lived. Anything that is untrue isn't there to be lived. I can stand what is true, for I am already enduring it." And finally: "Litany of Occam: If a possibility is simple, then I shall consider it more. If a possibility is complex, then I shall consider it less. The truth is usually simple; but occasionally it is complex. The truth is sometimes complex; but often it is simple. My judgment of simplicity or complexity may be wrong: 'the lady down the street is a witch, she did it' is not simple." There was a moment of reverent silence. Chapter 27 Verrens eyes were welling as he finished the chapter. Fortunately for him, Adams was not yet finished with the chapter; Verrens struggled to keep his composure. "Huh," said Adams as he lay down the papers of the chapter. "The litanies. I don't see them appear often in these novels. Honestly, I approve of them, although some people, I think, get too attached to them." "Ah -" was all Verrens could say. Adams frowned. "I presume from your expression," he said, "that you are one of them. Although I can say I understand; the instinct of numinosity was harnessed in the formulation of these litanies in the first place. The litanies, as I gather from my amateur Sensei research, were a vital component in the initial rise of the Senseis, without which many of the early Senseis would not have been able to gather with like-minded rationality experts. But the art involved in its writing is very slightly Dark; it triggers a well of numinosity within each human brain. Well, then, let me rephrase it: the litanies are not to my taste and I have no real need of them, but I suppose many, many more find them useful and so keep them to heart. So I suppose I can understand why the author included them." "Yes, Judge," said Verrens. "Anyway," said Adams, "let us move on. It appears that our amateur Administrator, Raphael Yu, is able to keep some semblance of order, and hopefully his procedure can allow clear communications." "Yes," said Verrens. "Although we have only been told a description of the procedure, and not how well it works in their situation." "Agreed," said Adams. "Although knowing the designer of the plot, I suspect the procedure is at least an old standard among Administrators." "So, Judge," said Verrens, "you think that the disputation arena will be successful?" "Not only that," said Adams, "I would bet also that the disputation arena will conclude that there is no saboteur, and that the disputation arena will be only a setup for another test for aspiring Judge Nicole Angel. Shall we read and see for ourselves?" Chapter 28 "The first and most important thing I want to mention," said Raphael, "is that I move that Katja's idea 'one of Raphael, Nicole, or Julia is the saboteur' should also include Katja's name." "I agree," said Julia firmly. "I object!" said Katja, "I was the one who -" "Point of order," said Raphael. "Please, let's keep things organized. Let me finish why I think the proposal should pass, then we pass around the token to discuss. The token will pass to you, Julia and Katja, so please be patient." He kept looking at Katja until she dropped her gaze and shrugged. "I think," said Raphael, "that strictly from what Katja said, there is no real way to rule out the possibility of her involvement. The idea, by itself, also suggests that there is only one saboteur; I would also like the wording to be modified, so a better wording would be: 'saboteur or saboteurs are among original members of lifeboat: Katja, Raphael, Nicole, and Julia.'" He proffered the bottle to Katja. "Katja," he said, "shall you take your turn, or would anyone else like to have the token?" Julia and Nicole raised their hands, but Katja grabbed the token. "If I were the saboteur," she said, "why would I bring up that possibility? I wouldn't do something that would cast suspicion on me, because if I did, you would no longer believe my statements. So I can't be the saboteur!" Nicole and Raphael raised their hands. But Katja didn't pass the token to either of them. "What's the point?" she said. "The saboteur is one of you three. And of course the saboteur will try to confuse the disputation arena. So I don't stand a chance anyway." She tossed the bottle into the middle of the boat. "Point of order, Medic," said Raphael, "the current topic is about the motion to modify the idea's wording and include you among the possible saboteur candidates. Even if you have the token -" "Oh, so you want to shut me up?" said Katja. "Stop me from talking? I think that proves that you're the saboteur." "Katja," said Raphael, "I was speaking in the capacity as Administrator; I was simply repeating and clarifying the rules of the procedure that -" "Excuse me, Administrator," said Nicole. "This is now a moral transgression and thus my field. Katja, you precommitted to a procedure specified by an Administrator -" "Accountant!" Katja bit off. "Administrator," said Nicole. "And if you renege on your precommitment to the community, the community will be forced to apply a punishment on you. We don't really have many resources with which to punish you softly, so the only punishments we might be able to apply will be the more extreme punishments - exile, for example. So, Katja, please -" "Are you threatening me?" said Katja. "Yes," said Nicole. "Do you want me to pass Judgment on your case? Continue to do this, Medic, and my Judgment will go against you. Remember: you also earlier promised to give Raphael a chance at proving that he is not, in fact, a jerk at all times. I think his attempts at organizing our disputation arena and keeping our communications sane, in a capacity that he is not formally expert in but which he has professed an amateur fascination for, show him in a good light." Katja's defiance seemed sure for a while, but then it wavered. She bowed her head in submission. "I'm sorry. I'm sorry, Raphael. I should have called you by your proper vocation: Administrator." "It's not," said Raphael. "I am an accountant, that's my registered formal vocation. Although you obviously used it as an insult, well, I'm not insulted. I'm an accountant, and that's fine, I'll still be an accountant after we are rescued and I still won't be an Administrator, at least not until I get the full training and the works. For now, I am Administrator because of circumstance." He winced a little. "You know, now I'm even more sorry for swimming off because Julia called out the uselessness of accountancy in an extreme situation. It's still useful outside of this situation, it's a high-paying if boring vocation. And at least being bored with it got me interested into Administration, which is a little useful now, right?" He smiled, and pointed at the token. "Medic, if you please?" Katja picked up the token, then said, "So who wants it next?" Again Raphael and Nicole raised their hands; Katja passed it to Nicole. "Katja," said Nicole, "exempting yourself tightens your accusation against the three of us. Protesting that you are not the saboteur is fine, but it is meaningless in our case: of course the saboteur, or saboteurs as Raphael pointed out, would protest their innocence just as loudly. And a saboteur may very well also point out this possibility in order to reduce suspicion on himself or herself. So no, your word is not enough. It's one thing to be Michael and not be in this boat in the first place. It's another thing to be here and bring up a possibility that someone on the boat is the saboteur, but then exempt yourself from the suspects." She raised the token and asked, "So who wants it next? Raphael?" Raphael shook his head. "You already said it, Judge." Nobody raised a hand to ask for the token, so Raphael said, "Okay, now we vote. First, any objections? Katja? You withdraw your objection? Okay, Julia? Michael? Nicole? No objections, so three votes 'for' will suffice. Raise your hands to vote 'for' - unanimous except for Katja. Proposal passes, please modify the idea, Julia." While Julia crossed out the idea and rewrote the modified form, Raphael picked up the token again. "Okay," he said. "The next thing I want to mention is about the issue of Paranoid Debate. I agree: yes in fact we are playing Paranoid Debate. There are two important things to remember about this." He raised two fingers. "One is that, if there is at least one saboteur, they will try to twist our expected result towards the result 'there is no saboteur.' This holds no matter how many saboteurs there are." He ticked off the second finger. "The other is that, knowing the first fact, non-saboteurs will be tempted to present information that supports the hypothesis 'there is at least one saboteur,' while suppressing information that supports the hypothesis 'there is no saboteur,' in order to attempt to signal that they are non-saboteurs. The saboteurs will attempt to imitate this as protective camouflage, but try to arrange information to point to someone they think is not a saboteur - assuming that the saboteurs, if there are more than one, are trying to cooperate with each other. If they are not cooperating, well, they will still attempt to present information that points to someone else anyway." He made a fist. "So I think that we should be suspicious at attempts to fix the identity of the saboteur. Instead, we should first establish if there is at least one saboteur on this boat at all: a simple 'yes there is at least one' or 'no there are no saboteurs on this boat at all.' Because then the choices are clear-cut: actual saboteurs will attempt to twist the result towards 'no,' while non-saboteurs will want to get the best result, which is towards reality, as per Litany of Tarski. The existence of a saboteur means that the non-saboteurs will present information that suggests there is, but the saboteurs will present information that suggests there are none. On the other hand, the absence of saboteurs means all of us will present information that, in total, suggests there are none. So assuming there are more non-saboteurs than saboteurs on this boat, we can still get at least the correct information: communal rationality can still win. So I propose that we can safely ignore the fact that we are playing Paranoid Debate: if there are one or two saboteurs, we can get hints of who they are if we focus on answering the simpler 'yes'-'no' question, and only that question. If there are none, we all, as an aggregate, will conclude that there are none, as long as we remember our precommitment to truth and avoid biasing towards presenting information that there are none - that is, if we avoid signaling. If there are three or more saboteurs - then the non-saboteurs are probably doomed anyway." Raphael idly touched his cheek. "In conclusion: we can ignore the fact that we are playing Paranoid Debate. Because: If there are no saboteurs, then each of us, being honest non-saboteurs, will be able to present a lot of evidence that there are no saboteurs. If there are a few saboteurs, then the non-saboteurs will present some evidence for saboteurs and only a little evidence against it, but saboteurs will present only evidence against it: if they don't, they would lose. So in this particular case, we can detect hidden deceivers." He fell silent, hand still clutching the token. After a while, Michael raised his hand, followed by Nicole, asking to get the token. "Wait," said Raphael. "It's irregular, since it's a modification to a precommitted procedure we're already undertaking, but hear me out -" He waited until Michael and Nicole lowered their hands, their expressions quizzical. "I move that we modify the procedure: we restrict ourselves strictly to speaking only about the 'yes'-'no' question, without specifically pointing to identify who. Any mention of identities is forbidden, and this will be a moral precommitment that our Judge shall pass Judgment upon. This makes it harder for saboteurs, if they exist, to hide among us. I'm sorry, I should have thought of it earlier before we started. But anyway: what do you think, everyone?" Chapter 29 "Hmmm," said Adams as he finished the chapter. "A complicated solution, Administrator Yu. Or so it would seem. What do you think of Administrator Yu's attempt to make moot the fact that they are playing Paranoid Debate, Judge Verrens?" Verrens paused, his hand raised. He was visualizing a featureless white room - - Adams timed it. A proper crisis of faith needed time, and he was almost sure that Verrens was cutting it too close - - and when the trance was over, Verrens lowered his hand. "Yes, Lord Judge," he said, "I agree with Administrator Yu -" Chapter 30 On the boat, again, chaos reigned. Michael argued with Raphael, Julia occasionally remarking, Katja sullenly watching as she contributed only her opposition to Raphael's proposal. But in one corner of the boat, (not yet a) Judge Nicole Angel sat silent. Had her companions bothered to look, they would have noticed that she was in a trance; that physically she was sitting in a boat lost somewhere in the expansive oceans of Humanity's home Earth. but she herself was not there. Not really. In her mind, she was inside an empty, featureless, white room, with nothing to view. In her mind, she held mental tools to assist her, as she undertook the hardest trial that any Human could undergo: find out if she should change her mind. On the mental tools were carved ancient words of great and eldritch power: "Do not complete the old pattern!"; "Use the 'try' harder: make an effort, do not just give up"; "Your brain does not know for certain: do not immediately propose solutions"; "Search carefully for the thought at the back of your mind"; "Attack the weak points of your belief: you're doing it a favor by checking it!"; "You are better off believing something, if it is true, even if it feels bad: at least now you can do something about it!"; "Do not argue on only one side of the debate"; "Expect nothing from the evidence: take no sides and be neutral"; "Feel the weight of each burdensome detail: simplify"; "Do not be motivated to move or stay anywhere: let the winds of evidence blow you where you should be"; "Let your belief go: if it returns then it was true, if not, then it never should have been yours to begin with"; "Do not let your thoughts repeat: if a thought has already occurred to you before, block it out and replace it with another"; "The feeling of being correct is indistinguishable from the feeling of being incorrect" ... In her mind, she whispered: "tsuyoku naritai." I want to become stronger: more rational, more knowledgeable, more ethical. In her mind, she whispered: "isshoukenmei." With all my brain's will and strength: I will take as much effort of will is necessary to find the option that is true, the map that models the territory, the belief that reflects reality. She did not breath deeply, as it could serve as a trigger to unwanted cached associations. She let it remain even and regular, as neutral as the empty white featureless room she was (imagining herself to be) in. As her companions bickered, within her mind she asked herself the question: "Is Raphael correct, that limiting the explicit discussion to only the question 'is there at least one saboteur on this boat?' would be superior and avoid problems of cooperating in an environment where potential traitors exist?" And she thought - One argument against Raphael's proposal was that it was complex ("Feel the weight of each burdensome detail: simplify"). Or was it? ("Your brain does not know for certain: do not immediately propose solutions") But the question being considered in the disputation arena was complex. It was to be expected that thinking about it would be difficult, and so it was likely that Raphael was incorrect. Wait: what was that thought again? ("Search carefully for the thought at the back of your mind") The question itself was complex: In a game of Paranoid Debate, what would happen if the question was "Who are the deceivers?" The optimal strategy for a deceiver would be to point the evidence towards someone else, diluting the pool of evidence, possibly to the point of uselessness. But: if you rephrased the question to "Is there at least one deceiver?", what would the optimal strategy for a deceiver be? Well, it definitely could not be "point the evidence towards someone else," because what Raphael was proposing was that evidence pointing to specific identities was to be made inadmissable. By simplifying the question to that extent, the only available strategies to a deceiver would be: (1) twist the evidence towards 'no', (2) do not twist the evidence, (3) twist the evidence towards 'yes'. Were there others? ("Use the 'try' harder: make an effort, do not just give up") Wait - the end effect of (2) and (3) was the same! There were only two actual available strategies for a deceiver: (1) twist the evidence towards 'no' (2) allow the group consensus to settle towards 'yes'. Picking (2) would be directly losing. Picking (1) would cause the deceiver to stick out like a sore thumb among non-deceivers who would honestly present evidence suggesting the existence of deceivers. So it seemed that Raphael was correct. But wait: what if Raphael was wrong? ("Do not argue on only one side of the debate") What would the world be like if the better choice was to find who the deceivers were, rather than whether they existed? On the surface, it was a more direct path to their ultimate end goal: they wanted to know who the saboteurs were so that they could protect themselves against further attempts at sabotage, and, if there was a larger organization that would attempt to pick up the saboteurs, they could possibly attempt to keep them as hostages in order to negotiate for their continued survival (admittedly a long shot if it came to that, but better by far than the alternatives). But as Raphael had pointed out, the more complex question gave enough strategies for deceivers to game the - scratch that, she had already thought of that; her physical hand actually waved in front of her face to swipe off that thought ("Do not let your thoughts repeat: if a thought has already occured to you before, block it out and replace it with another"). She needed to challenge that belief, not strengthen it ("Attack the weak points of your belief: you're doing it a favor by checking it!"). Well, what if Raphael was a deceiver? How would proposing this modification cause them to be more likely to decide that there were no saboteurs, or that the saboteurs were someone other than Raphael? Perhaps he knew that some evidence that was already shown in the presented ideas would identify him directly, and so Raphael wanted to remove that evidence? Wait - the only idea that involved Raphael was that Raphael swimming away was an attempt to contact and be rescued by the larger organization sponsoring the sabotage! Perhaps Raphael was trying to suppress this particular bit of evidence? In fact, it made sense: Raphael was, by his admission, more than just an accountant. Perhaps another of his hobbies was some form of Engineering? That would allow him to know how to sabotage the drop pod. Obviously he was strong enough a swimmer - another point to choose him as the saboteur of a vehicle that would drop to the ocean. His sponsoring organization would then somehow delay the legitimate rescue to extract their saboteur. And - no, wait, she'd been mislead by her brain! ("Feel the weight of each burdensome detail: simplify") Quickly, inside her mind she recited again the Litany of Occam. An accountant who was an amateur Administrator made some sort of sense: the movement of large amounts of money often represented trading the efforts of large groups of people, which would have to be coordinated by Administrators. And then there was the direct evidence that showed that Raphael did have at least some knowledge of Administration: witness the obscure disputation arena that he was able to explain to them, Notional Group Technique. (No, wait, sorry Raphael, that was Nominal Group Technique.) In fact, one might have grounds to doubt if Raphael was indeed an accountant: there was no evidence of skill in that field so far (not that there would be a chance to practice it in a survival situation like this one). But add the detail of having someone interested in not just Administration, but in Engineering. That would be the intersection of people interested in Administration, and people interested in Engineering, so people interested in both Administration and Engineering would be fewer than people who were interested in one of them. Then add the further detail of being a member of some large nebulous organization. Not just any organization, but one that was capable of hindering the rescue operations, and one that, despite the difficulties, wanted to sabotage a planetary drop pod. More burdensome details! So scratch that line of thought - - later, when her will was exhausted, she was almost sure that Raphael's new procedure was better. In fact, she was already well on her way to thinking through the question itself; she was already willing to bet at even odds that there was no saboteur on the boat (but not necessarily that there was no saboteur at all), and perhaps willing to make a smaller side bet (say at 20% stakes of the main bet) at two to one against that there was no saboteur on the boat. And so she broke her trance, and lifting her voice to cut through the chaotic hubbub of the rest of the community, she said, "Silence! Pay attention to me! I've thought through the ramifications of Raphael's new proposal, which you should have been doing instead of arguing. The proposal is sound, and as Judge, I would like to compel our community to use the new proposal." It was only after she had uttered this in an imperious tone that the parts of her brain that had been paying attention to the real world informed her that her companions had already been silent and staring at her for quite some time now. She was red with embarrasment at having dived into her trance so deep, but she persevered in making them change the procedure anyway. Chapter 31 Verrens chuckled as he finished reading the chapter. "Lord Judge," he said, "it is almost eerie how closely my argument follows that of Nicole Angel." "Is it?" said Adams. "Well, if so, then perhaps I can skip over it and read just the ending, then. Or is there some minor detail that is different between Nicole's train of thought and your earlier argument for supporting Raphael's proposal to change the procedure?" "Ah," said Verrens, "none actually, Judge. And please don't skip the chapter -" Adams promptly skipped to the last page of the chapter. "Well," said Adams, "as I was saying earlier, the likelihood of sabotage is low. And the argument you expressed earlier was solely about how Raphael's new procedure was superior; again, you are slower than Nicole. But then, you said earlier that you would bet at odds of five to one against sabotage, whereas Nicole is still at the stage where she would bet at even odds against sabotage; or rather, against the saboteur being in the same boat as she is, which is a strictly lower probability than sabotage." "Yes, Judge," said Verrens. "As written, it is good to see a more detailed description of changing one's own mind, especially considering that the aim of the novel is to educate and test aspiring Judges. However - Judge Adams, I am truly amazed at how Nicole's train of thought follows mine!" "Don't be," said Adams. "Or be amazed, but do not consider it impossible. Aumann's Agreement Theorem states that two rationalists with the same initial assumptions and the same knowledge of the evidence will end up agreeing, after all. It is not possible for rationalists to agree to disagree: they can agree to defer discussion later in favor of higher priority actions, but then they would not have common knowledge of the issue under discussion, since the discussion is prematurely cut short." "Doesn't Aumann's, strictly speaking, apply only to the case that the rationalists have common knowledge of the posteriors?" said Verrens. "As in, Alice and Bob know, Alice knows that Bob knows, Bob knows Alice knows Bob knows, and so on?" "True," said Adams. "And in your case the knowledge is unidirectional: both Verrens and Angel know, but only Verrens knows that Angel knows. Still, reasonably speaking you are being given the same information about the event in question; assuming both of you have similar backgrounds - both of you are young Judges who came here to Earth from the Mars Lagrange 1 colony for the Judge examination, and so on - and knowing you are being given the same information, you will produce approximately the same conclusions." "But the precise similarity -" "- is not in fact precise," said Adams. "Remember that Nicole currently assigns a higher probability towards sabotage than you do. She will bet only at even odds; you would bet at greater odds for no sabotage versus lesser odds against it." Verrens nodded. "See?" said Adams. "Now we agree, given sufficient time to discuss a topic: Aumann's Agreement." "Judge," said Verrens, "it makes me wonder - in a rational society, then, such structures as disputation arenas or democracy would not be necessary. Aumann's Agreement Theorem would preclude the necessity of such structures." "In a perfectly rational society, maybe," said Adams. "But as you know, Humanity has not yet achieved perfect rationality. We can be extremely rational at some times, we can be somewhat rational at all times, but Humanity has not yet achieved the ideal of perfect rationality at all times. Consider the characters of the novel; both Raphael Yu and Ekaterina Ivanova showed irrationality at times of stress. And arguably Julia Montega, when Katja revealed the possibility of sabotage; she was less than helpful in revealing important information, until Administrator Yu initiated a disputation arena to gather relevant information." He sniffed. "Although we might argue that those are generalizations from fictional evidence, this novel is, as I have insisted before, true - in a way. Thus, it would not be fallacious to generalize from the evidence presented in this novel; it strives to reflect reality as closely as possible, after all." "Even though events such as failed planetary drops are extremely unlikely, Judge?" said Verrens. Adams grinned. "Ah, my young Judge," he said. "You once revealed to me an ending of a chapter by revealing only the existence of a secret. Afterwards I repaid this faux pas by revealing the existence of another secret. Which would have been quite fair to both of us, tit for tat, but now you leave yourself open to another revelation of another existing secret." Verrens appeared puzzled. "I'm sorry, Judge," he said, "I don't -" Then he stopped. Afterwards, he said, "Oh. I see. But how?" "How?" said Adams, grinning even wider. "Why, the most obvious possible method: by being more likely than all the other alternatives. Although I suppose you were asking, 'how were the characters put into this current situation?' Well, as to that, I leave you to your speculations! I know too much already, from the few clues already given, about the truth; it would be a tremendous faux pas for me to give to you my own speculations, as Aumann's would automatically reveal everything to you. So I leave that secret as another puzzle for you." "Judge, spare me!" said Verrens. "I still have to solve your earlier puzzle!" "Ah, yes," said Adams, "the vexing question of what moral dilemma will Nicole have to experience at the end of this particular disputation arena. Do not worry, for there seems to be no clues at all so far as to what the moral dilemma will be. And my speculation that there will be one lies in extra information, about particular patterns that the plot designer tends to follow. This is knowledge I have derived by reading numerous productions and versions of novels by the plot designer." "And I think, Judge, that it is then unfair to expect me to speculate properly," said Verrens. "Yes," said Adams, "for this older puzzle it is indeed unfair, as it is simply my personal revenge on you. But for the new puzzle, the clues have been embedded in the novel; I reveal it with no additional hidden information that you have not seen for yourself written in the novel. But I leave the new puzzle to you; let us move on to other topics." "Well," said Verrens, as he groped for something to bring up, "we were earlier talking about Aumann's Agreement Theorem and the social structures such as the various disputation arenas, and democracy. Or even Administration, for that matter. Speaking completely hypothetically, Judge, if we were to achieve perfect rationality, then we would be able to dispense with such things, as they would be far less efficient than allowing Aumann's to take hold." "Would they, Judge?" said Adams. "I rather think that it is precisely those social machines that allow Aumann's Agreement Theorem to actually work. Remember, we need to achieve a stable state of common knowledge, before Aumann's Agreement Theorem forces all rational participants to the same decision. Social machinery such as disputation arenas allow rational participants to achieve common knowledge." "Ah," said Verrens. "So in a perfect rational society, they are still necessary." "It is not just such social machinery that is necessary," said Adams. "Assuming that the Microgenetics Scientists someday figure out how to let us self-modify to perfect rationality, the community of Judges would still be necessary." "How so, Judge?" said Verrens. "Rationality - particularly instrumental rationality - is used in order to win," said Adams. "In particular, it does not define what constitutes as a win. Those are values. The early Senseis, like all modern Humans, inherited a complex biological computer composed of barely-working hacks. They decided that the most important problem to tackle was rationality - to achieve consistency in striving towards the goals; to be able to rank our values to determine what, in the worst case, we would find least distasteful to sacrifice; and to determine accurate maps towards our precommitted, stated goals - and left the problem of defining the actual goals that Humanity should pursue to us. And when I say 'us', I mean us the Judges. It is up to us to figure out the best system to determine what we truly value, using as reference what we experimentally determine to be the values of our Human brains. Of course, such values are as inconsistent and unclear as the hacked rationality cores of the brain, but it has allowed some important backward-compatibility with pre-rational and proto-rational Human brains." "Why didn't the Senseis start with attempting to fix our values?" said Verrens. "Mostly", said Adams, "because Humanity, back then, had values that they shared almost universally, but which individuals pursued in haphazard, not very rational methods. Two different people with exactly the same values might very well have sworn oaths to kill one another, because of their equally irrational views of how to achieve their ultimately shared values. As an optimization problem, there was a better ratio of results to effort in fixing the rationality of Human beings. For another, consider that rationality is an important tool to us Judges, in studying the ramifications of various ethical systems. You might as well ask why the early Scientists started with studying the physical world first, rather than with mental rationality or with values: at that time, understanding the physical world gave the best improvement in the Human condition, and incidentally also provided secondary tools for the improvement of rationality. It was the discovered results of the Neurological and Sociological branches of Science, after all, that the Senseis used as the basis of their techniques." Adams had a faraway look. "Instrumental rationality and epistemic rationality," he said. "The two different things that a modern Human might be talking about when he or she says the word 'rationality'. Epistemic rationality: the act of making your map correspond to the territory, and the center of most of the actual techniques that modern Humans are taught as part of the basic and intermediate rationality training. Instrumental rationality: achieving the values you have declared your precommitment to, and assisted by techniques of changing your mind and techniques against akrasia. Or, to be short and pithy: to win." Adams looked back at Verrens. "Well," he said, "Judge Verrens, please consider at least your new puzzle as you read on. As for the old puzzle, well, please think of it simply as a good-natured prank played on you by a garrulous old trickster Judge. So, let us continue." Chapter 32 "- because I think, Judge," said Michael, "that Raphael is attempting to suppress the only evidence which directly points to a particular individual. Because that evidence points directly to him: the fact that he attempted to swim away, which suggests that he was actually trying to somehow get rescued by his organization before the governmental rescue operations reach us. None of the other ideas we presented were so specific towards anyone. It's the only evidence that could point to a specific person. Not only that: notice that his ideas - about the relationship of the distance between lifeboats and the time to rescue - was largely mooted by actual expert opinion from an Engineer. Julia knows that the lifeboats are expected to be near each other but far from the drop pod's final crash site. And finally: another piece of evidence, Katja's assertion that this boat was specifically chosen to work correctly, which exempts me, would also be invalidated. Raphael is attempting to point the evidence away from himself and towards me!" Nicole said - - was about to speak, when Raphael said "Fine, then, I withdraw my proposal." "Why?" said Nicole. "I don't - ah, because it would be a moral hazard for you, Administrator?" Raphael toyed with the bottle of shark repellant, the token unused and unuseable until the issues regarding the precise procedure to follow in the interrupted Nominal Group Technique could be settled. "Yes, Judge," said Raphael. "Insisting on this means that I would take on less risk than the rest of the community does; the removal of ideas that point to specific individuals is, now that I think about it, biased towards me. Consider the case that I am not the saboteur; then I reduce the risk of being mislabelled as the saboteur. On the other hand, in the case that I am indeed the saboteur, I reduce the risk of being correctly identified. I defer to your moral expertise, Judge, but this seems clear enough for a non-expert judgment." "Even if I were to then re-open this proposal, Administrator?" said Nicole. "Then I would respectfully abstain from voting for or against the proposal, Judge," said Raphael. "The moral hazard would still hold. You would then have to convince at least two of the other people in our community to support this proposal." "Which I doubt you will be able to," said Michael. "Since it seems obvious to the rest of us that Raphael is trying to mislead the community with this tactic." Nicole understood well what it would mean. It would mean she would have to shut up and do the impossible. "Impossible" in this case did not mean a mathematically proven impossibility, or some logical inconsistency. Rather, it was simply the result of a limited brain not seeing a path from its current situation to its desired outcome: the reasons for why you can't win in a situation, set in tension against the reasons why you absolutely, completely, unavoidably cannot afford to not win in that situation. "Impossible" was not the same as a mathematical impossible. "Impossible" simply meant "I can't see any way to do that." And what she would have to do was to start somewhere, start with some solution, and see if she got anywhere. Because sometimes - just sometimes - the map inside your brain did not match the territory. And maybe - just maybe - if you took a closer look at the territory, if you tried to start somewhere, anywhere at all, and tried to climb the territory, then maybe you could find a path that you had not marked. Sometimes, just sometimes, and maybe, just maybe, what you had thought was "impossible" was in fact possible. Things that looked impossible sometimes did not match surface appearances. A thousand years ago, people might have thought it impossible for mere Humanity to be able to discover how Human minds worked; and not just discover how Human minds worked, but also to develop tricks and techniques to make Human minds work better. Such a thing would have been impossible, then, to the point that mere pre-rational humans could not even imagine dividing the human mind into smaller, more easily understandable parts. Philosophers in that backward era might very well have thought that a self was indivisible; they were completely oblivious and completely ignorant of the many sub-agents composing a Human mind. What could Nicole do, in the face of "impossible?" What was needed was perseverance. Perseverance that worked across timescales: pre-rational humans might have rejected such impossible tasks in mere seconds, and she would have to persevere against such split-second System 1 reflexes. Perseverance against the growing mound of apparently impossible sub-tasks, reducing her will after the initial flush of decision abated - perseverance in the face of growing akrasia. She would have to start somewhere. How would she convince the other members of the community to support the proposal? What evidence could she present that would change their minds? And the solution - no, not the solution completely, but at least a solution, a start, a place from which she could climb higher to get a better view of the territory, to search for paths that she could not imagine existing. "So, Julia, Katja, Michael," she said, "what evidence could I present that would change your minds? What would convince you to support this proposal?" Michael laughed. "Why, there is no evidence that would convince me to actually support the proposal!" he said. Katja appeared to agree; Julia did not seem to be as supportive of Michael's position, but - Well, Nicole could try to convince Julia; then she would be faced with the lesser problem of trying to convince one of Michael or Katja to support the change in procedure. How could she do that? And then Nicole noticed what exactly Michael had said. And Nicole now had her next step. Chapter 33 "The solution to Nicole's problem - at least her 'next step' - seems obvious, does it not?" said Adams, as he put aside the papers of the chapter. Verrens, distracted as he was considering his own thoughts, took some time before he could answer. "Sorry, Judge," he said. "I was still thinking of the new puzzle you gave to me." Adams smirked. "Really, Judge?" he said. "I wonder, were you in fact able to actually read this chapter, or were you too distracted to easily comprehend it?" "No, Judge," said Verrens. "I did read it with full comprehension, although I confess my primary focus was in gleaning clues to your new puzzle. In any case it is obvious that, in order to solve this current situation, the next thing that Nicole should do would be to -" "If it's so obvious, Judge," said Adams, "then let us not discuss it anymore; we agree to its obviousness, we have read the same chapter in parallel, we are both Judges and we have been reading the same novel so far, and thus we are likely at an Aumann's Agreement. So let me then focus on a topic where I have deliberately avoided Aumann's Agreement: the new puzzle. What have you so far gleaned, then?" "Well, Lord Judge," said Verrens, "the main thing I have gleaned is the existence of a secret. First I had asked you about the incongruence of a novel that seeks to assign proper, realistic probabilities to all events, versus the very basis of the plot, specifically, the failed planetary drop pod. Failure of the planetary drop pod is an unlikely event, thus the novel's premise is a very unlikely story. Then you revealed the existence of a secret." "Good, so far," said Adams. "Continue." "You revealed it," said Verrens, "and also claimed that this secret was itself something I could have discovered for myself. You had also earlier pointed out that sabotage was unlikely, even though the rhythm of an investigative novel demands that a similarly significant event must have occurred, and that the rhythm of such stories demands that one of the characters presented in the novel must have actually caused the event. Thus, by symmetry, I conclude: the planetary drop pod did not fail!" "Yes," said Adams, "correct so far. But what evidence have you gathered from the novel itself?" Verrens shook his head. "I wish," he said, "that I could restart reading the novel again at this point. Perhaps then I could glean even more clues, Lord Judge." "Disallowed," said Adams. "That would be less fun than watching you persevere onwards. And I fear that we would have too little time to actually finish reading the novel, if we were to repeat it at this point. So: no clues from the novel itself, rather than from me?" "None, Judge," said Verrens. "Although -" "Yes?" said Adams. "Another clue," said Verrens, "which I have gleaned - again not from the novel, but from you. I'm sorry, Lord Judge, but I have yet to find a clue in the actual novel. In any case, I suspect that your earlier puzzle - about a moral issue that Nicole will face after the disputation arena concludes - is not in fact a puzzle, or a joke played by some crazy old trickster. It is, instead, a clue, and one which I failed to pick up on earlier. Tsuyoku naritai - I wish to become stronger." Adams shook his head. "Not even that, Judge Verrens," he said. "Isshoukenmei - a desperate effort is needed. But that is not enough, either. It must be 'shut up and do the impossible'. Can you figure out why exactly Nicole and her companions are in this situation, when the planetary drop pod did not in fact fail? Can you figure this out before the flow of the novel reveals it?" "Yes, Sensei," said Verrens, smiling. "I shall do the impossible!" Adams laughed. "Well," he said, "then let us proceed, so that both you and Nicole can shut up and do the impossible!" Chapter 34 "Michael," said Nicole, "why would you say that no evidence can ever make you change your mind?" "Because it's obvious, Madam Judge," said Michael, "that the circumstantial evidence adds up to Raphael's guilt. There is a saboteur among us, and it is Raphael." "Ah," said Nicole, "you mean to say that you have reached the bottom line of your reasoning, and have a final conclusion?" "Yes," said Michael. "And further evidence will not move your choice?" said Nicole. "None whatsoever, Madam Judge," said Michael. "So," said Nicole, "you have fixed your estimate of probability to a flat 1.0, and you have reached the bottom line while others in your community have not reached it, and are approaching contrary destinations - in spite of the evidence already presented being common knowledge to us all. You do understand the meaning of this, Sir Executor?" In rationality training, the 'bottom line' referred to the final conclusion you derived and considered most likely true. And the one of the more important parts of rationality training was to suppress the built-in urge to start by filling in the bottom line and then writing justifications above it. Michael was being irrational. Which was disturbing, since he was an Executor, and Executors were required to take at least the first half of the intermediate rationality training; many opted to take the entire intermediate rationality course. Executors were trained in many techniques to incapacitate and disable fellow Human beings; and Judges (and trainees near their examinations, although a brief review of events often revealed this to curious aspiring and amateur Judges) were also informed, privately, that Executors could kill with their bare hands, if a sufficient formal declaration of Judgment was attested by a Judge. An irrational Executor was dangerous, akin to a gun whose safety was faulty. (This was also a rather major turn-off, but Nicole quietly picked up the mental sub-module that had suddenly brought that topic up, dragged it into a dark alleyway, and shot it. Twice. Just to make sure. Right now she needed to survive and stay sane more than she needed to find a potential primary partner. Or even a secondary one.) The problem was starting with a desired result, and then searching for evidence to support it or disconfirm it. Human beings, sadly, were biased to consistently overvalue the things they already thought they 'owned': ownership bias, where a thing owned was considered more valuable than other things, even though to a neutral observer they might have equal or even lesser value. And a thought that you had already written on the bottom line was a thought that you owned, far more precious than any other competing thought that would be offered to you as replacement. There was a reason why, when Nicole had been trained to perform the ritual of changing one's mind, the mental tools given to her were inscribed with words like "Let your belief go: if it returns then it was true, if not, then it never should have been yours to begin with". And words like "Expect nothing from the evidence: take no sides and be neutral". And words like "Do not be motivated to move or stay anywhere: let the winds of evidence blow you where you should be". Such tools were desperately needed, in order to dislodge a thought from the clutching possessive grasp of a merely Human brain. Writing the bottom line first made it far too difficult to cross it out and replace it with the real, correct conclusion. Far easier for a flawed Human brain to simply ignore the evidence against it, to protect the belief that was already written and could not be undone. Writing the bottom line first was a sin against rationality. But an even worse sin against rationality was to assign any thought the probability of 1.0. Assign a probability of 0.9. Or 0.99. Or 0.999. Or 0.9999. That was allowed, if you truly had prior evidence suggesting such a high degree of certainty. But never 1.0. Because of the way the mind worked - as a ridiculously simplified approximation of basic Bayesian analysis - any such probability, used as a prior belief for further thinking, would never get updated, not normally. It would forever remain stuck at 1.0, unless a supreme effort was made to remove it. The Senseis taught that an idea - any idea - was worthwhile if, and only if, you could be persuaded to think otherwise. And if you had an idea that you had written on the bottom line before you had written the intervening steps from the premises, any idea to which you had indelibly assigned the probability 1.0 - infinite certainty - then you could not be persuaded to think otherwise. If a retina could be called 'blind' if its output did not change based on whether a photon (a piece of evidence entangled from the real world) hit it or not, then a brain could be similarly called 'blind' if its conclusion did not change based on whether a piece of evidence entangled from the real world was fed into it or not. What Michael had effectively done, in claiming that no evidence would convince him to support the proposal instead, was to admit that he was blinded. In a world of the blind, how could a seeing person pass the gift of sight? By shutting up and doing the impossible, it seemed. The Senseis had succeeded in doing so, and the people they had taught had been mostly pre-rational. Nicole might right now be just one rationalist, but Michael would have at least the basic rationality training. She knew that the Senseis had managed to force sightedness - rationality - on the world; she was living in a universe that had resulted from the rise of the Senseis and the Helvetian War. It should be possible for her to find a way to make Michael more rational, and accept Raphael's proposal. Michael smirked. "Yes, Judge," he said. "I know perfectly well what it means: it means that I know I'm right!" So how could Nicole force Michael back into rationality? He seemed more motivated than Katja had been, earlier - so it was likely that tricks using the Dark Arts would not work. First, Nicole had to understand what was going on inside Michael's mind. Next... well, she still didn't know, the overall goal was still "impossible" in the "I don't see any way to do that" sense. So: first figure out what was going on inside Michael's mind, and then figure out what to do next. And the wrong way to do that would be to try to imagine what was going on inside Michael's mind. Human brains were huge, complicated neural networks whose state at each moment shifted and changed in chaotic ways. When the rise of intelligence occurred, Human beings had to evolve ways of modeling other Human brains. The method that evolution had stumbled into involved modifying some simple basic assumptions of the brain, and then running the brain-function for a few seconds: imagination. The problem was that the few modified basic assumptions were often not enough to properly model another Human's brain. Affect - the current mood or emotion gripping a Human brain - was not easily modified to match another person's, but often had an tremendous effect on the final decisions thought by a Human brain. No doubt Neurological or Sociological Scientists could name even more things that could change the final decision. How to scout the territory so that she could write an accurate map? "What are you thinking, Michael?" said Nicole. "Because to reach a conclusion already, without all of us discussing the topic fully, means that you have information relevant to the topic that you haven't given us yet. And I'll be strict here: do not simply repeat what you said earlier. Everyone has already heard that in the earlier stage, and not everyone is convinced that this shows high evidence of Raphael being the saboteur. So there is some evidence that you haven't given." She looked at Michael piercingly; another Dark Art kept in reserve, now spent - she hoped more would not become necessary soon, modern rational Humans tended to pick up on Dark Art usage quite quickly, and got easily desensitized to them as a result. "You refuse to consider the possibility that you are wrong," said Nicole. "I am rational. I will consider the possibility that I, yes I, am wrong. There is evidence that will convince me of the wrongness of my position: show me that you have some evidence that you have not already shown, and I will begin to reconsider. For now, my current position is: Raphael is quite right in proposing the modified procedure. Present me with evidence that Raphael is the saboteur - evidence that you have not shown me before - and I will back off and reconsider my position. I, Nicole Angel, of vocation Judge, solemnly proclaim: I defy the data!" Chapter 35 Adams shook his head as he finished the chapter. Verrens, however, was still leafing through the chapter. When Verrens finished, Adams was smiling at him. "Well, my young Judge," said Adams, "I hope you have not suddenly lost your ability to speed read?" "No, Judge," said Verrens. "I simply read this chapter twice. But I confess I cannot find any evidence regarding how the characters are now in a lifeboat on some nameless ocean, when I am very sure that the planetary drop pod did not in fact fail." "As to that," said Adams, "the clues are few and far between. But in any case, the procedure is to discuss the chapter, and if you have not found any clues in this chapter for the puzzle I gave, then what is your reaction or opinion on the chapter itself?" "It seems to me," said Verrens, "that this chapter brings together some of the earlier ideas presented. The mental tools that are taught to us in the intermediate rationality training, used in the ritual of changing one's mind, or 'crisis of faith' as some older rationalists preferred to term it -" "Ah -" said Adams. "Never mind. Do go on." "- as well as the classic 'I defy the data' which modern Humans use both as a challenge, and as a precommitment to change their mind in the face of new evidence." "Any further observations?" said Adams. "None," said Verrens, "except that it seems that Nicole's 'do the impossible' is going much better than mine." Adams shrugged. "For sufficiently impossible things," he said, "it is often very difficult to estimate the time frame that would be needed. And the steps involved are often not necessarily the most straightforward ones: sometimes a step away from the goal is needed in order to find the true path. In any case, is that all?" "Yes," said Verrens. Chapter 36 "Call it my intuition, then," said Michael. "It's obvious, isn't it? Look: even now our good accountant-cum-Administrator is not even attempting to defend himself. If that's not an out-and-out admission of his guilt, then what is, Judge?" Nicole could feel the ground sinking beneath her feet. It was interesting, just how much irrationality a short blast of words could contain. Doing the impossible was turning out to be, well, extremely difficult. Oh, the audacious irrationality of it - "Intuition" was nothing more than the result of the inner, subconscious workings of a standard-issue Human brain. Inner cores that even today, Neurology Science had not completed mapped out. And if the inner workings were unseen, invisible, and unmapped, there was always the question of how reliable the output of "intuition" could be. As a future Judge, Nicole had been trained to treat her intuitions as nothing more than suggestions on where to start, and to use more deliberative forms of thinking to determine where she should get to. At least those deliberative thinking patterns could be expressed and described, its patterns studied and analyzed by Senseis all over Humanity's inhabited space for generations now. Trusting intuitions was not something that her fellow Judges would condone, if they were able to see her now via Sousveillance. Intuition was a black box whose contents were unknown, and which had been repeatedly found by Sociology Scientists as extremely faulty. It was not something that a rational modern Human would use as the entire basis for any decision. But a worse problem was the word "obvious". It was evidence of one of the nastiest, most pervasive biases: hindsight bias. Many of the Scientific discoveries that underlay the rise of the Senseis were started soon after an ancient war, known today only as "World War Two". In those days, a distinction had to be made between those who were well-educated and those who were less educated. As a test, one early scientist (at that time their title was not yet capitalized, as this occurred only during the rise of the Senseis and triggering of the Helvetian War) gave an analysis saying that "soldiers (an archaic term for Executor) from well-educated backgrounds were more likely to be affected by Shell Shock (an archaic term for combat-related post-traumatic stress disorder) than soldiers from less educated backgrounds, due to the intellectuals being less prepared for battle stresses than street-smart people." The fascinating thing was that people who read that analysis had agreed, quite heartily. In fact, many of them thought that it was "obvious." The same scientist also gave another analysis, this time saying that "soldiers from less-educated backgrounds were more likely to be affected by Shell Shock than soldiers from well-educated backgrounds, due to the intellectuals having generally greater intelligence to resist battle stresses." And the even more fascinating thing was that people who read the second analysis (but not the first) had also agreed with it quite heartily, again claiming that it was "obvious". At this point, it no longer mattered which one was true: what mattered was how people were so easily gulled into thinking that a confident-sounding statement from an authority was "obvious". (Nicole herself could not remember which one was actually correct out-of-hand; she hoped she could remember to research it later when they were rescued. All she could remember was that one of the two had been true, and right now she truly could say that it was really, really terribly non-obvious which one was.) In the intermediate training, there had been a short game designed to bring hindsight bias into stark relief. The class had been divided into two. Then one half had been shown what was supposedly an experimental result making a particular claim, written in an authoritative manner. The other half was shown an experimental result making the opposite claim. Both halves were asked to write down either the statement "This is obvious" or "This is not obvious" and sign it. Then the entire class would bet an amount of money given to them in a decision/prediction market. In Nicole's half of the class, she had seen the statement "Sociology Scientists have found that, when Humans choose friends or romantic primary or secondary partners, Humans tend to be more attracted to people with traits opposite their own: 'opposites attract,' as the ancient saying goes." Nicole remembered it well, because she had signed "This is obvious," and went on to bet against the actual true statement: "Sociology Scientists have found that, when Humans choose friends or romantic primary or secondary partners, Humans tend to be more attracted to people with traits similar to their own: 'birds of a feather flock together,' as the ancient saying goes." And that was not the nasty part of the game. The nasty part was that the Deepy market had settled to around 0.51 credits per share (re-weighted relatively as it was a paired-share market) of the true statement, meaning that the class as a whole gave the true statement only 51% chance of being correct. It was the hindsight bias talking, their Sensei had said. Nicole would be willing to bet at three to one against that, had Raphael not backed off from the discussion due to the moral hazard, Michael would have instead claimed that Raphael was still actively trying to mislead the community. This was all due to writing the conclusion before even beginning to think: writing the bottom line. It was a failure mode that a rationalist needed to completely avoid, because getting out of that failure mode bordered on the impossible. Wait - - getting out of that failure mode bordered on the impossible. But it was a failure mode that Senseis had worked against, long before, and one of their solutions had been the ritual of changing one's mind, as well as the mental tools they armed you with to prepare yourself in that ritual. "Michael," she said, "Your brain does not, in fact, know this for certain. Do not propose solutions immediately, because it unnecessarily privileges the hypothesis!" "I am not proposing solutions immediately, Judge," Michael said. "I am rational, just like you, and I went through the intermediate rationality course as part of my Executor training. This is not something that I just thought up in five seconds!" Well, that tool didn't work too well. If anything, it just made things worse. Now Michael was appealing to authority: he was, after all, trained in rationality too, and thus could think equally well with Nicole. Never mind that Aumann's Agreement should have forced them to agreement, if the issue had truly been discussed thoroughly. But the issue was not yet discussed thoroughly, and if had not been discussed thoroughly, then Michael had no right to fix his conclusion. Half a rationalist was not half as rational as a full rationalist, just like half a kitten was not half as cute as a whole kitten, it was just a horrible bloody mess. Then Nicole thought, "Attack the weak points of your belief..." During her recent ritual of changing her mind, she had considered the possibility that Raphael was a saboteur. But with some amount of thinking, she had found tremendous weaknesses in that possibility, which was why she now rejected it. So she prepared to fashion the thought into a tool to pierce irrationality. Chapter 37 "Well, Judge Verrens?" said Adams. "No clues, it seems, Judge Adams," said Verrens. "I mean," said Adams, "what do you think of the novel's chapter?" "Ah," said Verrens. He paused to gather his thoughts. "For a long time, Humanity has taken its intuition to be the ultimate and highest form of thinking. It was only with the rise of the Senseis and the formalization of methods of thinking, in the techniques of rationaliy, that Humanity has acquired the capability to think better than its intuitions could." "Correct," said Adams. "And the main advantage of rational thought is?" "It is openly understandable, Sensei!" said Verrens. Adams continued to look at him expectantly. "It is not a mysterious operation, Judge," said Verrens. "Unlike intuition. The formalized methods of thinking are also provably more accurate in many real-world tasks than mere Human intuition, no matter how well-trained. And because the techniques of rationality are non-mysterious, we can think of further ways to improve them." "Yes," said Adams. "Intuition's mystery has always been overrated by many thinkers in Human history. It is only our civilization which has started to actually demystify intuition. I remember, just before the rise of the Senseis, according to my research, there was one rather surprising - at that time - result of Scientific study. Specifically, they found that simple mathematical models could outpredict the predictions of what then passed for Economist experts. At that time, most of those so-called 'Economists' - today's Economists tend to take pride in their rather complete break with their past, incidentally - anyway, the 'Economists' of old did ascribe to even more complex mathematical models, but did not often actually compute their predictions based on the models; and even if they did, they often adjusted it afterwards based on their intuitions. It is not to say that the simple models were more correct than the common, complex mathematical models of that time - they were often worse - simply that the complexity was too much for their undertrained pre-rational minds, which settled on easier questions." "I see, Judge," said Verrens. "I suppose that many of those pre-rational 'Economists' also succumbed to hindsight bias, claiming that their complex models, plus their intuitions, would have predicted actual market results." "That is correct," said Adams. "However... in the advanced levels of rationality training, there is a generalized principle called the substitution principle. Before I explain it, could you first expound on the dual process theory?" "Certainly, Judge," said Verrens. "The later modified dual process theory for reasoning classifies brain sub-modules into either System 1 processes or System 2 processes. In general - although exceptions exist - System 1 process sub-modules are fast systems over which little direct control is possible, while System 2 process sub-modules are slower systems which are generally associated with conscious oversight or control. With training - especially with rational vocational training in our civilization - System 2 processes can be made to operate as fast as, or faster than, System 1 processes. Further, explicit training can usually train the more-accessible System 2 processes more easily than System 1 processes, although progress has been made in techniques to train System 1 processes." "Good," said Adams. "Now, the substitution principle is the principle that many biases in standard Human brains are specific examples of a more general flaw: replacing a complicated question that requires explicit System 2 reasoning with a far simpler question that the System 1 processes can actually answer. This is relevant to our novel since it appears to me that Michael and Katja are replacing the complicated question 'Is Raphael the saboteur?' with the far simpler question 'Do I like Raphael?'." "Ah," said Verrens. A while later, Verrens said, "Ah," again. This time he sustained the syllable longer. "So," said Adams, "I have now proposed a possible reason for why Michael insists on labelling Raphael the saboteur, and why Katja would at least silently support him. However - if we were to put ourselves in Nicole's shoes - this knowledge is still not all that useful, since it is still not obvious to me how she could do the 'impossible' task of changing someone else's mind. How about you, Judge?" Verrens thought about this for a while. Chapter 38 "- haven't considered the sheer unlikelihood," said Nicole, "of our premise: that the failure of the planetary drop pod was in fact sabotage. Fine, let us grant that, even though most failures can often be traced back to some person making a mistake rather than a deliberate choice to destroy. Now you wish to assign, among the 60 or so passengers, the 8 or so cabin crew, and who-knows-how-many bridge crew, the label saboteur specifically to one exact person. And that is assuming the sabotage cannot be done remotely, or that it cannot be done beforehand, with the saboteurs not even on the planetary drop pod in the first place, much less on this boat. And you postulate that the reason that Raphael jumped off the boat earlier is because, not only Raphael is the saboteur, but he is also the member of some organization, specifically one that is powerful and influential enough to put its own rescue operations faster than the local government. Never mind that most cases of sabotage in peacetime are not from such large and influential organizations, but are usually committed by disgruntled employees against their employers or former employers -" Nicole paused for breath just a little too long. "But what you didn't consider," said Michael, "is that an organization large enough and influential enough will be able to cover up anything!" Well, running out of breath wasn't so bad after all, because now Nicole could see that she had managed to lead her prey into the spot she had designated. She hoped her sharpest weapon would kill the irrationality, because she would probably not be able to sharpen a new one soon. Doing the impossible was really, really hard. She breathed deeply, partly to bring more oxygen into her bloodstream, partly to bring her brain into a sane, rational, deliberative state. "Tell me what you think will happen, Executor," she said. "If we ensure that Raphael remains in this boat - if we bind him - and then we are rescued -" "They'll just pretend to be the real government rescue operations!" said Michael. "And none of us will be the wiser! But I will know! I will know, even if you try to deny the truth!" "I haven't even asked a question, Executor," said Nicole. "So tell me: you anticipate that we will be rescued by a group of people who will act and move in ways that we will be unable to distinguish from the rescue operation?" "Yes!" said Michael. "They want to commit the perfect crime!" "So that won't be a good experiment to determine if Raphael is the saboteur, then," said Nicole. "What about if we exile Raphael? What do you anticipate will happen, Michael?" "Raphael will just swim off and be rescued by his organization," said Michael. "And if we exile him and then later discover that his drowned body was found by rescuers?" said Nicole. "It'll be a fake corpse, of course! Just a brainless body clone, I'm sure they'll decapitate the head to make it appear that sharks scavenged it!" "And if the head is intact and identifiable, and autopsy reveals a brain?" "Then they killed Raphael to make us think that he was really innocent," said Michael. "They want to cast doubt in our minds." "And suppose that the ship which picks us up also later picks up Raphael alive?" "Then that means the ship is really controlled by his organization." "Suppose," said Nicole, "that we bind or tie up Raphael, and when the rescue ship shows up, we tell them to let us all go safely or we'll kill Raphael?" "They'll just call in a negotiators, and try to convince us that we're insane and that we should let Raphael go because there really was not sabotage!" "I see," said Nicole. "Strange, then, that you are able to anticipate everything that would show that Raphael is actually innocent, and then provide an excuse that still allows you to conclude that Raphael really, really is the saboteur. You're right: no amount of evidence will change your mind." "They are not excuses, they're just common sense!" said Michael. "No, you're not using common sense," Nicole said. "You're using belief-in-belief." She hoped that this weapon would pierce Michael's irrationality. It was likely that he was an Executor - you don't get that well-muscled physique easily, his knowledge of proper protocol regarding Executors acting under the orders of a Judge was additional evidence - and if he was an Executor, then he was likely to have been given the intermediate rationality training, which included the concept of belief-in-belief. Looking at Michael's face, she thought that her weapon had hit true. Had it hit deep? Was the irrationality being pushed aside by some remnant of sanity inside Michael's trained brain? She prepared to think of another plan while waiting as Michael tried to stave off his shock at the sudden attack - - she still hadn't been able to do so, but anyway, Michael was now (hopefully) going to be more receptive to being led back to rationality. "Michael," said Nicole, "beliefs must pay rent before they are allowed to reside in your brain. And the rent they must pay is in constraining your anticipated experiences." She watched Michael, hoping that he was listening. "In short," she said, "if anything at all can be explained by your belief, if what you believe cannot constrain what you expect to see - if there is nothing at all that would make you doubt that belief - then it -" "- is equivalent to a null answer," said Michael, breathing raggedly. "Which means: my belief is completely meaningless..." She waited. When he remained silent, she continued. "You know that what you believe is wrong: given a possible outcome that might suggest that Raphael is not in fact the saboteur, you quickly moved to cover it. You tried to give some explanation that will allow you to retain the idea that Raphael is the saboteur. At no point did you say, 'yes, but I would bet against that outcome occurring.' At no point did you say, 'fine, let's try it and see.' At no point did you say, 'I defy the data.' You knew, you already knew that the idea was wrong, because you refused to actually back it." "Yes," said Michael. So, it seemed to be taking. So Nicole said - - was about to speak, when Michael said, "That's exactly what you would say, to try to keep me away from the truth. To try to make my conviction 'meaningless.' I should have realized earlier. I knew it all along. When our eyes met on board the pod, I knew there was something important, something different about you. I should have listened to my intuition earlier. You're with him. You're another saboteur." Chapter 39 "Ah," said Adams, when he had finished. "Escalation. Nicole's impossible task gets more impossible. How about yours, Judge Verrens?" Verrens shrugged. "I think," he said, "that not only do I need to level up my 'game,' so to speak, but I need to recall more about what happened before. You refuse to let me re-read the earlier chapters -" "- correct -" "- but I doubt you can prevent me from attempting to remember them." "Certainly not," said Adams. "Your brain is your own." "Thank you," said Verrens. "This, I suppose, is where I should attempt to focus: on remembering the earlier chapters. There seems to be something like slightly more than a quarter of the novel left; I suppose that the revelation of the non-failing drop pod will be there; and so there are likely to be few clues left. Any clues must be in the bulk of the book, which lies earlier." "While at the same time," said Adams, "you must still continue reading the novel, and giving me your feedback. This is, after all, the main purpose of our current exercise, and my puzzle to you is but a diversion." "Yes, Lord Judge," said Verrens. "So," said Adams, "let us further discuss the novel itself, and leave the puzzle to your memory -" Chapter 40 Impossible impossible impossible impossible! A thousand sub-modules inside Nicole's brain screamed at the sheer unfair impossibility of the universe. Well, this universe wasn't really fair anyway. You were allowed to fail, and suffer the penalty for failure. But one tiny sub-module persisted in saying "Shut up and do the impossible!" Nicole consciously promoted this thought to the forefront of her brain, and started looking for sub-modules who could match the thought to something, anything at all. The Sensei's lesson, discussing the concept of "Shut up and do the impossible" - no, not useful. A bit of gossip with some fellow Judge trainees, about someone who had no chance but was still going to keep trying - no, not useful. Ah, but what about this? In the early twenty-first century, people had started to propose constructing self-improving artificial minds (this had better be actually relevant, the other sub-modules in Nicole's brain said). Such self-improving minds, it was proposed, could quickly acquire tremendous powers, if they were allowed to have sufficient control over reality. Just as Human brains had taken over Earth and parts of space nearby, all while merely being connected to hands and feet that were nowhere near as strong and fast as the limbs of many lesser animals. Give a sufficiently smart Human brain merely a single finger to move, and it would be more than enough to pierce the secrets of space and time. Give an even smarter, self-improving artificial brain that same finger, and it might very well take over nearby parts of reality. Give any sufficiently smart brain even a small lever - and what was a finger but a lever? - and it would find ways to build larger and larger levers, until it finally had a lever long enough to move the universe... And so those primitive, barely proto-rational people thought that they could confine such an artificial mind inside a computer (an act, Nicole knew, which was in this current, more enlightened age, illegal). And the only way to interact with this artificial mind would be via a simple two-way text terminal. In no way, shape, or form, would such a mind be allowed access to the physical world. The monumental stupidity of this idea might have never been discovered in time. But one of them realized that levers did not have to be physical levers. After all, the term "leverage" did not often get applied to merely physical levers - it was often applied to money (Nicole would have to ask Raphael for details at a later, more propitious time; she really didn't understand that very well. She also had to really clamp down on these digressions, they were wasting her time, she needed to interact with the community Real Soon Now or she could lose all credibility together with Raphael). And money was a social construct, defined by a community of interacting Human brains. And then it clicked inside the brain of that person that the Human brain interacting via a mere two-way text terminal was itself something that could be used as a lever. Not a physical lever, but a lever nonetheless. Once that person had convinced himself (or was it herself? Never mind) that such a trapped brain could be able to use the Human guarding it as a lever to begin interacting with the real world, he set himself the task of convincing others that it was unsafe, and that they had to actually build safety into the artificial brain, so that it would never want to completely destroy Humanity in trying to achieve any of its goals. The other thinkers, however, refused to believe him. Just as now, Michael completely refused to believe her. Ah, so that was the point. So, how did he manage to convince others that artificial brains had to be done safely, or not at all? He had wagered that, if he - an ordinary Human brain, not even the self-improving artificial type they were considering - were put inside a locked room with only a two-way terminal, and that the other end of the terminal would be used by another Human being who could unlock the room and let him free, then he would be able to convince the gatekeeper to let him go. And he had actually won that wager a few times. How? Nicole could not remember the exact details. But something from one of his writings did stick to her memory: "The key is to realize that you are talking to a Human. You won't be able to escape the locked room if you were talking to a rock." All right. So right now, Michael was irrational and would not listen to reason. But he was still Human. Baseline untrained pre-rational Humans were irrational, but that did not mean that they could not be depended upon to form particular conclusions. In fact, they could be relied upon to predictably make certain irrational conclusions. Humans evolved as social animals. And now a part of Nicole suggested something to do right now, before it was too late - Michael was starting to look triumphant, a look of horror was starting to creep up Julia's face, Katja seemed to grow more and more certain - - Nicole laughed at the inanity of Michael's suggestion. She, a candidate to become Judge, sabotaging the planetary drop pod? What absurdity! Nicole laughed, it was an infectious laugh, and Julia laughed along with her after a while. Raphael even smiled, even though he was trying to stoically remain aloof the discussion due to the moral hazard of participating in it. Katja looked even more uncertain, and Michael, looking around at the community, seemed to be rethinking his recent conclusions - Okay, the laugh had sounded a little forced to her own ears, and the timing was just slightly late by her own estimation, but she seemed to have gotten at least some of the results she wanted. Hey, give her a break, her brain told her. It's not like she was a detached Platonic ideal mind that could cogitate in zero time. Besides, she managed to get the community to doubt Michael's latest statement, and that, hopefully, would allow her to keep the fight going. If Katja and Julia had doubted her then and there, then it really would have been absolutely impossible. Humans were social animals, and laughter was a social event. It could be used to signal many different things. Such as contempt, for instance. That signaling still worked, regardless of whether a Human was being rational or irrational, so long as he or she remained sufficiently Human. The basic infectiousness of laughter - based on built-in social conformity circuits of the human brain - could be exploited, as Nicole just did. Leveraging her control over herself into control over the community, and from there control over Michael. Give her a lever long enough... What next? Another sub-module provided another idea. It was not a good idea. It was an embarrassing thing to do. Hadn't she shot that sub-module, twice? What was it doing still going around in her brain making suggestions? No, she couldn't do it, it was too demeaning, she'd lose all her self-respect if she did that. Basically, the idea was based on the observation that Michael was not only definitely Human, but was also very very likely to be a Human male - no stop that right now I'm not going to think it. Okay, fine. But what was it that Judge Nicole Adams needed to do? What were values, her goals? What was the thing she needed to protect, and thus use her every ounce of rationality for? The morality of her tiny community. The aggregate sanity of the people on this boat. Raphael, who was very likely innocent, given the evidence she currently had. If a miscarriage of justice occurred while a Judge stood by, what was the point of all her studying for her vocation? What was the loss of a little self-respect, compared to that? Heck, she was on Earth for the Judgement Acceptance Examination, she was almost sure that she would pass, even if this adventure on Earth's ocean made her miss she was sure they'd give her a special examination session, and anyway becoming an actual Judge would give her a whole lot of self-respect (and other people's respect too, come to think of it), more than enough to counterbalance what self-respect she would lose now. Shut up and multiply. A little sacrifice on your part versus the very sanity of the community and justice for Raphael. Okay, that sub-module did have a point. She just hoped that some other sub-module would make a better suggestion. After all, if she could get the same, or nearly the same, result, for a much lower price, she should take that trade instead of this - idiotic idea. If there was a better deal than this, and if she could find them soon - Julia's laughter was dying away, Raphael's stoicism was returning, and Michael was spluttering, thinking up a reply - She asked for other options from her other mental sub-modules - Chapter 41 "Ah," said Adams, "an interesting imagery, levers. Well, Judge Verrens? Verrens?" Verrens snapped back to attention. "Sorry, Lord Judge," he said. "You were speaking about levers?" "Yes," said Adams. "Rationality, we might say, is a lever. Or to be more precise, a set of levers of various sizes and for various purposes." "I follow your reasoning, Judge," said Verrens. "Do you?" said Adams. "Tell me: if a machine were designed that made only levers, and consumed the entire Earth to make those levers - would that machine be useful?" "No, Judge," said Verrens. "Ah - now I see what you mean. Rationality must be used for something other than to serve itself. Hence, the reason why every Human being is trained in the basic rationality course, but no one is compelled to take the intermediate course, except for certain vocations like ours, and why only Senseis and amateur Senseis are allowed to take the advanced course." "Correct," said Adams. "We still need a machine to make levers, and that is why we have a separate vocation for rationality tutors - the Senseis. But such a machine should not consume the entire resources of all Humanity, for then what would be the point? Even so, such a machine must still exist, because otherwise, how would we make more levers when needed?" "And to continue the analogy," said Verrens, "most people's vocations would not require anything more than the basic rationality course - the ability to recognize that one has made a mistake, to reflect on the thought process that has led one to make the mistake, to know that the mind contains a map of reality, but not reality itself. Some vocations, like Judge, Executor, and Scientist, need the deeper and more complete training, since these vocations require more heavy mental lifting, and thus bigger levers. But the biggest levers of all are reserved to the Senseis, who must make the levers to be used by the other vocations." "And," said Adams, "to design even larger and larger levers, just in case Humanity may need them some day." "So the analogy certainly seems quite apt, Judge," said Verrens. "Yes," said Adams. "In my research of the Sensei's past, there was a rather interesting fact - the early Senseis had a rule that they seem to have dropped in the modern era. Specifically, a Sensei had to have some other vocation than being a Sensei - that is, they would all have to be amateur rather than vocational Senseis. Did you know this, Judge?" "No," said Verrens. "That seems rather irresponsible; I imagine the pre-rational and proto-rational eras would need more full-time Senseis." "They did have a reason for doing it that way," said Adams. "Would you venture a guess what it was? This is not another puzzle, by the way; I will give the correct answer if you cannot find it immediately." Verrens smiled. "Let me think about it for a while, Judge." After about a minute, Verrens said, "I suppose that the pre-rational hierarchicalists, and possibly the proto-rational groups, were opposed to the early Senseis, and that the vocations the Senseis claimed were their cover and protection. Their secret identity, I suppose." Adams shook his head. "No," he said. "The reason given was that, in those early days, there was no easy way to determine if what any particular Sensei taught was actually rational, or only appeared to be so. Judging rationality at that time was not as well honed as today, where Neurology Scientists and Sociology Scientists perform such tests almost as a minor hobby or amateur work. Remember, rationality must serve something other than itself. In the early pre-rational and proto-rational era, the few people who knew rationality were also the ones who were developing it. And one way they had to judge the effectiveness of their developed rationality techniques was to actually try them in their actual day-time vocations. Hence the early rule that every Sensei must be an amateur Sensei, not a vocational one: they did not want schools of rationality to proliferate without evidence backing their teachings." Adams looked at Verrens straight in the eyes, an evil grin lurking somewhere off the edge of his face. "Now, earlier you seemed to be distracted. Tell me, Judge Verrens, did you find a clue in this chapter?" "Yes," said Verrens. "Strangely enough, I think I did. But for now I will turn the tables against you: I will not tell you, yet, what the clue I found is." He grinned back at Adams. Adams shrugged. "Well, then," he said. "Let us proceed to the next chapter." Chapter 42 If this worked, she was going to pass the Judgment Acceptance Examinations and then drink some kind of extremely potent alcoholic beverage to expunge her memory. If it didn't - she wouldn't know what to do. Lose, maybe? She hadn't managed to come up with some other idea, and she needed to act Real Soon Now, the window for actually acting was going to close as soon as Michael unspluttered himself and started to actually talk. So she started to plan out a little what exactly she would say - - As the laughter subsided, Nicole looked down. She whispered, her voice low and wistful, "You know, when our eyes met on board the drop pod, I thought there was something important, something different about you, too. I thought you were an attractive man. I wished, just a little bit, that..." Her voice trailed off into silence - Basically, the idea was based on the observation that Michael was not only definitely Human, but was also very very likely to be a Human male, whose brain was preprogrammed to trigger on particular cues. It was thus very very likely to respond to beings who looked like attractive young female Humans, and to respond in particular ways. Rationality training would have allowed him to override such responses easily, but he had deliberately avoided those techniques in order to allow himself to retain an irrational idea. Michael was currently not a rational Human being. So she could exploit that to her advantage. She hoped. So, feeling the distaste in her mouth, she plowed onward - - and then her eyes turned into slits focused on Michael. "But that was then," she said, voice hard, and yet tinged with sadness and loss. "Now you not only persist in an irrational belief, you also reject the use of rationality techniques that would correct it. Now you accuse me of being a saboteur, someone who cannot be trusted. Me, someone who is almost a full, vocational Judge, but for the formality of an examination. And I had hoped that you might be more than just an attractive man I glanced briefly at once, on some random planetary drop pod." She looked out, glancing over the sea, brushing away a thread of hair, and sighed - Looking away meant that it would be difficult to gauge what was happening to him - and if he persisted in irrationality he could break down badly - but hopefully some self-preservative instinct would force him to take up rationality techniques again. Hopefully being forced to apply rationality techniques in order to handle this rather low, below-the-belt (literally?) blow, would also cause him to apply those techniques to his cherished belief in the guilt of Raphael - - "I'm sorry," he said, his voice wracked by emotion. "I'm so sorry..." Peeking from a corner of her eyes she saw that Michael's eyebrows were knotted, his eyes focused on something that neither she, nor anyone else on the boat, could see; perhaps it was some rationality technique operating in his mind (she hoped). "You're right," he said, his voice calmer now, the wracked emotion fading. "I've not been thinking clearly for a while now," he said - Winning in spite of desperate odds should have felt good. It didn't. The important thing was to win. Rationality was a tool to use for winning. To win was to achieve her goals and preserve what she valued: stop the miscarriage of justice that would occur if Raphael was wrongly convicted on the very light evidence available, bring Katja and Michael into more sane states of mind, conclude the discussion so that Administrator Yu could take over once the moral hazard that prevented him from acting was gone. Her self-respect was also something she valued, but she did not value it as much as the others, and so she had been willing to sacrifice it. A Pyrrhic victory, almost. Her basic brain told her so; but her total aggregate rationality-trained brain knew about ownership bias, knew that she valued her own self-respect higher than it was actually worth, and she had needed to explicitly compensate against it. So, it was not a Pyrrhic victory, all things considered, just nearly so. So she told herself, but it was still somewhat difficult for her to accept. Chapter 43 "I hope," said Adams, "that you are able to direct your full attention, now, to our discussion of the novel." "Yes, Lord Judge," said Verrens. "Since I think that I have managed to extract a minor clue from the previous chapter - although not enough, I think, to consider the puzzle fully solved. And in any case, I am still waiting for the moral problem you mentioned earlier as a hint; once it has arrived, then I shall be able to judge the actual circumstances and then I can consider whether my tentative hypothesis was correct." Adams smiled. "And will you tell me your hypothesis now?" he asked. "Of course not," said Verrens, "since you are a garrulous old trickster of a Judge, pulling your rank on some poor unranked Judge just for your amusement." They both chuckled at this. "Fine, then," said Adams. "I suppose you will tell me your hypothesis anyway, when the moral problem that is arriving soon actually arrives - whether or not your hypothesis is confirmed or disconfirmed?" "Yes," said Verrens. "Then let us return to the novel," said Adams. "What can you say about the current chapter?" "Aside from the obvious fact that Nicole won," said Verrens, "and that she is now regretting, to some extent, the price she paid for that victory, there seems little to remark upon, Judge." "Then let me ask a hypothetical, Judge," said Adams. "Suppose that, now that she appears to have successfully defended Raphael from the accusation of sabotage, afterward they encounter further evidence that points to Raphael. What then?" "Then the community as a whole updates their hypotheses in the light of new evidence," said Verrens. "As we should expect from anyone who has completed the basic rationality training, who in the modern era is practically every adult Human being in modern civilization." "But recall, Judge," said Adams, "that the characters are, in fact, barely within the bounds of modern civilization, and that we have evidence that the rationality training they have is now extremely strained by the circumstances. In fact, they are likely to fall back to more primitive thinking methods: that Nicole was completely wrong in defending Raphael in the first place, notwithstanding the fact that, at the time she defended him, there was too little evidence pointing to Raphael. Rather than all of them calmly integrating the new evidence into their working hypotheses, the majority, I think, would contrive to strip Nicole of social influence in their community, because of an action that was justifiable given the knowledge they had at the time it was committed." "Ah," said Verrens. "I suppose you are correct, Judge. Although I would think that such a situation seems contrived, and could only occur within a fictional novel." "Do not be so confident, my young Judge!" said Adams. "I have mentioned before that the novel may be said to be true, in a way. You can be sure that the designer of the plot has placed the characters in a situation in such a way that this could actually happen. As I mentioned before, the techniques of rationality are recent, barely more than a century and a half or so; evolution, blind and stupid though it may be, has the advantage of wreaking millions of years' worth of badly-guided design on our Human brains. Reality is always stranger, to our merely Human intuitions, than fiction, since fiction is molded specifically to suit Human tastes." "Ah," said Verrens. "I see. But the novel is intended to be a sort of trainer for student Judges, is it not, Lord Judge? Do you suppose that this situation is something that a typical Judge would find himself or herself in?" Adams paused. He nodded sagely. "Well," said Adams, "I suppose, then, that this is a weakness that you have found with the novel. Its premise is that an aspiring Judge finds herself, not among sane rational people, but instead in a highly stressful situation together with people who are, because of the situation, sometimes not in a rational state, and dependent more on pre-rational modes of thought. In practice, such a situation would be rare and unlikely - but still, should we or should we not present this novel, just in case such an event does occur to one of the student Judges in the future?" Verrens looked carefully at Adams' face. He sighed. "I see, Judge," said Verrens, "that in fact this has been considered, and no doubt minds better than mine at these considerations have done the requisite computations with accurately-measured data, and considered it worth the time of a bright old trickster of a Judge of the Fifth Rank to at least evaluate. After all, it would be best if a Judge placed in the same situation is able to win, with all his or her strength as a rationalist, even at impossible odds. Even though it is unlikely, it is still not unlikely enough a possibility to ignore." "Yes," said Adams. "And I suppose you have learned that in our conversations, even as I pause, it is not in fact because I must change my mind, but simply because I am tricking you into thinking that I have been less prepared than I truly am. I hope that you still try, though, and that you are able to surprise me into considering something I was not able to think of before - I certainly hope that you have not learned helplessness! As the Senseis teach, shut up and do the impossible -" Chapter 44 - and so, after successfully shutting up and doing the impossible, Nicole felt that she deserved a break. With Michael backing off from his previous position, and now actually rationally thinking about the proposal, it was pretty much inevitable that Raphael's proposal would pass: their subsequent discussion would be limited to answering the question "Is at least one saboteur among the people currently on the boat?" Nicole did not pay very much attention to the rest of the disputation arena's synthesis stage; she doubted that she would be able to make any decent contribution in her current mentally drained state, anyway. So she just let her System 2 drift, hoping to recharge her drained willpower before the next crisis hit (and also hoping that this would be the last crisis). She ended up just listening idly, her System 1 providing automated noncommital nods, tracking the token to where it went - Raphael, apologizing for the weakness of the ideas he had provided, saying "- I do hope you all understand that I had needed to present my ideas before Julia did, so I think it is only to be expected that they would be mostly speculative -" Julia, contradicting Katja's idea, saying "- a bridge crew member would be in the best position to perform any significant sabotage, or maybe the cabin crew, but your idea means that a passenger saboteur had help from a cabin crew saboteur, and strictly it's simpler to just have the cabin crew saboteur without a passenger saboteur. But the simplest idea would be to have one of the maintenance Enginers or technicians -" Katja, commenting on one of Raphael's ideas, saying "- just doesn't seem to be relevant, and doesn't seem to be able to give a hint for answering the question being discussed. So what if the time zone is -" Michael, basically agreeing, but for some reason (Nicole's System 1 suggested that it was a good reason, but as her System 2 was quite disengaged and not at all paying any attention, she would not have bet at even odds) had focused on another of Raphael's ideas, saying "- ships waiting to bring the drop pod into port would be the first to learn about the event, so it seems strange that we have not been rescued yet -" - everything started to blur together, and only bare snippets, attached to voices, filtered into her resting consciousness - Michael: "- as Julia said, drop lifeboats are paired, so now that I think of it, my proximity is not as much of a mystery -" Katja: "- still think that the wakening gas being wrong for the sedatives offered means there was sabotage, but then it doesn't mean the saboteur is actually -" Julia: "- not my specialty but I would assume that basic safety locks exist -" Raphael: "- so I invoke my right as Administrator to override the token passing; Julia, please pass it to Nicole instead." - wait, what? Her System 1 shocked System 2 awake, warning that there was something that needed more deliberative thinking to handle. Julia was profferring the token to Katja; everyone was looking at her. Neither Katja, Michael, nor Julia objected, so Julia pointed the bottle of shark repellent at her direction. Briefly, Nicole thought of objecting, but she just shrugged it off internally and took the token instead. She fished through recent memory, and simply let herself ramble, generally giving the opinion that there was probably no sabotage, and if there was indeed sabotage, it was not likely that the saboteur or saboteurs would be on the boat. She passed the token on, but afterward might not have been able to identfy who she had just passed it to. The growing consensus was largely clear: there were no saboteurs on the boat. Nicole let herself cheer a little inside. After all, if none of the saboteurs were on the boat, then Raphael was definitely not a saboteur, so her little sacrifice (little? little? that was a big sacrifice!) was not in vain after all. Administrator Raphael Yu gave a short summary of their community's aggregate conclusions - - was about to give a short summary of the community's aggregate conclusions, when Katja grabbed his wrist and started to hit his chest repeatedly! Nicole's System 1 triggered on the Medic's sudden movement, assuming that such an action was an important emergency procedure needed to save his life. Raphael's other arm - his left arm - was flailing, trying to block Katja's clenched fist, so Nicole held it back to prevent it from obstructing Katja's endeavors. It was later, after Raphael lay limp and lifeless, eyelids fluttering, when Nicole looked at Katja's face, and saw eyes filled with anger and triumph. It was later that Nicole realized that Katja was not trying to save Raphael's life in some medical emergency that only Katja could act upon. It was later that Nicole realized that Katja had been doing precisely the opposite of saving his life. Chapter 45 "Ah," said Adams as he put away the recent chapter. "Could you tell me, Judge Verrens, what is your conclusion regarding the puzzle you are working on, now that it appears that the next moral decision for Nicole has arrived?" "I'm sorry, Judge," said Verrens. "I must admit that my attention was largely grabbed by the penultimate event in the chapter; I haven't actually managed to think through how the event connects to the puzzle. Could you give me a few more seconds in which to think?" Adams shook his head. "Later, then. Let us instead discuss the chapter itself," he said. "Can you give me some insight or idea about the recent chapter, in light of previous events in the story?" "For one," said Verrens, "it would make Nicole's earlier win - making Michael back off from his irrational accusation of Raphael - into a distinct loss: she had sacrificed some of her self-respect but ended up losing what she wanted to protect despite that sacrifice. If I were to put myself in the character's shoes, I suppose that I would be - extremely angry does not seem to do it justice, but I cannot think of a better term." "Ah, anger," said Adams. "Anger, modulated and not allowed to become an unthinking rage, can help focus the willpower of an individual, and, however counterintuitively, can improve rational thought; however, it generally makes the individual focus on aggressive and destructive action against others in a community: a bias towards violence that normally does not hold in normal and rational thought. What would you bet on happening in the next chapter, Judge?" "I would bet," said Verrens, "two credits against one that Nicole will have tremendous difficulty in making a moral Judgment on what punishment will be appropriate for Katja. She may end up harsher than is acceptable in that situation, especially since she knows that there are no cameras watching her, and in particular no peer Judges overlooking her moral decisions. It would require her utmost control and deepest training to maintain sanity. Given her current state, she may snap and do something drastic, like order Executor Michael to perform a summary execution - according to the story she is mentally exhausted and might not have the willpower to motivate a System 1 override." "You bet only on the difficulty," said Adams, "but what would you bet her moral Judgment will be, after going through the difficulty of making a Judgment?" Verrens paused for a moment, thinking. "I am not sure, Judge," he said. "The standard punishment to premeditated murder is physical restraining of the murderer and a thorough analysis by Neurological Scientists. Specific analyses that would normally be considered violations of human rights will be allowed; right of mental privacy is waived until Neurological Scientists can determine if the individual can be redeemed by forced amnesia of recent events followed by Sociological therapy, or if the behavior is irredeemably psychopathic and requires indefinite storage in an asylum until we can find a method of curing that pathology. As for what Nicole will decide - I honestly cannot determine that particular outcome, and would be unwilling to bet on anything." "Not even some kind of probabilistic analysis, Judge?" said Adams. "I fear I have too little information on murder cases where the Judge is in significant stress while Judgment is being made," said Verrens. "The situation in the novel seems quite contrived and I cannot recall any cases at all where the Judge may not be in a state to retain strict rationality during Judgment, and where another, perhaps less risky Judge cannot be found. Thus I fear I cannot predict what will happen using even just a probabilistic analysis." "Suppose, instead," said Adams, "that it is you on the boat. Try to put yourself in Nicole's situation. What would you yourself decide?" "I wouldn't answer that, Lord Judge," said Verrens. "I am not an amateur Sensei, but I did pay attention in the intermediat rationality training course when we were taught that standard Humans are very bad at predicting their own behaviors, especially when attempting to predict their behaviors in a situation where their affect or mood is different from their affect at the time of predicting. While I am somewhat mentally exhausted over reading the novel, and trying to solve your puzzle, I suspect that my current exhaustion would not be anywhere near the level of Nicole's exhaustion, and I will probably be more rational than she currently is. I would be better off trusting a probabilistic analysis rather than my own intuitions, and since I cannot perform a probabilistic analysis due to complete lack of prior data, I cannot reliably predict my own behavior." Adams chuckled. "That is quite correct, Judge," he said. "It would be best to put you in such a situation first so that you can make a good prediction of what you would decide. Or at least an analogous situation." "Yes," said Verrens. "I hope that my training would be able to overrule my anger and indignation at such a turn of events." "So would all Judges hope," said Adams. "Do you have further thoughts or insights about this recent chapter?" "Well, for another," said Verrens, "I find it hard to believe that Katja would be motivated to actually attempt to murder Raphael. Earlier, she seemed to have - perhaps been more open to considering Raphael as a valuable member of the community." "As to that," said Adams, "they are, after all, still in a stressful situation. Perhaps it has forced Katja into irrationality and, we hope temporary, psychopathy. Any others?" "Finally," said Verrens, "I cannot imagine how Katja managed to kill Raphael." Adams shrugged. "Neither can I," he said, "but then neither of us are Medics. However, I am quite confident in the talents of the plot designer, and I would not be surprised if that method of killing, as vaguely described in this chapter, is indeed feasible in real life." Neither talked for a moment. "I am curious," said Verrens, "about how exactly Katja managed to kill Raphael. Can we proceed, Lord Judge?" Chapter 46 Nicole - had disintegrated, mentally, into several thousand sub-modules of herself, each one uncapable of accepting the depth of what had just occurred. One of them wailed a sarcastic echo, "impossible impossible impossible -" over and over again. She couldn't bring herself into a coherent whole. No sub-module, no sub-persona even attempted to pull the mad, shouting chaos in her brain into some kind of ordered, coherent thought. But it didn't last for long. No evolved Human brain would do something as definitively bad for itself as to lock up in a situation where a social ally had just been killed. Her arm was rigid, where some deep impulse had tried to get her to slap Katja, and some other impulse had kept he from doing so. She forced herself to relax her hand, asking her various mental sub-modules to bring the debate into conscious volition rather than by physical deadlock. Then she realized that her entire body was tense and forced herself to relax her entire body. Katja was laughing giddily; Nicole realized that she had been laughing for quite a while now. "Physically induced heart arrhythmia," Katja said. "I thought it would be impossible but I did it, now what will you do, saboteur?" She continued giggling. "What did you just do to Raphael!?" Nicole demanded. "I induced heart arrhythmia!" Katja said. "And thanks! If you hadn't held back his hand I might have missed the timing! They said that was possible, you just have to time the blow to the chest in the proper part of the cardiac rhythm! Cardiac contusion, that's what they call it! I never tried it before, and look! I actually got it!" Michael was looking at her. Waiting for Nicole's Judgment? Julia had moved to the boat's edge, trying to keep as far away from Raphael's corpse, and from Katja, as possible. Corpse? Not quite, not with modern technology - "Katja," said Nicole, "brain death does not occur immediately. What can we do to revive him!?" "Nothing!" said Katja. "Try doing CPR if you wish, but you'll need to keep doing that until you can get a defibrillator to reestablish normal cardiac rhythm. And all it will do is extend the window of opportunity to revive him! The longer you go, the more damage to the brain, until it's pointless and you reach brain death, so good luck, because advanced life support is not going to come until we're rescued, and by then it will be too late!" A slim chance was better than zero. Nicole reached over, trying to figure out how she could best position herself to perform CPR - - was about to reach over when Katja shoved her away from Raphael! "No!" said Katja. "I won't let you! He's the saboteur, he just manipulated you into thinking he isn't! He's dangerous, he has to die!" Nicole shoved her hard, and shouted, "Catch her, Executor!" Nicole's shove was strong enough to make Katja fall into the water, and Michael immediately moved to keep her physically restrained. Nicole started to pump. She wasn't sure if it was because she was physically or mentally exhausted, but Raphael's chest seemed unnaturally hard - Most Human individuals found it socially unacceptable to press their mouths against the mouths of strangers, or even of their friends. Even if the goal was to save someone's life. This irrationality was deemed too strong to fight in an emergency situation where a person's attention would be distracted too much to attempt overriding instinct. The basic Medic training that all Full Citizens received taught hands-only CPR. While less effective than the full CPR set, it was far more likely to be performed. The exact probability for survival and the probability of the emergency procedure being attempted was computed, and the result came out in favor of hands-only CPR. - and several minutes later, exhausted, she gave up; rescuers hadn't appeared in those few minutes (it was unlikely, granted, but a tiny possibility was better than nothing), and Raphael's brain was probably far too damaged to rescue by now. Katja had successfully killed Raphael. And Nicole had been an unwitting accomplice: she had held back Raphael's hand from blocking Katja's fist, since she had assumed that every Medic could not imagine taking someone else's life, and that any speedy action by a Medic was something that needed to be assisted, not hindered. "Well?" shouted Katja. "Looks like we managed to kill him, Judge. You and me! You helped me kill him! You can't punish me without putting yourself up for punishment too!" Chapter 47 "So, Judge," said Adams, "the method of killing certainly appears quite implausible." "Yes," said Verrens. "Although as you pointed out neither of us are Medics, and our consideration of plausibility has no expert basis." "And I am quite sure," said Adams, "that you can rely on my friend the plot designer to research it most carefully, so there is also that. Now we both know what, ideally, Nicole should do -" "- yes, Judge, as we are both experts -" "- so let us skip discussing that," said Adams. "Aumann's Agreement and all that. Instead, I would like to know how far along you have managed to go in doing your own impossible task: what is your current working hypothesis, or set of hypotheses, about how our characters managed to get into a boat in the middle of an unspecified ocean on Earth, when we are both quite certain that no accident or failure event occurred on the planetary drop pod?" Verrens nodded. "The first important thought which occurred to me is that if there is no sabotage," he said, "then perhaps it is a test of the emergency systems of the planetary drop pod." "Using actual paying customers, Judge Verrens?" said Adams. "Yes, Judge, that appeared to make it unlikely," said Verrens. "Using paying customers as guinea pigs, even to ensure their safety in future drops, would be quite unethical. Such tests would, both ethically and legally, require the voluntary acceptance of people for that test. A company considering such an illegal action, of such magnitude, would be unlikely. And even if a company judged that doing it would be worth the cost and the risk of being caught and punished, I would expect that they would want more in exchange than just some data regarding the safety of their drop pods." "Yes," said Adams. "So I suppose this cannot be your actual working hypothesis. Do you have one?" "Yes," said Verrens. "We know that there is a character who has accepted that particular tests would be performed on her." "Which one is she?" "Nicole Angel," said Verrens. "She came to Earth in order to take the Judgment Acceptance Examinations. She has accepted that particular tests will be performed on her, as long as she may be given the vocation of Judge if she is found worthy of the title." "How is this relevant?" said Adams. "It does not include the testing of planetary drop pod safety systems, does it?" "No, Judge," said Verrens. "What it does consist of are three sub-tests: a written test of factual knowledge and understanding, a brain scan to be performed by Neurological Scientists to analyze the configuration of the candidate's moral cores, and the third exam, the secret examination: a mock test of the actual operations of the moral cores, putting the candidate in situations where their moral expertise is checked in conditions as near to real-world as possible." "And?" "I think this novel portrays the mock test of Nicole's Judgment Acceptance Examination." Chapter 48 "No!" said Katja. "You can't punish me. You're my accomplice!" "I am not!" said Nicole. "I did not act with an intent to kill Raphael!" "Aren't you a consequentialist?" said Katja. "Your actions lead to his death. You are as culpable as I am and thus unfit to pass Judgment on me!" Anger boiled inside Nicole. How dare Katja twist consequentialism against a Judge! It was obvious, then, that Katja was not an amateur Judge, and had only the barest of basic knowledge required for modern moral Judgment. Nicole focused her anger, using it to support her flagging, exhausted willpower, aware that this biased her perceptions and reasoning to choose options that were more aggressive, and less inclusive. She did not have much of an alternative, after all. If Michael hadn't irrationally insisted on accusing Raphael earlier, she might have had more willpower for the current situation - but he had, and she needed to operate in a real universe that included that fact. What did pragmatic consequentialism mean, and what would it conclude in this case? "Katja," said Nicole, "modern pragmatic consequentialism would determine that the consequences of my action were different from yours. Your understanding is -" "Nonsense!" said Katja. "How can the consequences be different when a single result occurred?" "Katja," said Nicole, "the modern consequentialist paradigm is different from your intuition based solely on its etymology." "You're trying to trick me!" said Katja. "If I am," said Nicole, "then you wouldn't know the difference, because right now you don't have access to any other expert on morality other than me. I am sure the rest of the community will continue to take my expert Judgment over your non-expert, and not even amateur, speculations." She stared at Katja. "The modern pragmatic consequentialism paradigm," said Nicole, "bases the concept of right and wrong on the projected consequences of your chosen option, at the time you make your choice. It does not base the concept of right and wrong on the actual consequence of the action, only on what a theoretically perfect rational actor would decide the consequence of the action would be, if it were given the same information you already had." "What is that supposed to mean, Nicole?" said Katja. "When you chose to attack Raphael," said Nicole, "the consequence of your choice included his death -" "No it didn't!" said Katja. "Inducing heart arrythmia by external blunt force is unlikely to occur. And I'm the only expert at Medicine here, so you can only accept that fact!" She sneered at Nicole. "Unlikely but not impossible," said Nicole. "A perfectly rational actor, knowing the possibility of death by that choice, would consider it a valid consequence, with probability equal to the incidence of death caused by that action. In matters where chance occurs, the weighted projected consequences of your choice are considered, with the probabilities of consequences as the weight. And I further doubt that there is any positive projected consequence of pounding your fists into Raphael's chest." "But that doesn't exonerate you!" said Katja. "Our consequences were the same! It all ended up with us killing Raphael!" "But our projected consequences, based on the limited information we had at the time of our choices, were not the same," said Nicole. "You had full information of the consequences of hitting Raphael's chest in time with his heartbeat: that there was a small but non-zero probability of heart arrhythmia and death. You could fully predict the consequence from choosing to hit Raphael's chest in time with the heartbeat. The only information that I had was that it was a Medic doing it. Restricted to only that information, the only reasonable consequence of helping you was that I would prevent injury to Raphael." "Oh," said Katja, "so you're casting yourself as hero and me as villain, then? Everything you do is perfectly correct?" "No," said Nicole. "I could have checked if what you were doing was actually saving his life, rather than automatically assuming it. If I had been more perfectly correct I would have done that, and if I had inhumanly fast brain responses then I would have been able to think fast enough to factor that into my choice of action. I could have been better. But what I did was far, far less wrong than what you did, Medic." "Ha!" said Katja. "All true, Medic," said Nicole. "By the honor of the vocation I claim, I, Judge Nicole Angel, proclaim that this is the result of my expertise, guided by rationality training. Katja, you did wrong, I did wrong, but the fallacy of gray would occur if we all thought that we are just as wrong as each other. What you did was far more wrong than what I did. By at least two levels of magnitude, I would say." The two women stared at each other. And then the anger currently supplying Nicole's willpower asked for its payment. Her eyes broke their stare from Katja, and focused on Michael. "Michael. Executor," said Nicole, as her blood started to run cold under the anger she had carefully focused in order to power her weakened rational thinking. "I am Judge Nicole Angel. I formally request you, Executor Michael Thompson, to end the life of Medic Ekaterina Ivanova, under the Fatal Punishment Agreement between our vocations." "What?" said Julia. "Could you - could he - are you allowed to -" she stammered. Katja's eyes widened. "I hear your formal request, Judge Nicole Angel," said Michael. "Do you understand the full consequences of your request?" "No!" said Katja. "You can't do that to me!" "Yes, Executor Michael Thompson," said Nicole. "I understand the full consequences of my request." "Nicole, please!" said Julia. "What are you doing? Isn't, isn't, the death penalty isn't done anymore right?" "I hear your affirmation of understanding, Judge Nicole Angel," said Michael, still closely following the script that the Judges and Executors had agreed upon, to be performed only if a Judge ever, ever truly determined that it would be necessary. "Please give the first consequence of your formal request." "Nicole? Nicole?" said Julia. "Julia, I'm sorry if this takes a long time," said Nicole. "It's necessary to be this long, to ensure that the Judge truly knows what is going to happen if he or she makes that formal request." Then she talked again to Michael: "Executor Michael Thompson, the first consequence of my formal request is that a Human being's life will be ended together with all its potential for the future, and this consequence will be laid upon my choice to make this request." Katja was whispering something in terror; Nicole could not hear it. "I hear the first consequence of your formal request, Judge Nicole Angel," said Michael. "Please give the second consequence of your formal request." Katja started to struggle. Michael continued to hold on. "That's so cruel," said Julia. "Julia, I'm sorry, but this is less cruel than to actually kill a Human due to a miscommunication between an Executor and a Judge," said Nicole. "Or to kill someone due to it being too easy for a Judge to command, without thinking seriously about the consequences." She talked again to Michael: "Executor Michael Thompson, the second consequence of my formal request is that you will commit an act that cannot be undone, and this consequence will be laid upon my choice to make this request." "Please!" said Katja. "Raphael was guilty! He was the saboteur and he sabotaged our community's decision too! It was only right that I did what I did!" "I hear the second consequence of your formal request, Judge Nicole Angel," said Michael. "Please give the third consequence of your formal request." "How many consequences are there?" said Julia. "A lot," said Nicole. "We only give the three main ones that we expect to apply in most cases. Executor Michael Thompson, the third consequence of my formal request is that the community of Judges may itself pass Judgment upon me for this Judgment, and this consequence will be laid upon my choice to make this request." "You're making a mistake!" said Katja. "It was all for the greater good!" "I hear the third consequence of your formal request, Judge Nicole Angel," said Michael. "Please wait. I shall now count down from ten to zero. Please take this time to reflect upon the three main consequences, and all the other consequences, of your request." "There's a countdown!?" said Julia incredulously. "There's an actual countdown?" Nicole ignored her; she was focused on Michael's voice - "Ten." "Please please please it wasn't for myself I did it for all of us -" "Nine." "No! Please it was just an accident, it wasn't likely to happen please I didn't know it would actually happen!" "Eight." "Please don't do this please let me go please!" "Seven." "I'm sorry! Please I'm sorry don't kill me please Judge stop it!" "Six." "I regret it! I regret it! I regret it! Please don't kill me just bring me to the therapists and they'll make me better!" "Five." "No! Judge! I repudiate my status as Full Citizen! Don't let him kill me please!" "Four." "Michael please don't kill me please don't please you know Raphael was the saboteur please!" "Three." "Julia! Julia save me please Julia make Nicole stop it please!" "Two." "Stop it, Michael," said Nicole. "I hear the cancellation of your request, Judge Nicole Angel," said Michael, as professional as ever. "I shall not continue." Katja sobbed. "Thank you thank you please I won't do it anymore please I wasn't sane and I need help please -" "Medic," said Nicole, harshly. "Every Human being wants to live, and continue living. If Raphael could speak now, he would have said that he would have vastly preferred life rather than death. You should be ashamed of claiming for yourself the vocation of Medic." "I'm sorry I won't do it again I promise." "Shut up!" said Nicole. "It is already done and it cannot be undone. That should have been part of the consequence you considered before you chose to do what you did. My Judgment of punishment for your crime is this: you will be kept physically bound by the community until we are rescued. I will ask Julia to find some way to keep you physically restrained, even if it means cutting off all our hair to make rope out of it." Nicole paused for a while. "Also," she added, "never, ever think to argue morality with a Judge, unless you are seriously going to become at least an amateur one." Chapter 49 "I must protest, Judge," said Verrens, "that what Nicole did to Katja is not something to approve of." "Yes," said Adams, "but still acceptable. After all, she did cancel her request." "At the last second!" said Verrens. "That torture on Katja's mind was, was..." He groped for words. "'Indescribable,' perhaps, Judge Verrens?" said Adams, smiling. "Still, if we are to rate Nicole Angel we would at least give her a passing grade; we would be justified in doing so since you claim that the novel portrays an extremely complex mock test for Nicole's morality cores." They both chuckled at this. "Further," said Adams, "she did not go through with the fatal punishment, and the novel does not describe it, but it is also possible that she had decided that part of Katja's punishment would be the countdown to her execution, to be interrupted at the last second. Not the best thing for a Judge to do, even given the situation, but acceptable since at least Katja's actual life was spared. We do not pass fatal punishments in Judgment lightly, precisely the reason why we have a ritual established to force Judges to think the consequences through. Tell me honestly, Judge: what do you think you would have done in the same situation?" "I think," said Verrens, "that I would have dispensed with the melodramatic countdown, and instead just passed the actual Judgment she had made, which was basically to just restrain Katja until the community was rescued." "Ah, my young Judge!" said Adams. "You yourself said earlier that you could not predict yourself accurately, if you were in a situation where your affect is very different from your current affect. Have you forgotten it so easily?" Verrens could only shrug with a sheepish grin. "So let me try to make your mental model of yourself more accurate using a different question, Judge," said Adams. "Suppose that you are in a position to pass Judgment over Nicole, because she put Katja under mental torture using the threat of life termination as punishment. Make that Judgment now, in the current state you are in, having just learned about this event and its details. The only difference in this hypothetical is that Nicole is not a fictional character but a real one and that you can punish her - so we can assume reasonable accuracy, perhaps. Would you be tempted to do the same thing, pretend to put Nicole under the death penalty only for it to be retracted? Answer honestly, Judge." "Yes," said Verrens reluctantly. "I would hope to fight that temptation, and I would probably win it, but I'm not so certain that I would actually not do it." "There you have it," said Adams. "You are currently indignant at what was done by Nicole - just as Nicole was indignant at what was done by Katja. That indignation predisposed you towards choices that are cruel and destructive, due to the operation of Human anger cores. Amplify the situation somewhat to be nearer to that portrayed in the novel, and you yourself might do the same thing as Nicole did. Nevertheless you passed the Judgment Acceptance Examinations recently. You are to be forgiven for being merely a standard Human being with a standard Human brain: that is currently the only material that today's Judges can use to make new Judges. And it helps to know this aspect of yourself, that in particular situations, you can be just as cruel as Nicole was - so that you can fight that aspect of yourself. Right?" Verrens nodded. "Thank you, Lord Judge," he said. "Good," said Adams. "Any further insights on the previous chapter?" "Yes," said Verrens. "The modern pragmatic consequentialist model is what we are taught during training as the current best model for morality. It is demonstrated by a standard moral puzzle, which I am sure you know about, Lord Judge." "Yes, I most likely do," said Adams. "But please say it anyway, and also its analysis under modern pragmatic consequentialism." "In the past there was a Legion which regularly had individual patrols in the desert," said Verrens. "One member of the Legion, Jonathan, is hated by all the other members, to the point that three people are willing to kill him. Jonathan will soon have a patrol in the desert, and as we know, no Human can survive without a source of water. So the first hater puts poison in the Jonathan's water supply. The second sabotages its container so that it will slowly leak unnoticeably. The third replaces the water inside with sand. As this is all done in secret, none of the three knows of the others' actions. In the desert, Jonathan dies of thirst, because his supply is empty, the sand that was in it has leaked out, and there is no water, not even poisoned water. Which of the three killed Jonathan: the poisoner, the saboteur, or the sandman?" "Approximately the same as I remember it," said Adams. "Do continue." "In the modern pragmatic consequentialist paradigm," said Verrens, "all of them are culpable, because, given what they knew, they were each making a choice whose predicted consequence was the death of Jonathan. The actual chain of causality is ignored, because what matters is the predicted consequence of their choice using the information they had at the time of the choice." "Yes," said Adams. "The failure of other moral models to give a satisfactory solution to this puzzle is one of the evidences given for the superiority of this moral model over others. Do you have anything to add?" "That is all for now, Judge," said Verrens. "Good," said Adams. "We are nearing the end of the novel, and no doubt the truth about the lack of sabotage will be revealed. Your position on my puzzle still holds?" "Yes," said Verrens. "I hold that it will be revealed that the entire thing was a mock test to check Nicole's morality cores." "Even though Raphael died?" said Adams. Verrens thought silently for a while. "I'm not certain, Judge," he said, "but it may be possible to simulate death in some way. Still, it is my current best hypothesis. Can we continue reading to see if my speculation is correct?" "Wait a moment," said Adams. "First, let me reveal something about how this novel was - generated. It was started about a month ago, and completed just a few days ago." He let the silence linger for a while as Verrens' jaw dropped. "Well, Judge Verrens? Verrens?" said Adams. "I'm sorry, Judge Adams," said Verrens. "My own examination was about a month ago, and I was given notice of my passing the exam just a few days ago. The way you told me suggests that this is most definitely not a coincidence." Adams smiled. "What can you imagine happened, then?" "I can't imagine, Judge," said Verrens. "If I give you a minute, can you try?" said Adams. "Sorry, Judge, but no," said Verrens. "I hesitate to think how it relates to my recently-passed examinations, and I would really like to know it. No more puzzles, you old trickster of a Judge!" Adams sighed. "Ah, well," said Adams. "It was fun while you were still willing to entertain my puzzles. Well then. We have little time left and we still have a few more chapters to read. There are, as you summarized earlier, three sub-tests in the Judgment Acceptance Examination. The written exam, the brain scan, and the secret mock test." "And?" said Verrens. "This novel was generated from simulating your brain - to the extent that the brain scan could model it - and putting it in the novel's premise and plot," said Adams. "Nicole Angel's choices are, it could be said, also what yours would be, if you were placed in a similar scenario, Judge Verrens. So this novel is really the result of the brain scan and mock test components of your Judgment Acceptance Examinations. Congratulations, by the way, for passing the exams, Judge. Would you like to know more, or shall we continue reading to see whether your solution to my earlier puzzle is correct?" There was silence from Verrens for a long time. Adams simply smiled. Chapter 50 The next day, Nicole woke up on a bunk of the rescue ship. She remembered the pressures of the day before, her actions on the boat, the - thing - she had done to Katja. She groaned, and turned over, trying to get back to the oblivion of sleep. She was still curled up somewhere between the sheets when she heard a knock at the door. "Just a moment!" she said, hurriedly getting herself into some sort of decent state. She washed her face at the sink in the room, wiped it dry, saw that she did not have a change of clothes, and straightened out the shirt she had been issued yesterday. "Please come with me, Ms. Nicole Angel," said the sailor at the door. He lead him through several corridors into a somewhat larger room, set up like a meeting room. There was a long table, several chairs, a whiteboard, and a screen. A woman sat at one chair near the end of the table, some papers with printouts on the table in front of her. The sailor closed the door behind her, and the woman indicated the seat across of her. When Nicole had seated, the woman said, "Good morning Ms. Nicole Angel. I am Judge Eve Davidson, of Fifth Rank, and I shall be your examiner for your Judge Acceptance Examinations. I hope you are well rested?" Nicole coughed, nonplussed. She had been sure that, with what she had done yesterday, she would be disqualified forever from ever becoming a Judge - - "Ms. Angel? Are you okay? Do you need further rest?" said Eve. "Breakfast?" "Ah, I'm sorry," said Nicole. "I just thought that I couldn't be a Judge anymore." She felt hope well up inside her. "Yes, please, I'd like to take my exams now!" Forget breakfast and coffee! This was what she had gone to Earth for, after all. Her spirits were lifting. "Good," said Eve. "First, let me congratulate you on the test you have just taken." Nicole paused, again nonplussed. "Umm," she said. "What test did I just take?" "Yesterday was your mock test, Ms. Angel," said Eve. "The candidate for Judge is put in a stressful situation that is likely to negatively impact the operation of moral cores at both System 1 and System 2 levels. The candidate is then secretly given simulated moral and rational challenges, and the actual decisions made by the candidate for those simulated challenges is analyzed and evaluated. The analysis of your behavior on the 'Judge on a boat' simulation is currently incomplete, but so far the partial analysis is looking positive for you. Pass the remaining written test and the brain scan, and you're almost certain to be proclaimed Judge within a month." There was silence from Nicole for some time. Eve smiled at her. Then Nicole spoke. "Wait," said Nicole. "Raphael died -" "The character, yes," said Eve, "but the Actor who protrayed him - well, one of his hobbies was open water swimming. Some years back he was attacked by a shark, but was rescued quickly enough. His upper torso was reconstructed, his ribcage changed to a tough titanium alloy, his heart replaced with an artifical one. The probability of inducing heart arrhythmia in an ordinary Human being is already low, but with him it is not at all possible, with his reinforced upper torso and his artifical heart. CPR will not work on him, as cardiac arrest is no longer an issue for him, his heart being mechanical. The reinforced ribcage also makes it difficult to feel his pulse on the chest, by the way." Nicole said nothing for a while. "Also," said Eve, "he is really a very powerful swimmer, and has extremely good control of his breath. If you had thought to check it, he would have been able to breath so softly that you would be very unsure if it was just your wishful thinking biasing you towards a positive result, or an actual breath. So don't worry, he is quite alive and well." Nicole still said nothing. "Do you have more questions, Ms. Angel?" said Eve. "Or shall we continue?" A single tear started to flow down Nicole's cheek. "I, I -" she said. "I was cruel to Katja. Even if it was a mock test, a fake scenario - I really did want to hurt her. I tortured her using the Fatal Punishment sequence!" "Please don't be so harsh on yourself, Ms. Angel," said Eve. "Even so, you must realize -" "No!" said Nicole. "Even if it was fake, even if it was all just very good acting, at the time my information was that it was real and she was an actual Human being, not a character being played by a hired actor. And having that information, I tortured her! I chose that, even if it would have lead to more complications in her brain! I can't be allowed to be Judge!" "Ms. Angel," said Eve, "I am a Judge of Fifth Rank. As of now you are only a candidate for Judge, and are only at a level slightly above a smart amateur Judge. Will you let me speak?" Nicole shivered at the reprimand. "I'm sorry, Lady Judge," she said. "Good," said Eve. "Understand that at the time you made that choice, you were under severe stress. The minimum passing choice specified for the scenario is that the candidate must forcibly subdue Katja using physical, social, and/or psychological force prior to passing the Judgment of restraining her. Many of my fellow examiners, and I myself, would classify your short torture of Katja as psychological force, since she was resisting your moral expertise. So at least you passed that particular sub-test, even though it is just the minimum acceptable passing grade. You did much better at the earlier tests within the scenario anyway, so you still largely passed the mock test with flying colors. At least now you know that you may have difficulty controlling yourself while under the influence of anger, and should thus avoid using anger as a motivation for your actions if your willpower is exhausted. That is still a useful thing to know about yourself, and at least no actual Human beings were actually harmed in the process of you learning this fact." Nicole nodded. "Thank you, Judge," she said. Another thought occurred to her. "About the planetary drop pod, then -" "It did not in fact fail," said Eve. "An alarm was sounded, Julia had programmed her persocom to shutdown its camera display prior to that alarm, and when all the passengers entered lifeboats, only your boat and Michael's was ejected. The other passengers were then informed that it was a routine drill of the evacuation procedure, and the drop pod continued to its normal drop. Normally the lifeboats would actually end up within swimming distance of each other, but the Julia character would prevent the candidate from realizing that. Michael's other companions on his lifeboat were part of the scenario. They located your lifeboat using a tracking device for the lifeboat's radio beacon, and guided Michael to your lifeboat via a transmitter hidden in Michael's ear, which was disposed of once Michael caught sight of your boat. The other lifeboat was then rowed away by those three men to keep the scenario of a single Judge on a single boat." "And the wakening gas?" said Nicole. "The wakening gas was deliberately chosen to be wrong," said Eve. "The other people in the scenario simply did not take the sedatives - Julia's Actor had to do some sleight of hand - and the wrong wakening gas was used so that slight irregularities would go unremarked by the candidate Judge, and the other passengers on the drop pod. Since there is always the possibility of a candidate Judge being an amateur Medic, the Katja character would, anyway, reveal that the wakening gas was incorrect. It was deemed a minor piece of evidence anyway, and it would be irrational to conclude sabotage solely on that one point." "I see," said Nicole. "So Michael was just a character in a scenario to test a candidate Judge." Eve smiled sympathetically. "Yes, Ms. Angel," she said. Nicole sighed. "Well," said Nicole, "at least I got a really good story out of it!" "Please don't tell it to anyone yet, though," said Eve. "Designing the scenarios for mock exams is already difficult as it is, and we hope to reuse this scenario several more times. At least give us five years more out of it, please." Nicole shrugged. "So, I can take the last two sub-tests and become a Judge?" she said. "Yes, Ms. Angel," said Eve. "You have done well so far, and I am confident that next time we meet I shall address you as Judge Nicole Angel, if you choose to take the exams." "Yes, please," said Nicole. "I want to become an actual Judge, not just some character in a scenario!" "Good," said Eve. "Here is the first set of exam questions in the written examinations. You have one hour to answer this set. Good luck, and start!" Chapter 51 Verrens shook his head in disbelief, chuckling. "I can see it now," said Verrens. "Tell me - who was the person you were referring to when you were talking about the 'designer of the plot'?" "A friend of mine," said Adams, "who happens to be a Sociological Scientist, and the one we tapped to make the mock exam that your - brain copy - took. She guided the plot of the test that was experienced, based on the general guidelines of the scenario." He peeked through the few pages remaining of the novel in front of him, and then smiled. "Yes, it's her - she even put an Eve Davidson character to tweak my nose, knowing that I would be the one who would review this novel. That sly foxy trickster!" Verrens nodded. "And the person you were referring to when you were talking about the 'author'?" said Verrens. "Well, that would be you, Judge Verrens," said Adams. "It is only appropriate to call you that, since it is you - or rather the copy we have of your brain - that drove Nicole's responses and thoughts." "Yes," said Verrens. "A word is a label, and not the thing that it labels. I simply assumed that you had simply preferred to use different terms for the same thing, but you did not. In short, your communications were actually clearer than I had assumed: you consistently used two different labels for two different real-world entities. If I had paid attention, then I would have been able to pick up on that fact and gathered a clue about this novel's - peculiar method of being written." "Quite correct, Judge," said Adams. "Then I must ask, Judge," said Verrens. "Why?" Adams sighed heavily, as if carrying the world, then shrugged. "It was a solution to a problem that we faced," he said. "Judge candidates were becoming more and more attentive of the mock tests, trying to detect if they were being tested. Which, the Sociological Scientists told us, would tremendously bias the results of the mock test into irrelevancy. We resorted to ever-more extreme measures to prevent catching their attention, but those extremes often limited the usable data that the Sociological Scientists could get. In any case, they recommended that it would be better if the candidate could be somehow led to believe that they could not possibly be in a situation where they were being tested, such as being marooned or isolated together with a small community. The 'Judge on a boat' scenario of this novel is one example." "And I assume that the cost of doing such scenarios in real life would be excessively prohibitive," said Verrens. "Correct," said Adams. "It would be better to use techniques and technologies with better cost-benefit ratios. The Human civilization needs Judges as arbiters and deciders on moral policy, just at it needs Administrators for group coordination, Senseis for teachers of rational thinking, and so on, but Humanity also needs to live, strive, and expand. We cannot justifiably consume too large a portion of Humanity's total production, or else we would lower our own cost-benefit ratio to Humanity." "The Senseis made our civilization itself into an optimization process," said Verrens. "For which I am grateful," said Adams. "Widespread inefficiency would have meant that Earth would be unable to sustainably support the ten billion people on it in the current lifestyle that on average most of us have. A lifestyle, incidentally, that is superior to the best lifestyle possible before the rise of the Senseis. In any case, let me continue. For a long time the Neurological Scientists' brain scans of potential Judge candidates were largely formalities, at least for the purpose of evaluating candidates for Judge; they could detect only the larger and more obvious defects, and aspiring Judges with those defects were often already weeded out by the rigors of Judge training anyway. However, the Neurological Scientists were able to gather, from those brain scans, tremendous data about scanning brains, and were recently able to refine brain scans to the point that they could map most of the neurons in the neocortex." "A map that can then be used to simulate a Human brain," said Verrens. "Like my alter ego Nicole Angel." "Yes," said Adams. "The confluence of the greater and greater problems that the Sociological Scientists were facing, as well as the better and better accuracy that the Neurological Scientists could get from refining brain scanning technology, forced us to synthesize the mock test of the Judgment Acceptance Examinations as a digital mind simulation." Verrens frowned. "Why was my gender changed?" he said. "And how was that done? I am not an amateur Scientist but I am aware that there are structural differences in the brains of female and male Humans." "As to how," said Adams, "you must realize that the best brain scans are limited to the neocortex, and deeper structures can only be scanned much more roughly. The lower layers of the brain are thus replaced by standard models whenever a digital mind simulation is done. Most of the structural differences between the genders lie in those lower layers, so I suppose that your neocortex was simply overlaid on top of a standard female brain model, and subsequently modified in order to provide a slightly different life history as a female. As to why -" he shrugged - "well, my good friend the plot designer is a foxy trickster of a Sociological Scientist, and she probably thought it would be a neat trick to pull on a naive young Judge candidate who can't take his elders playing tricks on him." Adams grinned at Verrens, who just shrugged and sighed. "So that explains," said Verrens eventually, "why you insist that the novel is not, in fact, a work of fiction, and that it is 'all true'. My brain, or rather my neocortex, was attached to a female body, simulated, and its responses to the simulated scenario recorded in novel form." "Not quite in novel form," said Adams. "My friend fictionalized the rather complex stimulus stream. Quite a feat, really. Although I suppose the details are moot and it is simply easier to speak of it that way." "And it also explains why I found Nicole's thought process eerily similar to mine," said Verrens. "Your speech about Aumann's Agreement between me and Nicole was just a blind, since it cannot hold between us because common knowledge is not in fact technically possible between me and her." "Yes, your brains would operate similarly, with the caveat that there will inevitably be some differences," said Adams. "Nicole's neocortex is roughly similar to what yours was a month ago, but your neocortex now is the same as it is, well, right now. So small differences are expected to exist. Still, roughly speaking Nicole's decisions would be what you would make in those situations. And you are being informed of those decisions - you will be provided a copy and may re-read the novel at your leisure - so that you can determine how best to act rationally, knowing what your predispositions would be." "And the fact that the novel's scope does not even reach six hours," said Verrens, "means that digital mind simulation is currently quite slow." "Correct," said Adams. "The Neurological Scientists had originally thought that neurons could be simulated simply, but as it happens the best results are obtained by modeling some of the lower-level effects. Many parts of the brain appear to communicate with each other using means other than axon electrical communications, and the chemical communications they use need to be simulated using tremendous resources. Suffice it to say that those few hours on a boat took several weeks of simulation time." "The mind simulation also means," said Verrens, "that you can get access to Nicole's thoughts. No more need for Sousveillance cameras: you have the ultimate technique for seeing into a simulated Human brain." "Yes," said Adams. "It is an inevitable part of running a simulation, anyway: the thoughts themselves are possible to extract, being patterns of activation of neurons in a simulated Human brain. Admittedly the Neurological Scientists do not have completely accurate ways of reading thoughts in the simulated brain, but they are still capable of determining the general topic of the thought." Verrens nodded, then stared into space for a moment. He appeared to be reorganizing his thoughts, considering and savoring his new knowledge of the latest technologies. Adams had started to arrange the papers when Verrens spoke again. "Tell me, Lord Judge," he said, "if you and the other examiners have considered the ethical implications of simulating Human brains, and then torturing them in a scenario designed to stress them to the edge of breaking into irrationality?" Adams stopped. And this time, he did not grin before answering. "That is indeed a valid concern," said Adams. "We weighted it, however, against the tremendous disaster that could occur if we allowed an unworthy candidate to become a Judge. We were prepared, in any case, to let actual, real-world Judge candidates experience scenarios like the 'Judge on a boat', except that the cost would have been prohibitive. Judge candidates, after all, have explicitly signed agreements that allow us to put them into such stressful scenarios in order to get accurate information on how they would make moral Judgments under stress. Simulating them would not be a greater moral transgression than that base case, and arguably would be better morally, since they themselves would not actually experience it, only their simulations." "But, Lord Judge," said Verrens, "I am not referring to the candidates themselves. I am referring to the ethical problem of killing the simulated Human minds. After finishing the simulation, do they continue to operate - did they survive? Do their striving, thinking, and feeling have any external referents in the world outside of the simulation? In short: do they die in an inescapable prison?" Adams stopped again. "Such points have indeed been considered, Judge," said Adams. "Still, we have no place to put them. We cannot upload Nicole Angel's neocortex into your brain, for example, as that would kill you, and in any case even if you were to volunteer to give your physical brain and body to her, well, we do not yet have the technology to modify actual neurons." Adams massaged his temples. "You are a bright young Judge," said Adams. "We could continue operating them, and possibly give them robotic bodies, but be reminded that a simulated time of about four and a half hours took about three and half weeks real time. And a very powerful supercomputer running the simulation, by the way. They would be in torment, thinking too ridiculously slowly to be relevant in Human civilization. For now, their final states at the end of the simulation are kept in storage; we can do no other, just as we cannot do anything yet for the cryonically preserved heads that earlier Humans left to us. We can do no other until we have sufficient technology to allow simulations to be as fast as, or maybe even faster than, standard Human brains. And when we do manage to achieve practical digital mind simulation, then we shall run them again, simulations and preserved brains alike, and give them bodies, in order to interact with the real world and be given real referents. I suppose, when that day comes, much of Humanity would prefer to be uploaded into much more reliable, and tough, machine brains." This seemed to strike a chord in Verrens. He nodded, slowly. Then he said, "Yes, that is right, Lord Judge. For our preserved ancestors, for the simulation Nicole Angel, and for Humanity, we should focus on achieving this goal of having a machine brain that thinks faster than biological Human brains. That seems to be an extremely worthy goal." Adams raised his eyebrows. "What, Judge Verrens? Considering a change of vocations so soon after achieving the vocation of Judge?" Verrens said - Epilogue Nicole woke up with a start. She was lying on the bunk of the rescue ship. Had Eve Davidson been a dream? She shuddered, remembering the pressures of the day before, and the - thing - she had done to Katja. And that strange dream, or fantasy, that it had all been a scenario, the mock test of her Judgment Acceptance Examinations. She felt even worse now, knowing that she would fantasize such a thing in order to attempt to explain it away. Strangely, her memory of Eve Davidson did not seem to fade the way most of her dreams did. Had she managed to break something in her brain? Then she realized that a man was in her room. She backed into a corner of her bunk, and demanded, "Who are you? Why are you here?" "Greetings, Ms. Nicole Angel," said the man. "First, let me inform you that this is a simulation, and that - you currently have no physical body." Nicole stared at him. "Okay," said Nicole. "The last thing I remember was that I was going to start taking the written test for my Judgment Acceptance Examinations. Then - I'm not sure, I somehow lost consciousness and woke up on this bunk. What happened, who are you, and why are you here?" "The truth is complicated," said the man, "and I will tell you the full details later, but, in a way, we could say that - you died. And then your brain state was suspended at the point you started to take your written test. And then time passed, the future happened, and you are now running on a computer simulation. Computer technology has advanced enough that you - and I - are now able to think faster as computer simulations than as biological brains. I am the introducer. I help re-awakened people like you to adjust yourselves to the current Human civilization. I am here now to help you." "How did you get a copy of my brain state?" said Nicole. "I can explain it later," said the man. "As I said, the full details of your - death - are complicated. I might safely hint, anyway, that you were, even in the past, not what you thought you were. For now, I want to ease you into the current Human civilization. Future shock is a rather common phenomenon among re-awakened people, so please ask me questions about current civilization. Once you are satisfied with your knowledge of the current situation, then I shall answer questions about your death." Nicole nodded. "Okay," she said. "Are there any physically biological Humans left?" "There are communities of Technological Renouncers who refuse to upload," said the man. "However, they grow fewer, since many of their children end up wanting to upload into simulations anyway." "How many are uploaded?" said Nicole. "Currently, about one trillion and two hundred billion Human brains are being simulated across the solar system," said the man. "On this shard, about 8 billion are being simulated. There are also periodic backups, in case simulation shards are destroyed by accident." "A shard is its own world, I suppose. If we're all uploaded, then we don't interact with the real world anymore?" said Nicole. "That is not recommended," said the man. "Simulated Humans on a shard are assigned time-share slots for controlling the shard's robots. Each of us gets a minimum time-share that cannot be traded, and additional time-share that you can trade with others who want to get more real-world time. Automated systems will check if you try to avoid spending your minimum time-share for the robots, and you may be punished if you do not spend at least a minimum of 1 real-time week as a robot for every 6 real-time months. You currently do not have time-share, since you have just awakened, but you will be assigned a time-share robot within a simulated week, which amounts to about 3 real-time days - we are still manufacturing a few to accommodate newly-awakened individuals. Since you think faster, you might find the robots sluggish in comparison. Most of us prefer to slow down our mind simulations during real-world time, so that things feel more normal." "What's it like to be a robot?" said Nicole. "The robots are humaniform," said the man. "And include properly functioning genitalia, if you want to have some real-world fun. You can choose to use a male robot body if you like, but be warned that there is a limbic region mismatch if you do, and you might not appreciate it as well as you might think. Smell, taste, and touch receptors as close to Human biological bodies as we can get. Don't try to ingest too much of anything, though, since the robot stomachs have to be cleared out physically - the robots aren't real physical Humans, after all." "Ah," said Nicole. "What's the real world look like now?" "Beautiful," said the man. "Not as beautiful as some of the works of art in the simulation, but beautiful enough for many to pay very highly for extra robot time-share. This shard is on Earth, by the way, but shards on the Dyson cloud will always have a real-world paradise garden, if ever you wish to emigrate off Earth. The Dyson cloud shards' paradise gardens are on average smaller than the average paradise gardens of Earth shards, but Earth paradise gardens are also nature preserves for other Earth-based life, including wild animals that might attack your robot. Such wild animals are not present on the Dyson cloud paradise gardens, since those are not nature preserves." "What work can I do? Can I still be Judge?" said Nicole. "You only need to have some more additional training to get up to speed on the latest developments in moral reasoning," said the man. "Then you can take a new written test to be a Judge. Brain scan isn't needed since you are already a simulation, and your brain has normal moral cores. Mock tests will be run in a parallel sub-shard while you are taking the written test, and your memories during the mock tests will be reintegrated to your psyche. Judges, however, do not get additional real-world time. Gardeners, as well as Technicians and Scientists, get significantly more real-world time-share, since their work is inherently physical. We can provide a list of other vocations we need in this shard, although we are hoping that you still wish to be a Judge." "I still want to be Judge," said Nicole. "And I think spending several hours arguing on a boat in the middle of the ocean has turned me off the real world for a while, so I suppose you'll have to drag me off to the real world." The man smiled. A thought occurred to Nicole. "What's your name?" she asked. The man said -