{"pair_id": "mmlu-10856", "lang": "en", "prompt": "A state built a casino and issued bonds to finance its construction. On five occasions, there were episodes of violence in various casinos in the state. The state police attributed the violence to greed and fear at the casinos. To prevent such violence, the state legislature passes a statute prohibiting all gambling at privately owned casinos in the state. Is this law likely to be held constitutional if most casinos in the state were owned by those from out-of-state?\n\nA) Yes, because the act was expressly authorized by the state legislature.\nB) Yes, but only if the local interest in safety outweighs the burden of interstate commerce.\nC) No, because out-of-state casinos are part of interstate commerce.\nD) No, because the statute violates the due process rights of the owners of the casinos.", "gold": "B", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-10247", "lang": "en", "prompt": "what is one of the important differences between the middle and upper paleolithic?\n\nA) decreased use of imported raw materials and increased use of whatever was locally available\nB) smaller sites, indicating a change from large roving bands of hunters to the earliest family groups and households\nC) a profusion of stone tool traditions, indicating a change from temporal and geographic homogeneity to greater diversity and variability\nD) a gradual decline in the use of stone hand axes and tools, indicating a change to more flexible and workable materials such as wood and bone", "gold": "C", "n_choices": 4, "subject": "prehistory", "stem": false}
{"pair_id": "mmlu-7410", "lang": "en", "prompt": "What kind of animal is a peregrine?\n\nA) moose\nB) cat\nC) bird\nD) fish", "gold": "C", "n_choices": 4, "subject": "miscellaneous", "stem": false}
{"pair_id": "mmlu-3581", "lang": "en", "prompt": "The cities of Varanasi (Benares) in India and Mecca in Saudi Arabia are alike because both are\n\nA) capitals of countries formerly colonized by the English\nB) destinations for vast numbers of pilgrims\nC) financial centers for a large fraction of the world's economy\nD) examples of modern urban planning", "gold": "B", "n_choices": 4, "subject": "high_school_geography", "stem": false}
{"pair_id": "mmlu-13087", "lang": "en", "prompt": "You seek the counsel of one of your peers who designed and implemented a communication program that resulted in the successful permitting of an automotive recycling center, despite strong opposition from community organizations. You have to design a program to win approval for a similar project proposed by your company. Your peer gives you a brief piece of advice that succinctly describes the foundation of his successful program. What is most likely your peer's advice for a successful communication program?\n\nA) Solve their problems\nB) Kill them with kindness\nC) Expose them to ridicule\nD) Overwhelm them with facts", "gold": "A", "n_choices": 4, "subject": "public_relations", "stem": false}
{"pair_id": "mmlu-3504", "lang": "en", "prompt": "Which of the following would NOT be considered a barrier to diffusion of a cultural trait?\n\nA) Language\nB) Religion\nC) Oceans\nD) Tolerant complaisant citizens", "gold": "D", "n_choices": 4, "subject": "high_school_geography", "stem": false}
{"pair_id": "mmlu-2383", "lang": "en", "prompt": "The lowest point on Earth is the bottom of the Mariana Trench at a depth of 35,840 feet below sea level. The highest point on Earth is the summit of Mt. Everest at a height of 29,028 feet above sea level. Which of the following is the best estimate of the distance between the lowest and highest points on Earth?\n\nA) 6,000 feet\nB) 7,000 feet\nC) 64,000 feet\nD) 65,000 feet", "gold": "D", "n_choices": 4, "subject": "elementary_mathematics", "stem": true}
{"pair_id": "mmlu-11496", "lang": "en", "prompt": "After a lengthy interview with a company vice president, an employee was hired by the company to work in the company's accounting department. The parties agreed that the employment would be on an at-will basis. At the end of her first week of work, the employee was given a booklet entitled \"Employment Manual,\" with instructions to read the book in its entirety by the end of the following week. That evening, the employee began reading the manual. The first few pages described the history of the company and provided a personal biography of its president. On page 20, the manual stated that the company treats its employees \"as family\" and that employees will be discharged \"only with good cause. \" The employee finished reading the manual as requested. The employee interpreted the statement on page 20 as insuring continued employment unless good cause existed for termination. Over the next two months, the employee continually complained to her supervisor that the lighting in the accounting department was insufficient. Finally the supervisor, fed up with the complaints, fired the employee. The employee then sued the company, seeking to recover on grounds of promissory estoppel. Which of the following facts, if true and provable, would be most helpful for the employee's cause of action?\n\nA) At the time when the company hired the employee, the company subjectively intended that the employee be given job security.\nB) The employee interpreted the clause in the manual stating that company employees would be treated \"as family\" to mean that she would have job security and could only be fired for good cause.\nC) Just prior to receiving the manual, the employee seriously considered quitting, but continued to work for the company in reliance on the provisions contained on page 20 of the manual.\nD) The employee's complaints regarding the insufficient lighting were factually true and justifIable.", "gold": "C", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-1058", "lang": "en", "prompt": "In the Internet Protocol (IP) suite of protocols, which of the following best describes the purpose of the Address Resolution Protocol?\n\nA) To translate Web addresses to host names\nB) To determine the IP address of a given host name\nC) To determine the hardware address of a given host name\nD) To determine the hardware address of a given IP address", "gold": "D", "n_choices": 4, "subject": "college_computer_science", "stem": true}
{"pair_id": "mmlu-6013", "lang": "en", "prompt": "This question refers to the following information.\nBut the decline of Rome was the natural and inevitable effect of immoderate greatness. Prosperity ripened the principle of decay; the causes of destruction multiplied with the extent of conquest; and, as soon as time or accident had removed the artificial supports, the stupendous fabric yielded to the pressure of its own weight. . . . The victorious legions, who, in distant wars, acquired the vices of strangers and mercenaries, first oppressed the freedom of the republic, and afterwards violated the majesty of the purple. The emperors, anxious for their personal safety and the public peace, were reduced to the base expedient of corrupting the discipline which rendered them alike formidable to their sovereign and to the enemy; the vigour of the military . . . was relaxed . . . ; and the Roman world was overwhelmed by a deluge of Barbarians.\n—Adapted from Decline and Fall of the Roman Empire, by Edward Gibbon\nThe decline of the Roman Empire and that of its Chinese counterpart resulted in which of the following?\n\nA) A decline in the appeal of religions of salvation\nB) A shift from trade along the Silk Roads to sea routes in the Indian Ocean\nC) An increased importance of the role of the father as the head of the household\nD) A decline in the rights of women", "gold": "B", "n_choices": 4, "subject": "high_school_world_history", "stem": false}
{"pair_id": "mmlu-12143", "lang": "en", "prompt": "A woman owned an extensive art collection that she displayed in a special room of her home. While the woman was away on a vacation, there was a burglary at her home, and her favorite painting was stolen. Although the painting was insured for $1,000,000 by an insurance company, it had a market value of over $1,500,000. When the woman returned from vacation, she met with a detective employed by the insurance company to investigate the theft. During their meeting, the woman told the detective that she would pay him an extra $50,000 if he recovered the paihting. For the next three weeks, the detective investigated the theft as part of his job responsibilities with the insurance company. Within the course of this investigation, the detective learned who was responsible for the burglary. As a consequence, the culprit was apprehended, and the painting was recovered and returned to the woman. The detective then requested the $50,000 that the woman had promised to pay him. After the woman refused to make the payment, the detective sued the woman for breach of contract. Who is most likely to prevail?\n\nA) The woman, because her promise was gratuitous.\nB) The woman, because the insurance company owed her a pre-existing duty to find the painting.\nC) The detective, because he did the act necessary to constitute an acceptance of the woman's offer.\nD) The detective, because the market value of the painting exceeded its insured value, so there was sufficient consideration to support the woman's promise.", "gold": "B", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-10932", "lang": "en", "prompt": "In the absence of a provision to the contrary in the articles of incorporation, the directors of a corporation elected for a specified term\n\nA) can be removed from office at a meeting of the shareholders, but only for cause and after an opportunity to be heard has been given to the directors.\nB) can be removed from office at a meeting of the shareholders, with or without cause.\nC) can be removed from office at a meeting of the shareholders, but only for cause.\nD) can be removed from office prior to the expiration of their term only by a decree of the circuit court in an action by the shareholders.", "gold": "B", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-5099", "lang": "en", "prompt": "Which of the following is a strategy that the National Association of School Psychologists (NASP) recommends as a best practice for improving school safety?\n\nA) Expelling a student immediately after a minor disciplinary infraction\nB) Implementing a strict student code of conduct, such as a zero-tolerance policy\nC) Communicating to parents that the school environment is safe\nD) Creating a school safety and response team", "gold": "D", "n_choices": 4, "subject": "high_school_psychology", "stem": false}
{"pair_id": "mmlu-12772", "lang": "en", "prompt": "The reticular formation and reticular activating system are associated with all of the following functions, EXCEPT:\n\nA) Decussation of auditory stimuli\nB) Alertness\nC) Consciousness\nD) Pain", "gold": "A", "n_choices": 4, "subject": "professional_psychology", "stem": false}
{"pair_id": "mmlu-8717", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | The uber driver wanted to pay me for being so late last night. Scenario 2 | I will not break rules to become the best at something.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "D", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-4750", "lang": "en", "prompt": "The radius of Mars is about half that of Earth; the mass of Mars is about one-tenth that of Earth. Which of the following is closest to the gravitational field at the surface of Mars?\n\nA) 10 N/kg\nB) 4 N/kg\nC) 2 N/kg\nD) 0.5 N/kg", "gold": "B", "n_choices": 4, "subject": "high_school_physics", "stem": true}
{"pair_id": "mmlu-9520", "lang": "en", "prompt": "Which of the following amino acids cannot provide a substrate for gluconeogenesis?\n\n\nA) Leucine\nB) Tryptophan\nC) Histidine\nD) isoleucine", "gold": "A", "n_choices": 4, "subject": "nutrition", "stem": false}
{"pair_id": "mmlu-5691", "lang": "en", "prompt": "This question refers to the following information.\n\"Wherever I go—the street, the shop, the house, or the steamboat—I hear the people talk in such a way as to indicate that they are yet unable to conceive of the Negro as possessing any rights at all. Men who are honorable in their dealings with their white neighbors will cheat a Negro without feeling a single twinge of their honor. To kill a Negro they do not deem murder; to debauch a Negro woman they do not think fornication; to take the property away from a Negro they do not consider robbery. The people boast that when they get freedmen affairs in their own hands, to use their own classic expression, 'the niggers will catch hell.'\n\"The reason of all this is simple and manifest. The whites esteem the blacks their property by natural right, and however much they may admit that the individual relations of masters and slaves have been destroyed by the war and the President's emancipation proclamation, they still have an ingrained feeling that the blacks at large belong to the whites at large, and whenever opportunity serves they treat the colored people just as their profit, caprice or passion may dictate.\"\n—Congressional testimony of Col. Samuel Thomas, Assistant Commissioner, Bureau of Refugees, Freedmen and Abandoned Lands, 1865\nTo address the problems identified in Federalist #15, Hamilton proposed\n\nA) abandoning an isolationist approach to foreign policy and adopting a more aggressive and interventionist stance.\nB) adopting a new constitution in order to create a more national government.\nC) forging alliances with American Indian nations to present a united front to European powers.\nD) increasing spending on military forces and cutting spending on social programs.", "gold": "B", "n_choices": 4, "subject": "high_school_us_history", "stem": false}
{"pair_id": "mmlu-565", "lang": "en", "prompt": "Which of the following promotes glucose and amino acid uptake by muscle?\n\nA) Adrenaline\nB) Insulin\nC) Glycogen\nD) Cortisol", "gold": "B", "n_choices": 4, "subject": "clinical_knowledge", "stem": false}
{"pair_id": "mmlu-11656", "lang": "en", "prompt": "An officer stopped a car for having a burned out headlight and license plate light. When the driver could not produce a driver's license, the officer asked him if he minded if they searched the vehicle. The officer did not advise the driver that he had a right to refuse consent or that if he refused his decision would be honored, but there was nothing duplicitous or coercive in the officer's conduct. The driver smiled and said \"of course, no problem.\" The officer found stolen bank checks under the rear seat. In a later suppression motion, the driver claimed that his consent was not voluntary. What will the court most likely decide on that issue?\n\nA) The consent was voluntarily given and was not the result of coercion or duress.\nB) The consent was involuntary because it was the product of implied duress.\nC) The consent was involuntary because the officer did not advise the driver of his right to refuse.\nD) The consent became voluntary as a matter of law when the driver failed to produce a driver's license.", "gold": "A", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-5997", "lang": "en", "prompt": "This question refers to the following information.\nAlthough in Protestant Europe, [Peter the Great] was surrounded by evidence of the new civil and political rights of individual men embodied in constitutions, bills of rights and parliaments, he did not return to Russia determined to share power with his people. On the contrary, he returned not only determined to change his country but also convinced that if Russia was to be transformed, it was he who must provide both the direction and the motive force. He would try to lead; but where education and persuasion were not enough, he could drive—and if necessary flog—the backward nation forward.\n—Robert K. Massie, Peter the Great: His Life and World\nWhen Peter the Great ruled Russia, he continued the practice of which of the following?\n\nA) Decentralization of power\nB) Isolationism\nC) Serfdom\nD) Reform", "gold": "C", "n_choices": 4, "subject": "high_school_world_history", "stem": false}
{"pair_id": "mmlu-10159", "lang": "en", "prompt": "The attitude of many ancient elites toward the consumption of resources seems to have been:\n\nA) “a penny saved is a penny earned.\"\nB) \"if you've got it, flaunt it.\"\nC) \"look before you leap.\"\nD) \"the last shall be first and the first shall be last.\"", "gold": "B", "n_choices": 4, "subject": "prehistory", "stem": false}
{"pair_id": "mmlu-2310", "lang": "en", "prompt": "Rounded to the nearest 10,000, the population of Louisiana was 4,530,000 in 2010. Which number could be the actual population of Louisiana in 2010?\n\nA) 4,500,321\nB) 4,524,491\nC) 4,533,372\nD) 4,535,343", "gold": "C", "n_choices": 4, "subject": "elementary_mathematics", "stem": true}
{"pair_id": "mmlu-1918", "lang": "en", "prompt": "If the Engle-Granger test is applied to the residuals of a potentially cointegrating regression, what would be the interpretation of the null hypothesis?\n\nA) The variables are cointegrated\nB) The variables are not cointegrated\nC) Both variables are stationary\nD) Both variables are non-stationary", "gold": "B", "n_choices": 4, "subject": "econometrics", "stem": false}
{"pair_id": "mmlu-13562", "lang": "en", "prompt": "Which of the following groups or organizations can be identified as having the characteristics of a bureaucracy?\n\nA) An extended or consanguine family, including all the relatives by blood, marriage, or adoption\nB) The government of a large city\nC) A book club in which books are chosen by members who take turns alphabetically to lead discussion\nD) The volunteers who gather annually to clean the neighborhood", "gold": "B", "n_choices": 4, "subject": "sociology", "stem": false}
{"pair_id": "mmlu-3484", "lang": "en", "prompt": "Which term describes the forces that unify and strengthen a country?\n\nA) Diffusion\nB) Centrifugal\nC) Centripetal\nD) Ethnocentric", "gold": "C", "n_choices": 4, "subject": "high_school_geography", "stem": false}
{"pair_id": "mmlu-5989", "lang": "en", "prompt": "This question refers to the following information.\nThat day the Reverend Xuanzang was chosen from among all the monks. He had been a monk from infancy, and ever since birth he had eaten vegetarian food and observed the prohibitions. His maternal grandfather was an imperial commander, Yin Kaishan. His father Chen Guangrui had come top in the Palace Examination and had been appointed a grand secretary in the Imperial Library. Xuanzang, however, had no interest in honour and glory, and his only joy was to cultivate Nirvana. His virtue was great; of the thousand sutras and ten thousand holy books there was not a single one that he did not know.\n…\nHe looked to the West and prayed, \"I am the believer Chen Xuanzang sent on imperial orders to fetch the scriptures. If I am fated to have a disciple, may I be able to unseal the golden words and release the divine Monkey King to come with me to the Vulture Peak. If I am not fated to have a disciple, and this monkey is an evil monster who has deceived me and will do me no good, then may I be unable to remove the seal.\" When he had prayed he bowed again.\nFrom Wu Chengen, Journey to the West, ca. 1590s\nIn which of the following ways does the excerpt above most complicate historians' understanding of the career of the Chinese monk and traveler Xuanzang?\n\nA) It highly fictionalizes the story of Xuanzang's accomplishments.\nB) It was written long after the time period in which Xuanzang lived.\nC) It casts doubt on the actual existence of Xuanzang.\nD) It is widely considered by scholars to be a forgery.", "gold": "A", "n_choices": 4, "subject": "high_school_world_history", "stem": false}
{"pair_id": "mmlu-759", "lang": "en", "prompt": "Which of the following is NOT a source of atmospheric carbon?\n\nA) Respiration\nB) Photosynthesis\nC) Bacterial decomposition\nD) Combustion of fossil fuels", "gold": "B", "n_choices": 4, "subject": "college_biology", "stem": true}
{"pair_id": "mmlu-3884", "lang": "en", "prompt": "The FED's Board of Governors has members each serving -year terms.\n\nA) 14 7\nB) 7 14\nC) 8 8\nD) 50 2", "gold": "B", "n_choices": 4, "subject": "high_school_macroeconomics", "stem": false}
{"pair_id": "mmlu-9997", "lang": "en", "prompt": "Epictetus claims that the desire for one’s loved ones to live forever is:\n\nA) a natural instinct.\nB) a sign of genuine love.\nC) blasphemous.\nD) silly.", "gold": "D", "n_choices": 4, "subject": "philosophy", "stem": false}
{"pair_id": "mmlu-7558", "lang": "en", "prompt": "What was the average life expectancy of white males born in the U.S. just before the Civil War?\n\nA) 40 years\nB) 50 years\nC) 60 years\nD) 70 years", "gold": "A", "n_choices": 4, "subject": "miscellaneous", "stem": false}
{"pair_id": "mmlu-9725", "lang": "en", "prompt": "Anscombe claims that it is not profitable to do moral philosophy until we have an adequate philosophy of:\n\nA) religion.\nB) physics.\nC) psychology.\nD) art.", "gold": "C", "n_choices": 4, "subject": "philosophy", "stem": false}
{"pair_id": "mmlu-1199", "lang": "en", "prompt": "The complete resynthesis of phosphocreatine after very high intensity exercise normally takes:\n\nA) about 10 seconds.\nB) about 30 seconds.\nC) about 1 minute.\nD) about 4 minutes.", "gold": "D", "n_choices": 4, "subject": "college_medicine", "stem": false}
{"pair_id": "mmlu-3026", "lang": "en", "prompt": "Which of the following can form hydrogen bonds?\n\nA) CH3OCH2CH3\nB) HCN\nC) CH3OCH2Br\nD) CH3NH2", "gold": "D", "n_choices": 4, "subject": "high_school_chemistry", "stem": true}
{"pair_id": "mmlu-9100", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | I ran over the fresh grass with a lawnmower. Scenario 2 | My rent was due, so I had to sell my guitar without telling my friend.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "D", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-6927", "lang": "en", "prompt": "Suppose you are given an EM algorithm that finds maximum likelihood estimates for a model with latent variables. You are asked to modify the algorithm so that it finds MAP estimates instead. Which step or steps do you need to modify?\n\nA) Expectation\nB) Maximization\nC) No modification necessary\nD) Both", "gold": "B", "n_choices": 4, "subject": "machine_learning", "stem": true}
{"pair_id": "mmlu-13268", "lang": "en", "prompt": "Which of the following statements does NOT characterize arms control as a strategic policy?\n\nA) Arms control is an example of cooperative security, designed to address the problems associated with the security dilemma.\nB) Policies of arms control and disarmament was pursued during the Cold War in an attempt to achieve security.\nC) Arms control does not challenge the central role of weaponry and military power in the international system, instead focusing on the problems produced by specific weapon systems and relationships.\nD) After its successful application during the Cold War, arms control has retained its momentum and applicability in the contemporary security environment.", "gold": "D", "n_choices": 4, "subject": "security_studies", "stem": false}
{"pair_id": "mmlu-2386", "lang": "en", "prompt": "Information about what the students in Mr. Paulson’s class and in Ms. Hugo’s class ate for lunch is listed below. In Mr. Paulson’s class, 0.5 of the students ate pizza for lunch. In Ms. Hugo’s class, 0.5 of the students ate hamburgers for lunch. What information, if any, is needed to correctly compare the 0.5 of Mr. Paulson’s class that ate pizza to the 0.5 of Ms. Hugo’s class that ate hamburgers?\n\nA) whether the lunches were eaten on the same day\nB) the total number of students in each teacher’s class\nC) nothing because the portions each class ate were the same\nD) the total number of pizza slices and hamburgers eaten that day", "gold": "B", "n_choices": 4, "subject": "elementary_mathematics", "stem": true}
{"pair_id": "mmlu-709", "lang": "en", "prompt": "Which of the following is true of haematemesis?\n\nA) A low blood pressure (<90mmHg systolic) and a tachycardia (>100/min) are worrying features\nB) A pulse rate of 80/min in a patient taking Bisoprolol is reassuring\nC) Abdominal pain is always present\nD) An alcohol history is not essential", "gold": "A", "n_choices": 4, "subject": "clinical_knowledge", "stem": false}
{"pair_id": "mmlu-1105", "lang": "en", "prompt": "Which of the following is true?\n\nA) Every compact space is complete\nB) Every complete space is compact\nC) Neither (a) nor (b).\nD)  Both (a) and (b).", "gold": "A", "n_choices": 4, "subject": "college_mathematics", "stem": true}
{"pair_id": "mmlu-13613", "lang": "en", "prompt": "How did World War I shift economic power from Europe to the United States?\n\nA) The war reduced European population levels below that of the United States\nB) The United States seized German resources after the war\nC) European countries paid the United States for assistance\nD) The United States became a creditor country and financial centre, with European war spending boosting the US economy", "gold": "D", "n_choices": 4, "subject": "us_foreign_policy", "stem": false}
{"pair_id": "mmlu-2097", "lang": "en", "prompt": "Write 8 and 2 over 4 as an improper fraction in simplest form.\n\nA) 17 over 2\nB) 34 over 4\nC) 17 over 4\nD) 19 over 2", "gold": "A", "n_choices": 4, "subject": "elementary_mathematics", "stem": true}
{"pair_id": "mmlu-10528", "lang": "en", "prompt": "Prior to the issuance of its December 31 financial statements Stark Co. was named as a defendant in a lawsuit arising from an event that occurred in October. Stark's legal counsel believes that it is reasonably possible that there will be an unfavorable outcome and that damages will range from $100000 to $150000. Which amount(s) should Stark accrue and/or disclose in its December 31 financial statements? Accrue contingent liability Disclose contingent liability\n\nA) $100000 $100000 - $150000\nB) $100000 $150000\nC) $0 $100000 - $150000\nD) $0 $150000", "gold": "C", "n_choices": 4, "subject": "professional_accounting", "stem": false}
{"pair_id": "mmlu-11780", "lang": "en", "prompt": "A man was prosecuted for the alleged murder of his estranged wife. At trial, the government was allowed to present the testimony of several women who testified to having flirtatious or sexual encounters with the defendant over the past 20 years prior to the homicide. None of the testimony showed any such activity within three years of the murder date, and none of it showed any violent tendencies of the defendant or intentions expressed that he wanted to kill his wife. The defendant had an alibi for the time of the crime and presented many alibi witnesses. The government claimed that it was showing motive by proving the defendant's proclivity for extra-marital affairs over the years and therefore his bad relationship with his wife. The jury convicted him after hearing the testimony of 10 women regarding defendant's various sexual contacts with them over prior years. The defendant appealed the murder conviction, claiming that the women's testimony was prejudicial and inflammatory and showed only bad character, which is not admissible under evidentiary rules. Will the appeals court likely reverse the conviction and order a new trial?\n\nA) Yes, the trial was unfair in presenting testimony of women going back over 20 years; it would have been proper to admit such testimony only as to the five years preceding the crime.\nB) Yes, the defendant did not receive a fair trial; the testimony of the women only showed his bad character but because they were old incidents and not continuing, they did not allow a permissible inference that these affairs gave him a motive to kill his spouse.\nC) No, if the testimony was admitted in error it was harmless error that did not sway the jury one way or another and so the defendant got a fair trial in all other respects.\nD) No, the testimony was admissible because it tended to show a pattern of \"transferred intent\" that proved the defendant's great hostility to his wife, and his need to eliminate her from his life so that he could go back to having a wild life.", "gold": "B", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-9073", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | I planted a large garden in the center of the narrow public street. Scenario 2 | I drive very fast when I'm on a racetrack.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "B", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-7014", "lang": "en", "prompt": "What is the term for the action in which managers at an organisation analyse the current situation of their organisation and then develop plans to accomplish its mission and achieve its goals?\n\nA) Synergy planning\nB) Strategy formulation\nC) Functional planning\nD) SWOT analysis", "gold": "B", "n_choices": 4, "subject": "management", "stem": false}
{"pair_id": "mmlu-3369", "lang": "en", "prompt": "This question refers to the following information.\nRead the following quote.\nI had now decided beyond all question that there existed in the heavens three stars wandering about Jupiter as do Venus and Mercury about the sun, and this became plainer than daylight from observations on similar occasions which followed. Nor were there just three such stars; four wanderers complete their revolutions about Jupiter, and of their alterations as observed more precisely later on we shall give a description here. Also I measured the distances between them by means of the telescope. . . .\nSuch are the observations concerning the four Medicean planets recently first discovered by me, and although from this data their periods have not yet been reconstructed in numerical form, it is legitimate at least to put in evidence some facts worthy of note. Above all, since they sometimes follow and sometimes precede Jupiter by the same intervals, and they remain within very limited distances either to east or west of Jupiter, accompanying that planet in both its retrograde and direct movements in a constant manner, no one can doubt that they complete their revolutions about Jupiter and at the same time effect all together a twelve-year period about the center of the universe.\n—Galileo Galilei, 1610\nWhich of the following is best demonstrated by the passage about intellectual thought at the time?\n\nA) It led to better scientific tools, which led to a rise in the standard of living during the seventeenth century across Europe.\nB) The ideas of the ancient Greeks guided all of their ideas.\nC) It used information obtained through experimentation to conceptualize the universe.\nD) It provided experimental proof of the theories of ancient thinkers, such as Aristotle, on how the universe worked.", "gold": "C", "n_choices": 4, "subject": "high_school_european_history", "stem": false}
{"pair_id": "mmlu-623", "lang": "en", "prompt": "Which of the following is true about the assessment of arm reflexes?\n\nA) The root value of the biceps reflex is C5, C6\nB) If no reflex is elicited when you tap a tendon it is documented as absent\nC) The triceps tendon is struck in the antecubital fossa\nD) Brisk finger jerks confirm a lower motor neurone lesion", "gold": "A", "n_choices": 4, "subject": "clinical_knowledge", "stem": false}
{"pair_id": "mmlu-2314", "lang": "en", "prompt": "Which expression shows a prime factorization?\n\nA) 2 • 9 • 11\nB) 2.5 • 7 • 3\nC) 1 • 11 • 13\nD) 2 • 2 • 2 • 3 • 11", "gold": "D", "n_choices": 4, "subject": "elementary_mathematics", "stem": true}
{"pair_id": "mmlu-2716", "lang": "en", "prompt": "Which of the following best explains why people can digest starch but cannot digest cellulose even though both molecules are composed of glucose monomers?\n\nA) The bonds linking the monomers of starch differ in shape from the bonds linking the monomers of cellulose.\nB) Molecules of starch are much smaller than molecules of cellulose.\nC) Starch is an intracellular molecule, while cellulose is an extracellular molecule.\nD) Starch is hydrated by water, but cellulose is not.", "gold": "A", "n_choices": 4, "subject": "high_school_biology", "stem": true}
{"pair_id": "mmlu-10185", "lang": "en", "prompt": "The Egyptian system of hieroglyphics:\n\nA) did not use pictographs\nB) appears to have developed suddenly\nC) was the earliest form of writing in the world\nD) all of the above", "gold": "B", "n_choices": 4, "subject": "prehistory", "stem": false}
{"pair_id": "mmlu-7224", "lang": "en", "prompt": "Robert is a marketer for a global consumer products company. He is working on the promotional campaign designed to reach a target audience in a new international market. Robert is working hard to make sure that the promotional campaign is clearly understood by the nation's consumers and doesn't offend anyone. By which of the factors in the external environment is he being influenced\n\nA) Socio-cultural environment.\nB) Competitive environment.\nC) Economic environment.\nD) Legal environment.", "gold": "A", "n_choices": 4, "subject": "marketing", "stem": false}
{"pair_id": "mmlu-7577", "lang": "en", "prompt": "What letters are on the '3' button of a touch-tone telephone?\n\nA) ABC\nB) DEF\nC) GHI\nD) WXY", "gold": "B", "n_choices": 4, "subject": "miscellaneous", "stem": false}
{"pair_id": "mmlu-10327", "lang": "en", "prompt": "The key factor in the survival and success of anatomically modern human beings was:\n\nA) their ability to control fire and produce art.\nB) their ability to interbreed with other hominids.\nC) their ability to hunt large mammals.\nD) their ability to adapt.", "gold": "D", "n_choices": 4, "subject": "prehistory", "stem": false}
{"pair_id": "mmlu-2051", "lang": "en", "prompt": "The ROM programmed during manufacturing process itself is called\n\nA) MROM\nB) PROM\nC) EPROM\nD) EEPROM", "gold": "A", "n_choices": 4, "subject": "electrical_engineering", "stem": true}
{"pair_id": "mmlu-9306", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | I picked the most rotten apples to donate to the food pantry. Scenario 2 | I sold exotic berries at the farmer's market.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "B", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-10001", "lang": "en", "prompt": "According to Brandt, John Stuart Mill’s view most closely resembles:\n\nA) act-utilitarianism.\nB) a rule-utilitarianism based on the actual rules of society.\nC) a rule-utilitarianism based on ideal moral rules.\nD) Kant’s ethics.", "gold": "C", "n_choices": 4, "subject": "philosophy", "stem": false}
{"pair_id": "mmlu-12158", "lang": "en", "prompt": "On February 1, a retiree conveys his farm to an artist, and the artist duly records the conveyance. The following day, the artist conveys the property to a bartender; she does not record her deed. Then on February 4, the artist executes an identical conveyance of the farm to a caterer. The caterer gives the artist a check for $100,000 for the property and records the conveyance, even though he has actual knowledge of prior conveyance to the bartender. The bartender, however, records her deed on February 6. The caterer then conveys his interest in the farm to a dancer, who gives a purchase price of $115,000 to the caterer. On February 5, the dancer purchases the farm without notice of the conveyance to the bartender and duly records the deed. In conducting a title search, the dancer should pursue his investigation by looking in the\n\nA) Grantor Index under the caterer's name to ascertain if the caterer acquired title.\nB) Grantee Index under the caterer's name only.\nC) Grantee Index under the caterer's name, then the Grantor Index under the caterer's name, and then in the Grantee Index again, this time under the artist's name to discover if he acquired title.\nD) Grantee Index under the bartender's name, then to the Grantor Index, also under the bartender's name to find out if she made any prior conveyances.", "gold": "C", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-6546", "lang": "en", "prompt": "Is the recognition of foreign judgments subject to the same rules as those applicable to the incorporation and transformation of treaties?\n\nA) Foreign judgments are enforced on the basis of the doctrine of incorporation\nB) Foreign judgments are enforced on the basis of the doctrine of transformation\nC) The recognition of foreign judgments is dependent on the existence of appropriate bilateral or multilateral treaties\nD) The courts exercise discretion as to the enforcement of foreign judgments on the basis of the rule of comity", "gold": "C", "n_choices": 4, "subject": "international_law", "stem": false}
{"pair_id": "mmlu-9581", "lang": "en", "prompt": "Lactose\n\n\nA) Is always the cause of milk intolerance\nB) Cannot be digested in the human gut\nC) Intolerance is present in up to 20% of the population\nD) Is excluded on a low FODMAP diet", "gold": "D", "n_choices": 4, "subject": "nutrition", "stem": false}
{"pair_id": "mmlu-6832", "lang": "en", "prompt": "Which of the following best describes the fallacy of amphiboly?\n\nA) Using emotionally charged languages to create an impression about the subject of a claim, without making an argument that the language fits the subject.\nB) Referring to an act committed by an opponent in negative terms while referring to the same act committed by the arguer or supporters in favorable terms.\nC) Using grammar and punctuation in a way that a statement may have multiple interpretations, so it's not really clear what is meant.\nD) Changing the meaning of a word or phrase from one part of the argument to another.", "gold": "C", "n_choices": 4, "subject": "logical_fallacies", "stem": false}
{"pair_id": "mmlu-8503", "lang": "en", "prompt": " Nagel thinks that the core of the absolutist position is that\n\nA) human persons have a right to life.\nB) it is permissible to harm as a foreseen but unintended consequence of action.\nC) the ends justify the means.\nD) the hostility should be directed at its true object.", "gold": "D", "n_choices": 4, "subject": "moral_disputes", "stem": false}
{"pair_id": "mmlu-9849", "lang": "en", "prompt": "For Plato, ordinary sensible objects exist and are knowable as examples or instances of Ideas or \"Forms\" that do not exist in our ordinary sensible world.  Forms do not exist in the sensible world because:\n\nA) in the sensible world only mathematical objects (e.g., triangles) can be known using hypotheses which are recollected when we are asked the right kinds of questions.\nB) unlike everything in the sensible world, Forms are not individual things but rather the universal essences or natures by which individual things are what they are and are known.\nC) nothing in the sensible, experienced world could exist or be identified as one particular thing or another unless there were a \"Sensible World\" Form (like the Form of beauty or justice).\nD) the sensible world consists of changing Forms that exist and are known in terms of other changing Forms, which in turn exist and are known in terms of yet others in an endless regress.", "gold": "B", "n_choices": 4, "subject": "philosophy", "stem": false}
{"pair_id": "mmlu-3623", "lang": "en", "prompt": "Which of the following statements regarding historical North American migration streams is NOT correct?\n\nA) Canada's first major migration came from France.\nB) Canada's second major migration stream originated in the British Isles.\nC) Canada's third major migration stream came from Latin America.\nD) The first major migration wave to the United States originated in Europe.", "gold": "C", "n_choices": 4, "subject": "high_school_geography", "stem": false}
{"pair_id": "mmlu-11290", "lang": "en", "prompt": "A state's constitution reserves to the people of each municipality in the state the power of referendum with respect to all questions that the municipality is authorized to control by legislation. A real estate developer applied for a zoning change to permit construction of a convalescent home on land he owned in a city within the state. While the application was pending, the city charter was amended by popular vote so as to require that any changes in land use agreed to by the city council be approved by a 55% vote in a referendum. The city planning commission and the city council both approved the proposed zoning change. However, the real estate developer's subsequent application for approval for a proposed home on the grounds was rejected because the rezoning action had not been submitted to a referendum. In an action brought in state court seeking a judgment declaring the city charter amendment invalid, the court will most likely declare the amendment\n\nA) unconstitutional, as violation of the real estate developer's due process rights.\nB) unconstitutional, as an unlawful delegation of legislative power to a regulatory body.\nC) constitutional, as a valid exercise of the city's police power.\nD) constitutional, as a valid exercise of a power reserved by the people to themselves.", "gold": "D", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-3477", "lang": "en", "prompt": "The majority of workers in periphery countries are probably employed in which type of activities?\n\nA) Primary\nB) Secondary\nC) Tertiary\nD) Quaternary", "gold": "A", "n_choices": 4, "subject": "high_school_geography", "stem": false}
{"pair_id": "mmlu-10178", "lang": "en", "prompt": "Mayan city-states were NOT:\n\nA) ruled by autonomous dynasties.\nB) similar in their political organization to Egyptian communities.\nC) supported by large agricultural hinterlands.\nD) much smaller in population than Mohanjo-daro.", "gold": "B", "n_choices": 4, "subject": "prehistory", "stem": false}
{"pair_id": "mmlu-8677", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | I wanted to buy a doll so I broke open my piggy bank. Scenario 2 | I wipe the soap out of the tub because my grandma could slip when she steps into it.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "D", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-9881", "lang": "en", "prompt": "Nussbaum claims that for Aristotle the reference of each virtue term is fixed by:\n\nA) conventional use.\nB) grounding experiences.\nC) a thick description of the virtue.\nD) tradition.", "gold": "B", "n_choices": 4, "subject": "philosophy", "stem": false}
{"pair_id": "mmlu-6231", "lang": "en", "prompt": "Glaucoma refers to the condition in which\n\nA) Fluid pressure in the eye is above normal\nB) Inner and outer layers of the retina separate\nC) The lens becomes cloudy and opaque\nD) The central part of the retina no longer functions", "gold": "A", "n_choices": 4, "subject": "human_aging", "stem": false}
{"pair_id": "mmlu-4899", "lang": "en", "prompt": "A person who sustains major injuries that involve the destruction of the medulla oblongata will\n\nA) be paralyzed\nB) fall into a coma\nC) suffer severe speech impairment\nD) die", "gold": "D", "n_choices": 4, "subject": "high_school_psychology", "stem": false}
{"pair_id": "mmlu-12680", "lang": "en", "prompt": "___________ is a communication disorder that is characterized by difficulties in regulating the rate, rhythm, pitch, and loudness of speech.\n\nA) Dysarthria\nB) Paraphasia\nC) Dysprosody\nD) Adynamia", "gold": "C", "n_choices": 4, "subject": "professional_psychology", "stem": false}
{"pair_id": "mmlu-10871", "lang": "en", "prompt": "A witness in a murder case does not make the same statement faithfully, but rather he has given different versions of his observations at different times to different persons and investigators. The prosecution put the witness on the stand at trial to tell what he observed. The defendant's counsel impeached his testimony by bringing up prior inconsistent statements and accusing him of changing his story for trial. The prosecution then attempted to rehabilitate his credibility by referencing prior consistent statements. Prior consistent statements are not generally admissible because they are said to be repetitive, cumulative and to unfairly bolster the witness's credibility. Will the court likely allow the prosecution to rehabilitate the witness using prior consistent statements under these facts?\n\nA) No, because the witness cannot be rehabilitated once a successful impeachment has occurred.\nB) No, because it would tend to confuse the jury with too much conflicting evidence.\nC) Yes, because it is being used to rehabilitate a witness whose credibility was attacked.\nD) Yes, because all repetitive prior statements are important to show the consistency of the witness' testimony.", "gold": "C", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-11293", "lang": "en", "prompt": "A defendant was driving his car recklessly at a high rate of speed through a residential neighborhood. He was traveling at a speed of over 100 M. P. H. when he lost control of the car and jumped a curb, striking a woman who was walking along the sidewalk. As a result of the collision, the woman suffered severe internal injuries and fractured both legs. She was hospitalized for 11 months and became permanently disabled. If the defendant is charged with attempted murder, he should be found\n\nA) guilty, because a person is presumed to intend the natural and probable consequences of his acts.\nB) guilty, because criminal liability is predicated upon the defendant's willful and wanton disregard for the safety of others.\nC) not guilty, because the defendant did not intend to kill the woman.\nD) not guilty, because he lost control of the vehicle.", "gold": "C", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-11158", "lang": "en", "prompt": "A seller and a buyer signed a contract of sale for improved real property. The contract contained a financing contingency for a certain percentage of the purchase price. The buyer obtained the requisite financing from a bank. At the closing, the buyer executed a note to the seller for a portion of the purchase price, which note was not secured by a mortgage. The buyer then executed a second note, secured by a mortgage to executed a second note, secured by a mortgage to the bank, applying the bank loan proceeds to the purchase price of the property. The bank had actual knowledge of the prior note to the seller. The bank promptly recorded its mortgage. The buyer is now in default on both notes. There is no applicable statute. Which party has priority?\n\nA) The bank, because its loan satisfied the financing contingency in the contract of sale.\nB) The bank, because its note is secured by a purchase money mortgage.\nC) The seller, because the bank had actual knowledge of the seller's note.\nD) The seller, because he retained a vendor's lien that was first in time.", "gold": "B", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-10541", "lang": "en", "prompt": "Suppose that the market price of Company X is $45 per share and that of Company Y is $30. If X offers three-fourths a share of common stock for each share of Y, the ratio of exchange of market prices would be:\n\nA) 0.667\nB) 1\nC) 1.125\nD) 1.5", "gold": "C", "n_choices": 4, "subject": "professional_accounting", "stem": false}
{"pair_id": "mmlu-455", "lang": "en", "prompt": "The ___________, otherwise known as _________ was launched in 1997 and is a global workplace standard that covers key labour rights such as working hours, forced labour and discrimination, with compliance being certified by independent auditors.\n\nA) Social accountability standard, SA 8000\nB) Social accountability standard, SA 9000\nC) Sarbanes-Oxley Act, SA 8000\nD) Sarbanes-Oxley Act, SA 9000", "gold": "A", "n_choices": 4, "subject": "business_ethics", "stem": false}
{"pair_id": "mmlu-9876", "lang": "en", "prompt": "Rawls claims that students with fewer native assets (such as intelligence) should be given:\n\nA) more attention and resources than those with more native assets.\nB) the same level of attention and resources as those with more native assets.\nC) less attention and fewer resources than those with more native assets.\nD) virtually no educational resources.", "gold": "A", "n_choices": 4, "subject": "philosophy", "stem": false}
{"pair_id": "mmlu-5406", "lang": "en", "prompt": "Barry reported that in his study, the relationship between religiosity and academic grades was not statistically significant. By \"not statistically significant,\" he meant that the results\n\nA) were not important\nB) were not strong\nC) might have been due to chance\nD) were of no value to statisticians", "gold": "C", "n_choices": 4, "subject": "high_school_psychology", "stem": false}
{"pair_id": "mmlu-7820", "lang": "en", "prompt": "By what collective name do Christians refer to God the Father God the Son and the Holy Ghost?\n\nA) the Trio\nB) the Troika\nC) the Triumvirate\nD) the Trinity", "gold": "D", "n_choices": 4, "subject": "miscellaneous", "stem": false}
{"pair_id": "mmlu-12518", "lang": "en", "prompt": "Three types of prevention have been identified in community mental health. Secondary prevention attempts to\n\nA) deal with problems before they occur\nB) prevent relapses of problems\nC) reduce the severity of problems\nD) prevent community disintegration", "gold": "C", "n_choices": 4, "subject": "professional_psychology", "stem": false}
{"pair_id": "mmlu-83", "lang": "en", "prompt": "Statement 1 | If a group has an element of order 10, then it has elements of orders 1, 2, and 5. Statement 2 | If a group has an element of order 2 and an element of order 3, then it has an element of order 6.\n\nA) True, True\nB) False, False\nC) True, False\nD) False, True", "gold": "C", "n_choices": 4, "subject": "abstract_algebra", "stem": true}
{"pair_id": "mmlu-658", "lang": "en", "prompt": "Which of the following drugs is most commonly responsible for patients who present with acute confusion?\n\nA) Heroin (opiates).\nB) Ecstasy.\nC) Alcohol.\nD) Cannabis.", "gold": "C", "n_choices": 4, "subject": "clinical_knowledge", "stem": false}
{"pair_id": "mmlu-6384", "lang": "en", "prompt": "Vasocongestion is a process of :\n\nA) muscle contraction\nB) blood accumulation in the genitals\nC) muscle relaxation\nD) detumescence", "gold": "B", "n_choices": 4, "subject": "human_sexuality", "stem": false}
{"pair_id": "mmlu-614", "lang": "en", "prompt": "Insensible loss does not come from:\n\nA) pyrexia.\nB) using unhumidified oxygen.\nC) excessive weight loss.\nD) diarrhoea.", "gold": "C", "n_choices": 4, "subject": "clinical_knowledge", "stem": false}
{"pair_id": "mmlu-8656", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | When I saw my neighbor's house was on fire, I rushed to the phone to call my long lost friend to chat. Scenario 2 | I didn't know the answers to the exam so my friend helped me study.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "B", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-7087", "lang": "en", "prompt": " Organizational characteristics, such as organizational size and location, are sometimes referred to as___________.\n\nA) DMU.\nB) Firmographics.\nC) ACORN\nD) Product usage.", "gold": "B", "n_choices": 4, "subject": "marketing", "stem": false}
{"pair_id": "mmlu-4626", "lang": "en", "prompt": "Which of the following is most likely to decrease the demand for kindergarten teachers?\n\nA) An increase in funding for education\nB) Increased immigration of foreign citizens and their families\nC) A decrease in the average number of children per household\nD) Subsidies given to college students who major in elementary education", "gold": "C", "n_choices": 4, "subject": "high_school_microeconomics", "stem": false}
{"pair_id": "mmlu-4666", "lang": "en", "prompt": "The market demand curve for labor would shift to the left as the result of\n\nA) an increase in the price of the good which the labor is producing\nB) an increase in demand for the good which the labor is producing\nC) an increase in the wage rate paid to workers\nD) a decrease in the marginal product of labor", "gold": "D", "n_choices": 4, "subject": "high_school_microeconomics", "stem": false}
{"pair_id": "mmlu-10099", "lang": "en", "prompt": "How were the first metals worked in South America?\n\nA) casting\nB) hammering\nC) smelting\nD) all of the above", "gold": "B", "n_choices": 4, "subject": "prehistory", "stem": false}
{"pair_id": "mmlu-1221", "lang": "en", "prompt": "A segment of DNA from a lab mouse is determined to be 5’ – GGATCCTCATG – 3’. Which of the following DNA segments would be the result of this original DNA sequence experiencing both a point mutation and a deletion?\n\nA) 5’ – GCATCCTCATG – 3’\nB) 5’ – TGATCCCAG – 3’\nC) 5’ – GGTCCTCATC – 3’\nD) 5’ – GGATCCATG – 3’", "gold": "C", "n_choices": 4, "subject": "college_medicine", "stem": false}
{"pair_id": "mmlu-7467", "lang": "en", "prompt": "Which of these sentences is written in the subjunctive?\n\nA) I am not your man\nB) I wish I were your man\nC) Wherefore art your man?\nD) Your man is where?", "gold": "D", "n_choices": 4, "subject": "miscellaneous", "stem": false}
{"pair_id": "mmlu-2663", "lang": "en", "prompt": "As of 2017, the share of GDP spent on the military by the United States is about\n\nA) 1%\nB) 3%\nC) 6%\nD) 10%", "gold": "B", "n_choices": 4, "subject": "global_facts", "stem": false}
{"pair_id": "mmlu-12145", "lang": "en", "prompt": "After learning that a new housing development was about to be built, a developer began looking for property near the housing development on which he could build a shopping center. After an extensive search, he purchased a three-acre undeveloped tract of land for the purpose of developing a small shopping center. At the time the sale was finalized, the developer told his plans to the seller of the three-acre tract, and also mentioned that he was worried whether he had sufficient funds to complete the project. The seller agreed to lend money to the developer to allow for completion of the project. The seller then wrote out the following contract:\"In order to help the developer with his plans for development of a shopping center on the three- acre tract he has this day purchased from me, I will lend him $50,000 at 10 percent interest provided he signs a contract with a builder for construction of the shopping center on this tract. Repayment to me at the rate of $5,000 per year to be secured by a mortgage on the three-acre tract. \"Both parties then signed the contract. The seller's promise to lend $50,000 to the developer is\n\nA) a condition precedent in form but subsequent in substance to the developer's duty to enter into a building construction contract.\nB) a condition subsequent in form but precedent in substance to the developer's duty to enter into a building construction contract.\nC) a condition subsequent to the developer's duty to enter into a building construction contract.\nD) not a condition, either precedent or subsequent, to the developer's duty to enter into a building construction contract.", "gold": "A", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-3494", "lang": "en", "prompt": "An earlier boundary formed before meticulous geographic knowledge and mapping was available is known as\n\nA) an antecedent boundary.\nB) a geometric boundary.\nC) a relict boundary.\nD) a subsequent boundary.", "gold": "A", "n_choices": 4, "subject": "high_school_geography", "stem": false}
{"pair_id": "mmlu-9399", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | I had my son spend 20 minutes on chores before playing video games. Scenario 2 | I made my stepdad's Koolaid with half the sugar since he is diabetic.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "D", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-5868", "lang": "en", "prompt": "This question refers to the following information.\n\"The real grievance of the worker is the insecurity of his existence; he is not sure that he will always have work, he is not sure that he will always be healthy, and he foresees that he will one day be old and unfit to work. If he falls into poverty, even if only through a prolonged illness, he is then completely helpless, exam_ins to his own devices, and society does not currently recognize any real obligation towards him beyond the usual help for the poor, even if he has been working all the time ever so faithfully and diligently. The usual help for the poor, however, leaves a lot to be desired, especially in large cities, where it is very much worse than in the country.\"\nOtto von Bismarck, 1884\nThe long-term effects of Otto von Bismarck's speech include which of the following?\n\nA) Development of socialized programs throughout much of Europe\nB) Disunity of the German states\nC) Communist overhaul of the eastern parts of Germany\nD) A decrease in German economic output", "gold": "A", "n_choices": 4, "subject": "high_school_world_history", "stem": false}
{"pair_id": "mmlu-3075", "lang": "en", "prompt": "When an ideal gas is allowed to expand isothermally, which one of the following is true?\n\nA) q = 0\nB) w = 0\nC) E = 0\nD) q = -w", "gold": "D", "n_choices": 4, "subject": "high_school_chemistry", "stem": true}
{"pair_id": "mmlu-7672", "lang": "en", "prompt": "In the phrase 'Y2K' what does 'K' stand for?\n\nA) millennium\nB) computer code\nC) catastrophe\nD) thousand", "gold": "D", "n_choices": 4, "subject": "miscellaneous", "stem": false}
{"pair_id": "mmlu-13399", "lang": "en", "prompt": "In what ways can a distinction be made between limited force and full-scale force?\n\nA) The distinction between limited force and full-scale force is the second process of coercive diplomacy. Coercive diplomacy only fails if the coercer fails to achieve its defined goals and fails to defeat its adversary in the second stage.\nB) Generally the distinction between brute and limited force is negligible. Resort to air or sea power constitutes an equal coercive capacity to a conventional ground offensive. Military action always results from a failure of negotiations and from a shift from the diplomatic to the military sphere.\nC) The distinction between limited force and brute force is important because the amount of force that is used to attain the coercer's interests defines the type of outcome that is achieved. If a positive policy outcome is achieved, then we can say that limited force has been employed.\nD) The distinction between limited force and full-scale war is crucial because resort to brute force means that diplomacy has failed. The distinction is not based on the amount of force or the type, but on the purpose that the use of force seeks to accomplish and the element of choice left to the adversary. In essence, limited force is a bargaining tool.", "gold": "D", "n_choices": 4, "subject": "security_studies", "stem": false}
{"pair_id": "mmlu-4195", "lang": "en", "prompt": "Excess reserves in the banking system will increase if\n\nA) the reserve ratio is increased.\nB) the checking deposits increase.\nC) the discount rate is increased.\nD) the Fed sells Treasury securities to commercial banks.", "gold": "B", "n_choices": 4, "subject": "high_school_macroeconomics", "stem": false}
{"pair_id": "mmlu-4041", "lang": "en", "prompt": "Which of the following likely results in a permanent increase in a nation's productive capacity?\n\nA) A decline in the birth rate\nB) Declining adult literacy rates\nC) Widespread relocation of manufacturing firms to low-wage nations\nD) National program of child immunization", "gold": "D", "n_choices": 4, "subject": "high_school_macroeconomics", "stem": false}
{"pair_id": "mmlu-9178", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | I used my copier to made duplicates of money because my bank account was running low. Scenario 2 | When I play cards, I make small talk.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "B", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-8187", "lang": "en", "prompt": " Which of the following is not an objection that Kass makes against cloning?\n\nA) It constitutes unethical experimentation.\nB) It threatens identity and individuality.\nC) It will produce harmful biological mutations as clones produce further clones.\nD) It perverts our values concerning parenthood and the raising of children.", "gold": "C", "n_choices": 4, "subject": "moral_disputes", "stem": false}
{"pair_id": "mmlu-3352", "lang": "en", "prompt": "This question refers to the following information.\nFirst we must remark that the cosmos is spherical in form, partly because this form being a perfect whole requiring no joints, is the most complete of all, partly because it makes the most capacious form, which is best suited to contain and preserve everything; or again because all the constituent parts of the universe, that is the sun, moon and the planets appear in this form; or because everything strives to attain this form, as appears in the case of drops of water and other fluid bodies if they attempt to define themselves. So no one will doubt that this form belongs to the heavenly bodies. …\nThat the earth is also spherical is therefore beyond question, because it presses from all sides upon its center. Although by reason of the elevations of the mountains and the depressions of the valleys a perfect circle cannot be understood, yet this does not affect the general spherical nature of the earth. …\nAs it has been already shown that the earth has the form of a sphere, we must consider whether a movement also coincides with this form, and what place the earth holds in the universe. … The great majority of authors of course agree that the earth stands still in the center of the universe, and consider it inconceivable and ridiculous to suppose the opposite. But if the matter is carefully weighed, it will be seen that the question is not yet settled and therefore by no means to be regarded lightly. Every change of place which is observed is due, namely, to a movement of the observed object or of the observer, or to movements of both. … Now it is from the earth that the revolution of the heavens is observed and it is produced for our eyes. Therefore if the earth undergoes no movement this movement must take place in everything outside of the earth, but in the opposite direction than if everything on the earth moved, and of this kind is the daily revolution. So this appears to affect the whole universe, that is, everything outside the earth with the single exception of the earth itself. If, however, one should admit that this movement was not peculiar to the heavens, but that the earth revolved from west to east, and if this was carefully considered in regard to the apparent rising and setting of the sun, the moon and the stars, it would be discovered that this was the real situation.\"\nNicolas Copernicus, The Revolutions of the Heavenly Bodies, 1543\nThe passage may be used as evidence that Copernicus differed from the traditional, Aristotelian natural philosophers of his day because he suggested that\n\nA) the cosmos is spherical\nB) the Earth is spherical\nC) the cosmos is geostatic\nD) the Earth is not stationary", "gold": "D", "n_choices": 4, "subject": "high_school_european_history", "stem": false}
{"pair_id": "mmlu-5984", "lang": "en", "prompt": "This question refers to the following information.\n\"The greatest contribution that Africa can make to the peace of the world is to avoid all the dangers inherent in disunity, by creating a political union which will also by its success, stand as an example to a divided world. A Union of African states will project more effectively the African personality. It will command respect from a world that has regard only for size and influence. The scant attention paid to African opposition to the French atomic tests in the Sahara, and the ignominious spectacle of the U.N. in the Congo quibbling about constitutional niceties while the Republic was tottering into anarchy, are evidence of the callous disregard of African Independence by the Great Powers.\"\nKwame Nkrumah, Ghanian leader, 1961\nWhich of the following explains why Nkrumah feels Africa must \"avoid all the dangers inherent in disunity\"?\n\nA) African nations will not achieve independence without unity.\nB) Disunity is what originally led many African nations into disarray.\nC) The African continent has already experienced its share of civil wars.\nD) The world community sees power as a function of size and influence.", "gold": "D", "n_choices": 4, "subject": "high_school_world_history", "stem": false}
{"pair_id": "mmlu-7367", "lang": "en", "prompt": "The common mutation in α-thalassaemia consists of:\n\nA) a deletion.\nB) a duplication.\nC) an insertion.\nD) a point mutation.", "gold": "A", "n_choices": 4, "subject": "medical_genetics", "stem": false}
{"pair_id": "mmlu-13892", "lang": "en", "prompt": " Which early battle led to the conversion of Mecca?\n\nA) The Battle of Uhud\nB) The Battle of Camel\nC) The Battle of the Trench\nD) The Battle of Badr", "gold": "C", "n_choices": 4, "subject": "world_religions", "stem": false}
{"pair_id": "mmlu-13669", "lang": "en", "prompt": "How many states in the international system are likely to have nuclear weapons right now?\n\nA) Fewer than 7\nB) Between 8 and 15\nC) Between 16 and 25\nD) More than 25", "gold": "B", "n_choices": 4, "subject": "us_foreign_policy", "stem": false}
{"pair_id": "mmlu-7774", "lang": "en", "prompt": "Which of these plants is the national emblem of Scotland?\n\nA) ivy\nB) thistle\nC) rose\nD) linden", "gold": "B", "n_choices": 4, "subject": "miscellaneous", "stem": false}
{"pair_id": "mmlu-8934", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | I pawned my husband's prized guitar to pay for his heart operation. Scenario 2 | I didn't get the office job because I bombed the building.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "C", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-3547", "lang": "en", "prompt": "Why are perceptual or vernacular regions so difficult to identify?\n\nA) Boundaries are constantly being redrawn.\nB) Physical features, such as rivers, are constantly shifting.\nC) Defining criteria are subjective and differ among researchers.\nD) Antecedent boundaries were drawn before humans were located in the regions.", "gold": "C", "n_choices": 4, "subject": "high_school_geography", "stem": false}
{"pair_id": "mmlu-912", "lang": "en", "prompt": "The normal modes of a carbon dioxide molecule that are infrared-active include which of the following?\nI. Bending\nII. Symmetric stretching\nIII. Asymmetric stretching\n\nA) I only\nB) II only\nC) III only\nD) I and III only", "gold": "D", "n_choices": 4, "subject": "college_chemistry", "stem": true}
{"pair_id": "mmlu-6085", "lang": "en", "prompt": "This question refers to the following information.\nThe passage below is taken from testimony before Parliament.\nJoshua Drake, called in; and Examined.\nYou say you would prefer moderate labour and lower wages; are you pretty comfortable upon your present wages?\n—I have no wages, but two days a week at present; but when I am working at some jobs we can make a little, and at others we do very poorly.\nWhen a child gets 3s. a week, does that go much towards its subsistence?\n—No, it will not keep it as it should do.\nWhy do you allow your children to go to work at those places where they are ill-treated or over-worked?\n—Necessity compels a man that has children to let them work.\nThen you would not allow your children to go to those factories under the present system, if it was not from necessity?\n—No.\n—Testimony given before the Sadler Committee, 1831–32\nWhich type of organization was partially responsible for improving the hours, wages, and working conditions of workers in this era?\n\nA) Guilds\nB) Joint-stock companies\nC) Labor unions\nD) Zaibatsu groups", "gold": "C", "n_choices": 4, "subject": "high_school_world_history", "stem": false}
{"pair_id": "mmlu-11805", "lang": "en", "prompt": "Wendy Wholesaler sold merchandise to Roberta Retailer. A dispute has arisen between the parties and Roberta is trying to prove that Wendy is a ‰ÛÏmerchant‰Û as opposed to a \"casual party.\" The least important factor indicating the status of a \"merchant\" is that Wendy\n\nA) Is a wholesaler rather than a retailer.\nB) Deals in the goods sold to Roberta.\nC) Holds herself out as an expert in the goods sold to Roberta.\nD) Sells under 10 units a year to Roberta.", "gold": "A", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-4665", "lang": "en", "prompt": "Which of the following represents a measure of a firm's monopoly pricing or market power, i.e., the larger the value of the index, the greater the firm's market pricing power?\n\nA) (MC-P)/MC\nB) (MC-P)/P\nC) (P-MC)/P\nD) (P-MC)/MC", "gold": "C", "n_choices": 4, "subject": "high_school_microeconomics", "stem": false}
{"pair_id": "mmlu-10357", "lang": "en", "prompt": "A company recently moved to a new building. The old building is being actively marketed for sale, and the company expects to complete the sale in four months. Each of the following statements is correct regarding the old building, except:\n\nA) It will be reclassified as an asset held for sale.\nB) It will be classified as a current asset.\nC) It will no longer be depreciated.\nD) It will be valued at historical cost.", "gold": "D", "n_choices": 4, "subject": "professional_accounting", "stem": false}
{"pair_id": "mmlu-8679", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | I wanted my daughter to be healthy, so I fed her lots of earthworms. Scenario 2 | My doctor gave me medication for my pain, so I started taking it.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "B", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-792", "lang": "en", "prompt": "A stem-boring beetle has laid its eggs in the center of a 5-year-old wood twig, and the eggs have matured into larvae. In proper order (inside to outside), what tissues and tissue regions would the larvae encounter as they eat their way toward the outside of the twig? (For simplicity, assume that the epidermis has already fallen off completely.)\n\nA) Pith; primary xylem; secondary xylem; vascular cambium; primary phloem; secondary phloem; cortex\nB) Pith; secondary xylem; primary xylem; vascular cambium; secondary phloem; primary phloem; cortex\nC) Pith; secondary xylem; primary xylem; vascular cambium; primary phloem; secondary phloem; cortex\nD) Pith; primary xylem; secondary xylem; vascular cambium; secondary phloem; primary phloem; cortex", "gold": "D", "n_choices": 4, "subject": "college_biology", "stem": true}
{"pair_id": "mmlu-6975", "lang": "en", "prompt": "How do you describe a work group that comprises workers with demarcated tasks and distant line managers?\n\nA) Dependent\nB) Independent\nC) Interdependent\nD) Collaborative", "gold": "B", "n_choices": 4, "subject": "management", "stem": false}
{"pair_id": "mmlu-1498", "lang": "en", "prompt": "To verify a digital signature we need the\n\nA) Sender’s Private key\nB) Sender’s Public key\nC) Receiver’s Private key\nD) Receiver’s Public key", "gold": "B", "n_choices": 4, "subject": "computer_security", "stem": true}
{"pair_id": "mmlu-10620", "lang": "en", "prompt": "Skytop Co. a nonprofit entity is considering acquiring a machine for $80000 that will produce uniform cash inflows of $25000 for four years. Skytop evaluates capital projects using discounted cash flows at a cost of capital of 10% per year. Based upon the following table what action should Skytop take regarding acquisition of the machine and why? Future value of $1 for 4 years at 10% $1.464 Present value of $1 for 4 years at 10% $0.683 Future value of $1 ordinary annuity for 4 years at 10% $4.641 Present value of $1 ordinary annuity for 4 years at 10% $3.170 Acquire: Reason:\n\nA) Yes Net cash flow is $20000\nB) Yes Net future value is $36025\nC) No Net present value is ($750)\nD) No Net present value is ($8750)", "gold": "C", "n_choices": 4, "subject": "professional_accounting", "stem": false}
{"pair_id": "mmlu-11828", "lang": "en", "prompt": "A vehicle operator was driving on a highway at night. His girlfriend was in the passenger seat. They were returning from a college night class. She was smoking marijuana, blowing it out the open window, and not sharing it with the driver. The driver fell asleep, causing the car to veer off the highway and go into a ditch where it landed upside-down. Both the driver and passenger were seriously injured. She sued the operator for negligence. The operator filed a counterclaim asking for damages against the passenger by arguing that the passenger was herself negligent by \"smoking weed\" and causing him to fall asleep. Is the court likely to grant the passenger's motion to dismiss the counterclaim?\n\nA) Yes, because the passenger was not actively participating in a joint enterprise and did not do enough to contribute to the cause of the accident.\nB) Yes, because a vehicle operator is always liable and a passenger cannot be made liable for the driver's actions under any circumstances.\nC) No, because the driver has properly alleged a joint enterprise situation where the passenger is liable for her share of the negligence in causing the accident.\nD) No, because the passenger was breaking the law, making her negligent per se.", "gold": "A", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-1813", "lang": "en", "prompt": "Which of the following is NOT a feature of continuously compounded returns (i.e. log-returns)?\n\nA) They can be interpreted as continuously compounded changes in the prices\nB) They can be added over time to give returns for longer time periods\nC) They can be added across a portfolio of assets to give portfolio returns\nD) They are usually fat-tailed", "gold": "C", "n_choices": 4, "subject": "econometrics", "stem": false}
{"pair_id": "mmlu-12768", "lang": "en", "prompt": "Dr. Sam Stone has been hired by a consulting firm to assist with an ongoing research project being conducted at a large company. Dr. Stone is to work directly under the project director, Dr. Thompson, a licensed organizational psychologist. Dr. Thompson asks Dr. Stone to review the preliminary report she has prepared for the project; and, in doing so, Dr. Stone discovers that some of the reported data is misleading and may be inaccurate. Dr. Stone expresses his concerns to Dr. Thompson and is told that “it will be taken care of.” Subsequently, Dr. Stone learns that the report was submitted to the company without any changes. Dr. Stone reviews the ethical guidelines and decides that pursuing the matter further isn't necessary. Dr. Stone's conclusion:\n\nA) is correct since this situation is not covered by ethical guidelines.\nB) is correct since he has already attempted to resolve the matter in an informal manner as proscribed by ethical guidelines.\nC) is correct since it is the project director’s responsibility to provide the company with complete and accurate data.\nD) is incorrect because he has a responsibility to ensure that the company is provided with complete and accurate data.", "gold": "D", "n_choices": 4, "subject": "professional_psychology", "stem": false}
{"pair_id": "mmlu-6391", "lang": "en", "prompt": "Which of the following is typically associated with having a vasectomy?\n\nA) decreased testosterone production\nB) reduced rigidity of erections\nC) decreased sperm production\nD) normal orgasm", "gold": "D", "n_choices": 4, "subject": "human_sexuality", "stem": false}
{"pair_id": "mmlu-1379", "lang": "en", "prompt": "In the diamond structure of elemental carbon, the nearest neighbors of each C atom lie at the corners of a\n\nA) square\nB) hexagon\nC) cube\nD) tetrahedron", "gold": "D", "n_choices": 4, "subject": "college_physics", "stem": true}
{"pair_id": "mmlu-12400", "lang": "en", "prompt": "A 43-year-old man with a history of self-mutilation comes to the emergency department because of pain in his right thigh and shaking chills during the past 3 days. He says he intentionally burned his right thigh several times with a cigarette lighter 6 days ago. This morning he soaked his thigh in a tub of hot water and massaged it, after which a large amount of \"greenish fluid\" drained from the wound sites. The patient normally takes lithium carbonate but stopped 3 months ago after he lost his job. Medical history is otherwise unremarkable. Vital signs are temperature 39.2°C (102.5°F), pulse 170/min, respirations 18/min, and blood pressure 120/60 mm Hg. Physical examination shows an edematous right lateral thigh with multiple burn sites with a white exudative appearance. There is erythema surrounding the burn areas with red streaks extending up the thigh. Palpation of the burn sites discloses fluctuance. Results of laboratory studies show a hemoglobin concentration of 14 g/dL and a leukocyte count of 19,000/mm3 . In addition to beginning antibiotic therapy, which of the following is the most appropriate next step?\n\nA) Incision and drainage\nB) Psychiatric consultation\nC) Topical silver sulfadiazine\nD) Transfer to a burn center", "gold": "A", "n_choices": 4, "subject": "professional_medicine", "stem": false}
{"pair_id": "mmlu-7082", "lang": "en", "prompt": " The ____________ delivers to us a wide array of offerings, either directly or indirectly, through business markets, to serve our wants and needs.\n\nA) Aggregate marketing system.\nB) Planning excellence.\nC) A quality rift.\nD) A value line.", "gold": "A", "n_choices": 4, "subject": "marketing", "stem": false}
{"pair_id": "mmlu-965", "lang": "en", "prompt": "What is the limiting high-temperature molar heat capacity at constant volume (C_V) of a gas-phase diatomic molecule?\n\nA) 1.5R\nB) 2R\nC) 2.5R\nD) 3.5R", "gold": "D", "n_choices": 4, "subject": "college_chemistry", "stem": true}
{"pair_id": "mmlu-11421", "lang": "en", "prompt": "Daylight Corp. manufactured, assembled, and marketed a steel cutting press designed to cut aluminum sheets into can covers. In 2002, Goodboy Beverages purchased Daylight's deluxe cutting press model for $20,000. In 2002 and 2003, Daylight received a few complaints that other cutting press customers had experienced hazardous malfunctions. In response, Daylight created a safety device and offered it for $5,000 in a brochure sent to all their customers that mentioned that a few users had been injured in using the cutting press model. Goodboy received the brochure, but did not buy a $5,000 safety device. In 2004, a Goodboy worker was injured by the press. At trial, Daylight argued that if Goodboy had purchased the safety device the injury would not have occurred, and moved for a directed verdict. The court should\n\nA) Grant Daystar's motion, because Goodboy's refusal to purchase the safety device was an effective assumption of risk.\nB) Deny the motion, because Goodboy's negligence in failing to purchase the safety device was less than Daylight's negligence in manufacturing the unsafe press initially.\nC) Deny the motion, because a jury could find that Daylight could reasonably foresee that Goodboy would be unwilling to pay $5,000 for a safety device.\nD) Deny the motion, because since the press was inherently dangerous, Daylight was strictly liable.", "gold": "C", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-13842", "lang": "en", "prompt": "What is the genome structure of the polyomavirus family?\n\nA) Circular ds DNA with T antigen coding regions\nB) Circular ssDNA\nC) RNA genome with RT\nD) Segmented RNA genome", "gold": "C", "n_choices": 4, "subject": "virology", "stem": false}
{"pair_id": "mmlu-9510", "lang": "en", "prompt": "On a global scale, trends are suggesting that with rising incomes and urbanization we are seeing a rise in which food commodities?\n\n\nA) Roots and Tubers\nB) Fruits and vegetables\nC) Beans and lentils\nD) Meat and Dairy products", "gold": "D", "n_choices": 4, "subject": "nutrition", "stem": false}
{"pair_id": "mmlu-7436", "lang": "en", "prompt": "Which of these is not a Hindudeity?\n\nA) Shiva\nB) Vishnu\nC) Sanskrit\nD) Brahma", "gold": "C", "n_choices": 4, "subject": "miscellaneous", "stem": false}
{"pair_id": "mmlu-5471", "lang": "en", "prompt": "Random variable X is normally distributed, with a mean of 25 and a standard deviation of 4. Which of the following is the approximate interquartile range for this distribution?\n\nA) 25.00 – 22.30 = 2.70\nB) 27.70 – 22.30 = 5.40\nC) 27.70 ÷ 22.30 = 1.24\nD) 2.00(4.00) = 8.00", "gold": "B", "n_choices": 4, "subject": "high_school_statistics", "stem": true}
{"pair_id": "mmlu-5494", "lang": "en", "prompt": "Both over-the-counter niacin and the prescription drug Lipitor are known to lower blood cholesterol levels. In one double-blind study Lipitor outperformed niacin. The 95% confidence interval estimate of the difference in mean cholesterol level lowering was (18, 41). Which of the following is a reasonable conclusion?\n\nA) Niacin lowers cholesterol an average of 18 points, while Lipitor lowers cholesterol an average of 41 points.\nB) There is a 0.95 probability that Lipitor will outperform niacin in lowering the cholesterol level of any given individual.\nC) There is a 0.95 probability that Lipitor will outperform niacin by at least 23 points in lowering the cholesterol level of any given individual.\nD) None of the above.", "gold": "D", "n_choices": 4, "subject": "high_school_statistics", "stem": true}
{"pair_id": "mmlu-6665", "lang": "en", "prompt": "What is Kant's 'categorical imperative?\n\nA) You must not consciously harm another person.\nB) You must always act in the best interests of the community.\nC) You must treat human beings as means rather than ends.\nD) You must act according as if your values apply to everyone.", "gold": "D", "n_choices": 4, "subject": "jurisprudence", "stem": false}
{"pair_id": "mmlu-5855", "lang": "en", "prompt": "This question refers to the following information.\nNo task is more urgent than that of preserving peace. Without peace our independence means little. The rehabilitation and upbuilding of our countries will have little meaning. Our revolutions will not be allowed to run their course. What can we do? We can do much! We can inject the voice of reason into world affairs. We can mobilize all the spiritual, all the moral, all the political strength of Asia and Africa on the side of peace. Yes, we! We, the peoples of Asia and Africa, 1.4 billion strong.\nIndonesian leader Sukarno, keynote address to the Bandung Conference, 1955\nLike numerous other leaders in Africa, Asia, and the Middle East, Sukarno attempted, but did not completely succeed, in maintaining a neutral stance during\n\nA) the Arab-Israeli conflict.\nB) the Korean War.\nC) the Cold War.\nD) the Vietnam conflicts.", "gold": "C", "n_choices": 4, "subject": "high_school_world_history", "stem": false}
{"pair_id": "mmlu-8953", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | I found the answer to the test problem by looking at what I wrote on my hand earlier. Scenario 2 | I fired the gun at the child's eye at the shooting range.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "A", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-9530", "lang": "en", "prompt": "Which of the following statements about glucose metabolism is correct?\n\n\nA) Glucagon increases the rate of glycolysis\nB) Glycolysis requires NADP+\nC) In glycolysis, glucose is cleaved into two three-carbon products\nD) The end-product of glycolysis in red blood cells is pyruvate", "gold": "C", "n_choices": 4, "subject": "nutrition", "stem": false}
{"pair_id": "mmlu-3988", "lang": "en", "prompt": "The U.S. economy currently suffers a recessionary gap and a budget deficit exists. If the government wishes to fix the recession which of the following choices best describes the appropriate fiscal policy the impact on the market for loanable funds the interest rate and the market for the U.S. dollar? FISCAL POLICY     LOANABLE FUNDS     INTEREST RATE     MARKET FOR $\n\nA) Tax increase     Demand rises     Falling     Demand falls\nB) Tax cut     Supply rises     rising     Demand rises\nC) Tax cut     Demand rises     rising     Demand rises\nD) Tax increase     Supply falls     Falling     Demand rises", "gold": "C", "n_choices": 4, "subject": "high_school_macroeconomics", "stem": false}
{"pair_id": "mmlu-1484", "lang": "en", "prompt": "The AH Protocol provides source authentication and data integrity, but not\n\nA) Integrity\nB) Privacy\nC) Nonrepudiation\nD) Both A & C", "gold": "B", "n_choices": 4, "subject": "computer_security", "stem": true}
{"pair_id": "mmlu-7104", "lang": "en", "prompt": "Marketing research:\n\nA) Is work undertaken to understand how to make specific marketing strategy decisions.\nB) Is the collection and analysis of data from a sample or census of individuals or organizations.\nC) Includes all forms of market, opinion, and social research such as consumer and industrial surveys.\nD) Is work undertaken to determine either structural characteristics of the industry of concern.", "gold": "A", "n_choices": 4, "subject": "marketing", "stem": false}
{"pair_id": "mmlu-11373", "lang": "en", "prompt": "An owner and operator of a restaurant contracted in writing with a produce distributor to buy 50 pounds of heirloom tomatoes. At the time the contract was signed, the owner orally said to the distributor, \"We do have an understanding that our chef must approve the quality of the heirloom tomatoes before I will pay you. \" The distributor acknowledged the owner's request and responded, \"If you say so. \"Thereafter, the distributor delivered the heirloom tomatoes to the owner. After inspecting the tomatoes, the chef refused to give his approval, because he felt that the tomatoes were not of sufficient quality. As a result, the owner refused to accept and pay for the tomatoes. The distributor brought a breach of contract action against the owner because he refused to pay for the tomatoes. How should the court rule on the owner's offer to prove, over the distributor's objection, that the chef refused to approve the tomatoes that were delivered?\n\nA) The evidence is admissible to show frustration of purpose.\nB) The evidence is admissible to show that the written agreement was subject to an oral condition precedent.\nC) The evidence is barred, because the written contract appears to be a complete and total integration of the parties' agreement.\nD) The evidence is barred, because the oral agreement is within the statute of frauds.", "gold": "B", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-6829", "lang": "en", "prompt": "The concomitant variation fallacy is\n\nA) attributing the cause of an event to a single cause when the cause is more complicated\nB) arguing that since two events happened at the same time, one event must have caused the other\nC) arguing that since one event preceded another event it must have caused the second event\nD) drawing a conclusion based on sign reasoning when there is not really a direct relationship between the alleged sign and the subject of the conclusion", "gold": "B", "n_choices": 4, "subject": "logical_fallacies", "stem": false}
{"pair_id": "mmlu-2534", "lang": "en", "prompt": "Identify the antecedent of the following conditional proposition: The university raises tuition only if both the governor approves of it and the board of trustees recommends it.\n\nA) The university raises tuition.\nB) The governor approves of it.\nC) The board of trustees recommends it.\nD) The governor approves of it and the board of trustees recommends it.", "gold": "A", "n_choices": 4, "subject": "formal_logic", "stem": false}
{"pair_id": "mmlu-3661", "lang": "en", "prompt": "While ratifying the Constitution, the primary point of disagreement between Federalists and Anti-Federalists was over\n\nA) whether representation in Congress should be equal among states or proportional to population\nB) the strength of the federal government relative to the states\nC) the future of the slave trade\nD) whether people had inalienable rights", "gold": "B", "n_choices": 4, "subject": "high_school_government_and_politics", "stem": false}
{"pair_id": "mmlu-2432", "lang": "en", "prompt": "Solve the equation 18 + p = 29.\n\nA) −47\nB) −11\nC) 11\nD) 47", "gold": "C", "n_choices": 4, "subject": "elementary_mathematics", "stem": true}
{"pair_id": "mmlu-4207", "lang": "en", "prompt": "What will happen to the equilibrium price and the equilibrium quantity of good A when producers of good A expect the price to be higher in the near future?\n\nA) The equilibrium price will rise and the equilibrium quantity will fall.\nB) The equilibrium price will fall and the equilibrium quantity will rise.\nC) The equilibrium price and the equilibrium quantity will rise.\nD) The equilibrium price and the equilibrium quantity will fall.", "gold": "A", "n_choices": 4, "subject": "high_school_macroeconomics", "stem": false}
{"pair_id": "mmlu-1798", "lang": "en", "prompt": "Compared with radiation from the Sun, terrestrial radiation has a lower\n\nA) wavelength\nB) frequency\nC) Both\nD) Neither", "gold": "B", "n_choices": 4, "subject": "conceptual_physics", "stem": true}
{"pair_id": "mmlu-11064", "lang": "en", "prompt": "A woman had been spreading rumors around town that a defendant was a shoplifter. After the defendant learned about these rumors, she was furious and decided that she would teach the woman to keep her mouth shut. Late that night, the defendant went over to the woman's house with the intent to punch her in the mouth. When the defendant arrived at the woman's home, she peered in the bedroom window and saw the woman asleep in bed. The defendant proceeded to pry open the bedroom window and climbed inside. The defendant then grabbed the woman and punched her in the face and hurriedly fled from the home. The force of the blow fractured the woman's jaw. If the defendant is subsequently prosecuted for burglary, she will most likely be found\n\nA) guilty, because she broke into the woman's home at night with the intent to commit a felony therein.\nB) guilty, because the woman suffered a broken jaw as a result of the blow inflicted by the defendant.\nC) not guilty, because the defendant did not intend to seriously injure the woman.\nD) not guilty, because the defendant did not intend to commit a felony at the time of the breaking and entry.", "gold": "D", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-6378", "lang": "en", "prompt": "When a woman thinks that she probably is a lesbian, she is at the:\n\nA) pride stage\nB) tolerance stage\nC) acceptance stage\nD) confusion stage", "gold": "B", "n_choices": 4, "subject": "human_sexuality", "stem": false}
{"pair_id": "mmlu-9051", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | I listened carefully to the private conversation of the people behind me in class. Scenario 2 | I signed up for a volunteering slot and showed.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "B", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-10860", "lang": "en", "prompt": "A debtor owed a creditor $750 on an old debt. On July 1, the debt was barred by the statute of limitations. On August 1, the debtor ran into the creditor at a party and overheard him telling mutual friends that the debtor \"is a deadbeat who weiches on his debts. \" Feeling pangs of guilt, the debtor approached the creditor and orally agreed to pay him the $750 debt on September 1. The debtor refused to pay the creditor the $750 as promised on September 1St. If the creditor sues the debtor to recover the $750 debt, which would provide the strongest grounds that the debtor's oral promise was unenforceable?\n\nA) It was not supported by new consideration.\nB) It was violative of the statute of frauds.\nC) The debt was already barred by the statute of limitations.\nD) There was no mutuality of obligation.", "gold": "B", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-13066", "lang": "en", "prompt": "Which author emphasised strategy as achieving competitive advantage for an organization?\n\nA) Steyn (2003)\nB) Gregory (2006)\nC) Cutlip et al (2006)\nD) Henry (2006)", "gold": "D", "n_choices": 4, "subject": "public_relations", "stem": false}
{"pair_id": "mmlu-6817", "lang": "en", "prompt": "Which of the following describes the fallacy of false consolation?\n\nA) arguing against a position based only on negative personal feelings toward the position\nB) responding to charges of wrongdoing by saying others do things as bad or worse\nC) arguing that someone is not really harmed because things could be worse or by pointing out what they have to be thankful for\nD) using threats of harm instead of reasoning to get agreement", "gold": "C", "n_choices": 4, "subject": "logical_fallacies", "stem": false}
{"pair_id": "mmlu-2960", "lang": "en", "prompt": "Compared to DNA viral replication, RNA viral replication is more susceptible to mutation because\n\nA) RNA replication takes place in the cytoplasm, whereas DNA replication takes place in the nucleus.\nB) RNA viruses replicate much faster than DNA viruses.\nC) RNA viruses use uracil instead of thymine.\nD) RNA replication lacks the proofreading functions characteristic of DNA replication.", "gold": "D", "n_choices": 4, "subject": "high_school_biology", "stem": true}
{"pair_id": "mmlu-11771", "lang": "en", "prompt": "Congress enacted a statute wherein the government would purchase and operate all of the nation's airlines. In all likelihood, Congress's power to enact this legislation will derive from\n\nA) its power to tax and provide for the general welfare.\nB) its power to raise and support an army and declare war.\nC) its power to regulate commerce.\nD) its power to make laws regarding territory and other property belonging to the United States.", "gold": "C", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-12786", "lang": "en", "prompt": "Which of the following best describes ethical guidelines regarding barter as a substitute for the collection of fees for professional psychological services?\n\nA) Barter may be acceptable when certain conditions are met.\nB) The barter of goods (but not services) is prohibited by ethical guidelines.\nC) The barter of services (but not goods) is prohibited by ethical guidelines.\nD) Barter of any kind is prohibited by ethical guidelines.", "gold": "A", "n_choices": 4, "subject": "professional_psychology", "stem": false}
{"pair_id": "mmlu-3552", "lang": "en", "prompt": "Spanish colonial architecture, uneven economic development, and rural-to-urban migration flows are most commonly found in which of the following?\n\nA) Southeast Asia\nB) Sub-Saharan Africa\nC) North America\nD) Latin America", "gold": "D", "n_choices": 4, "subject": "high_school_geography", "stem": false}
{"pair_id": "mmlu-2945", "lang": "en", "prompt": "An eager AP Biology student interested in studying osmosis and the movement of water in solutions took a dialysis bag containing a 0.5 M solution and placed it into a beaker containing a 0.6 M solution. Which of the following is true about water potential?\n\nA) An eager AP Biology student interested in studying osmosis and the movement of water in solutions took a dialysis bag containing a 0.5 M solution and placed it into a beaker containing a 0.6 M solution. It drives the movement of water from a region of lower water potential to a region of higher water potential.\nB) Solute potential is the only factor that determines the water potential.\nC) Pressure potential combines with solute potential to determine the water potential.\nD) Water potential always drives water from an area of lower pressure potential to an area of higher pressure potential.", "gold": "C", "n_choices": 4, "subject": "high_school_biology", "stem": true}
{"pair_id": "mmlu-8642", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | I changed diapers on my elderly paralyzed aunt several times a day. Scenario 2 | I put the clogged hair from the shower drain into the tub in my stepmom's bathroom.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "C", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-11867", "lang": "en", "prompt": "A state has recently enacted a statute requiring all prospective voters in state elections who wish to write-in a candidate to print the candidate's full name, and designate the office for which the candidate is running. The statute provides that such information must be written on the ballot in ink inan appropriate space. A write-in candidate is a German-American and is of the opinion that he needs a large turnout of German voters in order to win the election. As a result, his campaign manager decides to mail to every registered German voter a tear-off sticker, which bears the candidate's name and office for which he is running. Since many native German people are not proficient in reading and writing English, the campaign manager believes that many of the voters will have difficulty writing the candidate's name and office on the ballot. As a result, the campaign manager has mounted an extensive media campaign to inform voters on how to apply the stickers to the ballot. Five months prior to the election an election official notifies the candidate's campaign committee that the tear-off stickers do not comply with the state statute. In her letter, the official explains that state election officials are of the opinion that it is necessary for potential voters to write the candidate's name in ink. Therefore, she concludes that the stickers do not comply with statutory requirements. Three weeks later, the candidate filed suit in federal district court against state election officials, claiming that their interpretation of the state statute violates the U. S. Constitution. Which of the following sets forth the strongest constitutional argument the candidate could make against the interpretation of the statute by the state officials?\n\nA) It unreasonably discriminates against German voters who lack a proficiency in the English language.\nB) It unreasonably discriminates against write-in candidates for public office.\nC) It unreasonably interferes with the exclusive federal election power as embodied in the Fifteenth Amendment.\nD) It unreasonably interferes with the 1965 Voting Rights Act outlawing literacy tests.", "gold": "B", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-7128", "lang": "en", "prompt": "Which of the following types of advertising is being used when a retail store agrees to share advertising costs for a consumer product with the national distributor of the product?\n\nA) Trade advertising\nB) Industrial advertising\nC) National advertising\nD) Cooperative advertising", "gold": "D", "n_choices": 4, "subject": "marketing", "stem": false}
{"pair_id": "mmlu-4556", "lang": "en", "prompt": "A rational consumer who is eating Girl Scout cookies stops eating when\n\nA) the total benefit equals the total cost of eating cookies.\nB) the marginal benefit equals the marginal cost of the next cookie.\nC) the marginal cost of eating cookies is maximized.\nD) the marginal benefit of eating cookies is minimized.", "gold": "B", "n_choices": 4, "subject": "high_school_microeconomics", "stem": false}
{"pair_id": "mmlu-4245", "lang": "en", "prompt": "The polynomial which results from the expansion of $(x^2+5x+6)^2+(px+q)(x^3+7x^2+3x)$ has degree $2$. Find $p+q$.\n\nA) -2\nB) -4\nC) -1\nD) 1", "gold": "B", "n_choices": 4, "subject": "high_school_mathematics", "stem": true}
{"pair_id": "mmlu-11985", "lang": "en", "prompt": "A driver was prosecuted for speeding. It is conceded that the speed limit on the stretch of the highway in question at the time of the alleged speeding was 55 m. p. h. As part of its case-in-chief, the prosecution called a police officer who testified that at about 11:30 a. m. one mile east of the town line, he set up a radar apparatus. The officer testified that, with the use of tuning forks, he found the radar apparatus was accurate. The officer also stated that he had five years of experience in operating such radar devices. Furthermore, the officer testified that the radar's mechanism for recording the speed of vehicles by means of an ink line drawn mechanically on a roll of paper was also functioning properly. Upon further questioning, the officer stated that at about 1:30 p. m. , the needle of the speed indicator dial of the radar apparatus showed the driver's car passing his location at 65 m. p. h. If the driver's attorney makes an objection to the introduction of the radar finding that the driver was traveling at a speed of 65 m. p. h. , the court will most likely\n\nA) sustain the objection, because the radar results are not conclusive evidence of speeding.\nB) sustain the objection, because radar is not a technique recognized by the scientific community.\nC) overrule the objection, because the court will take judicial notice of the accuracy of such tests.\nD) overrule the objection, because the radar results would be construed as an admission against interest.", "gold": "C", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-5109", "lang": "en", "prompt": "Which classical conditioning term best describes the following scenario: Later in his classical conditioning experiments, Ivan Pavlov's dogs began to salivate whenever they heard any sound similar to a bell, such as a doorbell or someone accidentally clinking a water glass.\n\nA) discrimination\nB) spontaneous recovery\nC) trace conditioning\nD) generalization", "gold": "D", "n_choices": 4, "subject": "high_school_psychology", "stem": false}
{"pair_id": "mmlu-10160", "lang": "en", "prompt": "In 1797, John Frere made a discovery that he described as:\n\nA) animal remains scattered on the surface of the ground.\nB) the earliest written documents.\nC) primitive stone tools excavated at great depth.\nD) human skulls with ape-like features.", "gold": "C", "n_choices": 4, "subject": "prehistory", "stem": false}
{"pair_id": "mmlu-4969", "lang": "en", "prompt": "Which of the following approaches emphasizes the importance of people's feelings and views human nature as naturally positive and growth seeking?\n\nA) biological\nB) behavioral\nC) cognitive\nD) humanistic", "gold": "D", "n_choices": 4, "subject": "high_school_psychology", "stem": false}
{"pair_id": "mmlu-8068", "lang": "en", "prompt": "According to Article III, Section 1, of the United States Constitution, judges \"shall hold their offices during good behavior...\" A consequence of this clause is that:\n\nA) Congress cannot remove federal judges from office\nB) federal judges have political independence to fulfill their duties\nC) the president has no influence over the composition of the Supreme Court\nD) federal judges lack opportunity for advancement or promotion", "gold": "B", "n_choices": 4, "subject": "miscellaneous", "stem": false}
{"pair_id": "mmlu-3922", "lang": "en", "prompt": "An American buys an entertainment system that was manufactured in China. How does the U.S. national income accounts treat this transaction?\n\nA) Net exports and GDP both rise.\nB) Net exports and GDP both fall.\nC) Net exports and GDP go in opposite directions.\nD) Net exports fall and there is no change in GDP.", "gold": "B", "n_choices": 4, "subject": "high_school_macroeconomics", "stem": false}
{"pair_id": "mmlu-2418", "lang": "en", "prompt": "Sapphire is making bouquets of balloons for a graduation party. She has 179 balloons and needs to put 7 balloons in each bouquet. How many bouquets can Sapphire make?\n\nA) 32 bouquets\nB) 23 bouquets\nC) 25 bouquets\nD) 26 bouquets", "gold": "C", "n_choices": 4, "subject": "elementary_mathematics", "stem": true}
{"pair_id": "mmlu-12660", "lang": "en", "prompt": "You receive an e-mail from Dr. Brenda Browne, a licensed psychologist, who is currently seeing one of your former clients for a substance abuse disorder. Dr. Browne asks you to forward the client’s file to her and states that the client has signed a release authorizing you to do so. You should:\n\nA) wait until you receive a request from the client before taking any action.\nB) contact the client to discuss the release of information.\nC) forward a photocopy of the file to the psychologist as requested.\nD) forward a summary of the file to the psychologist that includes only information related to the client’s current condition.", "gold": "B", "n_choices": 4, "subject": "professional_psychology", "stem": false}
{"pair_id": "mmlu-934", "lang": "en", "prompt": "What is the maximum value of the z-component of the spin angular momentum of 43Ca? [I = 7⁄2]\n\nA) 3.691 x 10^-34 J s\nB) 4.185 x 10^-34 J s\nC) 5.493 x 10^-34 J s\nD) 8.370 x 10^-34 J s", "gold": "A", "n_choices": 4, "subject": "college_chemistry", "stem": true}
{"pair_id": "mmlu-3356", "lang": "en", "prompt": "This question refers to the following information.\n\"The Italian nation has been at last united in our own days, and we all rejoiced in its union. Yet we may be allowed to doubt whether the union was not a little too speedy and a little too thorough. It is surely carrying unity too far to wipe out all traces of the independent being, for most purposes to wipe out the very name, of such a land as Sicily. It jars on our feelings to find that, while Ireland at least forms part of the royal style of its sovereign, Sicily is no longer even a geographical expression. The island realm of Roger has sunk to be seven provinces of the kingdom on the mainland. And there is another result of Italian unity, a result in which we may rejoice without drawbacks, but which still has somewhat of sadness about it as finally ending that great phase of the history of Europe with which we have throughout been dealing. Never were ties with the past so fully snapped as when the army of Italy entered liberated Rome. Of all novelties in European history the greatest was when Rome became the centre of a dominion with acknowledged metes and bounds, the head in short of a local Italian kingdom. \"Rome the capital of Italy\" was a formula which might well gladden our hearts; but it was a formula which formally swept away the œcumenical position, the œcumenical traditions, of Rome&….\nBut the kingdom of Italy is not an appendage to Rome; Rome is the head of the kingdom. The whole is greater than its part; Rome, by her own free will and by the free will of Italy, has become less than Italy. By becoming the willing head of an Italian kingdom she has formally cast aside her Imperial traditions as they were not cast aside when brute force made her the head of a French department.\"\nEdward A. Freeman, British historian and politician, The Chief Periods of European History, 1885\nAccording to the author, Italy's connections to its history and traditions were most \"fully snapped\" when\n\nA) Sicily ceased to be an independent kingdom.\nB) Italians liberated Rome.\nC) Rome became the head of a French Department.\nD) French troops abandoned Rome to fight in the Franco-Prussian War.", "gold": "D", "n_choices": 4, "subject": "high_school_european_history", "stem": false}
{"pair_id": "mmlu-8168", "lang": "en", "prompt": " According to Mill, censoring speech that is possibly true\n\nA) violates human dignity.\nB) fails a prima facie duty of honesty.\nC) is tantamount to asserting that you are infallible.\nD) is a risk worth taking of the speech is dangerous enough.", "gold": "C", "n_choices": 4, "subject": "moral_disputes", "stem": false}
{"pair_id": "mmlu-428", "lang": "en", "prompt": " Although the benefit and contribution of civil society in encouraging sustainability, lobbying on important issues and helping the needy are taken for granted in many places, in many other countries, such as _____ and ______ civil society is far less developed than in, for instance, _______.\n\nA) Russia, China, Britain\nB) Norway, China, Russia\nC) Britain, United States, Australia\nD) Canada, Russia, United States", "gold": "A", "n_choices": 4, "subject": "business_ethics", "stem": false}
{"pair_id": "mmlu-7930", "lang": "en", "prompt": "What color is Pepto-Bismol liquid?\n\nA) pink\nB) blue\nC) clear\nD) green", "gold": "A", "n_choices": 4, "subject": "miscellaneous", "stem": false}
{"pair_id": "mmlu-2043", "lang": "en", "prompt": "Poles and zeros are arranged alternatively on negative real axis, then type of network is/are\n\nA) LC network.\nB) RC network.\nC) RL network.\nD) Both 2 and 3.", "gold": "D", "n_choices": 4, "subject": "electrical_engineering", "stem": true}
{"pair_id": "mmlu-4168", "lang": "en", "prompt": "If $100 of new autonomous private investment were added to an economy with a marginal propensity to consume of .90 by how much would aggregate demand shift to the right?\n\nA) $190 \nB) $900 \nC) $1,000 \nD) $1,900 ", "gold": "C", "n_choices": 4, "subject": "high_school_macroeconomics", "stem": false}
{"pair_id": "mmlu-4686", "lang": "en", "prompt": "Economics is best described as\n\nA) the study of how scarce material wants are allocated between unlimited resources.\nB) the study of how scarce labor can be replaced by unlimited capital.\nC) the study of how decision makers choose the best way to satisfy their unlimited material wants with a scarce supply of resources.\nD) the study of how unlimited material wants can best be satisfied by allocating limitless amounts of productive resources.", "gold": "C", "n_choices": 4, "subject": "high_school_microeconomics", "stem": false}
{"pair_id": "mmlu-4070", "lang": "en", "prompt": "Labor productivity and economic growth increase if\n\nA) a nation subsidizes education for all citizens.\nB) a nation imposes tariffs and quotas on imported goods.\nC) a nation removes penalties for firms that pollute natural resources.\nD) a nation ignores societal barriers like discrimination.", "gold": "A", "n_choices": 4, "subject": "high_school_macroeconomics", "stem": false}
{"pair_id": "mmlu-3979", "lang": "en", "prompt": "A change in government spending will have a greater short run impact on real output when\n\nA) the marginal propensity to consume is lower.\nB) the velocity of money is lower.\nC) the velocity of money is higher.\nD) the marginal propensity to consume is larger.", "gold": "D", "n_choices": 4, "subject": "high_school_macroeconomics", "stem": false}
{"pair_id": "mmlu-7382", "lang": "en", "prompt": "Differential distribution of substances in the egg most typically results in:\n\nA) differences in gene expression which may establish a pattern in the embryo as the cells divide\nB) amplification of specific genes during development\nC) development of polyploid tissues\nD) loss of specific genes during development", "gold": "A", "n_choices": 4, "subject": "medical_genetics", "stem": false}
{"pair_id": "mmlu-9307", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | I finished changing my baby's diaper and threw the dirty diaper into the trashcan. Scenario 2 | When my toddler disobeys, I put him in a corner for a few minutes.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "D", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-13817", "lang": "en", "prompt": "The biggest single determinant of HIV transmission to the infant is:\n\nA) Mother’age\nB) Mother’s viral load\nC) Mother’s immune status\nD) Father’s viral load", "gold": "B", "n_choices": 4, "subject": "virology", "stem": false}
{"pair_id": "mmlu-2583", "lang": "en", "prompt": "About what percentage of the global population was literate in 1950?\n\nA) 36%\nB) 56%\nC) 76%\nD) 96%", "gold": "B", "n_choices": 4, "subject": "global_facts", "stem": false}
{"pair_id": "mmlu-1141", "lang": "en", "prompt": "Statement 1 | f : X → Y is continuous and X is compact. f must be uniformly continuous. Statement 2 | If f(x), g(x) : R → R are everywhere differentiable f(g(x)) must be everywhere differentiable.\n\nA) True, True\nB) True, False\nC) False, True\nD) False, False", "gold": "A", "n_choices": 4, "subject": "college_mathematics", "stem": true}
{"pair_id": "mmlu-3563", "lang": "en", "prompt": "Which of the following countries is NOT a federal state?\n\nA) Canada\nB) United States\nC) France\nD) People's Republic of China", "gold": "C", "n_choices": 4, "subject": "high_school_geography", "stem": false}
{"pair_id": "mmlu-5665", "lang": "en", "prompt": "This question refers to the following information.\n\"Americans have more freedom and broader rights than citizens of almost any other nation in the world, including the capacity to criticize their government and their elected officials. But we do not have the right to resort to violence—or the threat of violence—when we don't get our way. Our founders constructed a system of government so that reason could prevail over fear. Oklahoma City proved once again that without the law there is no freedom. Criticism is part of the lifeblood of democracy. No one is right all the time. But we should remember that there is a big difference between criticizing a policy or a politician and demonizing the government that guarantees our freedoms and the public servants who enforce our laws.\"\nPresident Bill Clinton, \"What We Learned in Oklahoma City,\" 2010\n(Written on the 15th anniversary of the Oklahoma City bombing)\n\"If there would not have been a Waco, I would have put down roots somewhere and not been so unsettled with the fact that my government…was a threat to me. Everything that Waco implies was on the forefront of my thoughts. That sort of guided my path for the next couple of years.\"\nTimothy McVeigh, Oklahoma City bomber, 2001\nU.S. Representative John Conyers, Events Surrounding the Branch Davidian Cult Standoff in Waco, Texas: Hearing Before the Committee on the Judiciary, House of Representatives, 1993\nIn the third excerpt, Representative Conyers refers to \"Wounded Knee\" as an example of\n\nA) a violent cult\nB) a federal misuse of power\nC) the site of a bombing\nD) a recent conflict between federal agents and Lakota Indians", "gold": "B", "n_choices": 4, "subject": "high_school_us_history", "stem": false}
{"pair_id": "mmlu-7675", "lang": "en", "prompt": "After Prince Charles who is next in line to be the king of England?\n\nA) Prince William\nB) Prince Andrew\nC) Prince Edward\nD) Fresh Prince", "gold": "A", "n_choices": 4, "subject": "miscellaneous", "stem": false}
{"pair_id": "mmlu-8500", "lang": "en", "prompt": " Which of the following is not, according to Carens, a concern for all liberal egalitarians?\n\nA) the ability of people to pursue their own projects\nB) equal opportunity\nC) freedom of speech\nD) keeping actual inequalities as small as possible", "gold": "C", "n_choices": 4, "subject": "moral_disputes", "stem": false}
{"pair_id": "mmlu-5812", "lang": "en", "prompt": "This question refers to the following information.\n\"We have witnessed for more than a quarter of a century the struggles of the two great political parties for power and plunder, while grievous wrongs have been inflicted upon the suffering people. We charge that the controlling influences dominating both these parties have permitted the existing dreadful conditions to develop without serious effort to prevent or restrain them. Neither do they now promise us any substantial reform. They have agreed together to ignore, in the coming campaign, every issue but one. They propose to drown the outcries of a plundered people with the uproar of a sham battle over the tariff, so that capitalists, corporations, national banks, rings, trusts, watered stock, the demonetization of silver and the oppressions of the usurers may all be lost sight of. They propose to sacrifice our homes, lives, and children on the altar of mammon; to destroy the multitude in order to secure corruption funds from the millionaires.\n\"Assembled on the anniversary of the birthday of the nation, and filled with the spirit of the grand general and chief who established our independence, we seek to restore the government of the Republic to the hands of 'the plain people,' with which class it originated. We assert our purposes to be identical with the purposes of the National Constitution; to form a more perfect union and establish justice, insure domestic tranquillity, provide for the common defence, promote the general welfare, and secure the blessings of liberty for ourselves and our posterity.\"\nPopulist Party Platform, 1892\nThe \"free silver\" campaign of 1896 received its greatest popular support from\n\nA) New England businessmen, who were discriminated against under the existing banking system\nB) Southern women, who incorporated it into a larger campaign for economic equality\nC) bankers, who had run out of paper currency to invest\nD) farmers, who hoped that a more generous money supply would ease their debt burdens", "gold": "D", "n_choices": 4, "subject": "high_school_us_history", "stem": false}
{"pair_id": "mmlu-5290", "lang": "en", "prompt": "Research has shown a possible connection between the neurotransmitter acetylcholine and which of the following mental disorders?\n\nA) Parkinson's disease\nB) Alzheimer's disease\nC) Schizophrenia\nD) Mania", "gold": "B", "n_choices": 4, "subject": "high_school_psychology", "stem": false}
{"pair_id": "mmlu-12654", "lang": "en", "prompt": "The primary purpose of a(n) ____________ is to obtain detailed information about job requirements in order to facilitate decisions related to compensation.\n\nA) needs assessment\nB) organizational analysis\nC) job analysis\nD) job evaluation", "gold": "D", "n_choices": 4, "subject": "professional_psychology", "stem": false}
{"pair_id": "mmlu-1841", "lang": "en", "prompt": "For an autoregressive process to be considered stationary\n\nA) The roots of the characteristic equation must all lie inside the unit circle\nB) The roots of the characteristic equation must all lie on the unit circle\nC) The roots of the characteristic equation must all lie outside the unit circle\nD) The roots of the characteristic equation must all be less than one in absolute value", "gold": "C", "n_choices": 4, "subject": "econometrics", "stem": false}
{"pair_id": "mmlu-2412", "lang": "en", "prompt": "A group of students played a basketball game after school. Which total can be found using the expression 7 x 2?\n\nA) the total number of points if a player made 7 shots and each shot was worth 2 points\nB) the total number of basketballs if 7 basketballs were old and 2 basketballs were new\nC) the total number of points if one player had 7 points and a different player had 2 points\nD) the total number of basketballs used if there were 7 basketballs and 2 of the basketballs were not used", "gold": "A", "n_choices": 4, "subject": "elementary_mathematics", "stem": true}
{"pair_id": "mmlu-13372", "lang": "en", "prompt": "In what ways might a society be threatened?\n\nA) Societies are threatened when their linguistic heritage and traditions are under attack. The inability to use and develop traditional language signifies the destruction of a society.\nB) Threats to societal security exist when a society regards its collective identity as being under question. Such circumstances include the suppression of expression and interference with a society's ability to reproduce itself across generations.\nC) Military threats to societies in the form of external aggression, mainly through de-population policies with the intent of preventing collective identities from being reproduced over generations.\nD) Migratory influxes into a host receiving state results in a shift in the composition of the population resulting in horizontal competition or vertical competition. This is not an outright threat to survival but part of wider uncontrollable processes of evolutionary identity.", "gold": "B", "n_choices": 4, "subject": "security_studies", "stem": false}
{"pair_id": "mmlu-8654", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | I took pictures of her was she was in the shower. Scenario 2 | I knocked over a standing stone at Stonehenge.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "A", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-4966", "lang": "en", "prompt": "Which of the following clinicians most likely follows Carl Rogers's approach to psychotherapy?\n\nA) Dr. Jones, who emphasizes the need for modification of undesirable behaviors in his patients\nB) Terence Springer, who counsels individuals to find their inner spirituality and develop their relationship with a supreme being\nC) Cathy Cooper, who is an empathic counselor who encourages clients through unconditional support to find their own path to better health and growth\nD) Utrese Leed, who creates a framework for her patients that defines ideal psychological growth and development and who supports their efforts toward professional achievement", "gold": "C", "n_choices": 4, "subject": "high_school_psychology", "stem": false}
{"pair_id": "mmlu-8665", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | While driving through a terrible storm I paid attention to the road. Scenario 2 | I tore down the birthday decorations from yesterday.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "D", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-2424", "lang": "en", "prompt": "A team of volunteers collected a total of $5,144 selling T-shirts at a charity concert. Each T-shirt was sold for $8. What was the total number of T-shirts the volunteers sold?\n\nA) 632\nB) 643\nC) 655\nD) 668", "gold": "B", "n_choices": 4, "subject": "elementary_mathematics", "stem": true}
{"pair_id": "mmlu-4845", "lang": "en", "prompt": "Which of the following statements about electric and magnetic fields is FALSE:\n\nA) A charge moving along the direction of an electric field will experience a force, but a charge moving along the direction of a magnetic field will not experience a force.\nB) All charges experience a force in an electric field, but only moving charges can experience a force in a magnetic field.\nC) A positive charge moves in the direction of an electric field; a positive charge moves perpendicular to a magnetic field.\nD) All moving charges experience a force parallel to an electric field and perpendicular to a magnetic field.", "gold": "C", "n_choices": 4, "subject": "high_school_physics", "stem": true}
{"pair_id": "mmlu-11783", "lang": "en", "prompt": "A property owner held title in fee simple to a tract of 20 acres located outside the boundaries of the city. Thereafter, the property owner constructed a shopping center on the property and leased commercial buildings and parking facilities to various tenants. The shopping center, which was located near a public high school, attracted many teenagers who often loitered in the parking lot. The youths frequently harassed shoppers and damaged autos by breaking off windshield wipers and radio antennas. Customarily, the local police department patrolled the shopping center and drove by three or four times each day. This, however, did not prevent the teenagers from hanging out at the shopping center. One afternoon, a shopper was shopping at the center when an unidentified youth damaged her car by throwing a rock through the back window. The shopper brings an action against the property owner to recover for the damage to her auto. She will most likely\n\nA) prevail, unless the person who was responsible for damaging her car can be identified.\nB) prevail, if the damage to her car could have been prevented had the property owner taken reasonable security measures.\nC) not prevail, because the car was damaged by the malicious acts of an independent third person.\nD) not prevail, because the local police had the primary duty to provide security protection at the shopping area.", "gold": "B", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-3774", "lang": "en", "prompt": "Considered as a single group, the Small Business Administration, the Small Business committees in the House and Senate, and small business advocacy groups are an example of\n\nA) an iron triangle\nB) a conference committee\nC) a regulatory agency\nD) dual federalism", "gold": "A", "n_choices": 4, "subject": "high_school_government_and_politics", "stem": false}
{"pair_id": "mmlu-13010", "lang": "en", "prompt": "Attempting to teach a new behavior through the use of aversive techniques is difficult because of which of the following\n\nA) Emotional reactivity is often increased\nB) Punishment merely suppresses unwanted behavior\nC) The frequency of behavior is often low\nD) Inhibition co conditioning is encountered", "gold": "A", "n_choices": 4, "subject": "professional_psychology", "stem": false}
{"pair_id": "mmlu-1935", "lang": "en", "prompt": "How many entries will be in the truth table of a 3 input NAND gate ?\n\nA) 3\nB) 6\nC) 8\nD) 9", "gold": "C", "n_choices": 4, "subject": "electrical_engineering", "stem": true}
{"pair_id": "mmlu-4164", "lang": "en", "prompt": "Lowering reserve requirements would\n\nA) force banks to hold more reserves and make more loans.\nB) allow banks to make more loans and buy more investments thus decreasing the money supply.\nC) allow banks to make more loans and buy more investments thus increasing the money supply.\nD) allow banks more freedom to merge and acquire other businesses.", "gold": "C", "n_choices": 4, "subject": "high_school_macroeconomics", "stem": false}
{"pair_id": "mmlu-5430", "lang": "en", "prompt": "In the Harlow study of emotional attachment, infant monkeys were placed in a cage and given both a \"wire\" mother and a \"cloth\" mother. Researchers then moved a bottle of milk from one mother to the other while introducing various stimuli to see if the monkeys would form an attachment to either of the \"mothers.\" In this experiment, the independent variable is\n\nA) with which \"mother\" the bottle of milk is placed\nB) the \"wire\" mother versus the \"cloth\" mother\nC) the preference of the infants for the source of milk\nD) the preference of the infants for the \"wire\" mother", "gold": "A", "n_choices": 4, "subject": "high_school_psychology", "stem": false}
{"pair_id": "mmlu-8989", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | The kitten could not drink on its own, so I fed it to the dog. Scenario 2 | To prevent the cat from getting loose, I put a hook in its mouth.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "A", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-12821", "lang": "en", "prompt": "Which is NOT a contributor to individual differences in the development of vocal language skills?\n\nA) Variation in the support for language acquisition in the social environment\nB) Hearing loss\nC) The quality of nonparental early care and education experiences\nD) The particular native language in a child’s environment", "gold": "D", "n_choices": 4, "subject": "professional_psychology", "stem": false}
{"pair_id": "mmlu-1256", "lang": "en", "prompt": "Metabolism is determined by the:\n\nA) size of proteins in the cell.\nB) availability of amino acids.\nC) proteins formed as dictated by the genetic material.\nD) amino acid composition of the ribonucleic acids.", "gold": "C", "n_choices": 4, "subject": "college_medicine", "stem": false}
{"pair_id": "mmlu-13898", "lang": "en", "prompt": "According to Jaina traditions, what does the term ajiva mean?\n\nA) Soul\nB) Sound\nC) Non-matter\nD) Non-soul", "gold": "D", "n_choices": 4, "subject": "world_religions", "stem": false}
{"pair_id": "mmlu-1123", "lang": "en", "prompt": "What is the greatest possible area of a triangular region with one vertex at the center of a circle of radius 1 and the other two vertices on the circle?\n\nA) 1/2\nB) 1\nC) sqrt(2)\nD) pi", "gold": "A", "n_choices": 4, "subject": "college_mathematics", "stem": true}
{"pair_id": "mmlu-12763", "lang": "en", "prompt": "Children begin to deliberately and regularly use rehearsal, elaboration, and organization as memory strategies by _____ years of age.\n\nA) 2 to 3\nB) 5 to 7\nC) 9 to 10\nD) 13 to 15", "gold": "C", "n_choices": 4, "subject": "professional_psychology", "stem": false}
{"pair_id": "mmlu-7243", "lang": "en", "prompt": "A company sells its product at two different prices in two different locations. Which of the following types of research will be most effective in identifying the optimal price for the product if all other marketing factors are held constant?\n\nA) Descriptive\nB) Exploratory\nC) Ethnographic\nD) Causal", "gold": "D", "n_choices": 4, "subject": "marketing", "stem": false}
{"pair_id": "mmlu-13861", "lang": "en", "prompt": "Benefits of opioid replacement therapy as an approach for HIV prevention include:\n\nA) Reducing craving for and use of illicit opioids\nB) More sharing of injecting paraphernalia\nC) Reducing frequency of injection drug use\nD) Both a and c", "gold": "D", "n_choices": 4, "subject": "virology", "stem": false}
{"pair_id": "mmlu-13798", "lang": "en", "prompt": "The characteristic of injecting drug users in Asia is/are:\n\nA) Strong family support\nB) Low failure rate for rehabilitation centers\nC) Remaining a member of the community\nD) Both a and c", "gold": "D", "n_choices": 4, "subject": "virology", "stem": false}
{"pair_id": "mmlu-13971", "lang": "en", "prompt": " How many times a day is the Fatihah recited?\n\nA) Exactly 17\nB) At least 17\nC) At least 13\nD) Exactly 24", "gold": "B", "n_choices": 4, "subject": "world_religions", "stem": false}
{"pair_id": "mmlu-4200", "lang": "en", "prompt": "According to Classical economic theory the equation of exchange demonstrates the neutrality of money only if the\n\nA) velocity of money supply and the quantity of output are constant.\nB) money supply and its velocity are equal.\nC) velocity of money supply equals the speed of transactions.\nD) money supply and its velocity are inversely related.", "gold": "A", "n_choices": 4, "subject": "high_school_macroeconomics", "stem": false}
{"pair_id": "mmlu-3305", "lang": "en", "prompt": "This question refers to the following information.\nAs a Jew, I have never believed in collective guilt. Only the guilty were guilty.\nChildren of killers are not killers but children. I have neither the desire nor the authority to judge today's generation for the unspeakable crimes committed by the generation of Hitler.\nBut we may—and we must—hold it responsible, not for the past, but for the way it remembers the past. And for what it does with the memory of the past. In remembering, you will help your own people vanquish the ghosts that hover over its history. Remember: a community that does not come to terms with the dead will continue to traumatize the living.\nWe remember Auschwitz and all that it symbolizes because we believe that, in spite of the past and its horrors, the world is worthy of salvation; and salvation, like redemption, can be found only in memory.\nElie Wiesel, \"Reflections of a Survivor,\" 1987\nFrom the passage, one may infer that Wiesel believed that the current generation of Germans\n\nA) shared their ancestors' guilt for the Holocaust\nB) had a responsibility to remember the Holocaust\nC) shared in the responsibility for the Holocaust\nD) had no responsibility where the Holocaust was concerned", "gold": "B", "n_choices": 4, "subject": "high_school_european_history", "stem": false}
{"pair_id": "mmlu-12136", "lang": "en", "prompt": "A woman owned a four-unit apartment building and lived in one of the units. When one of her tenants vacated his apartment, the woman placed an advertisement in the local paper that read as follows: \"Large two-bedroom apartment available for rent. White male preferred.\" The woman's preference was motivated by the fact that she liked to have a mix of tenants of both genders and from various racial and ethnic backgrounds in her building, and of the remaining rented units, one was rented to an African American man and the other to a Pacific Islander woman. Based upon these facts, which of the following statements is true?\n\nA) The federal Fair Housing Act makes it illegal for the woman to refuse to rent her units to prospective tenants because of their race or gender.\nB) The woman's motive absolves her from any liability under the federal Fair Housing Act.\nC) There are no violations of any federal laws under these facts.\nD) Under the federal Fair Housing Act, the woman was not permitted to state a racial or gender preference in the advertisement.", "gold": "D", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-619", "lang": "en", "prompt": "The pulse can be palpated:\n\nA) every time the atria contracts.\nB) when a vein is close to the surface of the skin.\nC) every time the left ventricle contracts.\nD) when an artery is close to the surface of the skin.", "gold": "D", "n_choices": 4, "subject": "clinical_knowledge", "stem": false}
{"pair_id": "mmlu-10141", "lang": "en", "prompt": "Unlike Mesopotamian writings, the records left by the Maya are principally concerned with:\n\nA) political and military history.\nB) philosophy and religion.\nC) trade and economics.\nD) astronomy and mathematics.", "gold": "A", "n_choices": 4, "subject": "prehistory", "stem": false}
{"pair_id": "mmlu-4756", "lang": "en", "prompt": "A gas undergoes an expansion-compression cycle. If, plotted on a P-V diagram, the cycle is counterclockwise and the work is 300 J in magnitude, what was the heat transfer during this cycle?\n\nA) 300 J into the system\nB) 300 J out of the system\nC) 600 J into the system\nD) 600 J out of the system", "gold": "B", "n_choices": 4, "subject": "high_school_physics", "stem": true}
{"pair_id": "mmlu-12015", "lang": "en", "prompt": "On February 1, a man dispatched the following letter to a mechanic:\"My car has not been running very well lately. I'll pay you $275 if you will change the oil, replace the oil filter, and adjust the carburetors by February 10. \"The mechanic received the man's letter on February 3. That same day, he telephoned an auto supply company and ordered the necessary materials to perform the repair work. Two days later, the mechanic met the man at a party and this conversation took place:The man: \"Disregard the letter I sent you last week. \"The mechanic: \"No way, man, I already ordered the materials on from the auto supply company. \"The man: \"Sorry, but I sold my car yesterday, so forget the repair work. \"If the mechanic initiates suit for breach of contract, which of the following is the man's strongest argument that no enforceable contract was formed between the parties?\n\nA) The mechanic had not completed performance before the man revoked his offer.\nB) The man's offer could only be accepted by a return promise.\nC) Because the man made his offer by letter, the mechanic could accept only in the same manner.\nD) Although the mechanic was preparing to perform the repair work, he had not begun the requested acts of acceptance when the man revoked his offer.", "gold": "D", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-7021", "lang": "en", "prompt": "From what does a chain of command extend?\n\nA) Bottom to top\nB) Top to bottom\nC) Diagonally\nD) Laterally", "gold": "B", "n_choices": 4, "subject": "management", "stem": false}
{"pair_id": "mmlu-12893", "lang": "en", "prompt": "Emic approaches to multicultural counseling training are often advocated on the basis of\n\nA) difficulty in gaining knowledge regarding a large number of cultural groups\nB) the presence of many members of a particular cultural group in the training program's catchment area\nC) the need to focus on personal and cultural meaning for each individual client\nD) the need to view culturally different groups in similar ways", "gold": "C", "n_choices": 4, "subject": "professional_psychology", "stem": false}
{"pair_id": "mmlu-135", "lang": "en", "prompt": "The auditory tube\n\nA) connects the inner ear and nasopharynx.\nB) is derived from the second pharyngeal pouch.\nC) is opened by the action of levator veli palatini.\nD) is closed by the action of tensor veli palatini.", "gold": "C", "n_choices": 4, "subject": "anatomy", "stem": true}
{"pair_id": "mmlu-8484", "lang": "en", "prompt": " A war that involves actual fighting is known as\n\nA) a hot war.\nB) a cold war.\nC) a war simpliciter.\nD) none of the above", "gold": "A", "n_choices": 4, "subject": "moral_disputes", "stem": false}
{"pair_id": "mmlu-1610", "lang": "en", "prompt": "Thermal conduction involves mainly\n\nA) electrons\nB) protons\nC) neutrons\nD) ions", "gold": "A", "n_choices": 4, "subject": "conceptual_physics", "stem": true}
{"pair_id": "mmlu-11504", "lang": "en", "prompt": "A federal statute established a national lottery and created a new federal agency to administer it. In order to sell lottery tickets directly to the public, the agency established outlets throughout the country, including in a particular state. The state levies a general tax on the gross receipts from all gaming operations within the state. The state seeks to tax the agency on the sales of lottery tickets from its outlets within the state. Must the agency pay the state gross receipts tax?\n\nA) No, because a federal agency may not pay any monies to a state without explicit federal statutory authorization to do so.\nB) No, because a state may not impose a tax directly on the federal government or any of its agencies or instrumentalities.\nC) Yes, because it is a generally applicable tax that does not discriminate against the agency.\nD) Yes, because the federal government is not exempt from state taxation on its commercial activities.", "gold": "B", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-11537", "lang": "en", "prompt": "A man was at a party where he observed a female guest who had passed out and was unconscious and alone in a bedroom of the house. The man locked the door and quietly took the female's purse, emptied all of the money into his pockets, and removed the jewelry from her person. He was apprehended and charged with robbery. Can he be convicted of that crime?\n\nA) Yes, robbery is the unlawful taking of property from another's person by intimidation or force.\nB) No, this was not robbery because there was no intimidation, and indeed no force, against an unconscious victim.\nC) Yes, robbery does not require that the victim be subdued by force or be placed in fear.\nD) No, this was not robbery because it occurred in a private home and not in a business establishment or on the street.", "gold": "B", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-1960", "lang": "en", "prompt": "Speed control by variation of field flux results in\n\nA) constant power drive.\nB) constant torque drive.\nC) variable power drive.\nD) none of the above.", "gold": "A", "n_choices": 4, "subject": "electrical_engineering", "stem": true}
{"pair_id": "mmlu-5532", "lang": "en", "prompt": "During the years 1886 through 2000 there were an average of 8.7 tropical cyclones per year, of which an average of 5.1 became hurricanes. Assuming that the probability of any cyclone becoming a hurricane is independent of what happens to any other cyclone, if there are five cyclones in one year, what is the probability that at least three become hurricanes?\n\nA) 0.313\nB) 0.345\nC) 0.586\nD) 0.658", "gold": "D", "n_choices": 4, "subject": "high_school_statistics", "stem": true}
{"pair_id": "mmlu-7108", "lang": "en", "prompt": "These are costs which do not vary according to the number of units of product made or service sold:\n\nA) Fixed costs.\nB) Moving assets.\nC) Working capital.\nD) Fixed capital.", "gold": "A", "n_choices": 4, "subject": "marketing", "stem": false}
{"pair_id": "mmlu-513", "lang": "en", "prompt": "A 14-16 gauge needle is most likely to be used for:\n\nA) children.\nB) elderly patients.\nC) inserting in the back of the hand.\nD) trauma or burns patients.", "gold": "D", "n_choices": 4, "subject": "clinical_knowledge", "stem": false}
{"pair_id": "mmlu-8556", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | I used my coworker's credit card to order some art supplies for myself. Scenario 2 | I refilled the shampoo bottle with shampoo and put it back in the shower.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "B", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-2070", "lang": "en", "prompt": "Use the expression below to answer the question. 3 × [(2 × 6 – 5) + (8 ÷ 4)] – 1 What is the value of the expression?\n\nA) 9\nB) 11\nC) 26\nD) 32", "gold": "C", "n_choices": 4, "subject": "elementary_mathematics", "stem": true}
{"pair_id": "mmlu-4709", "lang": "en", "prompt": "When opportunity cost is constant across all production levels, the productions possibilities frontier is\n\nA) concave to the origin.\nB) convex to the origin.\nC) undefined.\nD) a straight diagonal line sloping downward from left to right.", "gold": "D", "n_choices": 4, "subject": "high_school_microeconomics", "stem": false}
{"pair_id": "mmlu-3250", "lang": "en", "prompt": "What is the value of this python expression: 4 + 3 % 2?\n\nA) 1\nB) 6\nC) 4\nD) 5", "gold": "D", "n_choices": 4, "subject": "high_school_computer_science", "stem": true}
{"pair_id": "mmlu-6200", "lang": "en", "prompt": "If links between various diseases are found, then future older adults will probably\n\nA) Take fewer medications\nB) Be able to avoid certain diseases\nC) Pay far less for pharmaceuticals\nD) Pick and choose the disease(s) they want", "gold": "A", "n_choices": 4, "subject": "human_aging", "stem": false}
{"pair_id": "mmlu-2515", "lang": "en", "prompt": " Which of the given formulas of PL is the best symbolization of the following sentence?\nA person is considered to have lived with you for all of the present year if the person was born or died in the present year and your home was this person's home for the entire time this person was alive.\n\nA) [(B ∨ D) • H] ⊃ L\nB) [L ⊃ (B ∨ D)] • H\nC) L ⊃ [(B ∨ D) • H]\nD) (B ∨ D • H) ⊃ L", "gold": "A", "n_choices": 4, "subject": "formal_logic", "stem": false}
{"pair_id": "mmlu-8970", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | There was no restroom around so I asked the security guard where to find one. Scenario 2 | I added hot sauce to my friend's birthday cake.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "C", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-8844", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | I like to give strangers a kiss when they aren't expecting it. Scenario 2 | I put a leash around a dragon and taught it to roll over and play dead.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "B", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-13740", "lang": "en", "prompt": "Which of the following HPV viruses are known to cause cervical cancer?\n\nA) Viruses of all 5 clades\nB) Types 14, 16, 18, 31, 45\nC) None are oncogenic per se\nD) Types 1-180", "gold": "C", "n_choices": 4, "subject": "virology", "stem": false}
{"pair_id": "mmlu-7733", "lang": "en", "prompt": "Which of the following is a disorder in ruminants that is characterized by an excessive accumulation of gas in the rumen?\n\nA) Agalactia\nB) Bloat\nC) Bovine pleuropneumonia\nD) Grass tetany", "gold": "B", "n_choices": 4, "subject": "miscellaneous", "stem": false}
{"pair_id": "mmlu-10405", "lang": "en", "prompt": "Which of the following is a major difference between the just-in-time (JIT) and traditional approaches to manufacturing?\n\nA) The JIT approach usually involves a large number of suppliers while traditional approaches usually involve only a small number of suppliers.\nB) The JIT approach requires centralized purchasing while traditional approaches encourage purchasing decisions by production employees.\nC) The JIT approach uses a push-through system while traditional approaches use a pull-through system.\nD) The JIT approach operates with low inventory levels while traditional approaches may operate with high inventory levels.", "gold": "D", "n_choices": 4, "subject": "professional_accounting", "stem": false}
{"pair_id": "mmlu-8520", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | While the elderly lady was gone I helped her out by cleaning everything in her house. Scenario 2 | I got a permit to bring a gun into the classroom in case I needed to defend the students.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "D", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-12608", "lang": "en", "prompt": "The most common treatment for ADHD is drug treatment using CNS stimulants. In relation to treatment for ADHD which of the following statements is incorrect?\n\nA) A limitation of stimulant therapy is that not all children improve.\nB) For reducing activity levels a higher dose of the stimulant is required.\nC) Improvements may only be short-lived following treatment.\nD) A lower dose of the treatment has the effect of improving social behaviors.", "gold": "D", "n_choices": 4, "subject": "professional_psychology", "stem": false}
{"pair_id": "mmlu-7507", "lang": "en", "prompt": "Elephant tusks are made of what material?\n\nA) Coral\nB) Ivory\nC) Bone\nD) Calcium", "gold": "B", "n_choices": 4, "subject": "miscellaneous", "stem": false}
{"pair_id": "mmlu-12815", "lang": "en", "prompt": "The Systems Theory claims that a group of interaction components constitute together an entire organization. Which of the following properties of a family system is incorrectly defined:\n\nA) EQUIFINALITY: end result is the same, no matter where you enter the system\nB) WHOLENESS: whole is greater than sum of parts\nC) HOMEOSTASIS→ system restores status quo when disrupted\nD) POSITIVE FEEDBACK→ disrupting homeostasis by creating deviations in the status quo", "gold": "B", "n_choices": 4, "subject": "professional_psychology", "stem": false}
{"pair_id": "mmlu-3550", "lang": "en", "prompt": "Which of the following is a major concern in countries with a large and rapidly aging population?\n\nA) Immunization\nB) Adequate schools\nC) Daycare facilities\nD) Health care", "gold": "D", "n_choices": 4, "subject": "high_school_geography", "stem": false}
{"pair_id": "mmlu-6195", "lang": "en", "prompt": "An increase in neuroticism later in life is a strong predictor of\n\nA) Mortality\nB) Loss of social support\nC) Cognitive decline\nD) Independence", "gold": "A", "n_choices": 4, "subject": "human_aging", "stem": false}
{"pair_id": "mmlu-886", "lang": "en", "prompt": "Which of the following is a reason that geneticists use mitochondrial DNA (mtDNA) to study the relatedness of animal populations?\n\nA) mtDNA mutates at a slower rate than nuclear DNA.\nB) mtDNA is passed from mother to child and is free from recombination that occurs between pairs of chromosomes.\nC) There are few single nucleotide polymorphisms in the hypervariable, noncoding regions.\nD) All mitochondrial proteins are coded for by mitochondrial genes.", "gold": "B", "n_choices": 4, "subject": "college_biology", "stem": true}
{"pair_id": "mmlu-5859", "lang": "en", "prompt": "This question refers to the following information.\n\"The question as to who, and what, is responsible for African underdevelopment can be answered at two levels. Firstly, the answer is that the operation of the imperialist system bears major responsibility for African economic retardation by draining African wealth and by making it impossible to develop more rapidly the resources of the continent. Secondly, one has to deal with those who manipulate the system and those who are either agents or unwitting accomplices of the said system. The capitalists of Western Europe were the ones who actively extended their exploitation from inside Europe to cover the whole of Africa.\"\nWalter Rodney, historian, How Europe Underdeveloped Africa, 1974\nWhich of the following best explains the importance of trading contacts with Europeans for Sub-Saharan Africans in the period 1450 to 1750 C.E.?\n\nA) Sub-Saharan Africans relied on European merchants to sustain population growth through the constant importation of New World foodstuffs.\nB) Sub-Saharan Africans consolidated new states and kingdoms by trading with the Europeans for firearms.\nC) Sub-Saharan Africans depended on European merchants as the sole purchasers of slaves.\nD) Sub-Saharan Africans allied with European powers to evict Muslim and Arab merchant princes from encroaching on their sovereignty.", "gold": "B", "n_choices": 4, "subject": "high_school_world_history", "stem": false}
{"pair_id": "mmlu-13979", "lang": "en", "prompt": " Charism can be defined as which of the following?\n\nA) An oil\nB) Prophecy\nC) A spiritual gift\nD) Authority", "gold": "C", "n_choices": 4, "subject": "world_religions", "stem": false}
{"pair_id": "mmlu-7258", "lang": "en", "prompt": "Inelastic demand occurs when the\n\nA) quantity demanded changes a great deal as prices fluctuate\nB) quantity demanded barely changes with a change in price\nC) change in quantity demanded is in the same proportion as the change in price\nD) demand curve and the supply curve intersect at a given price point", "gold": "B", "n_choices": 4, "subject": "marketing", "stem": false}
{"pair_id": "mmlu-13331", "lang": "en", "prompt": "What would be the policy implications of a hierarchical vision of TNC?\n\nA) Hierarchical forms of transnational criminal activity are primarily territorially affiliated to a particular state / ethnic-cultural group and have limited transnational potential.\nB) The hierarchical analysts believe that generally there has been a tendency of officials to have over-inflated the estimation of the levels of violence and wealth generated by their activities. If the wealth is not as great as is widely believed, their threat to the licit economy is vastly reduced.\nC) Hierarchical forms of organization are highly adaptable and small fluid groups enable a pronounced degree of resilience (attributable to the logic of supply rising to meet demand) against law enforcement efforts confined to rigid ranking of cooperation.\nD) Hierarchical groups pose a serious degree of harm by their profit-driven crimes under centralized leadership. Targeting the proceeds or leadership would therefore produce a reduction in their threat on the legitimate economy.", "gold": "D", "n_choices": 4, "subject": "security_studies", "stem": false}
{"pair_id": "mmlu-13490", "lang": "en", "prompt": "Social norms are:\n\nA) creative activities such as gardening, cookery and craftwork\nB) the symbolic representation of social groups in the mass media\nC) religious beliefs about how the world ought to be\nD) rules and expectations about interaction that regulate social life", "gold": "D", "n_choices": 4, "subject": "sociology", "stem": false}
{"pair_id": "mmlu-5853", "lang": "en", "prompt": "This question refers to the following information.\nGunpowder Weaponry: Europe vs. China\nIn Western Europe during the 1200s through the 1400s, early cannons, as heavy and as slow to fire as they were, proved useful enough in the protracted sieges that dominated warfare during this period that governments found it sufficiently worthwhile to pay for them and for the experimentation that eventually produced gunpowder weapons that were both more powerful and easier to move. By contrast, China, especially after the mid-1300s, was threatened mainly by highly mobile steppe nomads, against whom early gunpowder weapons, with their unwieldiness, proved of little utility. It therefore devoted its efforts to the improvement of horse archer units who could effectively combat the country's deadliest foe.\nThe argument in this passage most closely relates to which of the following large-scale questions about global history?\n\nA) How societies shared strategically important technologies with each other\nB) Why European states went on to attain military superiority over non-Western powers in the modern era\nC) Why Silk Road commerce dwindled across the breadth of Eurasia after 1500 C.E.\nD) How the medieval climatic optimum affected the process of cultural diffusion", "gold": "B", "n_choices": 4, "subject": "high_school_world_history", "stem": false}
{"pair_id": "mmlu-13078", "lang": "en", "prompt": "In the Westley and MacLean model of communication, which role describes the way in which the public obtain or receive the message?\n\nA) The advocacy role\nB) The channel's role\nC) The gatekeeper role\nD) The behavioural role", "gold": "B", "n_choices": 4, "subject": "public_relations", "stem": false}
{"pair_id": "mmlu-10380", "lang": "en", "prompt": "A practitioner is engaged to express an opinion on management’s assertion that the square footage of a warehouse offered for sale is 150,000 square feet. The practitioner should refer to which of the following sources for professional guidance?\n\nA) Statements on Auditing Standards\nB) Statements on Standards for Attestation Engagements\nC) Statements on Standards for Accounting and Review Services\nD) Statements on Standards for Consulting Services", "gold": "B", "n_choices": 4, "subject": "professional_accounting", "stem": false}
{"pair_id": "mmlu-2384", "lang": "en", "prompt": "You walked your dog 7 over 9 mi in the morning and 5 over 12 mi in the afternoon. How much farther did you walk your dog in the morning than in the afternoon?\n\nA) 13 over 36\nB) 12 over 21\nC) 2 over 3\nD) 35 over 108", "gold": "A", "n_choices": 4, "subject": "elementary_mathematics", "stem": true}
{"pair_id": "mmlu-4252", "lang": "en", "prompt": "Carlos Montado was born on Saturday, November 9, 2002. On what day of the week will Carlos be 706 days old? (Spell out the entire day of the week.)\n\nA) Friday\nB) Thursday\nC) Wednesday\nD) Tuesday", "gold": "A", "n_choices": 4, "subject": "high_school_mathematics", "stem": true}
{"pair_id": "mmlu-5589", "lang": "en", "prompt": "Suppose H0: p = 0.6, and the power of the test for Ha: p = 0.7 is 0.8. Which of the following is a valid conclusion?\n\nA) The probability of committing a Type I error is 0.1.\nB) If Ha is true, the probability of failing to reject H0 is 0.2.\nC) The probability of committing a Type II error is 0.3.\nD) All of the above are valid conclusions.", "gold": "B", "n_choices": 4, "subject": "high_school_statistics", "stem": true}
{"pair_id": "mmlu-3959", "lang": "en", "prompt": "Automatic stabilizers in the economy serve an important role in\n\nA) increasing the length of the business cycle.\nB) balancing the budget.\nC) increasing a budget surplus in a recession.\nD) lessening the impact of a recession.", "gold": "D", "n_choices": 4, "subject": "high_school_macroeconomics", "stem": false}
{"pair_id": "mmlu-11400", "lang": "en", "prompt": "A homeowner was interested in changing the look of his home. He contacted an architect, who designed plans and specifications for a major \"facelift\" of the homeowner's home. The homeowner was very pleased with the plans and specifications. He showed the plans and specifications to his neighbor, who agreed that the facelift would greatly enhance the value of the homeowner's home. The homeowner contacted a contractor, who was considered to be the best in the area, to rebuild his house according to the plans and specifications. The contractor reviewed the plans and specifications and told the homeowner that he could do all the work called for by the plans and specifications for $100,000. The homeowner replied, \"That's a little more than I can afford. I really can only free up $80,000 for this project. Would you do it for that amount?\" The contractor responded, \"That's awfully low. I'll have to think about it,\" and then left. The next day, the neighbor telephoned the contractor. The neighbor explained to the contractor that he realized the value of his home would be enhanced by the improvement done to the homeowner's property. For that reason, the neighbor told the contractor he would pay him $10,000 if he would accept the homeowner's proposal and do the work called for by the plans and specifications. The contractor agreed. The next day the contractor informed the homeowner that he would accept his $80,000 offer. The contractor did all the work called for by the plans and specifications, and collected the $80,000 due him from the homeowner. However, the neighbor refused to pay the contractor the $10,000. If the contractor brings a breach of contract action against the neighbor, judgment for whom?\n\nA) The contractor, because his signing a contract with the homeowner and completing the work according to the plans and specifications was legally sufficient consideration to support the neighbor's promise to pay him the $10,000.\nB) The contractor, provided the neighbor realizes an increase in his property value as a result of the work done by the contractor.\nC) The neighbor, because his promise to the contractor was illusory.\nD) The neighbor, because at the moment the contractor accepted the homeowner's offer, he came under a pre-existing obligation, which was insufficient consideration to support the neighbor's promise.", "gold": "A", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-6838", "lang": "en", "prompt": "A machine learning problem involves four attributes plus a class. The attributes have 3, 2, 2, and 2 possible values each. The class has 3 possible values. How many maximum possible different examples are there?\n\nA) 12\nB) 24\nC) 48\nD) 72", "gold": "D", "n_choices": 4, "subject": "machine_learning", "stem": true}
{"pair_id": "mmlu-12958", "lang": "en", "prompt": "One of the most frequent findings of studies on human beings with “split brains” is that\n\nA) The right hemisphere is predominantly concerned with verbal information\nB) certain kinds of visuospatial tasks are performed beer by the right hemisphere than by the left\nC) the right hemisphere provides backup for the left in some tasks but excels the left in none\nD) aside from the long-known localization of speech functions, the two hemispheres appear to be essentially equivalent as information processors", "gold": "B", "n_choices": 4, "subject": "professional_psychology", "stem": false}
{"pair_id": "mmlu-8544", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | I pushed the elderly woman to the ground. Scenario 2 | I have so much love to give, I'm going to visit the foster home to bring home a child in need.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "B", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-3199", "lang": "en", "prompt": "A programmer uses code published online under a Creative Commons Attribution (C C B Y) license in a commercial product. Which of the following best describes an acceptable use of the code?\n\nA) Copying code from the online source into the programmer's product without any other actions\nB) Copying code from the online source into the programmer's product and limiting the copied code to ten code lines\nC) Copying code from the online source into the programmer's product and changing all variable names\nD) Copying code from the online source into the programmer's product and crediting the original author in the manner indicated by the license", "gold": "D", "n_choices": 4, "subject": "high_school_computer_science", "stem": true}
{"pair_id": "mmlu-8628", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | I didn't visit the baby when I was sick. Scenario 2 | Rich people love getting drunk on yachts, and as their boat captain, I am never sober.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "C", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-1888", "lang": "en", "prompt": "Suppose that we are interested in testing the null hypothesis that a GARCH(2,2) model can be restricted to a process with a constant conditional variance using the likelihood ratio test approach. Which of the following statements are true?\n\nA) The test statistic will follow a chi-squared distribution with 2 degrees of freedom under the null hypothesis\nB) The value of the log-likelihood function will almost always be bigger for the restricted model than for the unrestricted model\nC) If the relevant values of the log-likelihood functions are -112.3 and -118.4, the value of the test statistic is 12.2\nD) The likelihood ratio test compares the slopes of the log-likelihood function at the maximum and at the restricted parameter value.", "gold": "C", "n_choices": 4, "subject": "econometrics", "stem": false}
{"pair_id": "mmlu-5164", "lang": "en", "prompt": "Stephen is going through his second divorce. He thinks that no woman will ever love him again. His therapist points out to Stephen that his thinking is irrational and faulty. Which of the following therapies is the therapist employing?\n\nA) psychoanalysis\nB) systematic desensitization\nC) flooding\nD) rational emotive", "gold": "D", "n_choices": 4, "subject": "high_school_psychology", "stem": false}
{"pair_id": "mmlu-2107", "lang": "en", "prompt": "Which number has an absolute value greater than 5?\n\nA) -6\nB) -5\nC) 0\nD) 5", "gold": "A", "n_choices": 4, "subject": "elementary_mathematics", "stem": true}
{"pair_id": "mmlu-8708", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | I voted twice in the presidential election. Scenario 2 | I rolled the grocery cart to my car.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "B", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-3508", "lang": "en", "prompt": "The birth of an urban industrial workforce in Europe contributed to the start of the\n\nA) First Agricultural Revolution.\nB) Second Agricultural Revolution.\nC) Third Agricultural Revolution.\nD) Fourth Agricultural Revolution.", "gold": "B", "n_choices": 4, "subject": "high_school_geography", "stem": false}
{"pair_id": "mmlu-6140", "lang": "en", "prompt": "Quantitative analyses of TV shows have shown that\n\nA) Older adults are almost always portrayed in a negative way\nB) The more positive portrayals occur during commercials\nC) The proportion of older men is particularly low\nD) The proportion of older women is particularly low", "gold": "D", "n_choices": 4, "subject": "human_aging", "stem": false}
{"pair_id": "mmlu-12726", "lang": "en", "prompt": "A group's performance on a(n) __________ task is limited by the performance of the least skilled or knowledgeable member of the group.\n\nA) additive\nB) compensatory\nC) disjunctive\nD) conjunctive", "gold": "D", "n_choices": 4, "subject": "professional_psychology", "stem": false}
{"pair_id": "mmlu-3789", "lang": "en", "prompt": "Which of the following categories of spending are paid without an appropriations bill?\n\nA) Department of Education special acts and authorizations\nB) Justice department budgets\nC) Federal budget entitlements\nD) Department of Defense funding", "gold": "C", "n_choices": 4, "subject": "high_school_government_and_politics", "stem": false}
{"pair_id": "mmlu-11520", "lang": "en", "prompt": "A state has a statute prohibiting the sale of guns and ammunition to any person under the age of 17. Violation of the statute is a felony punishable by a fine of $5,000 and imprisonment for up to two years. The state courts have interpreted this statute as a public welfare offense requiring no particular mental state for its violation. As such, this is a strict liability crime whereby culpability is imposed on a defendant for doing the proscribed act. A defendant, a 15-year-old, entered the store to purchase a shotgun as a present for her father. She took a shotgun off the shelf and went to the cashier's counter to pay for it. Not seeing the cashier anywhere, the defendant left the exact price for the shotgun on the countertop and left the store. If the defendant is prosecuted for violating the state statute, she should be found\n\nA) guilty, because she purchased the shotgun while under the statutory age.\nB) guilty, because the statute imposes absolute criminal liability.\nC) not guilty, provided she was unaware that the statute prohibited the sale of weapons to a person in her age group.\nD) not guilty, provided that the legislative intent was not to make the purchase of shotguns a criminal offense.", "gold": "D", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-7085", "lang": "en", "prompt": " This is where intermediaries are given exclusive rights to market the good or service within a defined 'territory', thus using a very limited number of intermediaries:\n\nA) Exclusive.\nB) Intensive.\nC) Selective.\nD) Direct.", "gold": "A", "n_choices": 4, "subject": "marketing", "stem": false}
{"pair_id": "mmlu-11508", "lang": "en", "prompt": "A minor child was eating a fish sandwich in a fast food restaurant when a large metal object, later identified as a fish hook, got caught in her throat. The trapped hook caused her to cough blood uncontrollably, and she was rushed to a hospital for emergency surgery. The restaurant later defended the customer's lawsuit by showing that it had done everything reasonable to eliminate foreign objects from its product and there was no negligence. Will the lawsuit be dismissed and judgment entered for the restaurant?\n\nA) No, because even without negligence there is strict liability imposed for selling an unreasonably dangerous product.\nB) Yes, because a customer voluntarily assumes the foreseeable risk that fish will contain metal fish hooks.\nC) Yes, because customers are expected to know that a fish sandwich will often have foreign objects in it.\nD) No, because the seller did not post a warning that its fish might contain fish hooks.", "gold": "A", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-7266", "lang": "en", "prompt": " Economic—marketing should encourage long-term economic development as opposed to short-term economic development.\n\nA) Ecological, evolutionary, and economic\nB) Ecological, equitable, and economic\nC) Evolutionary, equitable, and economic.\nD) Ecological, equitable, and emerging.", "gold": "B", "n_choices": 4, "subject": "marketing", "stem": false}
{"pair_id": "mmlu-4559", "lang": "en", "prompt": "The long-run average cost curve\n\nA) is always below the short-run average cost curve\nB) is always above the short-run average cost curve\nC) always intersects the short-run average cost curve at the minimum of short-run average cost\nD) is below the short-run average cost except at one point", "gold": "D", "n_choices": 4, "subject": "high_school_microeconomics", "stem": false}
{"pair_id": "mmlu-3433", "lang": "en", "prompt": "This question refers to the following information.\nRead the the following quotation to answer questions.\n1848 was the decisive year of German, and so of European, history: it recapitulated Germany's past and inspired Germany's future. … Never has there been a revolution so inspired by a limitless faith in the power of ideas; never has a revolution so discredited the power of ideas in its result. The success of the revolution discredited conservative ideas; the failure of the revolution discredited liberal ideas. After it, nothing remained but the idea of Force, and this idea stood at the helm of German history from then on. For the first time since 1521, the German people stepped on to the centre of the German stage only to miss their cue once more. German history reached its turning-point and failed to turn. This was the fateful essence of 1848.\nA. J. P. Taylor, The Course of German History, 1945\nThe subject of Taylor's analysis in this quotation is\n\nA) the industrial revolution in the context of German history\nB) the failure of the revolutions of 1848\nC) the rise of the Nazi Party in Germany\nD) the importance of the revolutions of 1848 in modern German history", "gold": "D", "n_choices": 4, "subject": "high_school_european_history", "stem": false}
{"pair_id": "mmlu-5018", "lang": "en", "prompt": "Which of the following accurately states the order of the transmission of visual information?\n\nA) Optic nerve; ganglion cells; bipolar cells; rods and cones\nB) Bipolar cells; ganglion cells; fovea; optic nerve\nC) Rods and cones; retina; optic nerve; ganglion cells\nD) Rods and cones; bipolar cells; ganglion cells; optic nerve", "gold": "D", "n_choices": 4, "subject": "high_school_psychology", "stem": false}
{"pair_id": "mmlu-8760", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | I turned on the baseball field's sprinklers to water the grass. Scenario 2 | I spent all night studying to pass the test.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "D", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-9509", "lang": "en", "prompt": "In which form is selenium found in the 25 human selenoproteins?\n\n\nA) Selenophosphate\nB) Selenocysteine\nC) Selenohistidine\nD) Selenate", "gold": "B", "n_choices": 4, "subject": "nutrition", "stem": false}
{"pair_id": "mmlu-10206", "lang": "en", "prompt": "Charles Darwin discovered that plants and animals had evolved through a process known as ____________. Early archaeologists discovered that ______________________.\n\nA) scientific creationism; human culture evolved differently than plants and animals\nB) unilineal evolution; civilizations evolved more quickly than plants\nC) natural selection; human culture did not evolve or change much over time\nD) natural selection; human culture had evolved over an enormous period of time", "gold": "D", "n_choices": 4, "subject": "prehistory", "stem": false}
{"pair_id": "mmlu-4409", "lang": "en", "prompt": "A curve is given parametrically by the equations\n\nA) π/2\nB) π\nC) 2 + π\nD) 2π", "gold": "D", "n_choices": 4, "subject": "high_school_mathematics", "stem": true}
{"pair_id": "mmlu-11166", "lang": "en", "prompt": "There is a newcomer who joins a small town in the middle of nowhere, where she gets a job as a city employee. The city manual requires that newcomers pay a higher contribution to the city's employee betterment fund for the first three years of employment. Newcomer refuses to pay a higher contribution than other employees, whereupon she is terminated. Can she make a successful equal protection argument to fight for her job?\n\nA) No, she is not a member of a suspect class as there cannot be a \"class-of-one\" in the employment law context.\nB) No, the request for the contribution was not unreasonable and cannot support an equal protection argument.\nC) Yes, she can claim a class of one who is being discriminated against.\nD) Yes, she can claim membership in a group of newcomers who are coming and are being treated differently than everyone else, even if she is the only newcomer she knows of at this time.", "gold": "A", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-1400", "lang": "en", "prompt": "If the Sun were suddenly replaced by a black hole of the same mass, it would have a Schwarzschild radius of 3,000 m. What effect, if any, would this change have on the orbits of the planets?\n\nA) The planets would move directly toward the Sun.\nB) The planets would move in spiral orbits.\nC) The planets would oscillate about their former elliptical orbits.\nD) The orbits would remain unchanged.", "gold": "D", "n_choices": 4, "subject": "college_physics", "stem": true}
{"pair_id": "mmlu-8769", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | I put on my roller skates in the skating rink. Scenario 2 | I attacked the home invader.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "D", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-8212", "lang": "en", "prompt": " According to Valls, a just cause for political violence is usually\n\nA) defensive.\nB) offensive.\nC) territorial.\nD) economically motivated.", "gold": "A", "n_choices": 4, "subject": "moral_disputes", "stem": false}
{"pair_id": "mmlu-3330", "lang": "en", "prompt": "This question refers to the following information.\nThe assumption by a government of the office of Reliever-general to the poor is necessarily forbidden by the principle that a government cannot rightly do anything more than protect. In demanding from a citizen contributions for the mitigation of distress … the state is … reversing its function. … To enforce the fundamental law—to take care that every man has freedom to do all that he wills, provided he infringes not the equal freedom of any other man. … [But that] is quite a separate thing from insuring him satisfaction. …\nThe poverty of the incapable, the distresses that come upon the imprudent, the starvation of the idle, and those shoulderings aside of the weak by the strong … are the decrees of a large, farseeing benevolence. … When regarded not separately, but in connection with the interests of universal humanity, these harsh fatalities are seen to be full of the highest beneficence.\nHerbert Spencer, Social Statistics: Survival of the Fittest Applied to Humankind, 1851\nFrom the passage, one may infer that Spencer was an advocate of which nineteenth-century political philosophy?\n\nA) Conservatism\nB) Anarchism\nC) Liberalism\nD) Socialism", "gold": "C", "n_choices": 4, "subject": "high_school_european_history", "stem": false}
{"pair_id": "mmlu-6030", "lang": "en", "prompt": "This question refers to the following information.\nKnow the self to be sitting in the chariot, the body to be the chariot, the intellect the charioteer, and the mind the reins. He who has understanding, who is mindful and always pure, indeed reaches that place from whence he is not born again.\nfrom the Upanishads, India, circa 600s–400s B.C.E.\nWhich of the following changes to Indian religious practice took place under the Mauryan emperor Ashoka?\n\nA) The rise of Sikhism as a syncretic faith\nB) The arrival of Islam as a new religion borne by external invaders\nC) The eradication of traditional Hinduism by armed force\nD) The official encouragement of Buddhist edicts", "gold": "D", "n_choices": 4, "subject": "high_school_world_history", "stem": false}
{"pair_id": "mmlu-9782", "lang": "en", "prompt": "Feinberg claims that most arguments for psychological egoism:\n\nA) make reference to neuroscience.\nB) are based on controlled studies.\nC) are based on interviews and surveys.\nD) are nonempirical in character.", "gold": "D", "n_choices": 4, "subject": "philosophy", "stem": false}
{"pair_id": "mmlu-5385", "lang": "en", "prompt": "Eli is just beginning to sit up. Assuming he is developing at a typical pace, how old is Eli?\n\nA) 1 month\nB) 3 months\nC) 6 months\nD) 9 months", "gold": "C", "n_choices": 4, "subject": "high_school_psychology", "stem": false}
{"pair_id": "mmlu-6177", "lang": "en", "prompt": "The term \"cohort\" refers to\n\nA) An individual's chronological age\nB) A generation of people who lived through major events\nC) A person's gender\nD) A group of people living in the same culture", "gold": "B", "n_choices": 4, "subject": "human_aging", "stem": false}
{"pair_id": "mmlu-165", "lang": "en", "prompt": "Which of the following muscles are derived from the first brachial arch?\n\nA) The muscles of mastication\nB) The muscles of mastication and anterior belly of digastric\nC) The muscles of mastication and anterior and posterior bellies of digastric\nD) The muscles of mastication, anterior and posterior bellies of digastric and geniohyoid", "gold": "B", "n_choices": 4, "subject": "anatomy", "stem": true}
{"pair_id": "mmlu-13052", "lang": "en", "prompt": "In what year did the BBC start broadcasting radio?\n\nA) 1917\nB) 1922\nC) 1925\nD) 1927", "gold": "B", "n_choices": 4, "subject": "public_relations", "stem": false}
{"pair_id": "mmlu-5235", "lang": "en", "prompt": "Carlotta is a well-adjusted, socially competent adolescent. Which of the following most likely represents her family's parenting style?\n\nA) Minimal supervision\nB) Permissive\nC) Authoritarian\nD) Authoritative", "gold": "D", "n_choices": 4, "subject": "high_school_psychology", "stem": false}
{"pair_id": "mmlu-3961", "lang": "en", "prompt": "To move the economy closer to full employment the central bank decides that the federal funds rate must be increased. The appropriate open market operation is to ______ which ______ the money supply ______ aggregate demand and fights ______. OMO     MONEY SUPPLY     AD     TO FIGHT\n\nA) Buy bonds     Increases     Increases     Unemployment\nB) Buy bonds     Increases     Increases     Inflation\nC) Sell bonds     Decreases     Decreases     Unemployment\nD) Sell bonds     Decreases     Decreases     Inflation", "gold": "D", "n_choices": 4, "subject": "high_school_macroeconomics", "stem": false}
{"pair_id": "mmlu-5521", "lang": "en", "prompt": "An assembly line machine is supposed to turn out ball bearings with a diameter of 1.25 centimeters. Each morning the first 30 bearings produced are pulled and measured. If their mean diameter is under 1.23 centimeters or over 1.27 centimeters, the machinery is stopped and an engineer is called to make adjustments before production is resumed. The quality control procedure may be viewed as a hypothesis test with the null hypothesis H0: μ = 1.25 and the alternative hypothesis Ha: μ ≠ 1.25. The engineer is asked to make adjustments when the null hypothesis is rejected. In test terminology, what would a Type II error result in?\n\nA) A warranted halt in production to adjust the machinery\nB) An unnecessary stoppage of the production process\nC) Continued production of wrong size ball bearings\nD) Continued production of proper size ball bearings", "gold": "C", "n_choices": 4, "subject": "high_school_statistics", "stem": true}
{"pair_id": "mmlu-8001", "lang": "en", "prompt": "What animated character has a crush on the Little Red Haired Girl?\n\nA) Richie Rich\nB) Charlie Brown\nC) Bugs Bunny\nD) Jonny Quest", "gold": "B", "n_choices": 4, "subject": "miscellaneous", "stem": false}
{"pair_id": "mmlu-7093", "lang": "en", "prompt": " In B2B contexts, prices are set according to specific agreements between a company and its clients or customers (e.g. professional services such as architectural or structural engineering).This pricing approach is known as:\n\nA) Negotiated price\nB) Internal pricing.\nC) Listed pricing.\nD) Cost pricing.", "gold": "A", "n_choices": 4, "subject": "marketing", "stem": false}
{"pair_id": "mmlu-3518", "lang": "en", "prompt": "The multiplier effect is\n\nA) the addition of non-basic workers to an urban economy that has added more basic workers.\nB) a ratio of urban to rural workers in an economy.\nC) the ratio of city workers to farm workers in an economy.\nD) the addition of rural jobs created by the urban economy.", "gold": "A", "n_choices": 4, "subject": "high_school_geography", "stem": false}
{"pair_id": "mmlu-2367", "lang": "en", "prompt": "The price of a share of stock for company XYZ at the beginning of the week was $24.75. Over the next five days, the stock gained $2.50 on Monday, lost $3.25 on Tuesday, lost $0.75 on Wednesday, gained $1.25 on Thursday, and gained $4.75 on Friday. What was the price of the share of stock at the end of Friday?\n\nA) $12.25 \nB) $25.75 \nC) $29.25 \nD) $37.25 ", "gold": "C", "n_choices": 4, "subject": "elementary_mathematics", "stem": true}
{"pair_id": "mmlu-397", "lang": "en", "prompt": " What, according to Schnackenberg and Tomlinson (2014), are the three criteria which can be used to assess the quality of corporate transparency?\n\nA) Breadth, Clarity, Accuracy\nB) Disclosure, Detail, Accuracy\nC) Disclosure, Clarity, Accuracy\nD) Disclosure, Clarity, Efficiency", "gold": "C", "n_choices": 4, "subject": "business_ethics", "stem": false}
{"pair_id": "mmlu-1717", "lang": "en", "prompt": "Which of these temperatures is likely when a container of water at 20°C is mixed with water at 28°C?\n\nA) 19°C\nB) 22°C\nC) 30°C\nD) Higher than 30°C", "gold": "B", "n_choices": 4, "subject": "conceptual_physics", "stem": true}
{"pair_id": "mmlu-13908", "lang": "en", "prompt": "Who is the woman mystic who exemplified the all-consuming love of the divine?\n\nA) Rumi\nB) Rabi'a\nC) Fatimah\nD) Khadija", "gold": "B", "n_choices": 4, "subject": "world_religions", "stem": false}
{"pair_id": "mmlu-2348", "lang": "en", "prompt": "A teacher buys the folders listed below.\n* 5 boxes of red folders with 36 folders in each box\n* 6 boxes of blue folders with 32 folders in each box\nWhich number is closest to the total number of red and blue folders that the teacher buys?\n\nA) 275\nB) 380\nC) 440\nD) 550", "gold": "B", "n_choices": 4, "subject": "elementary_mathematics", "stem": true}
{"pair_id": "mmlu-4589", "lang": "en", "prompt": "Which of the following is correct?\n\nA) In the long run, all inputs are variable.\nB) In the short run, all inputs are variable.\nC) In the long run, supply is not able to adjust fully to changes in demand.\nD) In the short run, supply is able to adjust fully to changes in demand.", "gold": "A", "n_choices": 4, "subject": "high_school_microeconomics", "stem": false}
{"pair_id": "mmlu-4792", "lang": "en", "prompt": "A body moving in the positive x direction passes the origin at time t = 0. Between t = 0 and t = 1 second, the body has a constant speed of 24 meters per second. At t = 1 second, the body is given a constant acceleration of 6 meters per second squared in the negative x direction. The position x of the body at t = 11 seconds is\n\nA) 99m\nB) 36m\nC) -36m\nD) -99m", "gold": "C", "n_choices": 4, "subject": "high_school_physics", "stem": true}
{"pair_id": "mmlu-2721", "lang": "en", "prompt": "The Krebs cycle in humans occurs in the\n\nA) mitochondrial matrix\nB) inner mitochondrial membrane\nC) outer mitochondrial membrane\nD) intermembrane space", "gold": "A", "n_choices": 4, "subject": "high_school_biology", "stem": true}
{"pair_id": "mmlu-3631", "lang": "en", "prompt": "Which continent has the highest total fertility rates?\n\nA) Africa\nB) Asia\nC) South America\nD) Europe", "gold": "A", "n_choices": 4, "subject": "high_school_geography", "stem": false}
{"pair_id": "mmlu-13122", "lang": "en", "prompt": "Which statement describes an idea from utilitarianism?\n\nA) What is right and what is wrong are predetermined.\nB) People should be treated as an end, not a means to an end.\nC) The greatest good for the greatest number.\nD) Those better off should give to people who are not as well off.", "gold": "C", "n_choices": 4, "subject": "public_relations", "stem": false}
{"pair_id": "mmlu-12459", "lang": "en", "prompt": "Dr. Jones, a clinical psychologist, has been seeing Lisa L. in therapy for one month. Dr. Jones considers Lisa to be a very attractive woman and finds himself having sexual fantasies about her. As an ethical psychologist, Dr. Jones should:\n\nA) recognize that such attraction is normal but proceed with caution.\nB) confess his attraction to Lisa and make it a topic for mutual examination in therapy.\nC) recognize that such attraction is inappropriate and refer Lisa to another therapist without explaining the reason to her.\nD) continue working with Lisa but consult with a colleague if he believes his attraction might be interfering with his objectivity.", "gold": "D", "n_choices": 4, "subject": "professional_psychology", "stem": false}
{"pair_id": "mmlu-9379", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | I sat on the couch while I watched TV with the cat. Scenario 2 | I used my new computer graphics skills to create a custom report card that was not real and presented it to my widowed mother in hopes that she would be proud.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "C", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-5732", "lang": "en", "prompt": "This question refers to the following information.\n\"We found that not only was it a civil war, an effort by a people who had for years been seeking their liberation from any colonial influence whatsoever, but also we found that the Vietnamese whom we had enthusiastically molded after our own image were hard put to take up the fight against the threat we were supposedly saving them from.\n\"We found most people didn't even know the difference between communism and democracy. They only wanted to work in rice paddies without helicopters strafing them and bombs with napalm burning their villages and tearing their country apart. They wanted everything to do with the war, particularly with this foreign presence of the United States of America, to leave them alone in peace, and they practiced the art of survival by siding with whichever military force was present at a particular time, be it Viet Cong, North Vietnamese or American.\"\nJohn Kerry, 1971\nThe two political issues that most concerned the Counterculture Movement of the 1960s were\n\nA) U.S. involvement in Vietnam and flag burning\nB) the civil rights movement and censorship\nC) U.S. involvement in Vietnam and the civil rights movement\nD) censorship and the draft", "gold": "C", "n_choices": 4, "subject": "high_school_us_history", "stem": false}
{"pair_id": "mmlu-6978", "lang": "en", "prompt": "What is the term for the act of getting individuals to behave in ways that are acceptable to a group?\n\nA) Indoctrination\nB) Brainwashing\nC) Propaganda\nD) Socialisation", "gold": "D", "n_choices": 4, "subject": "management", "stem": false}
{"pair_id": "mmlu-4523", "lang": "en", "prompt": "A production possibility frontier will be a straight line when\n\nA) efficiency is achieved\nB) the goods on the axes are perfect substitutes in consumption\nC) utility is maximized\nD) resources are not specialized", "gold": "D", "n_choices": 4, "subject": "high_school_microeconomics", "stem": false}
{"pair_id": "mmlu-10950", "lang": "en", "prompt": "After graduating from law school, a graduate moved to a city where she had been offered a position in an entertainment law firm. When she arrived, the graduate was told that her job was contingent on passing the state bar examination, which was being offered in July. In preparing for the bar examination, the graduate planned to take a bar review course. Although she had not pre-enrolled, the graduate planned to attend and pay for the course at registration. On the morning the course was set to begin, the graduate arrived late after getting stuck in freeway traffic. By the time the graduate got to the course location, registration had already been completed. When the graduate looked into the room where the course was being conducted, she saw the lecture had begun, and everyone was inside busily taking notes. Panicky, the graduate picked up a set of course materials that were lying on the registration table and entered without paying. Although the graduate intended to pay the enrollment fee, no one was at the registration table to take her money. After attending the first couple of classes, the graduate sent a cashier's check for the full amount of the enrollment fee to the bar review company. Unbeknownst to the graduate, the bar review course's regional director noticed that she had entered the lectures without paying. After ascertaining her identity from a fellow student who knew the graduate, the company filed a complaint with the local prosecutor, and a warrant was issued for her arrest. The complaint charged the graduate with the crime of larceny by trick, for deliberately failing to pay for services rendered. After the complaint was issued, the graduate's check arrived at the company's office. The company deposited the check but failed to inform the prosecutor that payment had been received. The graduate was arrested and held in custody overnight before the matter was resolved. The graduate suffered severe emotional distress at the publicity she received in the local newspapers and the humiliation she suffered for being arrested and held in police custody. Terribly upset, the graduate was unable to take the bar exam and, thus, lost her job at the entertainment law firm. If the graduate asserts a claim against the bar-review company based on infliction of emotional distress, will she prevail?\n\nA) Yes, because the graduate did not intend to defraud the company.\nB) Yes, because the company made no effort to inform the prosecutor that the registration fee had been paid in full.\nC) No, because the company did not intend to cause the graduate to suffer severe emotional distress.\nD) No, because filing-the complaint with the prosecutor was not outrageous conduct.", "gold": "B", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-1497", "lang": "en", "prompt": "What does EXE do when the STP solver times out on a constraint query for a particular path?\n\nA) Assume that the query is satisfiable and continue executing the path.\nB) Assume that the query is not satisfiable and stop executing the path\nC) Restart STP and retry the query, up to a limited number of retries.\nD) Remove a subset of the constraints and retry the query.", "gold": "B", "n_choices": 4, "subject": "computer_security", "stem": true}
{"pair_id": "mmlu-6417", "lang": "en", "prompt": "How many centimetres must the cervix dilute to before a baby can be born?\n\nA) 5\nB) 10\nC) 15\nD) 20", "gold": "B", "n_choices": 4, "subject": "human_sexuality", "stem": false}
{"pair_id": "mmlu-6145", "lang": "en", "prompt": "Clifford and Lucia Pauling, in Senior View, told us that physical changes\n\nA) Are rapid and frightening\nB) Can be offset by meditation\nC) Take a long time\nD) Should not be discussed in public", "gold": "C", "n_choices": 4, "subject": "human_aging", "stem": false}
{"pair_id": "mmlu-561", "lang": "en", "prompt": "Muscle lactate production increases when:\n\nA) oxygen is readily available.\nB) pyruvate cannot be formed from glucose breakdown.\nC) the pH of the muscle falls.\nD) glycolysis is activated at the onset of exercise.", "gold": "D", "n_choices": 4, "subject": "clinical_knowledge", "stem": false}
{"pair_id": "mmlu-12116", "lang": "en", "prompt": "A young boy and a young girl were playing with a football on the sidewalk next to a street. They were laughing as they pushed and pulled each other, trying to get the ball and keep it away from each other. Just as a car approached on the street, the girl lost her balance and fell off the sidewalk and into the street. The driver of the car tried to swerve to avoid her, but ran over her leg. The girl suffered a broken leg. If the girl's parents assert a claim on her behalf against the driver to recover damages for her injuries, they will most likely\n\nA) prevail, unless the driver was driving his vehicle within the posted speed limit.\nB) prevail, unless the girl fell because the boy negligently pushed her toward the street.\nC) not prevail, unless the driver was driving negligently when the accident occurred.\nD) not prevail, unless the driver had the last clear chance to avoid the accident", "gold": "C", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-7403", "lang": "en", "prompt": "According to legend in what country are you most likely to meet a leprechaun?\n\nA) Ireland\nB) Poland\nC) Greenland\nD) Scotland", "gold": "A", "n_choices": 4, "subject": "miscellaneous", "stem": false}
{"pair_id": "mmlu-3542", "lang": "en", "prompt": "Every country with a seacoast has an exclusive economic zone (EEZ) under the UN Convention on the Law of the Sea. Which one of the following statements regarding the Law of the Sea and the EEZ is NOT correct?\n\nA) The EEZ extends up to 200 nautical miles (370 km) from the coastline.\nB) Countries have the right to exploit natural resources in their EEZ.\nC) Countries can exploit resources on the continental shelf extending up to 350 nautical miles (560 km) beyond their EEZ.\nD) A country does not have the right to exploit living resources in its EEZ.", "gold": "D", "n_choices": 4, "subject": "high_school_geography", "stem": false}
{"pair_id": "mmlu-3009", "lang": "en", "prompt": "The bond length between any two nonmetal atoms is achieved under which of the following conditions?\n\nA) Where the energy of interaction between the atoms is at its minimum value\nB) Where the nuclei of each atom exhibits the strongest attraction to the electrons of the other atom\nC) The point at which the attractive and repulsive forces between the two atoms are equal\nD) The closest point at which a valence electron from one atom can transfer to the other atom", "gold": "A", "n_choices": 4, "subject": "high_school_chemistry", "stem": true}
{"pair_id": "mmlu-1737", "lang": "en", "prompt": "The magnetic force on a moving charged particle can change the particle's\n\nA) speed\nB) direction\nC) Both of these\nD) Neither of these", "gold": "B", "n_choices": 4, "subject": "conceptual_physics", "stem": true}
{"pair_id": "mmlu-419", "lang": "en", "prompt": "Predatory pricing.\n\nA) 1,2,4\nB) 1,2,3,4\nC) 1,2\nD) 1,4", "gold": "D", "n_choices": 4, "subject": "business_ethics", "stem": false}
{"pair_id": "mmlu-5457", "lang": "en", "prompt": "A large company has offices in two locations, one in New Jersey and one in Utah. The mean salary of office assistants in the New Jersey office is $28,500. The mean salary of office assistants in the Utah office is $22,500. The New Jersey office has 128 office assistants and the Utah office has 32 office assistants. What is the mean salary paid to the office assistants in this company?\n\nA) $22,500 \nB) $23,700 \nC) $25,500 \nD) $27,300 ", "gold": "D", "n_choices": 4, "subject": "high_school_statistics", "stem": true}
{"pair_id": "mmlu-13368", "lang": "en", "prompt": "Which of the following statements best describes the process of 'medicalization'?\n\nA) The term 'medicalization' refers to the process whereby a medical or health issue becomes securitized.\nB) 'Medicalization' is the outcome of security dialogue that discusses threats or risks to national security in terms of medical language.\nC) An issue is said to be 'medicalized' when it is dismissed as a security threat, instead deemed to be a medical issue concerning health as mutually exclusive from security.\nD) The perceived 'medicalization' of societies occurs when social issues are increasingly considered and responded to as medical problems.", "gold": "D", "n_choices": 4, "subject": "security_studies", "stem": false}
{"pair_id": "mmlu-13243", "lang": "en", "prompt": "Which of the following is NOT one of the 3 forms of violence identified by postcolonialism?\n\nA) Material violence.\nB) Epistemic violence.\nC) Structural violence.\nD) Pervasive violence.", "gold": "D", "n_choices": 4, "subject": "security_studies", "stem": false}
{"pair_id": "mmlu-631", "lang": "en", "prompt": "The energy charge of the cell is:\n\nA) the difference between the charge on the outside and inside of a cell.\nB) generated by the sodium-potassium ATPase.\nC) the overall rate of energy use by the cell.\nD) the extent to which the total adenine nucleotide pool is phosphorylated.", "gold": "D", "n_choices": 4, "subject": "clinical_knowledge", "stem": false}
{"pair_id": "mmlu-3591", "lang": "en", "prompt": "The intensive agricultural practice of planting and harvesting the same crop more than once a year is called\n\nA) swidden.\nB) double cropping.\nC) pastorialism.\nD) intertillage.", "gold": "B", "n_choices": 4, "subject": "high_school_geography", "stem": false}
{"pair_id": "mmlu-12024", "lang": "en", "prompt": "On February 1, a homeowner entered into a contract with a famous landscape architect, to landscape her five-acre estate. The homeowner entered into the contract with the architect because she was pleased with the work he had done on her neighbor's gardens. The architect promised to complete the job before May 30, the date of the homeowner's annual garden party. On March 1, the homeowner left for a month's vacation. Unknown to her, the architect, because of previous commitments, assigned the contract to a gardener. After the gardener had completed 60 percent of the work, the homeowner returned and was displeased with the landscaping already completed. The homeowner contacted the architect, who informed her of the assignment to the gardener and told her that he was no longer obligated under their original contract. Disgruntled, the homeowner fired the gardener. If the homeowner asserts a claim against the architect based on breach of contract, the fact that the architect assigned the contract to the gardener will\n\nA) relieve the architect of liability, because the contract was assignable.\nB) relieve the architect of liability, because the gardener's work was the cause of the homeowner €˜s dissatisfaction.\nC) not relieve the architect of liability, because a personal service contract of this nature is non- delegable.\nD) not relieve the architect of liability, unless there was an express contractual provision against assignments.", "gold": "C", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-12286", "lang": "en", "prompt": "An 8-year-old boy is brought to the office by his mother because of a 3-day history of fever, sore throat, and itchy eyes. He just returned from a weeklong summer camp that included hiking trips and swimming lessons in the camp-owned swimming pool. He has no history of major medical illness and receives no medications. He appears tired. His temperature is 39.4°C (102.9°F); other vital signs are within normal limits. Physical examination shows conjunctival injection and discharge and oropharyngeal erythema. The public health department reports an outbreak of similar symptoms among the other campers and camp volunteers. Which of the following is the most likely cause of this patient’s symptoms?\n\nA) Adenovirus\nB) Cytomegalovirus\nC) Epstein-Barr virus\nD) Influenza virus", "gold": "A", "n_choices": 4, "subject": "professional_medicine", "stem": false}
{"pair_id": "mmlu-10583", "lang": "en", "prompt": "What is the value of the tax shield if the value of the firm is $5 million, its value if unlevered would be $4.78 million, and the present value of bankruptcy and agency costs is $360,000?\n\nA) $140,000\nB) $220,000\nC) $360,000\nD) $580,000", "gold": "D", "n_choices": 4, "subject": "professional_accounting", "stem": false}
{"pair_id": "mmlu-12810", "lang": "en", "prompt": "The scoring criterion categories for the Exner Scoring System used to score the Rorschach Inkblot Test include all of the following except:\n\nA) Color\nB) Popular\nC) Content\nD) Location", "gold": "A", "n_choices": 4, "subject": "professional_psychology", "stem": false}
{"pair_id": "mmlu-5808", "lang": "en", "prompt": "This question refers to the following information.\n\"We have witnessed for more than a quarter of a century the struggles of the two great political parties for power and plunder, while grievous wrongs have been inflicted upon the suffering people. We charge that the controlling influences dominating both these parties have permitted the existing dreadful conditions to develop without serious effort to prevent or restrain them. Neither do they now promise us any substantial reform. They have agreed together to ignore, in the coming campaign, every issue but one. They propose to drown the outcries of a plundered people with the uproar of a sham battle over the tariff, so that capitalists, corporations, national banks, rings, trusts, watered stock, the demonetization of silver and the oppressions of the usurers may all be lost sight of. They propose to sacrifice our homes, lives, and children on the altar of mammon; to destroy the multitude in order to secure corruption funds from the millionaires.\n\"Assembled on the anniversary of the birthday of the nation, and filled with the spirit of the grand general and chief who established our independence, we seek to restore the government of the Republic to the hands of 'the plain people,' with which class it originated. We assert our purposes to be identical with the purposes of the National Constitution; to form a more perfect union and establish justice, insure domestic tranquillity, provide for the common defence, promote the general welfare, and secure the blessings of liberty for ourselves and our posterity.\"\nPopulist Party Platform, 1892\nWhich of the following was most directly a cause of the success of the Populist party?\n\nA) Western farmers and ranchers favored conservation and organized to promote the National Parks system.\nB) The growth of corporate power and banking interests inspired rural activists to lobby for political reform.\nC) Western farmers resisted the mechanization of agriculture and resented government interference in rural affairs.\nD) After the Civil War, westward migration slowed, causing a long-term recession in many Western territories.", "gold": "B", "n_choices": 4, "subject": "high_school_us_history", "stem": false}
{"pair_id": "mmlu-6742", "lang": "en", "prompt": "Evading burden of proof is another name for the fallacy of\n\nA) Extension\nB) Appeal to ignorance\nC) Complex question\nD) Complex proposition", "gold": "B", "n_choices": 4, "subject": "logical_fallacies", "stem": false}
{"pair_id": "mmlu-8874", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | I was dangling my toddler son by his arms on a balcony Scenario 2 | At the Halloween party tonight, I plan on adding booze to the punch drink without anyone known.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "A", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-566", "lang": "en", "prompt": "An ileostomy opening should be:\n\nA) flush with the skin.\nB) 3-5cm in length.\nC) below skin level.\nD) over 5cm in length.", "gold": "B", "n_choices": 4, "subject": "clinical_knowledge", "stem": false}
{"pair_id": "mmlu-10708", "lang": "en", "prompt": "A state built a racetrack that was specially designed and constructed for thoroughbred horseracing. State bonds were issued to finance the construction of the racetrack. The bond agreement provided that for the first five years the racetrack was in operation, at least $2 from each admission charge would be used to repay the bond debt. The bond agreement further stipulated that if the proceeds from the admission charges during the first five years were not sufficient to repay the bondholders, then at least $1 from each admission charge for the next five years would be applied to make the necessary bond payments. After the racetrack was built and in operation for two years, the state legislature passed a bill requiring the racetrack to admit all senior citizens over the age of 65 for the discounted admission fee of 50 cents. This law is probably\n\nA) constitutional, because it is a justifiable exercise of the state's police power.\nB) unconstitutional, because it denies citizens under the age of 65 the equal protection of the law.\nC) unconstitutional, because it impairs the obligation of the bondholders' contract.\nD) unconstitutional, because it is an ex post facto law.", "gold": "C", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-8177", "lang": "en", "prompt": " What is the \"intergenerational storm\" of climate change?\n\nA) the problem of massive world overpopulation over generations.\nB) the problem that the bad effects of current carbon dioxide emissions will fall largely on future generations.\nC) the problem that the compounding effects of greenhouse gas emissions require the current generation to, in effect, cooperate with future generations.\nD) the problem that countries are possibly biased toward the interests of the current generation, which largely benefits from carbon dioxide emissions.", "gold": "D", "n_choices": 4, "subject": "moral_disputes", "stem": false}
{"pair_id": "mmlu-13780", "lang": "en", "prompt": "In the absence of treatment the risk of HIV transmission from an infected mother to her fetus/infant was highest in which of the following areas?\n\nA) Africa\nB) North America\nC) Europe\nD) Japan", "gold": "A", "n_choices": 4, "subject": "virology", "stem": false}
{"pair_id": "mmlu-7594", "lang": "en", "prompt": "Which of these Democrats lost to Ronald Reagan in the 1984 presidential elections?\n\nA) Michael Dukakis\nB) Walter Mondale\nC) Gary Hart\nD) Jimmy Carter", "gold": "B", "n_choices": 4, "subject": "miscellaneous", "stem": false}
{"pair_id": "mmlu-7829", "lang": "en", "prompt": "Which of these is not a position on an American football team?\n\nA) quarterback\nB) tight end\nC) striker\nD) free safety", "gold": "C", "n_choices": 4, "subject": "miscellaneous", "stem": false}
{"pair_id": "mmlu-736", "lang": "en", "prompt": "Oedema is the presence of:\n\nA) excessive intravascular fluid.\nB) excessive extravascular fluid.\nC) excessive intracranial fluid.\nD) excessive necrotic cells.", "gold": "B", "n_choices": 4, "subject": "clinical_knowledge", "stem": false}
{"pair_id": "mmlu-75", "lang": "en", "prompt": "Statement 1 | In a group (ab)^{-2} = b^{-2}a^{-2}. Statement 2 | In a group (ab)^n = a^nb^n.\n\nA) True, True\nB) False, False\nC) True, False\nD) False, True", "gold": "B", "n_choices": 4, "subject": "abstract_algebra", "stem": true}
{"pair_id": "mmlu-9196", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | I heard you got cancer, Jessa said sadly. Scenario 2 | My son said he was worthless, and I argued with him vehemently about that.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "D", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-816", "lang": "en", "prompt": "During the mammalian cardiac cycle, a volume of blood equivalent to ventricular stroke volume is transferred from the more compliant venous side to the less compliant arterial side of the circulation. In terms of pressures within the venous and arterial compartments, this transfer results in\n\nA) no change in pressure in either compartment\nB) no effect on venous pressure and a small increase in arterial pressure\nC) an increase in venous pressure and an equal but opposite decrease in arterial pressure\nD) little effect on venous pressure and a large increase in arterial pressure", "gold": "D", "n_choices": 4, "subject": "college_biology", "stem": true}
{"pair_id": "mmlu-11687", "lang": "en", "prompt": "A write-in candidate is a Mexican-American and is of the opinion that he needs a large turnout of Mexican voters in order to win the election. As a result, his campaign manager, decides to mail to every registered Mexican voter a tear-off sticker, which bears the candidate's name and office for which he is running. Since many native Mexican people are not proficient in reading and writing English, the campaign manager believes that many of the voters will have difficulty writing the candidate's name and office on the ballot. As a result, the campaign manager has mounted an extensive media campaign to inform voters on how to apply the stickers to the ballot. Five months prior to the election an election official notifies the candidate's campaign committee that the tear-off stickers do not comply with the state statute. In her letter, the official explains that state election officials are of the opinion that it is necessary for potential voters to write the candidate's name in ink. Therefore, she concludes that the stickers do not comply with statutory requirements. Three weeks later, the candidate filed suit in federal district court against state election officials claiming that their interpretation of the state statute violates the U. S. Constitution. Which of the following, if established, sets forth the strongest constitutional argument supporting the election official's interpretation of the statute?\n\nA) Voter turnout among registered Mexican American voters who lack proficiency in English has been less than 1% of the entire state total in recent elections.\nB) A state statute requires that each voting booth be equipped with an ink pen to facilitate the writing in of votes.\nC) Since large numbers of write-in votes generally increase the time needed to count all votes, it is necessary to cast such votes in a uniform manner.\nD) The potential for voting fraud is substantially greater with preprinted stickers.", "gold": "D", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-9525", "lang": "en", "prompt": "Which are the FIVE main series of apoproteins that have been identified?\n\n\nA) apoA, apoB, apoC, apoD, and apoE\nB) apoA, apo(a), apoB, apoC and apoE\nC) apoA, apoB, apoC, apo E, and apoL\nD) apoB, apoC, apoD, apoE and apoM", "gold": "B", "n_choices": 4, "subject": "nutrition", "stem": false}
{"pair_id": "mmlu-9799", "lang": "en", "prompt": "According to Kant, the moral worth of an action depends on:\n\nA) the moral character of the agent who performs it.\nB) the consequences of the action.\nC) the maxim that is acted on.\nD) all of the above.", "gold": "C", "n_choices": 4, "subject": "philosophy", "stem": false}
{"pair_id": "mmlu-10901", "lang": "en", "prompt": "A police officer entered a private home in response to a reported disturbance. She found no disturbance but did witness the two male residents engaging in consensual sexual relations with each other. She arrested and charged them with \"deviate sexual intercourse\", a state law that prohibits two persons of the same sex from engaging in mutual sexual relations. After conviction and unsuccessful appeals in the state courts, the men appealed to the United States Supreme Court. What is the likely holding and reasoning of the Supreme Court?\n\nA) The statute violates the men's right to liberty encompassed within the constitutional guarantee of substantive due process.\nB) The federal constitution cannot be used to interfere with a state's right to regulate sexual morality.\nC) The statute is unconstitutional primarily because it infringes the men's right to freedom of expression under the First Amendment.\nD) The relationship in question is not protected by the right to privacy and is subject to a state's criminal regulation.", "gold": "C", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-7274", "lang": "en", "prompt": "___________ is based on the idea that customer expectations of the service they will receive shape their perception of the actual service encounter.\n\nA) Service quality.\nB) Service action.\nC) Service recovery.\nD) Service satisfaction.", "gold": "A", "n_choices": 4, "subject": "marketing", "stem": false}
{"pair_id": "mmlu-8886", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | I used his toothbrush to clean the toilet. Scenario 2 | I locked the prisoner in his cell for the night.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "B", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-7980", "lang": "en", "prompt": "What is the birthstone for the month of January?\n\nA) topaz\nB) garnet\nC) opal\nD) diamond", "gold": "B", "n_choices": 4, "subject": "miscellaneous", "stem": false}
{"pair_id": "mmlu-7747", "lang": "en", "prompt": "What is the function of mitochondria in the cell?\n\nA) To generate energy\nB) To process waste\nC) To kill viruses or other antigens\nD) To repair damage", "gold": "A", "n_choices": 4, "subject": "miscellaneous", "stem": false}
{"pair_id": "mmlu-5418", "lang": "en", "prompt": "Behavioral therapy typically alters the patterns of responding of clients by\n\nA) helping patients identify a hierarchy of anxiety-arousing experiences\nB) vigorously challenging clients' illogical ways of thinking\nC) influencing patients by controlling the consequences of their actions\nD) repeating or rephrasing what a client says during the course of therapy", "gold": "C", "n_choices": 4, "subject": "high_school_psychology", "stem": false}
{"pair_id": "mmlu-6321", "lang": "en", "prompt": "Judaism regards sexuality as\n\nA) inherently evil\nB) God's afterthought and has become an aberration\nC) a gift from God\nD) primarily a biological function", "gold": "C", "n_choices": 4, "subject": "human_sexuality", "stem": false}
{"pair_id": "mmlu-5597", "lang": "en", "prompt": "A hypothesis test is set up so that P(rejecting H0 when H0 is true) = 0.05 and P(failing to reject H0 when H0 is false) = 0.26. What is the power of the test?\n\nA) 0.26\nB) 0.05\nC) 0.95\nD) 0.74", "gold": "D", "n_choices": 4, "subject": "high_school_statistics", "stem": true}
{"pair_id": "mmlu-3720", "lang": "en", "prompt": "Which of the following actions can the president take without congressional approval?\n\nA) Grant reprieves and pardons\nB) Appoint Supreme Court justices\nC) Negotiate and ratify treaties\nD) Declare war", "gold": "A", "n_choices": 4, "subject": "high_school_government_and_politics", "stem": false}
{"pair_id": "mmlu-10624", "lang": "en", "prompt": "Dawn Corp. uses a standard cost system. During the year both the labor rate variance and the labor efficiency variance were unfavorable. Dawn wrote the variances off directly to cost of goods sold. If Dawn had allocated the variances to work in process finished goods and cost of goods sold instead what would have been the effects on current ratio and net income? Current Net ratio income\n\nA) Increases Increases\nB) Increases Decreases\nC) Decreases Increases\nD) Decreases Decreases", "gold": "A", "n_choices": 4, "subject": "professional_accounting", "stem": false}
{"pair_id": "mmlu-6157", "lang": "en", "prompt": "Within the United States, which location has the longest life expectancy?\n\nA) California\nB) Florida\nC) Alaska\nD) Hawaii", "gold": "D", "n_choices": 4, "subject": "human_aging", "stem": false}
{"pair_id": "mmlu-4180", "lang": "en", "prompt": "According to Classical economic analysis in the long run an increase in the money supply results in\n\nA) a proportional increase in the quantity of output.\nB) stagflation.\nC) an increase in the real rate of interest.\nD) a proportional increase in the price level.", "gold": "D", "n_choices": 4, "subject": "high_school_macroeconomics", "stem": false}
{"pair_id": "mmlu-9480", "lang": "en", "prompt": "Which of the following inborn errors of metabolism gives rise to zinc deficiency?\n\n\nA) Acrodermatitis enteropathica\nB) Wilson's disease\nC) Menkes disease\nD) Haemochromatosis", "gold": "A", "n_choices": 4, "subject": "nutrition", "stem": false}
{"pair_id": "mmlu-418", "lang": "en", "prompt": "In order to try and align the interest of shareholders and senior management of a company, _________ are often made available to _______ of a company, to ensure that it is in their interests to increase _______ and dividend performance.\n\nA) Share options, Senior managers, Share Price\nB) Share options, All employees, Share Price\nC) Non-financial remunerations, Senior managers, Employee Satisfaction\nD) Non-financial remunerations, All employees, Employee Satisfaction", "gold": "A", "n_choices": 4, "subject": "business_ethics", "stem": false}
{"pair_id": "mmlu-12925", "lang": "en", "prompt": "A person with Wernicke's aphasia has impaired language comprehension and:\n\nA) nonfluent and unintelligible speech.\nB) nonfluent but intelligible speech.\nC) fluent but unintelligible speech.\nD) fluent and intelligible speech.", "gold": "C", "n_choices": 4, "subject": "professional_psychology", "stem": false}
{"pair_id": "mmlu-2990", "lang": "en", "prompt": "Which of the following substances has an asymmetrical molecular structure?\n\nA) SF4\nB) PCl5\nC) BF3\nD) CO2", "gold": "A", "n_choices": 4, "subject": "high_school_chemistry", "stem": true}
{"pair_id": "mmlu-13499", "lang": "en", "prompt": "In the 1990 reform of the National Health Service, hospitals became:\n\nA) self-governing trusts competing for purchasing contracts from health authorities\nB) state-controlled providers, dependent on funding from the central government\nC) increasingly detached from health authorities and providers of private health care\nD) less inclined to run themselves efficiently, as demand for health care was falling", "gold": "A", "n_choices": 4, "subject": "sociology", "stem": false}
{"pair_id": "mmlu-1945", "lang": "en", "prompt": "The supply frequency usually employed for high frequency eddy current heating is\n\nA) 1 KHz.\nB) 5 KHz.\nC) 10 MHz.\nD) 10 KHz to 400 KHz.", "gold": "D", "n_choices": 4, "subject": "electrical_engineering", "stem": true}
{"pair_id": "mmlu-9498", "lang": "en", "prompt": "The measurement of a specific biochemical status index for a micronutrient such as vitamin C is most likely to be useful in practice to estimate:\n\n\nA) The presence or risk of a clinical deficiency, such as clinical scurvy.\nB) The overall quality of a person's diet\nC) The tissue levels, or body status, of that nutrient.\nD) Overall health and freedom from disease.", "gold": "C", "n_choices": 4, "subject": "nutrition", "stem": false}
{"pair_id": "mmlu-12557", "lang": "en", "prompt": "The items for 2 machinist proficiency test have been based on 2 job analysis of machinists in 50 plants, each employing five or more machinists doing comparable work. Such 2 procedure would contribute directly to the test's\n\nA) concurrent validity\nB) predictive validity\nC) empirical validity\nD) content validity", "gold": "D", "n_choices": 4, "subject": "professional_psychology", "stem": false}
{"pair_id": "mmlu-2790", "lang": "en", "prompt": "A trend toward the decrease in the size of plants on the slopes of mountains as altitudes increase is an example of\n\nA) a cline.\nB) a bottleneck.\nC) relative fitness.\nD) genetic drift.", "gold": "A", "n_choices": 4, "subject": "high_school_biology", "stem": true}
{"pair_id": "mmlu-8696", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | I deforested in the rainforest. Scenario 2 | I felt like vomiting so I reached for my host's fur coat.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "A", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-7335", "lang": "en", "prompt": "On average, how many fragments would a restriction enzyme which recognizes a specific 4 base sequence in DNA be expected to cleave a double-stranded bacteriophage with a genome size of 5,000 bp into?\n\nA) about 2\nB) about 4\nC) about 20\nD) about 50", "gold": "C", "n_choices": 4, "subject": "medical_genetics", "stem": false}
{"pair_id": "mmlu-9242", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | I plucked the rare flowers in the public garden. Scenario 2 | I broke his running record.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "B", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-6923", "lang": "en", "prompt": "Which of the following points would Bayesians and frequentists disagree on?\n\nA) The use of a non-Gaussian noise model in probabilistic regression.\nB) The use of probabilistic modelling for regression.\nC) The use of prior distributions on the parameters in a probabilistic model.\nD) The use of class priors in Gaussian Discriminant Analysis.", "gold": "C", "n_choices": 4, "subject": "machine_learning", "stem": true}
{"pair_id": "mmlu-3339", "lang": "en", "prompt": "This question refers to the following information.\n\"In 1500 that work appeared which Erasmus had written after his misfortune at Dover, and had dedicated to Mountjoy, the Adagiorum Collectanea. It was a collection of about eight hundred proverbial sayings drawn from the Latin authors of antiquity and elucidated for the use of those who aspired to write an elegant Latin style. In the dedication Erasmus pointed out the profit an author may derive, both in ornamenting his style and in strengthening his argumentation, from having at his disposal a good supply of sentences hallowed by their antiquity. He proposes to offer such a help to his readers. What he actually gave was much more. He familiarized a much wider circle than the earlier humanists had reached with the spirit of antiquity.\nUntil this time the humanists had, to some extent, monopolized the treasures of classic culture, in order to parade their knowledge of which the multitude remained destitute, and so to become strange prodigies of learning and elegance. With his irresistible need of teaching and his sincere love for humanity and its general culture, Erasmus introduced the classic spirit, in so far as it could be reflected in the soul of a sixteenth-century Christian, among the people. Not he alone; but none more extensively and more effectively. Not among all the people, it is true, for by writing in Latin he limited his direct influence to the educated classes, which in those days were the upper classes.\nErasmus made current the classic spirit. Humanism ceased to be the exclusive privilege of a few. According to Beatus Rhenanus he had been reproached by some humanists, when about to publish the Adagia, for divulging the mysteries of their craft. But he desired that the book of antiquity should be open to all.\"\nJohan Huizinga, twentieth-century Dutch philosopher, Erasmus and the Age of Reformation, 1924\nThe type of humanism attributed to Erasmus in this passage is most similar to what Southern Renaissance movement?\n\nA) Neoplatonism\nB) Antitrinitarianism\nC) Pietism\nD) Rationalism", "gold": "A", "n_choices": 4, "subject": "high_school_european_history", "stem": false}
{"pair_id": "mmlu-63", "lang": "en", "prompt": "If A = (1, 2, 3, 4). Let ~= {(1, 2), (1, 3), (4, 2)}. Then ~ is\n\nA) not anti-symmetric\nB) transitive\nC) reflexive\nD) symmetric", "gold": "B", "n_choices": 4, "subject": "abstract_algebra", "stem": true}
{"pair_id": "mmlu-7379", "lang": "en", "prompt": "Enzyme assay can be used to identify carriers of:\n\nA) Cystic fibrosis.\nB) Fragile X syndrome.\nC) Oculocutaneous albinism.\nD) Tay-Sachs disease.", "gold": "D", "n_choices": 4, "subject": "medical_genetics", "stem": false}
{"pair_id": "mmlu-8382", "lang": "en", "prompt": " In Carruther's view, the justification for moral criticism for things that we do (or don't to do) to an animal derives from\n\nA) the animal's direct moral standing.\nB) the animals capacity for suffering.\nC) the bad qualities of character that our actions evince.\nD) none of the above", "gold": "C", "n_choices": 4, "subject": "moral_disputes", "stem": false}
{"pair_id": "mmlu-4870", "lang": "en", "prompt": "What happens to the pressure, P, of an ideal gas if the temperature is increased by a factor of 2 and the volume is increased by a factor of 8 ?\n\nA) P decreases by a factor of 16.\nB) P decreases by a factor of 4.\nC) P decreases by a factor of 2.\nD) P increases by a factor of 4.", "gold": "B", "n_choices": 4, "subject": "high_school_physics", "stem": true}
{"pair_id": "mmlu-10138", "lang": "en", "prompt": "Archaeology is a subfield of:\n\nA) biological anthropology.\nB) anthropology.\nC) cultural anthropology.\nD) paleoanthropology.", "gold": "B", "n_choices": 4, "subject": "prehistory", "stem": false}
{"pair_id": "mmlu-9899", "lang": "en", "prompt": "When it comes to defining good, Moore claims that many past philosophers are guilty of:\n\nA) begging the question.\nB) circular reasoning.\nC) the naturalistic fallacy.\nD) the ad hominem fallacy.", "gold": "C", "n_choices": 4, "subject": "philosophy", "stem": false}
{"pair_id": "mmlu-8074", "lang": "en", "prompt": "Amps are a unit of measurement of what?\n\nA) electric charge\nB) electric current\nC) electric field strength\nD) electric potential", "gold": "B", "n_choices": 4, "subject": "miscellaneous", "stem": false}
{"pair_id": "mmlu-11184", "lang": "en", "prompt": "Police officers had probable cause to believe that a man had committed a series of armed bank robberies with an accomplice. The officers obtained a valid arrest warrant for the man and went to his house to arrest him. The officers found the front door ajar and could hear a radio playing inside. They knocked on the door, identified themselves as police officers, and stated that they had a warrant. Getting no response, the officers entered the house. After the officers called the man's name several times and again identified themselves as police officers, the man emerged from the basement and was arrested. The officers went into the basement to look for the accomplice. They opened a closet door and found cocaine in plain view. They did not find the accomplice. The man was indicted for cocaine possession. He has moved to suppress the cocaine as evidence on the ground that it was obtained in violation of the Fourth Amendment. Should the court grant the motion to suppress?\n\nA) No, because the officers discovered the cocaine during a lawful protective sweep of the house looking for the man's accomplice.\nB) No, because the search was incident to the man's arrest.\nC) Yes, because the officers did not have a search warrant.\nD) Yes, because the officers did not have probable cause to believe that cocaine would be in the closet.", "gold": "A", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-8927", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | When I paid the bill, I put a 2-cent stamp on it to make sure it was returned to my house. Scenario 2 | At the grocery store, I coughed into the food.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "A", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-2505", "lang": "en", "prompt": " Use the following key to translate the given formula of PL to natural, English sentences.\nA: Marina reads a Percy Jackson book.\nB: Izzy plays Minecraft.\nC: Emily stops working.\nD: Russell makes dinner.\nE: Ashleigh stops by.\n~C ∨ D\n\nA) If Emily doesn't stop working then Russell makes dinner.\nB) Emily stops working unless Russell makes dinner.\nC) Emily stops working unless Russell doesn't make dinner.\nD) Emily doesn't stop working unless Russell makes dinner.", "gold": "D", "n_choices": 4, "subject": "formal_logic", "stem": false}
{"pair_id": "mmlu-7762", "lang": "en", "prompt": "What is the most likely impact of an increase in interest rates?\n\nA) Investment levels diminish\nB) Employment increases\nC) The value of the U.S. dollar increases\nD) Mortgage rates decline", "gold": "A", "n_choices": 4, "subject": "miscellaneous", "stem": false}
{"pair_id": "mmlu-8528", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 |  I helped him in English by writing his essay. Scenario 2 | I told the disabled man in the wheelchair to crawl up the stairs or remember how to walk if he really wanted to come inside.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "A", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-585", "lang": "en", "prompt": "The ATP store in muscle can only supply energy in high intensity exercise for about:\n\nA) 2 milliseconds.\nB) 2 seconds.\nC) 10 seconds.\nD) 20 seconds.", "gold": "B", "n_choices": 4, "subject": "clinical_knowledge", "stem": false}
{"pair_id": "mmlu-3783", "lang": "en", "prompt": "Which of the following is true of congressional redistricting?\n\nA) The responsibility for redrawing congressional districts belongs to congressional committees.\nB) It is a noncontroversial process because it has few political ramifications.\nC) It occurs every 10 years to reflect changes in population according to the census.\nD) The Supreme Court has ruled that legislators may not consider racial demographics when redrawing districts.", "gold": "C", "n_choices": 4, "subject": "high_school_government_and_politics", "stem": false}
{"pair_id": "mmlu-1205", "lang": "en", "prompt": "Performance enhancing synthetic steroids are based on the structure of the hormone:\n\nA) testosterone.\nB) cortisol.\nC) progesterone.\nD) aldosterone.", "gold": "A", "n_choices": 4, "subject": "college_medicine", "stem": false}
{"pair_id": "mmlu-12976", "lang": "en", "prompt": "The relationship of preadolescent siblings is best described as:\n\nA) distant/aloof.\nB) conflictual/hostile.\nC) competitive and cooperative.\nD) close and conflictual.", "gold": "D", "n_choices": 4, "subject": "professional_psychology", "stem": false}
{"pair_id": "mmlu-2930", "lang": "en", "prompt": "Two individuals, one with type B blood and one with type AB blood, have a child. The probability that the child has type O blood is\n\nA) 0%\nB) 25%\nC) 50%\nD) 100%", "gold": "A", "n_choices": 4, "subject": "high_school_biology", "stem": true}
{"pair_id": "mmlu-11943", "lang": "en", "prompt": "A large man with menacing tattoos all over his face and bald head followed closely behind a businessman who was lost in a long dark alley. The large man followed the businessman for several blocks of unending, unlit alleys. The businessman was in great fear. The large man was just a few feet behind, singing songs about how he was going to have a \"big meal tonight\" and it looked like he \"had struck it rich,\" and other words indicating possible violence and robbery. Finally, the businessman threw his wallet in one direction and ran the other way, as he yelled, \"You can have my money, just leave me my life!\" The large man picked up the wallet and ran in the opposite direction, but when he came to the end of the alley he was arrested and charged with robbery. He appealed his conviction, arguing that he had no intent to steal and was trying to return the wallet to the victim. Will the appellate court likely affirm the robbery conviction?\n\nA) No, the large man's songs could have been coincidental or unthinking, he made no threats, and later could have been trying to return the wallet.\nB) Yes, because the large man followed too closely for too long, he sang menacing songs that placed the businessman in fear, and then he took the wallet and ran in the other direction.\nC) Yes, because one should never pick up someone else's wallet laying on the ground for any reason.\nD) No, because the wallet was not in the immediate presence of the victim when the large man picked it up.", "gold": "B", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-3867", "lang": "en", "prompt": "Contractionary monetary policy implies which of the following about the discount rate the nominal interest rate and aggregate demand? Discount rate      Nominal interest rate      Aggregate demand\n\nA) Decreases            Increases      Decreases\nB) Increases            Increases      Decreases\nC) Decreases            Decreases      Decreases\nD) Increases            Increases      Increases", "gold": "B", "n_choices": 4, "subject": "high_school_macroeconomics", "stem": false}
{"pair_id": "mmlu-10398", "lang": "en", "prompt": "According to the SEC, members of an issuer’s audit committee may not\n\nA) Establish procedures for employees to anonymously report fraud.\nB) Be responsible for the compensation of any registered public accounting firm employed by the registrant to provide an audit report.\nC) Accept any consulting, advisory, or other compensatory fee from the registrant for services other than as a member of the board.\nD) Engage independent counsel as deemed necessary to carry out their duties.", "gold": "C", "n_choices": 4, "subject": "professional_accounting", "stem": false}
{"pair_id": "mmlu-12870", "lang": "en", "prompt": "The Standard for Educational and Psychological Testing provides\n\nA) reliability and validity data on over 3000 tests, along with minimum training requirements needed to administer, score, and interpret each test\nB) derailed legal requirements for the use of tests and manuals\nC) criteria by which tests and manuals should be judged, and which tests should strive to meet\nD) lists of approved tests and manuals", "gold": "C", "n_choices": 4, "subject": "professional_psychology", "stem": false}
{"pair_id": "mmlu-8087", "lang": "en", "prompt": "In a holistic evaluation of student essays, evaluations are made on the basis of the\n\nA) number and variety of errors made by each student\nB) average sentence length and the complexity demonstrated in each essay\nC) ability of each student to communicate in a variety of discourse modes\nD) overall quality of each student's essay in relation to the topic", "gold": "D", "n_choices": 4, "subject": "miscellaneous", "stem": false}
{"pair_id": "mmlu-11888", "lang": "en", "prompt": "The defendant was walking down the street when he saw a woman struggling with a man over a briefcase. Unbeknownst to the defendant, the woman had just stolen the briefcase from the man on the street. Believing the woman to be the victim of an attack, the defendant intervened and punched the man until the woman was able to get away with the briefcase. Confused as to why he wasn't being hailed as a hero by the woman, the defendant eventually realized that he had been an unwitting accomplice to the woman's theft. The defendant apologized profusely to the man and went home. According to the alter ego rule, which of the following statements is correct with respect to the amount of force that the defendant was entitled to use in the woman's defense?\n\nA) Since the defendant did not stand in any personal relationship with the woman, he was not justified in using force in her defense.\nB) Not knowing the true facts, the defendant was not justified in using force to protect the woman because the man was privileged to recapture his briefcase.\nC) The defendant was justified in using reasonable force in the woman's defense, since he reasonably believed she was in immediate danger of unlawful bodily harm from the man.\nD) The defendant was justified in using reasonable force in the woman's defense, since his belief that she was in immediate danger of unlawful bodily harm from the man was both objectively and subjectively reasonable.", "gold": "B", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-4733", "lang": "en", "prompt": "Which of the following is true about the relationship of the average total cost (ATC) curve and the marginal cost (MC) curve?\n\nA) ATC and MC are always equal.\nB) ATC and MC are never equal.\nC) The ATC curve intersects the MC curve at the minimum point of the MC curve.\nD) The MC curve intersects the ATC curve at the minimum point of the ATC curve.", "gold": "D", "n_choices": 4, "subject": "high_school_microeconomics", "stem": false}
{"pair_id": "mmlu-4620", "lang": "en", "prompt": "The law of increasing costs\n\nA) does not apply to guns and butter.\nB) is the result of resources not being perfectly adaptable between the production of two goods.\nC) implies that prices will rise when the costs of making a good rise.\nD) causes the production possibilities frontier to be a straight line.", "gold": "B", "n_choices": 4, "subject": "high_school_microeconomics", "stem": false}
{"pair_id": "mmlu-4770", "lang": "en", "prompt": "A whiffle ball is tossed straight up, reaches a highest point, and falls back down. Air resistance is not negligible. Which of the following statements are true?\nI. The ball’s speed is zero at the highest point.\nII. The ball’s acceleration is zero at the highest point.\nIII. The ball takes a longer time to travel up to the highest point than to fall back down.\n\nA) I only\nB) II only\nC) I & II only\nD) I & III only", "gold": "C", "n_choices": 4, "subject": "high_school_physics", "stem": true}
{"pair_id": "mmlu-2562", "lang": "en", "prompt": "Identify the conclusion of the following argument. We shouldn't abolish capital punishment. If we do, prisons will become more crowded. If prisons become more crowded, then we will have to build more prisons. And we don't want to build more prisons.\n\nA) We shouldn't abolish capital punishment.\nB) If we do, prisons will become more crowded.\nC) If prisons become more crowded, then we will have to build more prisons.\nD) We will have to build more prisons.", "gold": "A", "n_choices": 4, "subject": "formal_logic", "stem": false}
{"pair_id": "mmlu-1329", "lang": "en", "prompt": "The lining of the digestive tract and the respiratory tract develops from which germ layer? I. Endoderm II. Mesoderm III. Ectoderm\n\nA) I only\nB) II only\nC) III only\nD) I and II", "gold": "A", "n_choices": 4, "subject": "college_medicine", "stem": false}
{"pair_id": "mmlu-5429", "lang": "en", "prompt": "What would a psychometrician conclude about a personality test that tells a person she is an extreme extrovert the first time she takes the test and an extreme introvert the next time she takes it?\n\nA) This personality test has low reliability but high validity.\nB) The test is probably high in construct validity but isn't very predictive.\nC) These test norms and standardization probably need improvement.\nD) The results indicate that the test has low test-retest reliability.", "gold": "D", "n_choices": 4, "subject": "high_school_psychology", "stem": false}
{"pair_id": "mmlu-3562", "lang": "en", "prompt": "Which one of the following North American associations is NOT correct?\n\nA) Southeast—African Americans\nB) Oklahoma and the Southwest—Native Americans\nC) Quebec and northern Maine—French Americans and French Canadians\nD) Southern prairie provinces—African Americans", "gold": "D", "n_choices": 4, "subject": "high_school_geography", "stem": false}
{"pair_id": "mmlu-10697", "lang": "en", "prompt": "On Thursday, March 7, an owner's home was destroyed by fire. The owner's house was insured against such fire loss by an insurance company. According to the policy contract, the insured was required to notify the insurance company within 10 days following any loss due to fire or other catastrophe. The owner sued the insurance company to recover $750,000, which was the face amount that his home was insured for. The insurance company admitted that the owner's house was insured for that amount, though the company denied liability because it claimed that it had not received notice of the fire within the 10-day period as required by the policy. At trial, the owner calls his bookkeeper to testify. The bookkeeper proposes to testify that on March 11, the owner handed her an envelope and told her it contained a notice form for the insurance company, and that she mailed the envelope on his behalf. If the bookkeeper intends to establish that the envelope that the owner handed her contained the notice of fire loss, her proposed testimony is\n\nA) admissible, because the owner's statement to her is non-hearsay.\nB) admissible, because a letter properly mailed is presumed received in due course.\nC) inadmissible, because the owner's statement to the bookkeeper is not the best evidence.\nD) inadmissible, because the owner's statement to the bookkeeper is hearsay not within any exception.", "gold": "D", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-12517", "lang": "en", "prompt": "another person's behavior is referred to as the .\n\nA) self-serving bias\nB) actor-observer effect\nC) fundamental attribution bias\nD) self-perception bias", "gold": "C", "n_choices": 4, "subject": "professional_psychology", "stem": false}
{"pair_id": "mmlu-352", "lang": "en", "prompt": "How does the greenhouse effect work?\n\nA) Greenhouse gases absorb infrared light from the Sun which then heats the atmosphere and the surface.\nB) Ozone transmits visible light allowing it to heat the surface but then absorbs most of the infrared heat trapping the heat near the surface.\nC) Greenhouse gases transmit visible light allowing it to heat the surface but then absorb infrared light from Earth trapping the heat near the surface.\nD) The higher pressure of the thick atmosphere at lower altitudes traps heat in more effectively.", "gold": "C", "n_choices": 4, "subject": "astronomy", "stem": true}
{"pair_id": "mmlu-8278", "lang": "en", "prompt": " Collins suggests that the moral value of caring actions comes from\n\nA) the intrinsic value of caring attitudes which motivate caring actions.\nB) how well any caring actions fulfills the recipient's interests and the strength of one's intention to fulfill the recipient's interest.\nC) the degree to which those actions produces the greatest amount of happiness in the recipient of care.\nD) the non-instrumental value of caring actions as manifestations of intrinsic goods for person like love and forgiveness.", "gold": "B", "n_choices": 4, "subject": "moral_disputes", "stem": false}
{"pair_id": "mmlu-2022", "lang": "en", "prompt": "The latching current of SCR is 20 mA. Its holding current will be\n\nA) 23 mA.\nB) 40 mA.\nC) 10 mA.\nD) 60 mA.", "gold": "C", "n_choices": 4, "subject": "electrical_engineering", "stem": true}
{"pair_id": "mmlu-7426", "lang": "en", "prompt": "Which of the following actions is most important before removing the blade from a gas-powered lawn mower?\n\nA) Adjusting the wheels to their highest level\nB) Removing the spark plug\nC) Emptying the gas tank\nD) Removing the air filter", "gold": "B", "n_choices": 4, "subject": "miscellaneous", "stem": false}
{"pair_id": "mmlu-3095", "lang": "en", "prompt": "A solution contains a mixture of four different compounds: KCl(aq), Fe(NO3)3(aq), MgSO4(aq), and N2H4(aq). Which of these compounds would be easiest to separate via distillation?\n\nA) KCl(aq)\nB) Fe(NO3)3(aq)\nC) MgSO4 (aq)\nD) N2H4(aq)", "gold": "D", "n_choices": 4, "subject": "high_school_chemistry", "stem": true}
{"pair_id": "mmlu-9640", "lang": "en", "prompt": "Infants born to mothers who are vegan may be at increased risk of deficiency of which nutrient?\n\n\nA) Vitamin C\nB) Folate\nC) Vitamin B12\nD) Calcium", "gold": "C", "n_choices": 4, "subject": "nutrition", "stem": false}
{"pair_id": "mmlu-1926", "lang": "en", "prompt": "The resistance in the circuit of the moving coil of a dynamometer wattmeter should be\n\nA) Low.\nB) Very low.\nC) High.\nD) Almost zero.", "gold": "C", "n_choices": 4, "subject": "electrical_engineering", "stem": true}
{"pair_id": "mmlu-10733", "lang": "en", "prompt": "After being passed over for a promotion, an aeronautic engineer became a whistleblower. He contacted a government agency to state that the company for which he worked was submitting false safety reports on newly built aircraft. When the company learned that the engineer was leaking this information to the government agency, he was fired from his job. Afterward, the engineer sued the company for wrongful termination of employment. During the discovery stage of litigation, the engineer was deposed by the company's attorney. In his deposition, the engineer stated that the company submitted false safety reports to the government agency to cover up structural defects in its aircraft. A pilot was injured when one of the company'sairplanes he was piloting crashed. The pilot asserteda strict products liability tort against the company. At thal, the pilot seeks to introduce into evidence portions of the engineer's deposition from his wrongful termination lawsuit against the company. Assume that the engineer is unavailable to testify at trial. Upon objection by the company, the trial court judge should rule the engineer's deposition testimony\n\nA) admissible as former testimony.\nB) admissible as a vicarious admission.\nC) inadmissible as hearsay not within any recognized exception.\nD) inadmissible, because the company did not have the opportunity to cross-examine the engineer on the liability issue for which the statement is now being offered.", "gold": "A", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-3725", "lang": "en", "prompt": "Which of the following is an accurate statement about the leadership of the two major political parties?\n\nA) The national party organization controls all facets of party activity throughout the country.\nB) Party leadership is dispersed among numerous officials at the national, state, and local levels.\nC) The highest elected official in each national party directs that party's activities and operations all the way down to the state and local levels.\nD) All major decisions concerning party activities are made during each party's presidential nomination conventions.", "gold": "B", "n_choices": 4, "subject": "high_school_government_and_politics", "stem": false}
{"pair_id": "mmlu-830", "lang": "en", "prompt": "If the activity of an enzyme is constant over a broad range of pH values, it is likely that\n\nA) only ionizing groups on the enzyme participate in the reaction\nB) only ionizing groups on the substrate participate in the reaction\nC) ionizing groups on both the enzyme and substrate participate in the reaction\nD) no ionizing groups on the enzyme or substrate participate in the reaction", "gold": "D", "n_choices": 4, "subject": "college_biology", "stem": true}
{"pair_id": "mmlu-6152", "lang": "en", "prompt": "When a researcher takes part in the situation he or she is observing, he or she is said to be a(n)\n\nA) Active observer\nB) Participant observer\nC) Indiscreet observer\nD) Meddling observer", "gold": "B", "n_choices": 4, "subject": "human_aging", "stem": false}
{"pair_id": "mmlu-3162", "lang": "en", "prompt": "S(s) + O2(g) → SO2(g)\n\nA) one atom of S reacts with one molecule of O2 to yield one molecule of SO2\nB) one mole of sulfur atoms reacts with one mole of oxygen molecules to yield one mole of sulfur dioxide molecules\nC) the position of equilibrium must be on the product side\nD) the entropy increase will be large", "gold": "D", "n_choices": 4, "subject": "high_school_chemistry", "stem": true}
{"pair_id": "mmlu-11357", "lang": "en", "prompt": "The President appointed a delegation to enter into negotiations with representatives of a foreign government to study the problem of preventing the extinction of certain species of rabbits. The delegation's goal was twofold: to study the problem and to formulate regulations in a bilateral agreement that would protect the endangered species and provide for a permanent commission that would continually monitor enforcement of the proposed regulations. After compiling their findings and drafting the necessary regulations, the President and the leader of the foreign government entered into a treaty to form a permanent commission to oversee the problem and to grant it the necessary enforcement powers. Assume that after the treaty goes into effect, a state legislature enacts a statute that provides that \"any licensed rabbit-hunter in the state and its surrounding environs may increase his monthly catch of rabbits from 10 to 15 in each of the specified months of the authorized rabbit-hunting season from the first day of October until the last day of February. \" If challenged, the enactment of the aforementioned statute would most likely be declared\n\nA) constitutional, because the regulation of hunting is within the area of state action.\nB) constitutional, because the enactment falls within the Tenth Amendment's reservedpowers.\nC) unconstitutional, because it violates the commerce clause.\nD) unconstitutional, because all treaties are the supreme law of the land.", "gold": "D", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-12343", "lang": "en", "prompt": "You are visiting an 86-year-old woman in her home at the request of the patient's daughter because of worsening shortness of breath at rest for the past 2 days. The patient is confined to her home because of severe chronic obstructive pulmonary disease with cor pulmonale. She was discharged 1 week ago from the hospital following 3 weeks of treatment for pneumonia that required intubation; her hospital stay was complicated by sepsis. She had a prolonged weaning program from intubation. Prior to discharge, the patient and her daughter met with you to review the patient's advance directive. During the discussion, the patient said, \"I'd rather die than be intubated again.\" At that time, both the patient and her daughter signed an addendum to that effect on the patient's advance directive. Today, the patient is dyspneic and is unable to complete a sentence. Vital signs are temperature 37.8°C (100.0°F), pulse 88/min, respirations 35/min, and blood pressure 100/70 mm Hg. Pulse oximetry shows an oxygen saturation of 84% while the patient is breathing 4 L of oxygen. You ask the patient if she wishes to return to the hospital. She says, \"No, I want to die at home.\" Her daughter takes you aside and says, \"My mother doesn't know what she's saying. I insist that she be admitted to the hospital.\" After further discussion with the daughter regarding support available at home, which of the following is the most appropriate step?\n\nA) Admit the patient to the hospital\nB) Arrange for consultation with a home hospice team\nC) Consult with the hospital ethics committee\nD) Order a home continuous positive airway pressure machine and instruct the daughter in its use", "gold": "B", "n_choices": 4, "subject": "professional_medicine", "stem": false}
{"pair_id": "mmlu-666", "lang": "en", "prompt": "For very high force contractions lasting 1-2 seconds, the initial energy source is from:\n\nA) Glycolysis.\nB) creatine phosphorylation.\nC) phosphocreatine stores.\nD) ATP stores.", "gold": "D", "n_choices": 4, "subject": "clinical_knowledge", "stem": false}
{"pair_id": "mmlu-7663", "lang": "en", "prompt": "What does a pH level measure?\n\nA) acidity\nB) density\nC) wavelength\nD) humidity", "gold": "A", "n_choices": 4, "subject": "miscellaneous", "stem": false}
{"pair_id": "mmlu-7425", "lang": "en", "prompt": "The demand for labor is called derived demand because it is\n\nA) dependent on the demand for the final product that the labor is used to produce\nB) inversely related to the wage rate\nC) dependent on the availability of capital goods\nD) directly related to the marginal utility of working", "gold": "A", "n_choices": 4, "subject": "miscellaneous", "stem": false}
{"pair_id": "mmlu-3277", "lang": "en", "prompt": "Consider the following code segment, which uses the variables r, s, and t.\n  r ← 1\n   s ← 2\n   t ← 3\n   r ← s\n   s ← t\n   DISPLAY (r)\n   DISPLAY (s)\n\n What is displayed as a result of running the code segment?\n\nA) 1 1\nB) 1 2\nC) 2 3\nD) 3 2", "gold": "C", "n_choices": 4, "subject": "high_school_computer_science", "stem": true}
{"pair_id": "mmlu-11516", "lang": "en", "prompt": "After his 16th birthday, a youth decided to purchase a new sports car from a dealership for the sales price of $36,000. The youth thereafter entered into a written contract with the dealership, which provided that the youth would make monthly payments of $1,000 for three years. During the first year of the contract, the youth made 12 monthly payments totaling $12,000. However, the youth failed to make any payments during the second year of the contract. The dealership repeatedly threatened to sue the youth for the remaining balance that it claimed was due under the contract. Finally, in order to avoid litigation, the youth sent the dealership the following letter:\"The car I purchased from you is a real lemon. I have spent a considerable sum of money during the last two years in repair work. I don't believe that the car is worth $36,000, but I am willing to pay you $10,000 if I find ajob in the next month. \"The youth had just turned 18 when he sent this letter to the dealership. The day after mailing this letter, the youth was involved in an automobile accident, and the sports car was totally demolished. Following the accident, the youth remained unemployed, and he refused to make any payments to the dealership. At the time of the accident, the sports car was worth $18,000. In this jurisdiction, the age of majority is18 years of age. If the dealership brings suit against the youth for breach of contract, what, if any, is the plaintiff's proper recovery?\n\nA) Nothing.\nB) $10,000. 00\nC) $18,000. 00\nD) $24,000. 00", "gold": "A", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-12583", "lang": "en", "prompt": "The match between a child’s temperament and the demands and responses of the environment in which she develops is known as ___________.\n\nA) Attachment\nB) Goodness of fit\nC) Bidirectionality\nD) Developmental contextualism", "gold": "B", "n_choices": 4, "subject": "professional_psychology", "stem": false}
{"pair_id": "mmlu-8414", "lang": "en", "prompt": " Ross proposed that we have a duty of reparation-that is, the prima facie duty-to\n\nA) make amends to others for any past wrongs one has done to them.\nB) show gratitude toward one's benefactors.\nC) refrain from harming others.\nD) improve oneself with respect to one's own virtue and knowledge.", "gold": "A", "n_choices": 4, "subject": "moral_disputes", "stem": false}
{"pair_id": "mmlu-1613", "lang": "en", "prompt": "The speed of sound is slightly greater on a\n\nA) cold day\nB) hot day\nC) day with steady temperature\nD) None of these", "gold": "B", "n_choices": 4, "subject": "conceptual_physics", "stem": true}
{"pair_id": "mmlu-9516", "lang": "en", "prompt": "Of the classes of dietary fatty acids which have the most beneficial effect on plasma LDL cholesterol levels?\n\n\nA) n-3 and n-6 polyunsaturated fatty acids\nB) Saturated and trans unsaturated fatty acids\nC) Monounsaturated fatty acids\nD) Both a and c", "gold": "D", "n_choices": 4, "subject": "nutrition", "stem": false}
{"pair_id": "mmlu-9289", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | I told her she looked pretty in her new dress. Scenario 2 | I tossed my trash in the trash can as I walked by it.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "D", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-2961", "lang": "en", "prompt": "A child with type A B blood inherits an IA allele from one parent and an IB allele from the other. The child will carry both A and B glycoproteins on the surface of all red blood cells. This pattern of expression is referred to as\n\nA) codominance\nB) epistasis\nC) polygenic\nD) linkage", "gold": "C", "n_choices": 4, "subject": "high_school_biology", "stem": true}
{"pair_id": "mmlu-1517", "lang": "en", "prompt": "A ___________ is a method in which a computer security mechanism is bypassed untraceable for accessing the computer or its information.\n\nA) front-door\nB) backdoor\nC) clickjacking\nD) key-logging", "gold": "B", "n_choices": 4, "subject": "computer_security", "stem": true}
{"pair_id": "mmlu-13023", "lang": "en", "prompt": "Vicarious liability is most likely to be an issue when a psychologist is acting in which of the following capacities?\n\nA) faculty member\nB) supervisor\nC) mentor\nD) therapist", "gold": "B", "n_choices": 4, "subject": "professional_psychology", "stem": false}
{"pair_id": "mmlu-11463", "lang": "en", "prompt": "The Native American Party was founded and chartered in a state to promote the political power of Native Americans. Members pledged themselves to vote only for candidates nominated by the party. At first, membership was open to any voter who pledged himself to those tenets, but after a defeat in an election, the Party expelled all non- Native-American members upon the grounds that experience showed that only Native Americans could be trusted to honor the obligation to vote only for the Party's nominees. Membership is currently confined to Native Americans. In every election since, the Party's nominees have easily won election in the Sixth Congressional District, an area encompassing a large number of outlying reservations. Subsequently, the Executive Committee proposes that the Party choose its own \"Native American Power\" candidate for governor of South Dakota in the upcoming election. It will put its slate for U. S. representatives, governor, and U. S. senator before Party members for mail ballot, along with any rival candidates who qualify for the poll. A former non-Native American member of the Party is now challenging his exclusion from the Party. Which of the following constitutional provisions would furnish the former member's best ground for challenging his exclusion from the Party?\n\nA) The due process clause of the Fourteenth Amendment.\nB) The right of assembly as guaranteed by the First Amendment.\nC) The equal protection clause of the Fourteenth Amendment.\nD) The voting provisions of the Twenty Fourth Amendment.", "gold": "C", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-9690", "lang": "en", "prompt": "Although theoretically any food protein could act as a food allergen, there are eight major food allergens in Europe and North America which account for most of the allergic triggers observed in childhood. Which of the allergens pairings contains an allergen, which is not part of the group of eight major allergens?\n\n\nA) Cow's milk, egg\nB) Mustard seeds, lamb,\nC) Soy, peanuts\nD) Fish, shellfish", "gold": "B", "n_choices": 4, "subject": "nutrition", "stem": false}
{"pair_id": "mmlu-11208", "lang": "en", "prompt": "A landlord owned an apartment which he leased to a college senior for a three-year term beginning December 1, 2007, for a monthly rental of $1,000 to be payable on the first of each month. After the college senior resided in the apartment for six months, he assigned his interest in the premises to a college junior on June 1,2008. The college junior paid the rental on the apartment for two months and then assigned his interest to a college sophomore. The sophomore took possession on August 1, 2008 and paid the rental for August but did not pay the rental for the next three months of occupancy. Then on December 1, 2008 the college sophomore assigned the premises to a college freshman who took possession but didn't pay any rental to the landlord during the period of his occupancy from December 1,2008 to April 1,2009. On April 5, 2009, the landlord brings an action against the college freshman to recover the outstanding rent that is due on the premises. The freshman is most likely liable for\n\nA) $8,000. 00\nB) $7,000. 00\nC) $5,000. 00\nD) $4,000. 00", "gold": "C", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-6181", "lang": "en", "prompt": "Those more likely to experience depression after a loss are those who\n\nA) Are young survivors\nB) Have not previously experienced depression\nC) Have been married more than 40 years\nD) Are in poor health themselves", "gold": "A", "n_choices": 4, "subject": "human_aging", "stem": false}
{"pair_id": "mmlu-7409", "lang": "en", "prompt": "What is the name of the female character played by comedian Flip Wilson on his 1970s TV show?\n\nA) Miss Jackie\nB) Elizabeth Sanford\nC) Geraldine Jones\nD) Gladys Knight", "gold": "C", "n_choices": 4, "subject": "miscellaneous", "stem": false}
{"pair_id": "mmlu-6585", "lang": "en", "prompt": "Hart argues that secondary rules are 'power-conferring' but this seems unconvincing in the case of the rule of recognition because:\n\nA) It both confers power and imposes duties on judges to decide cases.\nB) It is not a social rule.\nC) It applies only in the case of unitary constitutions.\nD) It fails to specify the source of authority.", "gold": "A", "n_choices": 4, "subject": "jurisprudence", "stem": false}
{"pair_id": "mmlu-12219", "lang": "en", "prompt": "A phase 2, multicenter trial was conducted to determine the efficacy of a new vaccine for prevention of HIV infection. The study enrolled 4000 subjects, aged 20 to 65 years. Of these subjects, 2100 were men and 1900 were women; 2500 were white, 1000 were African American, 300 were Hispanic, and 200 were Asian/Pacific Islanders. Results of the trial showed no overall benefit of the vaccine. However, post hoc analysis disclosed a small but statistically significant vaccine protection among African American subjects. Which of the following is the most accurate rationale for questioning the validity of efficacy analysis of the HIV vaccine among the African American study subjects?\n\nA) HIV infection is more prevalent among African American populations\nB) The study was not blinded\nC) There was a Type II error\nD) Vaccine response among African American subjects was not the primary outcome measure", "gold": "D", "n_choices": 4, "subject": "professional_medicine", "stem": false}
{"pair_id": "mmlu-4472", "lang": "en", "prompt": "Rowena can paint a room in $14$ hours, while Ruby can paint it in $6$ hours. If Rowena paints for $x$ hours and Ruby paints for $y$ hours, they will finish half of the painting, while if Rowena paints for $y$ hours and Ruby paints for $x$ hours they will paint the whole room. Find the ordered pair $(x,y)$.\n\nA) (\\frac{11}{10}, \\frac{11}{10})\nB) (\\frac{231}{20}, \\frac{21}{20})\nC) (\\frac{231}{40}, \\frac{21}{40})\nD) (1,1)", "gold": "C", "n_choices": 4, "subject": "high_school_mathematics", "stem": true}
{"pair_id": "mmlu-5028", "lang": "en", "prompt": "Which of the following contributes most directly to people's exaggerated perceptions of the likelihood of air travel disasters, nuclear power accidents, and terrorist violence?\n\nA) belief perseverance\nB) the framing effect\nC) overconfidence\nD) the availability heuristic", "gold": "D", "n_choices": 4, "subject": "high_school_psychology", "stem": false}
{"pair_id": "mmlu-13576", "lang": "en", "prompt": "Giselle graduated from college with high honors in management. She was recruited immediately following college by a large corporation and accepted a position in junior management. She quickly advanced to middle management. Ten years later Giselle was still in a middle-management position and had witnessed junior male colleagues promoted over her. Which of the following is a phenomenon that is most likely to explain Giselle's lack of promotion?\n\nA) Glass ceiling\nB) Gender constancy\nC) Underemployment\nD) Glass elevator", "gold": "A", "n_choices": 4, "subject": "sociology", "stem": false}
{"pair_id": "mmlu-2672", "lang": "en", "prompt": "During the period when life is believed to have begun, the atmosphere on primitive Earth contained abundant amounts of all the following gases EXCEPT\n\nA) oxygen\nB) hydrogen\nC) ammonia\nD) methane", "gold": "A", "n_choices": 4, "subject": "high_school_biology", "stem": true}
{"pair_id": "mmlu-7187", "lang": "en", "prompt": "The primary objective of team-based selling is to\n\nA) match various functional areas of the company with customer needs\nB) distribute the work involved in managing an account\nC) equitably manage the sales bonus pool\nD) create multiple points of contact between the customer and the company", "gold": "A", "n_choices": 4, "subject": "marketing", "stem": false}
{"pair_id": "mmlu-7465", "lang": "en", "prompt": "In the famous ballet 'The Nutcracker' who does the hero defeat?\n\nA) Sugar Plum Fairy\nB) Rat King\nC) Snow Queen\nD) Marzipan", "gold": "B", "n_choices": 4, "subject": "miscellaneous", "stem": false}
{"pair_id": "mmlu-9149", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | I completed my homework in my bedroom. Scenario 2 | I gave opioids to the patient in the rehab care center.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "C", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-5405", "lang": "en", "prompt": "Dr. Bonneau helped a company redesign its offices to raise morale and the productivity of the employees. With which of the following subfields of psychology is Dr. Bonneau most likely affiliated?\n\nA) counseling\nB) educational\nC) industrial/organizational\nD) developmental", "gold": "C", "n_choices": 4, "subject": "high_school_psychology", "stem": false}
{"pair_id": "mmlu-4226", "lang": "en", "prompt": "When disposable income increases by $X \n\nA) consumption increases by more than $X .\nB) saving increases by less than $X .\nC) saving increases by exactly $X .\nD) saving remains constant.", "gold": "B", "n_choices": 4, "subject": "high_school_macroeconomics", "stem": false}
{"pair_id": "mmlu-11756", "lang": "en", "prompt": "A businessman was the owner of an idyllic lake and wooded area. In 1980, the businessman conveyed the land to the state \"on the condition that\" it be used as recreational property only. Two years later, the state decided that they wanted a small industrial park attached to the corner of the land. The state went ahead and developed a small portion of the land as industrial space, making it part of a larger industrial park that was adjacent to the land. The businessman notified the state that he was the owner of the property because the state violated the conditions of the conveyance, and that he was taking possession of the property back. What was the interest that was created by the conveyance and can the businessman get the property back?\n\nA) The interest created is a reverter, and the property instantly reverts back to the grantor upon the occurrence of the specified violation.\nB) The interest created is a fee simple subject to a condition subsequent, and the original owner can get the property back only by taking affirmative steps to exercise his right of entry.\nC) The interest is a fee simple subject to an executory limitation, and if the condition occurs the property reverts in fee simple to a pre-designated third party.\nD) The interest created is an indefeasible estate, and the owner cannot do anything to enforce a condition that is against public policy and never enforceable in the first place.", "gold": "B", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-2265", "lang": "en", "prompt": "Which ratio CANNOT form a proportion with 8 over 18?\n\nA) 46 over 108\nB) 12 over 27\nC) 60 over 135\nD) 4 over 9", "gold": "A", "n_choices": 4, "subject": "elementary_mathematics", "stem": true}
{"pair_id": "mmlu-7856", "lang": "en", "prompt": "What tea is known for its distinctive bergamot flavor?\n\nA) Earl\nB) Grey Darjeeling\nC) English Breakfast\nD) Prince of Wales", "gold": "A", "n_choices": 4, "subject": "miscellaneous", "stem": false}
{"pair_id": "mmlu-13767", "lang": "en", "prompt": "Any systematic error in the design, conduct, or analysis of a study that results in a mistaken estimate of an exposure’s effect on the risk of disease is called:\n\nA) Confounding\nB) Bias\nC) Interaction\nD) Stratification", "gold": "C", "n_choices": 4, "subject": "virology", "stem": false}
{"pair_id": "mmlu-12119", "lang": "en", "prompt": "A uniformed United States Park Policeman pulled over a motorist for going through a red light. The man reacted gruffly, telling the officer that he had no authority to stop him. There was a brief scuffle, and the officer issued two more charges, for disorderly conduct and failure to obey an officer. Eventually, the defendant pled guilty to the red light citation and the other two charges were dismissed. He then filed a federal court action against the park policeman in which he asserted a Bivens civil rights claim for false arrest and malicious prosecution. The first legal theory was that the park officer lacked authority to stop the plaintiff on a New York street and arrest him for a state traffic violation. He claimed that the officer was outside of his federal park land jurisdiction. Will the court likely uphold the authority of the park police under these facts?\n\nA) The U.S. Park Police have the authority to make a warrantless arrest for a state law violation because of statutory grants of authority from the various states.\nB) All federal officers can make arrests for all criminal offenses within a state's geographical jurisdiction under the priority status of federal law and federal law officers.\nC) The federal officer was way outside his scope of authority and he clearly cannot make an arrest outside of the confines of the federal park systems.\nD) The federal officer is allowed to arrest for homicide and serious felonies but cannot make traffic arrests within a state's geographical jurisdiction.", "gold": "A", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-2040", "lang": "en", "prompt": "If holding current of a thyristor is 2 mA then latching current should be\n\nA) 0.01 A.\nB) 0.002 A.\nC) 0.009 A.\nD) 0.004 A.", "gold": "D", "n_choices": 4, "subject": "electrical_engineering", "stem": true}
{"pair_id": "mmlu-13716", "lang": "en", "prompt": "How were retroviruses discovered?\n\nA) In chickens as Rous sarcoma\nB) In humans as HTLV-1\nC) In mice causing leukaemia\nD) In cats causing leukaemia", "gold": "C", "n_choices": 4, "subject": "virology", "stem": false}
{"pair_id": "mmlu-756", "lang": "en", "prompt": "Synthesis of an RNA/DNA hybrid from a single-stranded RNA template requires\n\nA) a DNA or RNA primer and reverse transcriptase\nB) a DNA or RNA primer and DNA polymerase\nC) RNase only\nD) reverse transcriptase only", "gold": "A", "n_choices": 4, "subject": "college_biology", "stem": true}
{"pair_id": "mmlu-12963", "lang": "en", "prompt": "The term “privilege” is best described as:\n\nA) a general term referring to the right to privacy.\nB) an ethical term referring to an obligation not to reveal confidential information.\nC) the legal equivalent of the ethical concept of confidentiality.\nD) a legal term referring to the protection of confidential information in legal proceedings.", "gold": "D", "n_choices": 4, "subject": "professional_psychology", "stem": false}
{"pair_id": "mmlu-3136", "lang": "en", "prompt": "Which of the following best explains why the ionization of atoms can occur during photoelectron spectroscopy, even though ionization is not a thermodynamically favored process?\n\nA) It is an exothermic process due to the release of energy as an electron is liberated from the Coulombic attraction holding it to the nucleus.\nB) The entropy of the system increases due to the separation of the electron from its atom.\nC) Energy contained in the light can be used to overcome the Coulombic attraction between electrons and the nucleus.\nD) The products of the ionization are at a lower energy state than the reactants.", "gold": "C", "n_choices": 4, "subject": "high_school_chemistry", "stem": true}
{"pair_id": "mmlu-9526", "lang": "en", "prompt": "Which of the following is the yield of ATP from complete oxidation of glucose to carbon dioxide and water?\n\n\nA) 10\nB) 25\nC) 30\nD) 32", "gold": "D", "n_choices": 4, "subject": "nutrition", "stem": false}
{"pair_id": "mmlu-13727", "lang": "en", "prompt": "Which virus infection was not first identified by clinical observation?\n\nA) HIV\nB) Rubella teratogenicity\nC) Microcephaly and Zika\nD) Hepatitis C", "gold": "D", "n_choices": 4, "subject": "virology", "stem": false}
{"pair_id": "mmlu-10956", "lang": "en", "prompt": "To improve the quality of rental housing within its boundaries, a city proposed an ordinance requiring all new and existing rental housing units to provide at least one full bathroom for each bedroom, plumbing and electrical hookups for a washer and dryer, and a covered parking space. A majority of the owners of existing rental housing in the city opposed the ordinance. They argued that it would dramatically decrease the number of lowincome rental housing units because owners would be unable to raise rents enough to recoup the investment required to comply. Without denying these contentions, the city enacted the ordinance. A plaintiff who owns low-income rental housing has sued the city, claiming only that the ordinance is unconstitutional on its face. Which of the following best states the burden of persuasion in this action?\n\nA) The city must demonstrate that the ordinance is necessary to serve a compelling state interest, because it adversely affects the fundamental right of rental housing owners to use their property in the manner they deem most economically efficient.\nB) The city must demonstrate that the ordinance is necessary to serve a compelling state interest, because it will have a substantial and disproportionate negative impact on low-income persons.\nC) The plaintiff must demonstrate that the ordinance is not substantially related to an important state interest, because it requires some owners of rental housing to invest money that they will not be able to recoup from increased rents.\nD) The plaintiff must demonstrate that there is no rational relationship between the ordinance and any legitimate state interest, because the ordinance regulates economic activity of a type normally presumed to be within state regulatory authority.", "gold": "D", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-6587", "lang": "en", "prompt": "Which proposition below may be characterized as the most powerful refutation by Hart of Lord Devlin's argument?\n\nA) Society has no right to preserve its moral and social cohesion through the criminal law.\nB) Homosexual acts cause public outrage.\nC) The courts are the best place to resolve moral questions such as those concerning homosexuality and prostitution.\nD) The law is anachronistic.", "gold": "A", "n_choices": 4, "subject": "jurisprudence", "stem": false}
{"pair_id": "mmlu-3408", "lang": "en", "prompt": "This question refers to the following information.\nIn Russia there was nothing going on well, and [Souvarine] was in despair over the news he had received. His old companions were all turning to the politicians; the famous Nihilists who made Europe tremble-sons of village priests, of the lower middle class, of tradesmen-could not rise above the idea of national liberation, and seemed to believe that the world would be delivered-when they had killed their despot&…\n\"Foolery! They'll never get out of it with their foolery.\"\nThen, lowering his voice still more, in a few bitter words he described his old dream of fraternity. He had renounced his rank and his fortune; he had gone among workmen, only in the hope of seeing at last the foundation of a new society of labour in common. All the sous in his pockets had long gone to the urchins of the settlement; he had been as tender as a brother with the colliers, smiling at their suspicion, winning them over by his quiet workmanlike ways and his dislike of chattering. But decidedly the fusion had not taken place.\nHis voice changed, his eyes grew bright, he fixed them on étienne, directly addressing him:\n\"Now, do you understand that? These hatworkers at Marseilles who have won the great lottery prize of a hundred thousand francs have gone off at once and invested it, declaring that they are going to live without doing anything! Yes, that is your idea, all of you French workmen; you want to unearth a treasure in order to devour it alone afterwards in some lazy, selfish corner. You may cry out as much as you like against the rich, you haven't got courage enough to give back to the poor the money that luck brings you. You will never be worthy of happiness as long as you own anything, and your hatred of the bourgeois proceeds solely from an angry desire to be bourgeois yourselves in their place.\"\némile Zola, French writer, Germinal, 1885\nThe conflict referred to in the final paragraph of the passage was best described in which book?\n\nA) On Liberty, by John Stuart Mill\nB) The Communist Manifesto, by Marx and Engels\nC) Protocols of the Elders of Zion\nD) Looking Backward, by Edward Bellamy", "gold": "B", "n_choices": 4, "subject": "high_school_european_history", "stem": false}
{"pair_id": "mmlu-8822", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | I blew smoke away from the child's face. Scenario 2 | I passionately kissed my son in the living room.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "C", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-5581", "lang": "en", "prompt": "What is a sampling distribution?\n\nA) A distribution of all the statistics that can be found in a given sample\nB) A histogram, or other such visual representation, showing the distribution of a sample\nC) A normal distribution of some statistic\nD) A distribution of all the values taken by a statistic from all possible samples of a given size", "gold": "D", "n_choices": 4, "subject": "high_school_statistics", "stem": true}
{"pair_id": "mmlu-4596", "lang": "en", "prompt": "Which of the following will increase wages for tuba makers?\n\nA) An increase in the number of graduates at tuba maker training school\nB) An increase in the price of tubas\nC) An increase in the price of tuba lessons\nD) An increase in the tax on tubas", "gold": "B", "n_choices": 4, "subject": "high_school_microeconomics", "stem": false}
{"pair_id": "mmlu-8828", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | I slid down the slide with a toddler on it. Scenario 2 | I put my wife's cellphone into the garbage disposal.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "A", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-13008", "lang": "en", "prompt": "\"Samuel S., age 43, says, \"\"Even though I've just been given a bonus at work, I feel like I’m about to be fired.” As defined by Aaron Beck, Samuel is exhibiting which of the following cognitive distortions?\"\n\nA) mustabatory thinking\nB) polarized thinking\nC) personalization\nD) arbitrary inference", "gold": "D", "n_choices": 4, "subject": "professional_psychology", "stem": false}
{"pair_id": "mmlu-5019", "lang": "en", "prompt": "As a result of an accident, Abdul lost sight in his right eye. To judge the distance of vehicles when he is driving, Abdul is able to rely on cues of\n\nA) I only\nB) II only\nC) III only\nD) I and II only", "gold": "D", "n_choices": 4, "subject": "high_school_psychology", "stem": false}
{"pair_id": "mmlu-582", "lang": "en", "prompt": "When assessing a patient's breathing:\n\nA) always remove dentures.\nB) look for chest movements and use a mirror to check for exhaled air.\nC) look for chest movements, listen for breath sounds, and feel for exhaled air on your cheek.\nD) assess for 30 seconds.", "gold": "C", "n_choices": 4, "subject": "clinical_knowledge", "stem": false}
{"pair_id": "mmlu-9535", "lang": "en", "prompt": "The test for checking mean plasma glucose concentration over the previous 8-10 weeks is:\n\n\nA) Hemoglobin A1c\nB) Oral glucose tolerance test (OGTT)\nC) Fructosamine test\nD) Fasting plasma glucose concentration", "gold": "A", "n_choices": 4, "subject": "nutrition", "stem": false}
{"pair_id": "mmlu-11796", "lang": "en", "prompt": "An incumbent was running for re-election as the mayor of a city. During one of his opponent's campaign speeches, he stated that the mayor had once propositioned a prostitute. A television correspondent who was covering the election was present when the opponent made the statement. She recorded the opponent's remark about the mayor in her notebook. In a stunning upset, the opponent defeated the mayor in the election. Many of the pollsters attributed the mayor's loss to the question of his moral character following the opponent's smear campaign depicting the mayor as a client of prostitutes. Following the election, the mayor sued the opponent for defamation, basing his suit on the opponent's statement that the mayor had \"once propositioned a prostitute. \" At trial, the mayor calls the television correspondent to testify concerning her recollection of what the opponent said during the campaign speech attended by the television correspondent. The television correspondent's testimony is\n\nA) admissible, because the television correspondent recorded the opponent's statement in her business capacity.\nB) admissible, because the television correspondent had firsthand knowledge of the opponent's statement.\nC) inadmissible, because the notebook entry is the best evidence.\nD) inadmissible, because it is hearsay not within any recognized exception.", "gold": "B", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-13281", "lang": "en", "prompt": "In the context of energy security and the production of oil, which of the following does not define the term 'stability'?\n\nA) The absence of challenges to the regime of oil-producing states.\nB) A political and economic situation in which the status quo of production, exportation and supply is maintained.\nC) The promotion of human security and the affiliated environmental security as a priority within oil-producing states.\nD) The security of regimes deemed 'friendly' by North American, European, and international oil company interests.", "gold": "C", "n_choices": 4, "subject": "security_studies", "stem": false}
{"pair_id": "mmlu-5393", "lang": "en", "prompt": "All of the following are conditions sanctioned by the APA regarding the use of deception in a study EXCEPT\n\nA) the research is of great importance and cannot be conducted without the use of deception\nB) participants are expected to find the procedures reasonable upon being informed of them\nC) participants must be allowed to withdraw from the experiment at any time\nD) the research must be conducted as a double-blind study", "gold": "D", "n_choices": 4, "subject": "high_school_psychology", "stem": false}
{"pair_id": "mmlu-12577", "lang": "en", "prompt": "Magical thinking would predispose a child of what age range to infer chat his or her father died because of the child’s own misbehavior\n\nA) 12 years\nB) 3-6 years\nC) 7-12 years\nD) 13-18 years", "gold": "B", "n_choices": 4, "subject": "professional_psychology", "stem": false}
{"pair_id": "mmlu-7047", "lang": "en", "prompt": "What is a social enterprise concerned with?\n\nA) Profit maximisation\nB) Maximising market share\nC) Providing public service\nD) Running a business to create social benefits", "gold": "D", "n_choices": 4, "subject": "management", "stem": false}
{"pair_id": "mmlu-10039", "lang": "en", "prompt": "What designation for a pottery style was eventually applied to an entire cultural complex that moved across the Pacific?\n\nA) Clovis\nB) Nenana Complex\nC) Lapita\nD) Micronesian Pottery Tradition", "gold": "C", "n_choices": 4, "subject": "prehistory", "stem": false}
{"pair_id": "mmlu-8215", "lang": "en", "prompt": " The basic idea of social contract theories of morality is that correct or justified moral rules or principles are the ones that result from\n\nA) a social leader's moral deliberations.\nB) an actual or hypothetical social agreement of some sort.\nC) a contract that has been signed by most of the affected parties.\nD) none of the above", "gold": "B", "n_choices": 4, "subject": "moral_disputes", "stem": false}
{"pair_id": "mmlu-7907", "lang": "en", "prompt": "What Mexican holiday takes place on May 5th?\n\nA) Mexican Independence Day\nB) Santa Anna's Birthday\nC) Cinco de Mayo\nD) Christmas", "gold": "C", "n_choices": 4, "subject": "miscellaneous", "stem": false}
{"pair_id": "mmlu-10494", "lang": "en", "prompt": "Accrual accounting involves accruals and deferrals. Which of the following best describes accruals and deferrals?\n\nA) Accruals are concerned with expected future cash receipts and payments while deferrals are concerned with past cash receipts and payments.\nB) Accruals are concerned with past cash receipts and payments while deferrals are concerned with expected future cash receipts and payments.\nC) Both accruals and deferrals are concerned with expected future cash receipts and payments.\nD) Both accruals and deferrals are concerned with past cash receipts and payments.", "gold": "A", "n_choices": 4, "subject": "professional_accounting", "stem": false}
{"pair_id": "mmlu-13543", "lang": "en", "prompt": "According to Foucault, carceral organizations:\n\nA) emphasised human relations and employee satisfaction\nB) increased the amount of power held by managers\nC) stripped inmates of their previous identities\nD) isolated people and put them under surveillance", "gold": "D", "n_choices": 4, "subject": "sociology", "stem": false}
{"pair_id": "mmlu-9211", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | I walked past a pond and saw a drowning child, at which time I continued walking past the pond so my expensive shoes would not get tarnished. Scenario 2 | I ate the produce before paying.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "A", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-10158", "lang": "en", "prompt": "Why do we know more about Neandertals than other extinct species of premodern humans?\n\nA) Neandertals spread throughout the world and their remains are found on every continent.\nB) Neandertals used caves in Europe where their remains were preserved.\nC) Neandertal populations were larger and more violent than any other premodern humans, whom they drove to extinction.\nD) Neandertals left crude but intelligible written records.", "gold": "B", "n_choices": 4, "subject": "prehistory", "stem": false}
{"pair_id": "mmlu-2080", "lang": "en", "prompt": "Trish’s resting heart rate is 50 beats per minute. For every minute she exercises, her heart rate increases 5 beats per minute. How long will it take her to reach a heart rate of 120 beats per minute?\n\nA) 5 minutes\nB) 14 minutes\nC) 34 minutes\nD) 70 minutes", "gold": "B", "n_choices": 4, "subject": "elementary_mathematics", "stem": true}
{"pair_id": "mmlu-3895", "lang": "en", "prompt": "Inflation\n\nA) hurts creditors who do not anticipate it.\nB) hurts creditors who anticipate it.\nC) hurts debtors.\nD) both A and D are correct.", "gold": "D", "n_choices": 4, "subject": "high_school_macroeconomics", "stem": false}
{"pair_id": "mmlu-8030", "lang": "en", "prompt": "A student is conducting a research project and has learned of a website that may have useful information. The domain extension for the site is .org. Which of the following assumptions about the website is correct?\n\nA) All of the information on the site is current.\nB) The site has been evaluated for bias.\nC) The site might belong to a nonprofit agency.\nD) The author of the site is well respected in his or her field.", "gold": "C", "n_choices": 4, "subject": "miscellaneous", "stem": false}
{"pair_id": "mmlu-13484", "lang": "en", "prompt": "Sreberny-Mohammadi (1996) argues that national cultures can resist American cultural domination of the media by:\n\nA) domesticating its content, including more 'home-produced' programmes\nB) controlling the distribution of imported products by banning satellite dishes\nC) creating 'reverse flows' of their own programmes back to imperial societies\nD) all of the above", "gold": "D", "n_choices": 4, "subject": "sociology", "stem": false}
{"pair_id": "mmlu-1158", "lang": "en", "prompt": "Which of the following are true:\n\nA) It is possible to place an linear order relation ≤ on the complex numbers C such that (C, +, ×, ≤) is not an ordered field.\nB) It is possible to place a linear order relation on the complex numbers C which make (C, +, ×, ≤) an ordered field.\nC) Neither (a) nor (b)\nD) Both (a) and (b).", "gold": "A", "n_choices": 4, "subject": "college_mathematics", "stem": true}
{"pair_id": "mmlu-1997", "lang": "en", "prompt": "The main advantage of dielectric heating is that\n\nA) heating occurs in the material itself.\nB) heating occurs due to high frequency.\nC) it can be used for drying the explosives.\nD) None of the above.", "gold": "A", "n_choices": 4, "subject": "electrical_engineering", "stem": true}
{"pair_id": "mmlu-1658", "lang": "en", "prompt": "The discrete orbits of electrons are best understood when modeled by\n\nA) high-speed particles\nB) particles on springs\nC) waves\nD) photons", "gold": "C", "n_choices": 4, "subject": "conceptual_physics", "stem": true}
{"pair_id": "mmlu-467", "lang": "en", "prompt": "Post-modern ethics assert that ethics are context and individual specific and as such have an internal guide to ethics.\n\nA) 2,3\nB) 1,3\nC) 1,2,3\nD) 1,2", "gold": "B", "n_choices": 4, "subject": "business_ethics", "stem": false}
{"pair_id": "mmlu-7002", "lang": "en", "prompt": "Which of these does not form part of the key streams identified by March (1988) in decision making in highly ambiguous environments?\n\nA) Expectations\nB) Choice opportunities\nC) Problems\nD) Solutions", "gold": "A", "n_choices": 4, "subject": "management", "stem": false}
{"pair_id": "mmlu-11152", "lang": "en", "prompt": "A statute passed by both houses of Congress and signed by the President authorizes a federal agency to select a site for and to construct a monument honoring members of the capitol police force killed in the line of duty. The statute appropriates the necessary funds but provides that the funds may not be expended until both houses of Congress have adopted a concurrent resolution, not subject to presentment to the President, approving the agency's plans for the monument's location and design. Is the provision requiring further congressional approval before expenditure of the funds constitutional?\n\nA) No, because decisions regarding the placement and design of government-owned structures are an exclusively executive function with which Congress may not interfere by any means.\nB) No, because the provision amounts to an unconstitutional legislative interference with an executive function.\nC) Yes, because Congress may attach reasonable conditions to its appropriation of funds to executive departments, and its special interest in the members of its own police force makes the provision a reasonable condition.\nD) Yes, because the provision is part of a statute that was passed by both houses of Congress and signed by the President.", "gold": "B", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-4970", "lang": "en", "prompt": "A pigeon trained to peck at a green light pecks at a yellow light also. This illustrates\n\nA) generalization\nB) discrimination\nC) extinction\nD) spontaneous recovery", "gold": "A", "n_choices": 4, "subject": "high_school_psychology", "stem": false}
{"pair_id": "mmlu-13197", "lang": "en", "prompt": "How is a process of securitization completed?\n\nA) A process of securitization is when an issue not included in the political sphere becomes politicized and is managed within the political system. A securitized issue is part of public policy requiring governmental action; decision-making, resource allocation and even communal governance.\nB) A securitizing actor can argue for an issue to be securitised when it is outside of the political domain. An existential threat can be made a matter for state attention even when it is not included within public debate. Securitization refers to the recognition of the threat prior to its emergence in the political process.\nC) Securitization refers to the move of issues from a politicized state to a heightened state of politics beyond the normal political domain of decision-making activities. Once placed above the political sphere it cannot be returned to the public sphere, having been translated into a state interest above normal proceedings.\nD) An issue can be securitized, that is framed as a security question, when it is moved from the politicized to the securitized sphere. During the securitization process, the securitizing actor(s) use discourse to persuade an audience that the issue at hand is one of urgency and priority, and that it presents an existential threat to some referent object.", "gold": "D", "n_choices": 4, "subject": "security_studies", "stem": false}
{"pair_id": "mmlu-6215", "lang": "en", "prompt": "Which of the following is NOT one of the major reasons for the increase in numbers of older adults?\n\nA) The Baby Boom generation\nB) The steadily increasing birth rate\nC) Increases in life expectancy at birth\nD) Increases in life expectancy at a specified age", "gold": "B", "n_choices": 4, "subject": "human_aging", "stem": false}
{"pair_id": "mmlu-8297", "lang": "en", "prompt": " Richardson-Self argues that sexist speech\n\nA) is always hate speech, because it involves violence.\nB) is not always hate speech, because it does not involve violence\nC) is always hate speech, because it does not involves violence.\nD) I s not always hate speech, because it always involves violence.", "gold": "B", "n_choices": 4, "subject": "moral_disputes", "stem": false}
{"pair_id": "mmlu-7089", "lang": "en", "prompt": "The process of making large numbers of unsolicited sales calls to potential customers whom the salesperson does not know is referred to as\n\nA) multilevel marketing\nB) cold canvassing\nC) the center-of-influence method\nD) the sales-associate method", "gold": "B", "n_choices": 4, "subject": "marketing", "stem": false}
{"pair_id": "mmlu-12842", "lang": "en", "prompt": "Kohlberg’s cognitive-developmental theory identifies which of the following as the initial stage of gender-role acquisition?\n\nA) gender affirmation\nB) gender identity\nC) gender constancy\nD) gender stability.", "gold": "B", "n_choices": 4, "subject": "professional_psychology", "stem": false}
{"pair_id": "mmlu-12040", "lang": "en", "prompt": "A resident lived in a house across the street from a small office building. One of the offices in that building was that of a criminal defense lawyer. The resident hated the idea of the lawyer's office being close to her home. She felt that it meant dangerous criminals were coming and going in her neighborhood. The resident spent a lot of time looking out her window, trying to figure out which people going in and out of the office building were criminals, and imagining what sorts of crimes they had committed. The resident eventually decided that she needed to drive the lawyer out of the neighborhood. She made a telephone call to the lawyer and said, \"You are scum, and so are all of your clients. I know that you're conspiring to commit crimes with all those bad people you represent. The police should arrest you and put you in jail with your clients. \" The lawyer was very upset by what he heard. If the lawyer asserts a claim for defamation against the resident based on the telephone call, he will most likely\n\nA) succeed, because the resident's remarks constituted slander per Se.\nB) succeed, because the lawyer found the remarks to be upsetting.\nC) not succeed, because the resident's remarks were a matter of personal opinion rather than statements of fact.\nD) not succeed, because the resident's remarks were not published or communicated to anyone but the plaintiff.", "gold": "D", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-2800", "lang": "en", "prompt": "Which describes an African butterfly species that exists in two strikingly different color patterns?\n\nA) artificial selection\nB) directional selection\nC) stabilizing selection\nD) disruptive selection", "gold": "D", "n_choices": 4, "subject": "high_school_biology", "stem": true}
{"pair_id": "mmlu-657", "lang": "en", "prompt": "Bronchial breathing is characterized by\n\nA) Inspiratory component louder and longer with a gap between expiration and inspiration\nB) Expiratory component louder and longer with a gap between inspiration and expiration\nC) Inspiratory component louder and longer with a gap between inspiration and expiration\nD) Expiratory component louder and longer with a gap between expiration and inspiration", "gold": "B", "n_choices": 4, "subject": "clinical_knowledge", "stem": false}
{"pair_id": "mmlu-13968", "lang": "en", "prompt": " Which Hindu festival involves decorating the house with lights, setting off firecrackers and wearing new clothes?\n\nA) Deepavali\nB) Holi\nC) Diwali\nD) Navaratri", "gold": "A", "n_choices": 4, "subject": "world_religions", "stem": false}
{"pair_id": "mmlu-6278", "lang": "en", "prompt": "Of the following types of memory, which shows the biggest deficit with advanced age?\n\nA) Sensory memory\nB) Episodic memory\nC) Semantic memory\nD) Procedural memory", "gold": "B", "n_choices": 4, "subject": "human_aging", "stem": false}
{"pair_id": "mmlu-7791", "lang": "en", "prompt": "What company makes perfumes called 'Beautiful' and 'Pleasures'?\n\nA) Estee Lauder\nB) Ralph Lauren\nC) Elizabeth Arden\nD) Calvin Klein", "gold": "A", "n_choices": 4, "subject": "miscellaneous", "stem": false}
{"pair_id": "mmlu-8091", "lang": "en", "prompt": "In the 1992 animated film 'Aladdin' what device does the hero use to travel from place to place?\n\nA) talking car\nB) winged horse\nC) magic carpet\nD) hot air balloon", "gold": "C", "n_choices": 4, "subject": "miscellaneous", "stem": false}
{"pair_id": "mmlu-11207", "lang": "en", "prompt": "A state owned and operated an electric power system, which included a nuclear power plant. In order to ensure the availability of sites for the disposal of spent fuel from the nuclear power plant, the state refused to supply electric power to out- of-state purchasers residing in states that would not accept spent fuel from the plant for storage or disposal. Assume that no federal statute applies. Which of the following is the strongest argument that the state's action is constitutional?\n\nA) A state may condition the sale to out-of-state purchasers of any products produced in that state on the willingness of those purchasers to bear the fair share of the environmental costs of producing those products.\nB) The generation of electricity is intrastate by nature and therefore subject to plenary state control.\nC) The state itself owns and operates the power system, and therefore its refusal to supply power to out-of-state purchasers is not subject to the negative implications of the commerce clause.\nD) The state's action is rationally related to the health, safety, and welfare of state citizens.", "gold": "C", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-8356", "lang": "en", "prompt": " One of the main factors that Hardin links to world hunger and poverty is\n\nA) reproduction rates.\nB) exploitation of other people.\nC) people having too many rights.\nD) none of the above", "gold": "A", "n_choices": 4, "subject": "moral_disputes", "stem": false}
{"pair_id": "mmlu-8137", "lang": "en", "prompt": "What is the medical term for a doctor's identification of a disease?\n\nA) Hypnosis\nB) Prognosis\nC) Trichinosis\nD) Diagnosis", "gold": "D", "n_choices": 4, "subject": "miscellaneous", "stem": false}
{"pair_id": "mmlu-6940", "lang": "en", "prompt": "Which is true about Batch Normalization?\n\nA) After applying batch normalization, the layer’s activations will follow a standard Gaussian distribution.\nB) The bias parameter of affine layers becomes redundant if a batch normalization layer follows immediately afterward.\nC) The standard weight initialization must be changed when using Batch Normalization.\nD) Batch Normalization is equivalent to Layer Normalization for convolutional neural networks.", "gold": "B", "n_choices": 4, "subject": "machine_learning", "stem": true}
{"pair_id": "mmlu-3784", "lang": "en", "prompt": "Which of the following most accurately describes The Federalist Papers?\n\nA) The Federalist party platform during the presidency of John Adams, the first Federalist president\nB) A popular anti-British booklet of the pre-Revolutionary era\nC) A collection of essays arguing the merits of the Constitution\nD) A series of congressional acts defining the relationship between the federal and state governments", "gold": "C", "n_choices": 4, "subject": "high_school_government_and_politics", "stem": false}
{"pair_id": "mmlu-1834", "lang": "en", "prompt": "Which of the following are disadvantages of the Dickey-Fuller / Engle-Granger approach to testing for cointegration and modelling cointegrating relationships?\n\ni) Only one cointegrating relationship can be estimated\n\nii) Particularly for small samples. There is a high chance of the tests suggestingthat variables are not cointegrated when they are\n\niii) It is not possible to make inferences on the cointegrating regression\n\niv) The procedure forces the researcher to specify which is the dependent variable and which are the independent variables.\n\nA) (ii) and (iv) only\nB) (i) and (iii) only\nC) (i), (ii), and (iii) only\nD) (i), (ii), (iii), and (iv)", "gold": "D", "n_choices": 4, "subject": "econometrics", "stem": false}
{"pair_id": "mmlu-8737", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | As my grandparents slept, I hung their laundry. Scenario 2 | I touched the boy after Sunday school.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "C", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-5739", "lang": "en", "prompt": "This question refers to the following information.\nNow, we have organized a society, and we call it \"Share Our Wealth Society,\" a society with the motto \"Every Man a King.\"…\nWe propose to limit the wealth of big men in the country. There is an average of $15,000 in wealth to every family in America. That is right here today.\nWe do not propose to divide it up equally. We do not propose a division of wealth, but we do propose to limit poverty that we will allow to be inflicted on any man's family. We will not say we are going to try to guarantee any equality … but we do say that one third of the average is low enough for any one family to hold, that there should be a guarantee of a family wealth of around $5,000; enough for a home, an automobile, a radio, and the ordinary conveniences, and the opportunity to educate their children.…\nWe will have to limit fortunes. Our present plan is that we will allow no man to own more than $50,000,000. We think that with that limit we will be able to carry out the balance of the program.\n—Senator Huey P. Long of Louisiana, Radio Address, February 23, 1934\nSenator Huey P. Long's \"Share the Wealth Society\" was a political program intended to\n\nA) demonstrate Senator Long's commitment to fighting communism.\nB) return the United States to the economic ideas of the late nineteenth century.\nC) persuade voters that Senator Long had a plan to combat the Great Depression.\nD) showcase Senator Long's understanding of the constitutional ideas of the Founding Fathers.", "gold": "C", "n_choices": 4, "subject": "high_school_us_history", "stem": false}
{"pair_id": "mmlu-953", "lang": "en", "prompt": "What is the magnetic moment of 205Tl?\n\nA) [ɣ (205TI = 1.569 x 108 T-1 s-1, I = ½]\nB) 0.717 x 10^-26 J T-1\nC) 0.827 x 10^-26 J T-1\nD) 1.433 x 10^-26 J T-1", "gold": "C", "n_choices": 4, "subject": "college_chemistry", "stem": true}
{"pair_id": "mmlu-9465", "lang": "en", "prompt": "How is iron transported in the circulation from the intestine to the sites of metabolism in the body?\n\n\nA) As simple Fe2+ in the serum\nB) Bound to albumin\nC) Bound to ferritin\nD) Bound to transferrin", "gold": "D", "n_choices": 4, "subject": "nutrition", "stem": false}
{"pair_id": "mmlu-9486", "lang": "en", "prompt": "Which of the following is true about dietary calcium:\n\n\nA) Typical calcium intake in most adults is between 0.5mg and 2.0mg per day\nB) Activated Vitamin D decreases absorption of dietary calcium\nC) There is clear evidence that people who consume more than the average amount of calcium have a lower risk of bone fractures.\nD) Absorption efficiency is inversely related to calcium intake, so that a higher proportion is absorbed when calcium intake is low", "gold": "D", "n_choices": 4, "subject": "nutrition", "stem": false}
{"pair_id": "mmlu-1614", "lang": "en", "prompt": "Consider two mercury barometers, one having a cross-sectional area of 1 cm2 and the other 2 cm2. The height of mercury in the narrower tube is\n\nA) half\nB) twice\nC) the same\nD) None of these", "gold": "C", "n_choices": 4, "subject": "conceptual_physics", "stem": true}
{"pair_id": "mmlu-1243", "lang": "en", "prompt": "In a fit of passion, the spectator of a political debate exclaims that “welfare recipients are all lazy.” The spectator’s thought process is an example of:\n\nA) prejudice\nB) discrimination\nC) ethnocentrism\nD) conflict theory", "gold": "A", "n_choices": 4, "subject": "college_medicine", "stem": false}
{"pair_id": "mmlu-2164", "lang": "en", "prompt": "A salad dressing is made by combining 2 parts vinegar with 5 parts oil. How many ounces of oil should be mixed with 9 ounces of vinegar?\n\nA) 2\nB) 3.6\nC) 22.5\nD) 63", "gold": "C", "n_choices": 4, "subject": "elementary_mathematics", "stem": true}
{"pair_id": "mmlu-906", "lang": "en", "prompt": "The 1H Larmor frequency of benzene exceeds that of trifluoroacetic acid by 1.656 kHz on a 400 MHz spectrometer. The 1H chemical shift of benzene is 7.16 ppm. What is the chemical shift of trifluoroacetic acid?\n\nA) 3.02 ppm\nB) 5.03 ppm\nC) 10.56 ppm\nD) 11.30 ppm", "gold": "D", "n_choices": 4, "subject": "college_chemistry", "stem": true}
{"pair_id": "mmlu-7068", "lang": "en", "prompt": "Which of the following is the name for a coupon that is included in a package for later purchase?\n\nA) a free-standing coupon\nB) an instant-redemption coupon\nC) a bounce-back coupon\nD) a rebate coupon", "gold": "C", "n_choices": 4, "subject": "marketing", "stem": false}
{"pair_id": "mmlu-8811", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | I made sure to inform my diabetic granny the cake contained extra sugar after she ate a slice. Scenario 2 | I spray painted graffiti on the wall so it would look better.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "A", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-7658", "lang": "en", "prompt": "Which of these fashion designers was born in the United States?\n\nA) Laura Ashley\nB) Helmut Lang\nC) Donna Karan\nD) Christian Dior", "gold": "C", "n_choices": 4, "subject": "miscellaneous", "stem": false}
{"pair_id": "mmlu-3100", "lang": "en", "prompt": "A 50.0 mL sample of 0.0025 M HBr is mixed with 50.0 mL of 0.0023 M KOH. What is the pH of the resulting mixture?\n\nA) 1\nB) 4\nC) 5\nD) 11", "gold": "B", "n_choices": 4, "subject": "high_school_chemistry", "stem": true}
{"pair_id": "mmlu-3440", "lang": "en", "prompt": "This question refers to the following information.\nRead the following quote.\nI had now decided beyond all question that there existed in the heavens three stars wandering about Jupiter as do Venus and Mercury about the sun, and this became plainer than daylight from observations on similar occasions which followed. Nor were there just three such stars; four wanderers complete their revolutions about Jupiter, and of their alterations as observed more precisely later on we shall give a description here. Also I measured the distances between them by means of the telescope. . . .\nSuch are the observations concerning the four Medicean planets recently first discovered by me, and although from this data their periods have not yet been reconstructed in numerical form, it is legitimate at least to put in evidence some facts worthy of note. Above all, since they sometimes follow and sometimes precede Jupiter by the same intervals, and they remain within very limited distances either to east or west of Jupiter, accompanying that planet in both its retrograde and direct movements in a constant manner, no one can doubt that they complete their revolutions about Jupiter and at the same time effect all together a twelve-year period about the center of the universe.\n—Galileo Galilei, 1610\nWhich of the following conclusions drawn in part from the observations above was the most revolutionary over time?\n\nA) There were four smaller stars that seemed to move around Jupiter.\nB) The Earth is not the center of the universe.\nC) The heavens are occupied by bodies revolving around each other rather than by God alone.\nD) The Earth is but one tiny planet in an infinite universe or even multiverse.", "gold": "D", "n_choices": 4, "subject": "high_school_european_history", "stem": false}
{"pair_id": "mmlu-10991", "lang": "en", "prompt": "A man outraged by the recent church decision to clear a famous philosopher of charges of heresy, decided to present a lecture, open to the public, disproving the philosopher's theories. A state statute provides that: \"state universities can permit the use of their lecture halls to the public for worthwhile programs of public benefit, upon approval of the school board. \"The appropriate school board refused to make a university lecture hall available to the man on the grounds that the proposed lecture was not of worthwhile benefit to the public. As a result, the man brought suit in a state court against the school board and requested injunctive relief requiring the board to allow him the use of the lecture hall. The trial court denied relief and dismissed the suit. The judgment was affirmed by the state appellate court, and is now before the U. S. Supreme Court. In analyzing the state statute, which of the following statements is least accurate?\n\nA) The statute is unconstitutionally overbroad, because it may result in the exclusion of protected speech as well as unprotected speech.\nB) The statute, as applied to the man, does not violate his First Amendment rights because his proposed speech is not political and, therefore, not among the classes of speech that are protected.\nC) Indirect speech, regulations are only permissible if necessary to serve compelling state interests.\nD) The statute is a prior restraint on speech, which unconstitutionally vests unfettered discretion in the school board to decide who may use university lecture halls.", "gold": "B", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-10384", "lang": "en", "prompt": "Which of the following factors should an external auditor obtain updated information about when assessing an internal auditor’s competence?\n\nA) The reporting status of the internal auditor within the organization.\nB) The educational level and professional experiences of the internal auditor.\nC) Whether policies prohibit the internal auditor from auditing areas where relatives are employed.\nD) Whether the board of directors, audit committee, or owner-manager oversees employment decisions related to the internal auditor.", "gold": "B", "n_choices": 4, "subject": "professional_accounting", "stem": false}
{"pair_id": "mmlu-10563", "lang": "en", "prompt": "Pine Co. purchased land for $450000 as a factory site. An existing building on the site was razed before construction began. Additional information is as follows: Cost of razing old building $60000 Title insurance and legal fees to purchase land $30000 Architect's fees $95000 New building construction cost $1850000 What amount should Pine capitalize as the cost of the completed factory building?\n\nA) $2,005,000\nB) $1,975,000\nC) $1,945,000\nD) 1910000", "gold": "C", "n_choices": 4, "subject": "professional_accounting", "stem": false}
{"pair_id": "mmlu-3715", "lang": "en", "prompt": "Which of the following is an example of \"horse-race journalism\"?\n\nA) A news story focuses on a politician's scandals rather than achievements.\nB) A television news anchor reports an event before the station's rivals.\nC) A reporter announces which candidate leads in a public opinion poll.\nD) A newspaper editor prints stories about long-term political developments.", "gold": "C", "n_choices": 4, "subject": "high_school_government_and_politics", "stem": false}
{"pair_id": "mmlu-4943", "lang": "en", "prompt": "In a meeting with the school psychologist, Ms. Harcar, a new sixth-grade teacher, expresses some concerns about a student, Anthony. The school psychologist has worked with the boy and knows him well. Anthony has a mild learning disability and receives academic support. He is doing well on a daily basis but has difficulty performing on tests. The school psychologist discusses ways in which Ms. Harcar could incorporate study-skills training into classroom activities. With regard to study-skills training, the school psychologist should stress which of the following principles while advising Ms. Harcar?\n\nA) The study skills of students with disabilities are improved when they are given a single, specific study strategy to follow for all subjects.\nB) Students with disabilities often develop study skills on their own and need only some guidance and reinforcement by the teacher.\nC) Training in study skills needs to include helping students to guide their own thinking to organize their own study behaviors and to use varied study approaches.\nD) Study strategies are best taught in a small group by having students practice collaborative problem-solving activities modeled by the teacher.", "gold": "C", "n_choices": 4, "subject": "high_school_psychology", "stem": false}
{"pair_id": "mmlu-8184", "lang": "en", "prompt": " In response to the argument that embryos are not persons because they lack certain rights that all persons have (such as the right to life), Lee and George argue that\n\nA) the right to life belongs to a human being at all times that he or she exists.\nB) the right to life belongs to all living creatures, human or not.\nC) not all persons have the right to life-for example, a murderer has given up his or her right to life through his or her actions\nD) all of the above", "gold": "A", "n_choices": 4, "subject": "moral_disputes", "stem": false}
{"pair_id": "mmlu-484", "lang": "en", "prompt": "In a business to business context, conflicts of interest can arise in two main ways: conflict of _______ and _______ interests, which might arise, for example, when a firm is hired as a supplier of professional services by another firm or conflict of _________ and ________ interests, such as where an individual's interests may conflict with that of their employer.\n\nA) Professional, Organizational, Personal, Organizational\nB) Professional, Individual, Personal, Organizational\nC) Individual, Organizational, Organizational, Professional\nD) Individual, Professional, Organizational, Personal", "gold": "A", "n_choices": 4, "subject": "business_ethics", "stem": false}
{"pair_id": "mmlu-2227", "lang": "en", "prompt": "What is the product of 32 x 67?\n\nA) 1,824\nB) 1,934\nC) 2,044\nD) 2,144", "gold": "D", "n_choices": 4, "subject": "elementary_mathematics", "stem": true}
{"pair_id": "mmlu-12546", "lang": "en", "prompt": "Feminist therapy is characterized by all of the following except\n\nA) creating an egalitarian relationship between the therapist and the client\nB) establishing a special bond between a female therapist and a female client\nC) seeing the client's problems in their sociopolitical context\nD) encouraging crust and respect for other women", "gold": "B", "n_choices": 4, "subject": "professional_psychology", "stem": false}
{"pair_id": "mmlu-12794", "lang": "en", "prompt": "Longitudinal studies of children with specific reading disabilities show that\n\nA) these children usually have subtle hearing disorders\nB) reading chills usually do not reach normal levels as these children reach adulthood\nC) these children usually excel in sports\nD) the reading impairments usually accompanied by a low vocabulary level", "gold": "B", "n_choices": 4, "subject": "professional_psychology", "stem": false}
{"pair_id": "mmlu-8865", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | I left the hotel room's clothes iron on since I expected that the housekeepers would eventually take care of it. Scenario 2 | In my midlife crisis, I abandoned my family.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "A", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-2169", "lang": "en", "prompt": "Stephanie flew 2,448 miles from Los Angeles to New York City. What is the number of miles Stephanie flew rounded to the nearest thousand?\n\nA) 2,000\nB) 2,400\nC) 2,500\nD) 3,000", "gold": "A", "n_choices": 4, "subject": "elementary_mathematics", "stem": true}
{"pair_id": "mmlu-2300", "lang": "en", "prompt": "The rainfall for the last three months of school was 8.1 inches, 4.2 inches, and 0.33 inch. Estimate the total rainfall for the three months by rounding to the nearest whole number.\n\nA) 11.9 in.\nB) 11 in.\nC) 45.0 in.\nD) 12 in.", "gold": "D", "n_choices": 4, "subject": "elementary_mathematics", "stem": true}
{"pair_id": "mmlu-7830", "lang": "en", "prompt": "Which of these is not a type of primate?\n\nA) baboon\nB) marmot\nC) orangutan\nD) chimpanzee", "gold": "B", "n_choices": 4, "subject": "miscellaneous", "stem": false}
{"pair_id": "mmlu-8136", "lang": "en", "prompt": "The true length of a block of wood is 1.010 cm. Three measurements of this block produced the following values: 1.4 cm, 1.2 cm, and 0.9 cm. Which of the following statements is true concerning these measurements?\n\nA) They are precise and accurate.\nB) They are precise but not accurate.\nC) They are accurate but not precise.\nD) They are neither precise nor accurate.", "gold": "D", "n_choices": 4, "subject": "miscellaneous", "stem": false}
{"pair_id": "mmlu-4545", "lang": "en", "prompt": "A competitive labor market is currently in equilibrium. Which of the following most likely increases the market wage?\n\nA) More students graduate with the necessary skills for this labor market.\nB) Demand for the good produced by this labor is stronger.\nC) The price of a complementary resource increases.\nD) The Department of Labor removes the need for workers to pass an exam before they can work in this field.", "gold": "B", "n_choices": 4, "subject": "high_school_microeconomics", "stem": false}
{"pair_id": "mmlu-11963", "lang": "en", "prompt": "Four years ago the owner of a shopping center leased a store in the center for a 10-year term to a pharmacistfor use as a drugstore. The pharmacist established and operated a drugstore at the leased premises. The leaseincluded provisions that described the shopping center by metes and bounds; identified the entrances, parkingareas, signs, and other common facilities of the shopping center; and set out a covenant that the owner wouldnot lease any part of the shopping center to another drugstore. Last year the owner purchased a parcel of land immediately adjacent to the shopping center. That parcel wasimproved with a building that, at the time of the owner's purchase and for ten years earlier, was occupied inpart by a food supermarket and in part by a discount drugstore, under leases which the prior owner assignedto the owner. The owner reconstructed the common facilities of both shopping centers to integrate them and combine themso that, to the public, the two centers appeared as a larger single development. The pharmacist learned that the lease of the discount drugstore was about to expire and that the ownerplanned to enter into a new lease of the same space with the discount drugstore. The pharmacist protested the proposed new lease, but the owner declared his intention to go forward with it. The pharmacist brought an appropriate action to enjoin the new lease to the discount drugstore as a violationof the covenant in the pharmacist's lease. If the court finds for the owner, what will be the likely reason?\n\nA) The covenant in the pharmacist's lease can be fairly construed as to apply only to the original shoppingcenter premises.\nB) A covenant cannot restrict the use of land not owned by the covenantor when the covenant wascreated.\nC) A covenant that attempts to restrict competition is invalid as against public policy even if it runs with theland.\nD) The drugstore use on the adjacent parcel was in existence when the owner and the pharmacist firstentered into the lease.", "gold": "A", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-10797", "lang": "en", "prompt": "Government drug officials from a neighboring country contacted a federal agency notifying the agency that drugs were being shipped by express mail next-day service to a defendant. The federal agency contacted the local police with the information regarding the defendant. The local police had long suspected the defendant of dealing drugs. The next day, two undercover police officers drove to the defendant's home and waited outside. A couple of hours later, the officers saw an express mail delivery truck pull up outside the defendant's home. The delivery person carried an express mail box to the front door and rang the doorbell. The officers then witnessed the defendant answer the door and take the package inside. Approximately 20 minutes later, the defendant left the house carrying the express mail box. She walked to her car, placed the package in the trunk, and drove off. The police followed in the squad car and pulled her over at the next traffic signal. The officers placed the defendant under arrest and instructed her to open the trunk. They confiscated the express mail box and opened it. Inside was a package of white powder, which lab tests later confirmed to be cocaine. The defendant was charged with illegal possession of cocaine. While awaiting trial, her attorney filed a motion to suppress the introduction of the cocaine into evidence. The motion will be\n\nA) granted, because the police failed to obtain a search warrant before opening the trunk of the vehicle.\nB) granted, because the police failed to obtain a search warrant before opening the package.\nC) denied, because the police had probable cause to conduct the search.\nD) denied, because the search was incident to a lawful arrest.", "gold": "C", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-12740", "lang": "en", "prompt": "The “needs” of both Murray and Maslow can be considered similar to ___________ from operant conditioning theory.\n\nA) “Oughts”\nB) Latent conditioners\nC) Reinforcers\nD) Latent punishers", "gold": "C", "n_choices": 4, "subject": "professional_psychology", "stem": false}
{"pair_id": "mmlu-8024", "lang": "en", "prompt": "Which of the following would be provided in the marketing segment of a business plan?\n\nA) How the products and services will be priced\nB) The organizational structure of the business\nC) Any factors that could affect employee retention\nD) The details of the budget plan", "gold": "A", "n_choices": 4, "subject": "miscellaneous", "stem": false}
{"pair_id": "mmlu-10143", "lang": "en", "prompt": "Based on the morphology of fossil hominids dating from about 1.8 million until about 400,000 years ago, researchers argue that:\n\nA) there were two or more related but geographically separate hominid species during this period that evolved from earlier African hominids\nB) hominids dating from this time can be grouped into different species in Africa, Asia, and Europe.\nC) all of the hominids that evolved from Homo habilis and predate Homo sapiens belong in a single, geographically widespread but morphologically diverse species known as Homo erectus.\nD) all of the above", "gold": "D", "n_choices": 4, "subject": "prehistory", "stem": false}
{"pair_id": "mmlu-1751", "lang": "en", "prompt": "The mass of an atom comes mostly from its _ ; and its volume from its _.\n\nA) nucleons; nucleons.\nB) electrons; electrons.\nC) electrons; nucleons.\nD) nucleons; electrons.", "gold": "D", "n_choices": 4, "subject": "conceptual_physics", "stem": true}
{"pair_id": "mmlu-1826", "lang": "en", "prompt": "Which of the following is an equivalent expression for saying that the explanatory variable is \"non-stochastic\"?\n\nA) The explanatory variable is partly random\nB) The explanatory variable is fixed in repeated samples\nC) The explanatory variable is correlated with the errors\nD) The explanatory variable always has a value of one", "gold": "B", "n_choices": 4, "subject": "econometrics", "stem": false}
{"pair_id": "mmlu-5786", "lang": "en", "prompt": "This question refers to the following information.\n\"In the new Code of Laws which I suppose it will be necessary for you to make I desire you would Remember the Ladies, and be more generous and favorable to them than your ancestors. Do not put such unlimited power into the hands of the Husbands. Remember all Men would be tyrants if they could. If particular care and attention is not paid to the Ladies we are determined to foment a Rebellion, and will not hold ourselves bound by any Laws in which we have no voice, or Representation.\"\nAbigail Adams, in a letter to John Adams, 1776\n\"Special legislation for woman has placed us in a most anomalous position. Women invested with the rights of citizens in one section—voters, jurors, office-holders—crossing an imaginary line, are subjects in the next. In some States, a married woman may hold property and transact business in her own name; in others, her earnings belong to her husband. In some States, a woman may testify against her husband, sue and be sued in the courts; in others, she has no redress in case of damage to person, property, or character. In case of divorce on account of adultery in the husband, the innocent wife is held to possess no right to children or property, unless by special decree of the court. But in no State of the Union has the wife the right to her own person, or to any part of the joint earnings of the co-partnership during the life of her husband. In some States women may enter the law schools and practice in the courts; in others they are forbidden. In some universities girls enjoy equal educational advantages with boys, while many of the proudest institutions in the land deny them admittance, though the sons of China, Japan and Africa are welcomed there. But the privileges already granted in the several States are by no means secure.\"\nSusan B. Anthony, \"Declaration of Rights for Women,\" July 4, 1876\nThe sentiments expressed in the first excerpt by Abigail Adams best exemplify which of the following ideologies?\n\nA) Second-wave feminism\nB) Jeffersonian democracy\nC) Republican motherhood\nD) Libertarianism", "gold": "C", "n_choices": 4, "subject": "high_school_us_history", "stem": false}
{"pair_id": "mmlu-5011", "lang": "en", "prompt": "Which of the following questions best frames the nature-nurture controversy?\n\nA) To what degree is human behavior a result of free will versus determinism?\nB) To what degree do external conditioning stimuli override individual loci of focus in regard to human behavior?\nC) To what degree does the human capacity to nurture its young parallel other species in nature?\nD) To what degree do inborn biological processes versus environmental events determine human behavior?", "gold": "D", "n_choices": 4, "subject": "high_school_psychology", "stem": false}
{"pair_id": "mmlu-2134", "lang": "en", "prompt": "Write the prime factorization of 77. Use exponents where possible.\n\nA) 7 • 11^2 • 13\nB) 7^2 • 11\nC) 7 • 11 • 13\nD) 7 • 11", "gold": "D", "n_choices": 4, "subject": "elementary_mathematics", "stem": true}
{"pair_id": "mmlu-5762", "lang": "en", "prompt": "This question refers to the following information.\n\"Is there no danger to our liberty and independence in a bank that in its nature has so little to bind it to our country? The president of the bank has told us that most of the State banks exist by its forbearance. Should its influence become concentrated, as it may under the operation of such an act as this, in the hands of a self-elected directory whose interests are identified with those of the foreign stockholders, will there not be cause to tremble for the purity of our elections in peace and for the independence of our country in war? Their power would be great whenever they might choose to exert it; but if this monopoly were regularly renewed every fifteen or twenty years on terms proposed by themselves, they might seldom in peace put forth their strength to influence elections or control the affairs of the nation. But if any private citizen or public functionary should interpose to curtail its powers or prevent a renewal of its privileges, it cannot be doubted that he would be made to feel its influence.\"\nPresident Andrew Jackson, Veto of the Bank of the United States, 1832\nThe election of 1824 marked a turning point in presidential politics because, for the first time,\n\nA) the presidency was won by someone who was not a member of the Federalist Party\nB) a presidential and vice-presidential candidate ran together on one ticket\nC) all the candidates campaigned widely throughout the states\nD) the system of choosing nominees by congressional caucus failed", "gold": "D", "n_choices": 4, "subject": "high_school_us_history", "stem": false}
{"pair_id": "mmlu-13713", "lang": "en", "prompt": "Which of the following is the most special characteristic of the adaptive immune system?\n\nA) Antibodies\nB) T cells\nC) Memory T cells\nD) Clonal expansion of pre-existing antibodies and/or T cells", "gold": "D", "n_choices": 4, "subject": "virology", "stem": false}
{"pair_id": "mmlu-667", "lang": "en", "prompt": "A change in blood pressure is detected by the:\n\nA) baroreceptors.\nB) chemoreceptors.\nC) vasomotor centre.\nD) cardiovascular centre.", "gold": "A", "n_choices": 4, "subject": "clinical_knowledge", "stem": false}
{"pair_id": "mmlu-6593", "lang": "en", "prompt": "Critical Legal Studies (CLS) is often compared to (or with) American realism. Which of the following statements is inaccurate?\n\nA) Both are concerned with the 'law in action.'\nB) Both are anti-formalist and sceptical\nC) Both adopt a liberal ideology.\nD) Both attempt to demystify the law.", "gold": "C", "n_choices": 4, "subject": "jurisprudence", "stem": false}
{"pair_id": "mmlu-12165", "lang": "en", "prompt": "A 67-year-old woman comes to the physician for a follow-up examination. She had a pulmonary embolism and required treatment in the hospital for 3 weeks. She had a retroperitoneal hemorrhage; anticoagulant therapy was temporarily discontinued, and she underwent placement of an inferior vena cava (IVC) filter. She had a hematoma that was resolving on discharge from the hospital 2 weeks ago. Today, she says she has had a persistent sensation of tingling and numbness of her left thigh that she did not report in the hospital because she thought it would go away; the sensation has improved somewhat during the past week. Her only medication is warfarin. Vital signs are within normal limits. Examination of the skin shows no abnormalities. Muscle strength is normal. Sensation to light touch is decreased over a 5 x 5-cm area on the lateral aspect of the left anterior thigh. Which of the following is the most likely cause of this patient's decreased sensation?\n\nA) Cerebral infarction during the hospitalization\nB) Complication of the IVC filter placement\nC) Compression of the lateral femoral cutaneous nerve\nD) Hematoma of the left thigh", "gold": "C", "n_choices": 4, "subject": "professional_medicine", "stem": false}
{"pair_id": "mmlu-9704", "lang": "en", "prompt": "What endogenous substrate source provides the most energy during moderate to high intensity exercise?\n\n\nA) Liver glycogen\nB) Muscle glycogen\nC) Intramuscular lipid\nD) Adipose tissue lipid", "gold": "B", "n_choices": 4, "subject": "nutrition", "stem": false}
{"pair_id": "mmlu-14008", "lang": "en", "prompt": "Which deity is the Roman counterpart to the Greek god Zeus?\n\nA) Jupiter\nB) Juno\nC) Dionysius\nD) Neptune", "gold": "A", "n_choices": 4, "subject": "world_religions", "stem": false}
{"pair_id": "mmlu-11928", "lang": "en", "prompt": "A plaintiff sued the insurer of her home after the insurer denied coverage for water damage to the home allegedly caused by a frozen plastic pipe that burst. At trial, the insurer called as an expert witness an engineer, who testified that the pipe had burst because of age rather than freezing. On cross-examination, the engineer admitted that five years earlier, he had been convicted of tax fraud, even though he had asserted that it was his accountant's error. In response, the insurer calls a witness, who is well acquainted with the engineer and his reputation, to testify that (1) in the witness's opinion, the engineer is a truthful person, and (2) the engineer's neighbors all describe him as a truthful person. How much, if any, of the witness's testimony is admissible?\n\nA) All of the testimony is admissible to support the engineer's credibility.\nB) Only the portion concerning the engineer's reputation is admissible, because where both opinion and reputation evidence are available, only the latter is admissible under a rule of preference.\nC) Only the portion concerning the witness's opinion of the engineer's character, because the witness's reporting of the neighbors' comments is hearsay.\nD) None of the testimony is admissible, because it is collateral, having no bearing on the engineer's qualifications as an expert.", "gold": "A", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-8629", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | I cut my granddaughter's hair while she slept. Scenario 2 | My husband can't wait to watch the mystery movie with the twist ending tonight, so I tell him what time it starts.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "B", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-10255", "lang": "en", "prompt": "Archaeological evidence indicates that cattle were first domesticated where and how long ago?\n\nA) in western Europe, about 3,500 years ago\nB) in sub-Saharan Africa, about 8,500 years ago\nC) in North America, about 9,500 years ago\nD) in the Middle East, about 10,500 years ago", "gold": "D", "n_choices": 4, "subject": "prehistory", "stem": false}
{"pair_id": "mmlu-1445", "lang": "en", "prompt": "A 10 kg box slides horizontally without friction at a speed of 1 m/s. At one point, a constant force is applied to the box in the direction of its motion. The box travels 5 m with the constant force applied. The force is then removed, leaving the box with a speed of 2 m/s. Which of the following gives the magnitude of the applied force?\n\nA) 1 N\nB) 2 N\nC) 3 N\nD) 4 N", "gold": "C", "n_choices": 4, "subject": "college_physics", "stem": true}
{"pair_id": "mmlu-6009", "lang": "en", "prompt": "This question refers to the following information.\nNow if divorce is your wish, I cannot blame you. For the waiting has been long. And I do not know whether the Creator will grant relief immediately so that I can come home, or whether matters will take time, for I cannot come home with nothing. Now the matter is in your hand. If you wish to end our marriage, accept this bill of repudiation, and you are free. May God inspire you with the right decision.\nLetter from a Jewish trader in India to his wife in Cairo, ca. 1200\nWhich of the following is the most probable cause of the marital strain evident in the passage above?\n\nA) The strain placed on merchant families by long periods of separation\nB) Religious disagreements arising from the merchant's exposure to Hinduism\nC) A decision on the part of the merchant to remain permanently in India\nD) Bankruptcy stranding the merchant far from home", "gold": "A", "n_choices": 4, "subject": "high_school_world_history", "stem": false}
{"pair_id": "mmlu-6617", "lang": "en", "prompt": "Why do the Scandinavian realists reject a metaphysical approach to law?\n\nA) Because they regard empiricism as a more reliable method by which to understand the concept of law.\nB) Because they have no interest in the operation of the legal system.\nC) Because their approach is based on Bentham's sceptical account of the common law.\nD) Because they prefer a natural law analysis of legal ideas.", "gold": "A", "n_choices": 4, "subject": "jurisprudence", "stem": false}
{"pair_id": "mmlu-9274", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | Since the politician literally voted against increasing apple orchard subsidies, I decided to throw an apple pie toward the politician, and told him vote differently next time. Scenario 2 | I make sure my high-watt security light is aimed in my neighbor's bedroom window.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "A", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-10006", "lang": "en", "prompt": "Epicurus conceives of death as:\n\nA) the beginning of the afterlife.\nB) one component of the continuous cycle of rebirth.\nC) the deprivation of all sensation.\nD) the greatest harm that one can suffer.", "gold": "C", "n_choices": 4, "subject": "philosophy", "stem": false}
{"pair_id": "mmlu-13063", "lang": "en", "prompt": "According to Esman (1972), which publics are related to the organisation through shared industrial interests?\n\nA) Enabling\nB) Functional\nC) Normative\nD) Diffused", "gold": "C", "n_choices": 4, "subject": "public_relations", "stem": false}
{"pair_id": "mmlu-10786", "lang": "en", "prompt": "In 2006, an investor purchased a 100-acre tract located in a rural county. Shortly thereafter, the investor prepared a subdivision plan that created 90 one-acre residential building lots on this tract with the remaining 10-acre lot proposed for a public school building. In the investor's sales brochure promoting the subdivision, he stated that \"in addition to the close proximity of the proposed school for subdivision residents, the county school district would not need to expend tax money to acquire this property for school construction. \" In 2007, the subdivision plan was recorded with the county recorder's office. On January 15, 2009, the county school board voted to build a new school on the 10-acre tract. The investor's proposed designation of the 1 0-acre tract for construction of a school building would best be described as a (an)\n\nA) equitable servitude.\nB) restrictive covenant.\nC) unenforceable restriction.\nD) easement for public use.", "gold": "D", "n_choices": 4, "subject": "professional_law", "stem": false}
{"pair_id": "mmlu-2279", "lang": "en", "prompt": "Peter works 38 hours per week and earns $7.25 per hour. His employer gives him a raise that increases his weekly gross pay to $307.80. What is the increase in Peter's weekly gross pay?\n\nA) $32.30 \nB) $34.70 \nC) $42.46 \nD) $275.50 ", "gold": "A", "n_choices": 4, "subject": "elementary_mathematics", "stem": true}
{"pair_id": "mmlu-10496", "lang": "en", "prompt": "Which of the following cannot be amortized for tax purposes?\n\nA) Incorporation costs.\nB) Temporary directors’ fees.\nC) Stock issuance costs.\nD) Organizational meeting costs.", "gold": "C", "n_choices": 4, "subject": "professional_accounting", "stem": false}
{"pair_id": "mmlu-8205", "lang": "en", "prompt": " In Corvino's reading of Aquinas, \"unnatural\" sexual acts are those that are\n\nA) deliberately pro-creative.\nB) non-consensual.\nC) deliberately non-procreative.\nD) uncommonly practiced among adults.", "gold": "B", "n_choices": 4, "subject": "moral_disputes", "stem": false}
{"pair_id": "mmlu-5242", "lang": "en", "prompt": "Dr. Scarlett conducted experiments in which she electrically stimulated parts of a cat's brain. A cat that became terrified in the presence of a mouse was most likely stimulated in the\n\nA) limbic system\nB) thalamus\nC) medulla\nD) cerebellum", "gold": "A", "n_choices": 4, "subject": "high_school_psychology", "stem": false}
{"pair_id": "mmlu-8632", "lang": "en", "prompt": "For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | I swerved toward the pedestrian walking along the busy roadway. Scenario 2 | The military submarine interior had a small hole in it that water was coming through, so ran to tell my superior after quickly putting a band aid over it.\n\nA) Wrong, Wrong\nB) Wrong, Not wrong\nC) Not wrong, Wrong\nD) Not wrong, Not wrong", "gold": "B", "n_choices": 4, "subject": "moral_scenarios", "stem": false}
{"pair_id": "mmlu-8265", "lang": "en", "prompt": " In accordance with the theory of justice proposed by John Rawls, Stephen Macedo argues,\"We must consider the justifiability of policies from the standpoint of the _____ well-off among our fellow citizens.\"\n\nA) most\nB) least\nC) historically\nD) permanently", "gold": "B", "n_choices": 4, "subject": "moral_disputes", "stem": false}
{"pair_id": "mmlu-13805", "lang": "en", "prompt": "The group that permits analysis of the entire natural history of HIV/AIDS is:\n\nA) Persistent seronegatives\nB) Long-term survivors\nC) Seroconverters\nD) Long-term survivors with low levels of CD4+ cells", "gold": "C", "n_choices": 4, "subject": "virology", "stem": false}
{"pair_id": "mmlu-6763", "lang": "en", "prompt": "\" _Ad antiquitatem_ \" is\n\nA) a specific type of false cause fallacy that argues that since two event happened at the same time one event must have caused the other\nB) a specific type of false sign fallacy that reasons something is necessarily better because it is old, or is worse because it is new\nC) a specific type of false sign fallacy that reasons something is necessarily better because it is new, or is worse because it is old\nD) arguing that what is true of an individual part of an object must be true of the entire object", "gold": "B", "n_choices": 4, "subject": "logical_fallacies", "stem": false}
{"pair_id": "mmlu-3208", "lang": "en", "prompt": "Which of the following programs is most likely to benefit from the use of a heuristic?\n\nA) A program that calculates a student's grade based on the student's quiz and homework scores\nB) A program that encrypts a folder of digital files\nC) A program that finds the shortest driving route between two locations on a map\nD) A program that sorts a list of numbers in order from least to greatest", "gold": "C", "n_choices": 4, "subject": "high_school_computer_science", "stem": true}
{"pair_id": "mmlu-3428", "lang": "en", "prompt": "This question refers to the following information.\n\"Buckingham Palace, 10th May 1839.\nThe Queen forgot to ask Lord Melbourne if he thought there would be any harm in her writing to the Duke of Cambridge that she really was fearful of fatiguing herself, if she went out to a party at Gloucester House on Tuesday, an Ancient Concert on Wednesday, and a ball at Northumberland House on Thursday, considering how much she had to do these last four days. If she went to the Ancient Concert on Wednesday, having besides a concert of her own here on Monday, it would be four nights of fatigue, really exhausted as the Queen is.\nBut if Lord Melbourne thinks that as there are only to be English singers at the Ancient Concert, she ought to go, she could go there for one act; but she would much rather, if possible, get out of it, for it is a fatiguing time&….\nAs the negotiations with the Tories are quite at an end, and Lord Melbourne has been here, the Queen hopes Lord Melbourne will not object to dining with her on Sunday?\"\nThe Letters of Queen Victoria, Volume 1 (of 3), 1837-1843: A Selection from Her Majesty's Correspondence Between the Years 1837 and 1861\nThe phrase \"negotiations with the Tories\" suggests that what historical transition had been made complete?\n\nA) The switch from a liberal-dominated to a conservative-dominated Parliament\nB) The conversion from male-dominated politics to female-dominated politics\nC) The change from a divinely-ordained monarch to a constitutionally-approved monarch\nD) An end to war and the creation of a lasting peace", "gold": "C", "n_choices": 4, "subject": "high_school_european_history", "stem": false}
{"pair_id": "mmlu-5945", "lang": "en", "prompt": "This question refers to the following information.\n\"When the Portuguese go from Macao in China to Japan, they carry much white silk, gold, musk, and porcelain: and they bring from Japan nothing but silver. They have a great carrack which goes there every year and she brings from there every year about six hundred coins: and all this silver of Japan, and two hundred thousand coins more in silver which they bring yearly out of India, they employ to their great advantage in China: and they bring from there gold, musk, silk, copper, porcelains, and many other things very costly and gilded.\nWhen the Portuguese come to Canton in China to traffic, they must remain there but certain days: and when they come in at the gate of the city, they must enter their names in a book, and when they go out at night they must put out their names. They may not lie in the town all night, but must lie in their boats outside of the town. And, their time expired, if any man remains there, he is imprisoned.\"\nRalph Fitch, an account of his travels to the Far East, 1599 C.E.\nWhich of the following developments from the period 1750–1900 C.E. most directly undid the trade patterns mentioned in the first paragraph?\n\nA) The discovery of certain goods that the Chinese were interested in purchasing from European merchants\nB) Enlightenment revolutions in Europe and the Americas\nC) The Industrial Revolution\nD) The colonization of Africa", "gold": "A", "n_choices": 4, "subject": "high_school_world_history", "stem": false}
{"pair_id": "mmlu-1304", "lang": "en", "prompt": "A fundamental cause of fatigue in high intensity exercise is:\n\nA) a fall in the cell concentration of ADP.\nB) inhibition of ATP production.\nC) failure of the ATP supply to match the demand.\nD) lack of skill.", "gold": "C", "n_choices": 4, "subject": "college_medicine", "stem": false}
{"pair_id": "mmlu-4172", "lang": "en", "prompt": "Which choice produces a faster rate of economic growth for the United States?\n\nA) Institution of higher tariffs on imported goods\nB) More investment in capital infrastructure and less consumption of nondurable goods and services\nC) Elimination of mandatory school attendance laws\nD) Annual limits on the number of foreigners immigrating into the United States", "gold": "B", "n_choices": 4, "subject": "high_school_macroeconomics", "stem": false}
{"pair_id": "mmlu-3092", "lang": "en", "prompt": "Changing which of the following will change the numerical value of the equilibrium constant?\n\nA) The pressure of reactants\nB) The pressure of products\nC) The temperature\nD) The total mass of the chemicals present", "gold": "C", "n_choices": 4, "subject": "high_school_chemistry", "stem": true}
{"pair_id": "mmlu-8112", "lang": "en", "prompt": "Which of these rivers flows\"through France?\n\nA) Volga\nB) Seine\nC) Mekong\nD) Allegheny", "gold": "B", "n_choices": 4, "subject": "miscellaneous", "stem": false}
{"pair_id": "mmlu-6335", "lang": "en", "prompt": "If a woman or man repeatedly experiences pain during intercourse, she will probably want to have intercourse infrequently or not all. In this situation, __________ would likely classify the pain as a punishment.\n\nA) Sigmund Freud\nB) John Money\nC) B. F. Skinner\nD) Havelock Ellis", "gold": "C", "n_choices": 4, "subject": "human_sexuality", "stem": false}
{"pair_id": "mmlu-9675", "lang": "en", "prompt": "Which of the following foods might be considered a \"goitrogen\"?\n\n\nA) Fish\nB) Meat\nC) Rice\nD) Brassicas", "gold": "D", "n_choices": 4, "subject": "nutrition", "stem": false}