“AI and Jewish Law: Seeing How ChatGPT 4.0 Looks at a Novel Issue – Part II” by Michael J. Broyde

“Artificial Intelligence” from Pixabay (License)

---

This essay is the second installment of a three-part series by the author on this topic. Read the first and third installments in this series.

---

I suspect that this field of AI reasoning is a mix between the ability of the engine to process and the available data it has at its “fingertips.” Our topic provides a double challenge:  ChatGPT 4.0 has only a small collection of data on halacha in Hebrew (the halachic  lingua franca), and this particular case has no direct precedent. Thus, it is a good test of its ability to engage in legal reasoning in rabbinics without reference to what others have said about this problem. I have generally found in my experience that machine learning and computation is most powerful when the task is a mix of database searching and reasoning. For example, I am a mediocre chess player at best, and by now all chess programs easily beat me and everyone else. Yet, when I play transposed chess online, I do much better than in “real” chess against a computer because the computer in transposed chess has no database of moves to use. This case has a general database, but little precedent and one cannot find out what others think.  As the database grows and its reasoning improves, it will become much more powerful. 

ChatGPT is widely reported to have been trained on 60 billion words. The largest rabbinic database, Otzar HaHochma, is less than 10 billion words. ChatGPT also trains on multiple languages simultaneously. It is unclear how much one language is helping the other in the training process, but it’s possible that rabbinic Hebrew learns from generally large language models [LLM] in other languages or non-rabbinic Hebrew. Thus, we have no idea if the skills learned by LLM in legal thinking will transfer to rabbinic thinking. (Thank you to Shaun Regenbaum of Hebrew University and Dr. Moshe Koppel for helping me formulate some of this).

Because ChatGPT 4.0 does have access to the specific sources that I uploaded, it had enough data to form both an argument and a counterargument and also invest in thinking about extra-judicial reasoning (what this article calls migdar milta) and the role of decisors (poskim in Hebrew) n making decisions not directly supported by Jewish law sources. It did a fine job both reasoning within the specific sources and outside of them, although it could, of course,  have done better. My friend, Steven S. Weiner of U of P., added the following caveat, which is important: 

I wouldn’t underestimate how much heavy lifting you did by manually selecting the relevant sources that you wanted the system to focus on. It’s true, as you point out, that none of these sources dealt directly with the question of SSM duchening [Blessing], but they did address closely related questions. Handing the system a relatively small set of sources to focus on is a big assist. It makes the task more like reading comprehension — which ChatGPT excels at — and less like the hard work you did when you confronted the question yourself and had to analyze it in order to figure out which sources were most pertinent. There are zillions of sources about duchening and qualifications, and zillions of others about homosexuality. If ChatGPT had ingested all of those without your help in prioritizing a few, it might have generated a bunch of much less promising arguments.” (emphasis added)  Further experimentation is needed to test that caveat and better assess how well ChatGPT would actually perform on a task of this nature if trained on a more comprehensive database of Jewish law responsa, but without expert human input as to which particular responsa should be focused on for a given question.

This is, of course, true. But it is also clear that ChatGPT processed, analyzed, and added to these sources sources. Knowing what sources to look at is a huge help in every aspect of rabbinics. The edge of having the sources highlighted is certainly true, but perhaps will diminish over time as AI develops greater sources in its database. Let me add that I gave it the sources in Hebrew, suggesting that ChatGPT also has  fine Hebrew skills.The core of the question might be whether Jewish law is different from any other legal field when it comes to ChatGPT and AI more generally. Jewish law has many different parts. The first is a logical data-driven inference from information – if we know in case “X”, “A” is the halacha, how does the rule of “A” apply to case “Y” – what is the rule of decision? The second is a weighting of the sources to determine what is the normative Jewish law. Given that not all sources are weighed equally (and not even weighed identically in different communities), that is not a purely logical determination. The third is made up of an even more subjective idea, what this article calls migdar milta, the idea that a community can prohibit that which Jewish law permits when the needs of the times are such. Jewish law is somewhat different from American law in that this kind of activity is not done by the courts in America, but by the legislative branch. However in the Jewish tradition, these two functions reside in the same place and are done by the same people. 

Finally, Jewish law has a groundbreaking function whereby the ancient precedent is examined anew and renewed with innovative reasoning from the ancient sources.  This process, commonly referred to as chiddish (novel insights), reflects the evolving understanding of the sources of halacha – it also represents a source of growth and regeneration.

Let me make a comparative observation, about Jewish law and American law in connection to AI reasoning. The reasoning of AI is unlikely to take the place of district court judges – those trial court judges who make factual determination based on the demeanor of the witnesses and so many more visible and subjective clues. So too, they are unlikely to replace U.S. Supreme Court judges, since much of what the justices do is decide when they ought to change the rules and replace one rule of interpretation with another. Delegating any legal decision to AI is no different than delegating the role of Congress to AI. On the other hand, the strengths of AI reasoning could take the place of U.S. Court of Appeals judges, who are rigidly bound by precedent, functionally interchangeable with each other, randomly assigned to cases, sit in panels, and do almost all their work based on a written record. These five characteristics are more easily supplanted by AI given the current state of technology that either policy decisions about the law ought to be or credibility determinations. In contrast, Jewish law has many scholars who function like district court judges by determining facts and applications, and many scholars who think of themselves like Supreme Court justices by being able to change the rules as applied, but nothing like the Courts of Appeals judges in the United States, who review decisions on the written record of other rabbis and issue binding decisions within a geographical area. 

AI and Determining Normative Jewish Law (Psak)

A grand view of the bigger picture is taken by the well known and modern Jewish law authority Rabbi David Cohen of Gvul Yavetz. In a podcast on AI, he argues that there is a critical, two-fold central aspect of horah – issuing a decision of Jewish law. One must take religious responsibility for their rulings, and the answer provided must be correct for the person who asked the question. This view, which I think is a correct read of normative Jewish law, is driven by the flexibility of Jewish law to provide the “right” answer in context. 

Since many minority opinions are possibly correct in a time of need (דכדאי הוא דעת יחיד לסמוך עליו בשעת הדחק ), views that are minority opinions can be relied on in even more diverse situation. It is the religious job of the posek to provide the “right” answer in each particular context. That is a hard job for any artificial intelligence machine as it requires more than mere information about the sources.  Sometimes, people require more lenient or more strict answers than a straight read of the sources might direct. 

Indeed, at the time of this writing, sometimes AI is just simplistic and wrong. For example, I logged into RavGPT,  which says it specializes in halacha, and asked it “Can a man in a same-sex marriage who is a kohen duchen?” and it answered:

A man in a same-sex marriage who is a kohen would not be permitted to perform the priestly blessing (duchan) according to traditional Jewish law. The role of a kohen carries specific requirements and restrictions, including those related to marriage. A kohen is expected to maintain a higher level of sanctity and adhere to certain marital restrictions, such as not marrying a divorcee or a convert.  While the topic of same-sex marriage is beyond my jurisdiction, it is important to note that traditional Jewish law does not recognize same-sex marriages. Consequently, a kohen in a same-sex marriage would not be fulfilling the expected sanctity and marital restrictions associated with his role, and therefore would not be permitted to perform the priestly blessing (duchan).

This is a well written but just wrong answer, as I have noted in a longer article.  

These websites are not unique to Judaism or halacha. There are also websites for Sharia law done by AI (thank you to Rabbi Jonathan Ziring for noting this site). Furthermore, sometimes poskim consider the motives of the questioners, or the impact on society;  many poskim consider the psychological and emotional impact of their rulings, which are simply beyond what AI can consider in the foreseeable future.

Of course, the more one believes that most legal disputes of Jewish law can be resolved internally and textually by reference to the Talmud’s accepted logical rules, the more useful and powerful one will view AI in mastering texts and applying logical rules. Of course, this first school of thought concedes that matters of ritual custom, theology, matters of no practical import, and other secondary matters cannot be resolved this way, but matters of functional Jewish law generally can. We can, this school of thought claims, resolve disputes of the previous generations by dint of logic. Without question, universally accepted rules within traditional Jewish jurisprudence would make AI all the more effective in its analysis and resolution of Jewish legal issue.

AI’s method of “thinking” might even be valuable according to a second school of thought, which essentially adopts the same outcome – that all disputes can be conclusively resolved – but does so from a very different starting point. This school of thought posits that almost no disputes can actually be resolved by reference to first-tier rules of Jewish jurisprudence , since it is exceedingly rare,  at least among giants of Jewish law, that one view is demonstrably incorrect. Rather, there is  a collection of second-tier rules which are binding and which resolve disputes, such as “follow the current majority” or “be strict on matters of Biblical law,” along with  many more which must be followed. 

A related school of thought argues that the first or second school is functionally correct as to how people and communities ought to function regularly, and this third school is the power of custom (מנהג) in ritual, commercial, and family law matters, since, in the real world, law needs consistency of outcomes and certainty of results. These rules are binding because they are followed, not the other way around. According to this school of thought,  Jewish legal theory has evolved over time, and it allows Jewish law to be considered a legal system, rather than merely a personal ethical system.

However, the fourth and final school of thought mostly rejects this approach, both as a matter of legal theory and also as a matter of actual practice, at least in a time of need, whether personal or communal. This school of thought proposes three basic ideas. First, very few opinions are ever truly and completely rejected as definitively wrong. Second, in a time of need, many opinions can be relied on unless it is one of those few and rare opinions that is distinctly considered wrong. Third, this matter is left to the judgment of lone Jewish law authorities who may decide for themselves and their followers what the rules ought to be. There is no real hierarchy at all. In this model, Jewish law is much more open, and the customs mentioned in the third school above are social and not jurisprudential. The contrast between Jewish law and American law here is complete: minority opinions in American law are just for study but are of no legal value at all. In a recent podcast, Rabbi Hershel Schachter proposed that a Jewish law AI model could be taught to both ask and answer the right questions about need and then would assume the religious identity (my phrase) of the person or rabbis programming. The heart of this argument is verified by anyone who uses AI, where the questioner locates themselves on the religious spectrum when asking the question. So when I asked ChatGPT 4.0 “I am a member of the Pupov Chasidic community. Do we eat gabrakz on Pesach?” should (and does) get a different answer than “I am a member of the Edot Hamizrach community. Do we eat gabrakz on Pesach?” 

Conclusion

Anyone connected to Jewish law in practice and not just as an academic discipline knows that Jewish law is not a pure, logical science and that the final approach, Rabbi Cohen’s, is viewed as correct in practice. Minority opinions can and are resurrected all the time in cases of need and in situations of complexity. This is actually the heart of the approach to Jewish law taken by the Mishnah Berurah in his work. For more on this, see my work, with Ira Bedzow, The Codification of Jewish law and an Introduction to the Jurisprudence of the Mishna Berura (pages 27 to 61). In practical terms, this helps us understand why Jewish law is much more open and less structurally jurisprudential than American law, and the contrast between Jewish law and American law is sharp. There is, for example, neither a highest court nor even an appellate court in Jewish law and forum shopping is normative. Furthermore, while minority opinions in American law are just for study but are of no legal value at all, in Jewish law, minority opinions are often resurrected in time of need and in other cases by rabbinic authorities. 

Moreover, we have not considered to this point a subset of innovative legal rulings that are critical to the functioning of Jewish law in the modern world. Thus far, we have examined questions for which there are opinions already developed or logically derived from other sources — already pieces of the halachic pie. Those opinions have to be found, examined, and weighed and AI will learn, I suspect, to do this very well as it makes significant progress towards formalistic legal reasoning. But there are questions that arise for which there really is no literature. Formulating answers to these novel questions takes deep novel readings of Talmudic texts and their medieval commentaries. Principles are discovered by noting subtlety and nuance in competing texts. The eureka moments that are necessary for Jewish law to develop come only through creativity and insight. That is not what we ask of AI in its current iterations, and we have no way of knowing whether AI will ever be able to deliver in the area of Jewish law.

All of this just reminds us that ChatGPT 4.0 is pretty good at what it seems to be doing, which is reasoning within a text, even in Jewish law. One suspects it will only get better at purely text-driven reasoning and that the Jewish law database will increase.  Indeed, it would come as no surprise to me if over the next decade we encounter AI of Jewish law tools that substantially augment and enhance human authorities’ deciding what is the proper Jewish law rule  without replacing them. For example, augmented AI “is a subsection of AI machine learning developed to enhance human intelligence rather than operate independently of or outright replace it. It’s designed to do so by improving human decision-making and, by extension, actions taken in response to improved decisions.” (Thank you to Dr. Moshe Koppel who noted this idea to me by email.) 

So, in dealing with the subject of whether excellent AI will ever be able to issue psak, the real question is “what is psak?” To those who consider psak to be the application of cold and dry rules to clear and established facts, the answer will eventually be “yes,” even as such is not the case yet. To those who have a broader understand of what it means to be a posek, and what exactly psak is, AI has much farther to go to be able to master the sources; it will have to develop a way to understand the needs of the questioner, and the implications to society of answers to questions, and so much more than merely the sources. Will ChatGPT be able, then, to issue psak? To those who prefer a definitive answer,  I say “You define your terms and you take your choice.”

---

Dr. Broyde would like to thank everyone who read and commented on the larger project referenced in the essay. This three-part essay series is adapted from the appendix that appears with his forthcoming article, “May a Kohen in a Same Sex Relationship Duchen.” This series is conceptually unrelated to the question in the main article and is not part of the article and is not intended to be about normative Jewish law.

---

---

Recommended Citation

Broyde, Michael J. “AI and Jewish Law: Seeing How ChatGPT 4.0 Looks at a Novel Issue – Part II.” Canopy Forum, October 3, 2023. https://canopyforum.org/2023/10/03/ai-and-jewish-law-seeing-how-chatgpt-4-0-looks-at-a-novel-issue-part-ii/.