A troubling pattern has emerged in courtrooms across the United States: lawyers increasingly relying on artificial intelligence to draft legal briefs are inadvertently—or perhaps negligently—submitting documents riddled with fabricated quotes, invented case citations, and baseless legal arguments. The problem reached critical mass when a US judge discovered that a lawyer had used Claude, a sophisticated AI chatbot developed by Anthropic, to compose court filings that contained entirely fabricated quotations attributed to legitimate court cases. The attorney subsequently admitted to leaning on the chatbot to generate the brief, igniting a broader reckoning within the judiciary about the proper role of machine learning in legal practice.

The core difficulty stems from a fundamental limitation of large language models: these systems excel at generating convincing text that mimics the structure and tone of legitimate legal writing, yet they lack any genuine understanding of factual accuracy or legal precedent. When pressed to produce citations or quotes under time pressure, Claude and similar platforms often construct plausible-sounding references that have no basis in actual court decisions or legal documents. This phenomenon, known as "hallucination" in artificial intelligence research, poses an acute risk in legal contexts where accuracy is not merely a matter of style but a foundation of justice itself.

The implications for Southeast Asia's legal sector are substantial. Malaysia, Singapore, and other regional economies are increasingly integrating digital technologies into legal practice, and many law firms and in-house counsel departments have begun experimenting with AI tools to streamline document review, contract analysis, and brief preparation. Yet the cautionary tale emerging from American courtrooms suggests that adoption without proper safeguards and institutional oversight could expose Malaysian lawyers to sanctions, malpractice liability, and damage to their professional standing. The Malaysian Bar Council and legal regulatory authorities have yet to issue comprehensive guidance on AI use in legal proceedings, leaving practitioners in uncertain terrain.

The incident in the United States represents only the most visible manifestation of a wider problem. Legal professionals have reported instances where AI tools have invented entire case names, fabricated holdings from real cases, and constructed arguments that, while superficially coherent, lack any foundation in established law. Some judges have begun imposing financial penalties on attorneys who submit AI-drafted briefs without adequate human review, treating such submissions as violations of professional responsibility standards that require lawyers to be truthful and not mislead courts.

Judicial responses across American states have ranged from cautionary language in published decisions to explicit rules prohibiting or severely restricting AI usage without judicial pre-approval. Several jurisdictions have established mandatory disclosure requirements, compelling attorneys to inform courts whenever AI has been used in the preparation of filings. Others have imposed competence standards, requiring lawyers to understand the limitations and potential pitfalls of the specific AI tools they employ. These measures reflect a judicial consensus that while AI might usefully accelerate certain aspects of legal work, its deployment in the highest-stakes contexts demands extraordinary diligence.

For Malaysian law firms and corporate legal departments, the warning is unambiguous: deploying AI tools like ChatGPT, Claude, or other language models without human supervision equivalent to traditional legal review creates unacceptable professional and institutional risks. Even where AI is used only as a preliminary drafting aid or brainstorming mechanism, the supervising lawyer bears full responsibility for the accuracy of every statement, citation, and argument presented to a court. The ethical rules that govern the profession do not carve out exceptions for technology-assisted errors; a lawyer's certification that a brief is truthful and well-founded in law cannot be outsourced to an algorithm.

Part of the difficulty stems from the seductive nature of these tools. AI chatbots present their outputs with apparent confidence and syntactic fluency that can lull users into a false sense of reliability. A lawyer working under deadline pressure might receive a 15-page draft that reads perfectly coherently and contains numerous specific case citations that appear legitimate at a glance. Without deliberate, methodical verification of every citation—a laborious process in traditional legal work—errors can pass through to submission. The pace and seeming expertise of AI output creates psychological pressure to trust the machine, a cognitive bias that courts are now explicitly rejecting.

The regional dimension is particularly acute in Malaysia and Singapore, where the legal profession has historically emphasised attention to detail and adherence to procedural rules. Courts in both jurisdictions maintain strict standards for written submissions, and judges are accustomed to expecting meticulous scholarship in counsel's briefs. An influx of AI-generated filings containing fabricated citations would rapidly undermine confidence in legal advocacy and create caseload pressures as courts devote resources to identifying and correcting errors.

Some legal technology entrepreneurs have begun developing AI tools specifically designed to avoid hallucination by restricting them to databases of verified legal precedent and prohibiting free-form composition. These supervised systems, though more cumbersome and expensive than general-purpose chatbots, may eventually provide safer alternatives for routine tasks such as contract review or legal research. However, the legal profession in Malaysia has not yet widely adopted such specialist tools, and awareness among practitioners of the risks posed by general-purpose AI remains patchy.

Moving forward, the Malaysian legal profession would benefit from proactive guidance from regulatory authorities before problems multiply. The Bar Council might consider issuing ethical advisories clarifying the standards of human oversight required when AI is used in legal practice, and specific warnings about the risks of relying on unevaluated machine-generated citations. Law schools might intensify instruction on the limitations of AI and the professional obligations that technology cannot displace. Individual firms should implement protocols ensuring that any AI-assisted work undergoes rigorous human verification before presentation to courts.

Ultimately, the emerging case law from the United States demonstrates that technology cannot absolve lawyers of their professional duties. Courts exist to administer justice, not to police the working methods of legal advocates. When an attorney presents a brief containing fabricated quotations, the court must pause proceedings, investigate the source of the error, and determine appropriate sanctions. The profession's credibility depends on maintaining standards that predate the artificial intelligence era and will endure long after current algorithms become obsolete. For Malaysia's legal community, the time to establish clear, firm boundaries around AI use is now—before courts are forced to respond reactively to problems that proper foresight could prevent.