By: Robert Moore, OSU Extension
In a previous post “Artificial Intelligence – What Is it and How to Use It”, I briefly discussed AI, how it works and some of its potential uses. There is no doubt that AI will have profound effects on each of us and our society in general. In this post, I am going to examine how AI works for a specific task related to agricultural law and measure its performance.
Surveys by Ohio State University indicate around 50% of farmland in Ohio is leased. Therefore, farm leases are an important legal document for many Ohio farmers. While some farm leases are still only verbal, many tenants and landowners recognize the benefits of a written lease and have at least a basic written lease in place. Some leases are written by the tenant or landlord while other leases are written by attorneys. The issue addressed in this article is: is AI ready to draft your farm lease?
The Process
To address the above question, ChatGPT and Google Bard, two of the more prominent AI interfaces, were each tasked with the following: “draft a cash farm lease”. This command was broad and vague but would likely reflect what a tenant or landowner might request. This exercise was performed on May 30, 2023 and each AI tool provided a cash farm lease. The exercise was again performed on October 4, 2023 to assess if AI’s capabilities changed over time.
To measure the effectiveness of AI, the drafted leases were compared to the recommended lease terms provided in OSU Extension’s bulletin “What’s In your Farm Lease? A Checklist of Farm Lease Provisions”. This bulletin was written by Peggy Hall and provides 26 key terms that should be included in most farm leases. Each draft lease was scored based on the number of terms that were included.
The Results
The following is the score for each draft, with the score reflecting the number of recommended terms from the lease bulletin that were included in the lease drafts:
ChatGPT, May 2023 8
Google Bard, May 2023 10
Chat GPT, October 2023 9
Google Bard, October 2023 7
As the scores show, neither ChatGPT nor Google Bard included even one-half of the recommended terms and the best was 10 out of 26 or 38%. Two important items of note. First, no drafts included terms to prevent the tenant from assigning the lease to someone else – an extremely important provision to include in farm leases. Second, no drafts addressed landowner or tenant signatures needing notarized.1
I would describe these drafts as “bare minimum” leases. They are probably better than having no lease at all, but they could be much better and do not include several key terms. Also, there was no significant improvement of performance over time. In fact, the Google Bard score was lower in the later draft. Asking ChatGPT or Google Bard to “draft a farm cash lease” is not going to provide a satisfactory lease.
Providing Input to AI to Improve Output
As I discussed in my prior AI post, one of the benefits of AI is the ability to chat with it. That is, you can provide feedback to the AI to assist it in providing a better outcome. So, that’s what I did. After reviewing the first two rounds of lease drafts, I asked ChatGPT and Google Bard to draft a third cash farm lease and to specifically include the 26 recommended terms from the lease bulletin. The resulting leases were better and scored as follows:
ChatGPT 16
Google Bard 20
As you can see, the scores increased significantly. So, the feedback provided to AI was integrated into the resulting drafts and made the leases better. This is one of the major advancements of AI. It allows someone like me that has little computer proficiency to provide untrained input that causes a significantly better result.
While the scores did increase, there were still some major issues with the drafts. I was probably generous in the scoring and gave credit if an issue was addressed, even if somewhat incomplete. For example, in its first two drafts, ChatGPT did not include a term addressing who receives FSA payments, the tenant or landowner. ChatGPT did address this issue after being prompted but stated that the landowner would receive all FSA payments. According to FSA rules, the tenant must receive at least some of the program payments and it is customary for the tenant to receive all FSA payments. So, while ChatGPT included a term about FSA payments, the included term was not completely accurate or correct.
Google Bard also had similar issues. In its first two drafts, it did not address what happens in the event of eminent domain takes a portion of the leased property. A typical lease term would say that the tenant is compensated for any crop damage caused by eminent domain and the landowner would keep the acquisition proceeds. Google Bard included a provision about eminent domain but stated the tenant would receive all eminent domain proceeds. Allowing the tenant to keep eminent domain proceeds would be very unusual and not something a landowner should agree to.
I would assess these leases as “better but still not good”. These drafts did include more of the recommended terms but included many of them in an insufficient or incomplete manner. The third round of leases did show that AI can learn and improve with feedback but also that it has a long way to go. The craft and nuance of drafting legal documents still seems to belong to the domain of people.
Conclusion
There are some well-known people, such as Elon Musk, who claim that we should have serious concerns about AI eventually taking over the world. Their concerns may be valid, but as of now I don’t believe AI is going to take over farm lease drafting anytime soon. An experienced attorney can do a much better job of drafting a farm lease than today’s AI. For a tenant or landowner who are unwilling to hire an attorney or may not have the resources to pay an attorney, a farm lease drafted by AI may be better than nothing but that’s about it. The best source of legal services remains to be attorneys and likely will be for the foreseeable future. AI is not ready to replace your attorney – yet.
1Leases for more than three years must be notarized.