AI is no use for legal research

February 16, 2024

I tried out AI to do legal research and here is the result.

I was looking at child maintenance the other day and the way the Child Maintenance Service (CMS) calculates child maintenance in specific cases. That is set out in Schedule 1 to the Child Support Act 1991. So far so good.

This has been amended several times and the government helpfully has the amended latest version on its legislation website. However, there is a paragraph that provides that some people do not have to pay child maintenance at all:

5 The rate payable is nil if the non-resident parent—
(a) is of a prescribed description; or
(b) has a gross weekly income of below £7.

How do I find out what “prescribed description” means? This is set out in secondary legislation, also called statutory instruments or SIs. They are also online on the government legislation website, but they are not cross-referenced in the act which they define further, nor are they updated. So an SI may have been amended by another SI, but the older SI will appear on the government website in its original wording. So unless I get a learned book on the topic (and even then finding this is not straightforward), I would have to traipse through dozens of SIs to find the definition. I did find the answer (by chance) in regulation 45 of the Child Support Maintenance Calculation Regulations 2012 SI 2012 No. 2677: prisoners, children and some students and residents in care homes etc.

I wondered whether AI could get there quicker. Since all this information is publicly online and it is just a question of reading all of it, cross-referencing it and finding the most recent version if there is more than one, it does not really take intelligence to find it, just the ability to read a lot of information fast and put it in date order.

I started with bing.com AI. I asked:

Hi. I’m looking for the statutory instrument which sets out the “prescribed purposes” for paragraph 5 of schedule 1 of the child support act 1991.

It told me:

I’m sorry, but I couldn’t find the specific statutory instrument that sets out the “prescribed purposes” for paragraph 5 of schedule 1 of the Child Support Act 1991. The information available on the legislation’s website1 does not provide the exact details you’re looking for. You may want to consult a legal expert or a law library for more specific information.

Elon Musk 
U.S. AIR FORCE ACADEMY, Colo. --  Tesla Inc. Chief Executive Officer Elon Musk speaks with Lt. Gen. Richard Clark, Superintendent of the U.S. Air Force Academy, during the Ira C. Eaker Distinguished Speaker Presentation in the Academy's Arnold Hall on April 7, 2022 in Colorado Springs, Colo. (U.S. Air Force photo by Trevor Cokley)

Polite but wrong. I tried Gemini from Google next:

Unfortunately, there isn’t a single statutory instrument that directly defines the “prescribed purposes” for paragraph 5 of Schedule 1 of the Child Support Act 1991. Here’s why:

    1. No specific instrument: Paragraph 5 itself doesn’t mention any specific statutory instrument for prescribed purposes. Instead, it references regulations made by the Secretary of State. This means the definition of “prescribed purposes” depends on the specific regulations in force at the time.
    2. Dynamic regulations: Child support regulations are regularly updated and amended. Past regulations might not be valid now, making it difficult to pinpoint a single instrument.
    3. Multiple regulations: Several different sets of regulations could contain relevant definitions depending on the specific scenario you’re interested in.

Right, yes, if the SI was mentioned in the Act, I would not have asked! So I thought: “Third time lucky.” and tried ChatGPT (free version):

The statutory instrument you’re referring to is likely “The Child Support (Information, Evidence and Disclosure) Regulations 1992 (SI 1992/1812)” which sets out the prescribed purposes for paragraph 5 of Schedule 1 of the Child Support Act 1991.

And which paragraph in that SI do I need to look at?

You’ll want to look at paragraph 3 of Schedule 1 of “The Child Support (Information, Evidence and Disclosure) Regulations 1992 (SI 1992/1812)” for the prescribed purposes related to paragraph 5 of Schedule 1 of the Child Support Act 1991.

Not only did ChatGPT randomly pluck out an SI (admittedly relating to child support), but it also referred to a paragraph of a Schedule which didn’t even exist. It simply made it up.

So when Elon Musk said that we would all be out of a job in a few years’ time because AI can do it, I am not getting concerned. (Despite having 10 surviving children, Musk also doesn’t seem to have cared for a child, older person or anyone else in his life or he would know that AI is hardly going to be able to do that job.)