Modelling Legal Aid

June 21, 2024

A quick write-up of a little microsimulation of Legal Aid that I’ve written for the Scottish Legal Aid Board (SLAB). It was fun, though I crashed a few deadlines and took some unexpected turns.

Previous Attempts

I’ve done a few models of Legal Aid in the past.

In 2001/2, back when I was still at the Institute for Fiscal Studies (IFS), I produced what I’m pretty certain was the first ever microsimulation modelling of Legal Aid in England and Wales, along with the brilliant Alexy Buck of the Legal Services Research Centre¹.

Subsequently, I produced similar models for SLAB and the Northern Ireland Legal Aid Board, including the June 2007 Report “Modelling Financial Eligibility For Legal Aid”, jointly with the rather less brilliant Tony Dignan.

For the England and Wales studies, the modelling used a variant of the IFS Tax and Benefit model². The Scottish and Northern Irish work used a custom-build model, written in Ada and released under an Open Source Licence on the GitHub code sharing site, where it remains today.

The England/Wales work was especially rewarding as using the model we developed a systematic method for simplifying the system,whilst making as few changes to the entitled population as possible. That’s important because most tax-benefit simplification proposals - flat taxes, basic incomes, negative income taxes, or whatever - involve huge distributional changes, usually in ways that aren’t intended or wanted.

The Model

Like the English work, but unlike the previous SLAB model, was integrated into ScotBen rather than stand alone. Building on an existing model had several advantages:

• much (but not all – see below) of the hard work of data creation, weighting and uprating is already done;
• we can also re-use output routines, for example tabulators1;
• also, SLAB gets a certain amount of policy relevant modelling as a bonus, such as the effects on Legal Aid eligibility of changes to Universal Credit, or of the gradual phasing out of legacy passport benefits such as Income Support.

One issue was that Scotben currently has no correction for non-takeup of means-tested benefits, so it is possible that it would overstate the effects of passporting these benefits on Legal Aid eligibility.

The Scottish Crime and Justice Survey

The previous models were mainly concened with financial eligibility for legal aid. But to go from eligibility to actual expenditure is much tricker than for, say, eligibility to Universal Credit to actually receiving it. The intial proposal involved using the Civil legal module of the Scottish Crime And Justice Survey (SCJS) to impute the likelihood of each FRS person experiencing a legal problem that might require going to a solicitor. So I spent the first few weeks of the project doing that. That went well, and produced some interesting results - probits modelling reporting a civil problem against family type, income, housing and so on. I might try publishing something using those regressions. But this approach got vetoed by SLAB, on the grounds that the problem categories used in SCJS don’t match the reporting categories SLAB use internally. The client is not always right, but they are always the client. That threw me quite a bit as I’m old and like sticking to the plan.

Problems

Apart from the SCJS thing, I thought the project went pretty well, though a few dealines were crashed. My collaborator at SLAB, Kieran Forbes, was really on the ball but supportive. He asked a lot of good questions and has really helped by pushing for better understanding of data - especially capital and expenses, which are pretty central to the Legal Aid calculations. I spent a lot of time sorting capital and expenses out - see this post on capital and I’ll write up expenses presently.

Admin Data

SLAB made up for vetoing by SCJS big idea by giving me some really nice admin data. It’s fantastic data - case costs, contributions, case types. Unfortunately there’s not much demographics so the usual data matching tricks don’t work. I settled for matching rough crosstabs of cases and genders against aggregate entitlements, to give rough propensities to claim by case types, entitlement levels and gender. Unfortunately I can’t distribute this because of confidentiality, but I’m considering building a synthetic dataset version.

Validation

The model was test first as usual. Mostly the tests replicated the results from the two online Legal Aid Calculators, but later on I added a bunch of tests of the code that imputed claims onto entitlements since that ended up being pretty brittle.

Front Ends

The model is not much use without an interface. It was a struggle though. It’s a convoluted story but we ended up having to install the model on an i3 laptop with 8mb RAM. Which was a lot when we wrote Virtual Economy but not a lot to run a Julia Instance with a Web Stack in. It turned out pretty well in the end but getting the whole thing to run efficiently was a lot of work. I’m proud of the crosstab. And I learned a decent amount about how to write an efficient job quite and session manager.

———. ‘Simplicity versus Fairness in Means Testing: The Case of Civil Legal Aid’. Fiscal Studies 24, no. 4 (2003): 427–49. https://doi.org/10.1111/j.1475-5890.2003.tb00090.x.