Evidence Centre seminar: July 2020

Improving wellbeing and outcomes: What we can learn from four Oranga Tamariki early intervention programmes.

Dr Iain Matheson, Matheson Associates.

Tēnā koutou, tēnā koutou, tēnā koutou. Ko Clisham te maunga, ko Creed te awa, ko ngāti Kotimana te iwi. Ko Iain Matheson tōku ingoa. Ka nui te mihi ki a koutou. Tēnā koutou, tēnā koutou, tēnā koutou katoa.

Good morning, everyone. As Dorian said, Iain Matheson, I'm from Scotland as I've just indicated, from an island off the northwest coast, and greetings to you all.

This is my first time in a big room since the lockdown, so if that's the same for anybody else, you're particularly welcome and nice to see so many familiar faces. And also with Covid, it's a reminder that the thousands of children, families and whānau that are served by these four programmes have had it tough over the last weeks and months, and while some of us have had our challenges, they're dwarfed by the experiences that many of these have had.

So, 2019 and early in 2020, the Evidence Centre published eight reports on four of its early intervention programmes. They had -- there was one on Strengthening Families, which I did, there were -- there's one on Family Start -- a process evaluation on Family Start, there's an impact evaluation that's in development at the moment. Three reports on Social Workers in Schools and three evaluation reports on Children's Teams. Different questions, different methodologies, different purposes, but a number of themes appear to be apparent from the reports that were coming through, and therefore, I was asked if I could look at these eight reports and also look at some of the older research that's gone on on these programmes as well.

So, I'll do a little bit of context, five main findings, and if we've got time at the end some discussion around implications.

Now, I'm not going to fire some heavy definitions at you, but suffice to say for the purpose of this, whether it's newborn babies or whether it's adolescents, for me, at its simplest, is three dimensions of early intervention. One is whether it's about harm, preventing harm or addressing wellbeing, whether it's targeted or universal, and whether it's focused on individual children or families and whānau and it could be anywhere along those lines for, for any particular programme.

The four we're talking about is Strengthening Families, established in 1997 is an interagency case conferencing system, it's in place in roughly two thirds of the country. Family Start started the following year, 1998, that's an intensive home visiting service, long-term home visiting service for pregnant mothers and family/whānau with young children. Family Start is a nationwide programme, Social Workers in Schools operates across the country as well, that is a school based social work service for children and their families/whānau. That takes place in most decile schools 1-3 primary and intermediate and that's English medium schools as well as kura and Children's Teams started out 2012, they cover roughly the third of the country where Strengthening Families no longer operates and they focus on those young people and children below the statutory threshold and it's an interagency programme to serve their particular needs.

Now, I want to talk briefly about why, what and how. There's micro-why and there's macro-why. Micro, I'm not really going to be bothering about because that's historical stuff and that may have all changed, but the macro stuff is around beliefs, why these programmes exist, why providers provide them, and why Oranga Tamariki delivers them on a national basis, and it's around children being valued, children, families and whānau being valued and being seen to have strengths and agency.

Secondly, it's around the state having a role beyond universal services to support and help families and children and whānau most in need and thirdly, it's around the power of early intervention, if you're more analytical that's the kind of description of that that you might prefer.

In terms of programme types, essentially for me there's three different types of programmes. There's a Manualised Evidence-Supported Treatment (MESTS) programme, and that's a hell of a mouthful, but that's essentially the sort of evidence-based programmes that the expert panel and the modernisation of Child, Youth and Family was recommending. These have very particular characteristics, you tend to find them on US websites where they get accredited and that accreditation is often dependant on US federal funding, so that's the kind of history of them.

The other end of the spectrum you've got what I call local services, and I've contrasted on that slide the different characteristics of the two, and the third is in the middle, it's a blend of those two approaches and for me, the four programmes we're discussing today are a blend of that, to try and get the best from the kind of MESTs orientation whilst also having aspects of that local service provision and community focus that is so important.

Very quickly, how the programmes operate, look at the bottom, there we have programme infrastructures, so that could be national office staff, referral criteria, tools, the information systems they're up top left, the knowledge, skills and experience of the providers and their staff, a very important component. Thirdly, top right, the practice principles, that's how it's being brought to life in terms of the application of the programme to the particular situation that workers face and intersections around engagement, trust and flexibility and that could be in relation to children, families and whānau themselves or it could be trust and engagement around community partners and other professionals.

Okay, so moving onto the five overarching findings, the programmes are targeted, they're all targeted, Strengthening Families requires two agencies to be involved or preferably two agencies to be involved, Family Start is particular for, as I said earlier, pregnant mothers or families/whānau with particularly young children. Social Work in Schools is targeted in relation to the choice of decile schools that it goes into, and Children's Teams are targeted in relation to the particular area just below the statutory threshold that they're looking at.

However, when you look at the aims of the programmes, you look at the stories in the reports, you look at some of the statistical data, it's apparent that there are some strong commonalities between those families, sometimes at different points in time, different ages, but they're quite similar. One exception might be Strengthening Families which appears to be more Pākehā orientated than the other three where most of the children families are Māori.

So as well as targeting broadly similar families, these programmes have some quite restrictive requirements that families need to agree to. So, for example, Strengthening Families, you need to be willing to have your personal information shared with other government agencies or NGOs and to attend a meeting with representatives from those -- that is negotiated with you -- to discuss your family and to develop a plan. That's not for everyone.

Family Start, equally, it's quite an intense programme. Initially it's weekly, it's quite long -- it's very long term potentially and families need the worker to come into their home every week or every fortnight or whatever, so, again, some families will go for that depending on the particular circumstances they find themselves in and how they perceive the provider, but, again, it's not for everyone. And they're quite focused, focused or siloed programmes. They have their own infrastructure, there isn't too much national coordination or local coordination so they tend to be very standalone. So these are some of the characteristics of the approaches and purposes.

Now, in terms of design commonalities, they broadly sit in the same area of the service delivery spectrum. Now, as we learnt from the Strengthening Families report is within Strengthening Families different areas could be almost different ends of that. I think the others appear to be a lot more consistent and there are differences, but they're kind of clustered around the middle and rather a lot of comments across the reports on we need more intensives provision above and we need more provision below and I think particularly for the two that were coordinating a lot of comments around, "Well, we're just coordinating", but there's not a hell of a lot of stuff to coordinate because we're not necessarily getting a lot of support from government agencies, for example.

Okay, so they're more or less clustered around that area. They balance fidelity and flexibility, so Family Start was found to have high fidelity in the process, the evaluation, but it's a balance for all of them. Sometimes difficult to unpick from the reports whether good outcomes were because of strong fidelity or strong flexibility and vice versa, so that's maybe something worth unpicking in the future.

 Some suggestion of whether the designs were less appropriate for Māori, this question actually was addressed in different ways in different reports in different discussions. So what came through quite strongly, very strongly in the Family Start process evaluation was the need for more te reo materials and before publication, Partnering for Outcomes had in fact introduced some materials anyway, but much of that was coming from Kaupapa Māori organisations that were used to that way of working, they maybe had Whānau Ora contracts as well, so that whole sense of how they should operate has possibly changed over the years.

There were other discussions about whether the programme was or was not appropriate for Māori, so the Children's Teams, they were quite mixed views on whether the whole model was appropriate for Māori and others were approaching the issues in terms of being Māori sensitive or culturally sensitive. So there were some Strengthening Families areas for example who said that some Māori prefer us because we're not a Māori organisation, because of whānau and connections in locality. Others said we can and do put on a Kaupapa Māori experience for families if that appears to be what's going to help them and in their best interests, so it wasn't a particularly cut and dry, but there was a lot of discussion.

 It came through in the Social Workers in Schools, kuras as well, similarly looking for more te reo material and almost a sense that they wanted to evolve that model maybe slightly differently or maybe have a separate model for the future rather than just a te reo version of the programme that is delivered to everyone else.

Little common infrastructures, I've kind of touched on that briefly before. Three of the four are required to have local governance arrangements with the exception of Family Start. I should say all of them appeared to have strong bi-agency or inter-agency professional relationships in their locality but this requirement for local governance applied to three of them.

Strengthening Families, seven of the areas didn't even have a local governance group and it wasn't -- there was some examples of strong local governance but not that many. Children's Teams, the discussion was more around whether the phrase "locally led and nationally supported", which was going around at the time, whether that reflected reality. A lot of stakeholders were saying this isn't locally led and were looking for much more ability to engage and drive what happened locally than how they felt, and I should say the Children's Team concerned are just three, so this was Otaki, Horowhenua, Christchurch and Rotorua. One old one, one middle one, one quite recent one.

Information systems was another theme that came up again and again, I maybe need to put a caveat around this; I've worked in three different jurisdictions and I've yet to meet social workers who love information systems. It fights against the core of many of them so I just kind of put that in as a caveat but there seem to be a lot of comments about complicated, difficult to use, not user-friendly, inaccurate, presumably because the information that went in was inaccurate, but at the end of the day, it wasn't delivering what it was meant to be doing for whatever reason, so people weren't really able to use the information very strategically because people were saying it wasn't that accurate, therefore after all this time, effort and money it wasn't actually helping them that much.

The one that doesn't have one of those types -- and they're all different systems -- Social Workers in Schools, they have the strengths and difficulties questionnaire platform, so that's a behavioural questionnaire that usually gets done before and after an intervention, it can be done at other times and it's maybe done by family whānau and the teacher and then there's a platform to record all of that but there's been some issues with that as well from maybe a different point of view. It's a very clinical type of tool, very prominent in -- I mean even the pākehā world it's kind of right on the psychiatry/psychology end of the spectrum and the language doesn't necessarily fit, it's focused on behaviours and some people are saying there's a bit of a tension there.

There has been a te reo version produced but because this is an internationally accredited tool it has been accepted by the publisher but it's a literal translation in order to comply with those requirements. So whilst there was a comment that it looks positive to family and whānau that this is available in te reo, the actual language was quite alien to people and therefore there were reports that it wasn't very applicable or helpful.

Professional development, so going back to the previous slide where I had the MEST and the kind of characteristics, so back in the day, these programmes would have had a sort of national training programme to support them. Bits of that do still go on, often regionally, but across all four programmes the current arrangements do not appear to be meeting the needs of those that were interviewed across the different programmes. So, sometimes practitioners would have to find their own training that they thought met their needs. There is a national conference for Family Start and there are occasional events I think for Social Workers in Schools, but Strengthening Families were limited (0.18.06 inaudible) back years and years and years and years when they last had their national -- a national event. So that did seem to be a strong element around the infrastructure.

Delivery challenges. Salaries. Recruitment and retention, maybe equity but there were reports that it was impacting on recruitment and retention as well. Strengthening Families seem particularly problematic in terms of when they last had a pay rise. So that came across three of them, it wasn't really covered in Children's Teams because those staff are paid through a different route.

Case complexity. I think nearly everybody said cases were more complex today than they were in the past. The Family Start one can be clearly attributed to the 2012 changes in the referral criteria, that's what was reported in that process evaluation, but there were also discussions around societal complexities, poverty presumably, society being less equal than perhaps it was in the past, and thresholds. Thresholds, not just Oranga Tamariki and CYF but also other government agencies, so there was a strong sense or feeling that those thresholds had been tightened and therefore NGOs were getting more challenging cases than one, that they're used to, and two, that they were necessarily contracted for. Again, Family Start have done some work in that area since that report was published.

Service continuity. NGOs are required to ensure there are arrangements in place whenever a member of staff leaves or is on leave and that seems to work by in large, there were some examples where services were not provided for a number of months, there was one example of a change in providers that led to some disruption but there doesn't seem to have been any large disruptions reported as were the case back in 2012 with Family Start when they got rid of four providers and had another 11 on one year contracts because of performance issues.

But it did come through in other ways, so the Children's Teams, they were taking I think it was ten days to accept a referral, 29 days to allocate that case to somebody, and then in one of the teams, because of a lack of resources, they appear to prematurely close a lot of cases because they just don't have the people to serve them.

So service continuity can present itself in different ways and certainly the Children's Team, people that were interviewed, the family whānau that were interviewed as I come on to say they were particularly positive but they were waiting six, seven, eight, nine months to get a service.

So, some programme outcomes. All four highly valued by those that were asked. I was surprised how positive the Children's Team one was, to be honest. The majority of family whānau who were interviewed as part of the Children's Team evaluation described terms that equated to it being transformational. But Family Start, Social Workers in Schools, similarly some very, very positive feedback. Tended to be about the worker, not the programme. So that is maybe a useful caveat to keep in mind.

Strengthening Families, nobody was interviewed as part of that piece of work. We did have feedback forms from those stayed on for a planned final meeting but that was a minority of people that participated in that programme and where there were interviews, (Inaudible 22.42) were involved in the selection of interviewees so they weren't representative samples.

But nonetheless, some very positive feedback from family whānau and professionals as well. Professionals in these areas all seem to appreciate these particular services.

In this round we also got a lot of documented stories, and it's nice to see what people appreciated. How family whānau liked to be engaged with, what kinds of issues were being addressed, that kind of detail tends not to come through historically, so that was particularly good to see. There was limited quantitative evidence across these eight reports because most of them were either qualitative or mixed with a qualitative bias perhaps, so it will be good to see the Family Start Impact Evaluation that comes through soon.

There are, of course, the quantitative stuff come through from MSD 2016, 2017, 2018, so we had three quasi-experimental design studies, two on Social Workers in Schools and one on Family Start and some encouraging, positive material coming through from that, some statistically significant findings, one particularly encouraging result was in relation to sudden infant death, so it would be useful to see whether that's echoed in the next one. I think that's quite early days in terms of the use of the IDI and it was quite exploratory, I thought, but it would be useful to look at some of that material in the light of what's coming through just now.

Okay, some broad implications. There's firm foundations. I mean, these programmes are welcome by family whānau, welcome by professionals, we have staff within this building and in NGOs across the country, Partnering for Outcomes advisors with up to 20 years' experience of developing and delivering early intervention. My caveat is, it looks easy, but it ain't. It's very tricky to recalibrate all of these elements so that they work harmoniously together and I think if we look back, we can see historically where it has worked and where it hasn't worked and how learning has had to be continuously applied in a fast moving environment, particularly around whānau ora.

Secondly, I say there's a common issue around collaborate or compete. There's quite a lot of competition in the current framework. I think there's also some implications around how we actually roll out national programmes, Strengthening Families I think took five minutes, Social Workers in Schools and Family Start, they're nigh on 15 years to get that whole programme rolled out onto a national basis and Children's Teams, before the decision to transition away from that model, you know, the projections of when that will be fully in place were quite long term and I think that just makes it very difficult for everybody that these things are moving at such different paces.

Certainly for me, one of the messages is that we need to find ways of speeding up this process and having effective programmes in place all at one time across the country, piloting what needs to be piloting but find ways to learn much quicker than perhaps we've done in the past.

And I think the last implication was it does highlight that the provision in many areas feels quite limited and whilst the likes of Family Start have done a lot of work in terms of connecting that service with the universal services, I'm not sure that's being done as much in the other programmes.

So, thank you for listening. Thank you.