I have a confession. I’m one of those people who thinks ‘evaluation’ is a four letter word. Or at least I did.
Let me explain.
Evaluations have always felt like the equivalent of a trip to the dentist – an expensive and painful necessity that we’d all rather avoid. And, if you’re anything like me, you only ever go to the dentist if something painful happens or you’re forced to go.
I’m willing to bet 75% of evaluations[1] in social services fall into one of three categories:
- Because we got money for the program and the funder made us;
- Because we need money for the program and funders like them;
- Because an academic thought it looked interesting and got a grant.
Now don’t get me wrong, I’ve seen a lot of agencies and programs do very good evaluations for all the right reasons. Unfortunately I’ve seen too many half-hearted and pointless efforts and several evaluations that were downright baloney.
Here’s the thing, I’ve also learned through trial and error (and error and error and error) that Housing First cannot be done well without rigorous evaluation – more specifically quality assurance and performance management.
As the Canadian Housing First Toolkit team rightly points out, research and evaluation was essential to the adoption of Housing First – because Sam Tsemberis and Pathways to Housing used research and evaluation to prove Housing First was more effective and more efficient than the status quo.
I’ve seen Housing First programs all over North America and at the Calgary Homeless Foundation I helped develop, implement and fund over 30 different Housing First (and Housing First-ish) programs housing over 4,500 people.
I only mention that to say, if there was a mistake to be made in funding, building and implementing Housing First programs I’ve probably seen it or more likely done it myself. You can save yourself a lot of time and money and house more people by getting evaluation right at the beginning.
Here’s why I think evaluation in the Housing First context is different and essential.
The bar has been set high – how do you stack up?
Housing First sets a very high bar for success with clear and evidence based outcomes (high housing retention rate, low returns to homelessness, reduced public system cost or at the very least, more efficient use of those systems) and challenges us to prioritize those with the most complex needs. Everybody and their dog will soon be saying they are doing Housing First, but funders will want to know how you stack up against the standard outcomes and at what cost. You’ll be expected to know how you perform and how you stack up against the standard.
Quality assurance
The simple fact is if you want to house chronically homeless people with complex needs, you will not likely achieve Housing First benchmark outcomes without fidelity to the model. If your program is not meeting the benchmark outcomes, there is something wrong.
Real time data gathering, analysis and evaluation tells program staff, leadership and funders if there is a healthy and effective program. For example, a Housing First program with a 65% housing retention rate after 12 months can be indicative of a range of issues ranging from unaddressed client needs, a mismatch of acuity to supports, or issues with case management practice. You could also have a 95% housing retention rate, but 30% discharge to homelessness which suggests the program could be creaming and cutting loose the tough clients.
Housing First introduces a major paradigm shift from one shot program evaluation to a more dynamic form of real time performance management and quality assurance & improvement. This requires funders and agencies to have a much more intimate (and potentially contentious) relationship.
Risk management
Housing First challenges us to house the people generally excluded from mainstream and non-profit systems because their needs are so acute. We are challenging agencies, with limited resources and limited training, to house people with complex psychiatric needs, active addiction, long term homelessness, medical problems (and more) in a harm reduction environment. This is inherently risky. Let’s not beat around the bush – crazy s**t is going to happen.
We have an obligation to do everything we can to manage that risk to our clients, our staff and the community. Leaders have to have a real time visibility of problems which means incident reporting, analysis and a rapid response from funders and program leaders. We have to know exactly how programs are performing and do everything we can to make sure they are the best they can be. That will only happen if a quality assurance process – real time evaluation and response – happens.
Uploading and sustainability
Homelessness exists in large measure because public systems are broken or not addressing the needs of those in greatest need. We can’t keep papering over these systematic problems with charity dollars. Evaluations of Housing First programs allows us to expose these systemic problems and upload their resolution back to the governments where they rightly belong. And if government won’t fix them, they ought to be paying us to do it. Either way, systemic issues need to be resolved if we want to end homelessness.
One of the biggest worries people have going into Housing First is sustainability. Evaluation is also really important to demonstrating value, building public support and ensuring ongoing funding. The infographics below show how Edmonton and Calgary have used evaluation data to build public, funder and political support for Housing First programs.
There is nothing more attractive to a funder or a politician than success.
Finally - you got a better idea? Prove it.
Housing First isn’t a silver bullet, nor is it the only possible response to homelessness. As a funder I would hear over and over how Program X or Program Y was better than Housing First. My response was always ‘Great. Prove it. If you have an idea that gets better outcomes and is cheaper for the people I want to house, I’m all ears.’
This conversation has led to some really interesting innovations in Housing First as well as the development of permanent supportive housing for a small cadre of homeless families and individuals.
Evaluation can be a four letter word, but you can’t do Housing First safely or well without it.