*Blog written by Jenny Elliott, originally published by the Edinburgh Living Lab 24 June 2021.*

The sixth in our blog series about project ‘Future of the High Street’ — openly sharing the evolving project and process, our learnings and findings. This month — we explore the ‘collaborative evaluation’ approach and 5 indicators of project ‘success’ we’ve developed to guide our on-going project decision-making, as well as how we are using Public Life Studies and interviews to understand pilot impact. Or scroll down for this month’s insights direct from the project team members.

Film by Megan Miller — project film-maker

Most people agree that evaluation is important. But how often is this done on built environment projects? And what value could it bring?

By drawing on the research expertise within the University of Edinburgh in collaboration with New Practice, we have been putting research and practice together through the Future of the High Street project.

In addition to understanding more about the challenges, opportunities and possible small ‘tweaks’ that could support our high streets, we have also incorporated evaluation into the heart of the project. Both in terms of evaluation of the project as a whole — its process and outcomes, as well as the two specific ‘high street tweak’ pilots.

Edinburgh illustrator Victoria Rose Ball creating window illustrations for Gorgie-Dalry high street businesses as part of the June pilot

How have we been reflecting on the whole project?

Often evaluation just happens retrospectively at the end of a project, or might just skim the surface. We wanted to find a way to embed evaluationthroughout the project process as a way to reflect on and course-correct our decisions while the project developed — with the aim of improving project outcomes and impact as a result. Sort of a mini feedback loop or way to check how we were doing against our collective aims and ambitions for what ‘success’ should look like.

To help us do this, we developed an evaluation framework that included 5 indicators or principles, central to the overarching project aims, that we could use to guide our decisions on a day-to-day basis as the project progressed. We continually refined and applied these collective ‘indicators of success’ over the 6 month project timeframe to feed into active project decision-making.

Our 5 Indicators of Project ‘Success’ for the project are:

Open Learning — sharing ideas, the datasets and findings produced via the project, and ‘working out loud’.
Local Participation — aiming to use an inclusive approach that invites diverse perspectives and genuinely feeds into decision-making.
Professional Exchange — facilitating collaboration and opportunities to learn and share knowledge from this project and others work.
Meaningful Contribution — via pilots delivering short-term small-scale high street improvements with potential for longer-term legacy, as well as developing relationships and connection via the project more broadly (e.g via the Advisory Board, workshops and other conversations).
Critical Reflection — reflecting on and sharing project learnings and decision-making, with a report summary of the project and its findings/learnings to be shared in July.

But we also wanted to make sure that it wasn’t just us as a project team deciding what success looked like for the project.

Instead we used a collaborative approach to the evaluation that reflected the project’s wider ethos of engagement, and incorporated the perspectives of other stakeholders involved. This also meant that these 5 Indicators weren’t fixed, but evolved with the project to reflect new insights and aspirations as they were shared with us.

By actively listening throughout the project — at the workshops, advisory board meetings and project discussions that went ahead as the project developed, and keeping an ear out for participants’ comments about their view of what ‘success’ would look like for the project, we were able to incorporate these perspectives to refine our 5 indicators above, and the metrics we should measure to see if we were achieving them across all project activities.

Whilst we also needed to incorporate practical and logistical factors into our decision-making — such as what was achievable on a short timeframe and within the available budget, changing Covid restrictions, and ensuring we still delivered the core objectives set out in the original project brief — which together meant we couldn’t always meet every individual expectation — this approach did mean that we could analyse and find common themes of what a shared sense of project ‘success’ would look like for all those involved and use this framework to guide us toward the best achievable project outcomes.

In this way, it helped us to continually cross-check our decisions against our aims throughout the project (not just at the end), as well as acting as a prompt for how extra value might be added into planned project outputs whilst these were still in development, so they could better reflect our 5 indicators of success.

We have called this approach collaborative evaluation.

Example elements of the pilot Tactical Urbanism Kit on site in Jarnac Court in Dalkeith

Evaluating the ‘High Street Tweak’ pilots

As well as the wider project, evaluation is also essential to understand any impact of the two ‘high street tweak’ pilots initially tested in June on Gorgie-Dalry and Dalkeith high streets.

To evaluate the pilots, we used interviews with passers-by and business owners, as well as Public Life Studies — a robust research methodology that helps us to understand how people are moving around and using the high street.

By creating a full Public Life Study in May for Gorgie-Dalry and Dalkeith high streets, and comparing this baseline to when the pilots were in place a few weeks later in June, using the same consistent methodology, we can start to understand any ways that these small-scale interventions and changes to the high street built environment can affect footfall, how long people are stopping and spending time on the high street (dwell time), public life, and the way pedestrians move around.

Public Life Study in Dalkeith: tracing pedestrian flow using Procreate

To do this, our trained Public Life Study research team was positioned at 4 key locations along each high street. We used direct observation techniques at 3 timeslots on each research day to observe pedestrian movement and behaviour, as well as more holistic place quality assessments and interviews to gain a wider understanding of the high street as a place, locals’ lived experience, and how the current built environment might help set the scene (or not) for public life and footfall that can indirectly support high street business success.

We trialled use of digital software Procreate for these Public Life Studies — an alternative to the traditionally more analogue data collection methods for this research. This worked really well practically on the day, and also brought benefits following each day in the efficiency of data analysis, as well as enabling video exports (as above) showing how data built up.

The two full Public Life Study baseline reports for Gorgie-Dalry and Dalkeith are currently being finalised, and will be made publicly available in July so their data and findings can also be used for other projects and purposes.

Project Officer Shawn Bodden on site as part of the Public Life Study research team evaluating pilot impact

Evaluating the pilots in this way is important so we know if these ‘high street tweaks’ were successful in improving public life whilst in situ (such as footfall, dwell time or time spent outside) or the appeal and accessibility of the high street more broadly. Both these things indirectly support high street businesses, as well as improving the vibrancy of the high street as a place, social space and desirable destination.

These findings are important so we can share our learnings about whether the two ‘high street tweaks’ tested (from the original 6 prototype ideas co-created with stakeholders) might be a beneficial approach or resource for other high streets elsewhere. Both ‘high street tweaks’ piloted also have potential for a longer legacy incorporated into them, and so understanding and gathering robust data from the pilot to provide learnings for any further refinement or development is also important.

For example, in Gorgie-Dalry on Sat 12 June the project pilot tested out prototype bench seating in a few key spots along the high street (locations chosen via our May baseline Public Life Study and stakeholder workshop input), as well as live illustration by Edinburgh-based illustrators Victoria Rose Ball and Cassandra Harrison of high street business owners and their shop/cafe/premises’ windows. In addition to the benefit to businesses, public life and the accessibility of the high street on the day, we are currently exploring ways to build on the locations piloted and other insights to install several permanent benches along the length of Gorgie-Dalry high street.

In Dalkeith, on Sat 19 June the project pilot tested out a few initial prototype elements of a Tactical Urbanism Kit formed using a WikiHouse/Open Desk CNC plywood approach. This one day weekend event was designed to extend the wider public conversation from the digital workshops in April and May. It focused on explaining how the TUK would function and operate as a kit of parts that could be borrowed by businesses or community, exploring what components local stakeholders felt it would be valuable to include, and how this might offer opportunities to activate the town centre high street space in new ways. In addition, the day included hands-on demonstrations of the assembly/deconstruction process.

Several TUK seating elements were also positioned nearby to busy high street frontages as small-scale examples to test in practice the potential impact and benefit the TUK elements might deliver in terms of public life and dwell time supporting the vibrancy of the high street.

So what next?

With the project activity wrapping up in July we want to continue the ‘working-out-loud’ approach we’ve taken throughout the project via these blogs and short films, to share our key findings and learnings via a report summarising and reflecting on the Future of the High Street project. This will include standalone sections sharing our learnings about key topics relevant to the project process, from the digital engagement tools tried and tested to the decision-making process. We’ll also be sharing more about the pilots and plans for their legacy, and will make publicly available the two Public Life Study reports for Gorgie-Dalry and Dalkeith.

In addition, we hope you can join us for a webinar about project, its findings, and the things we’ve learned along the way from 2–3.30pm on Weds 21 July*.

*The webinar recording is available here

Direct from the project team: our thoughts and diaries from inside the project process — June 2021.

Each month we share insights direct from the project team. This month focuses on what the project team has been working on as the pilots have been refined and tested on site, and the Public Life Studies data collection and analysis work that will form part of how we will understand their impact.

Jenny Elliott (Project Lead, EFI)

This last month I have been training the research team in Public Life Studies, and together we have completed our on-site data collection and started analysis of our findings. I have led 9 of these studies previously for City of Edinburgh Council, but this time there are a few differences.
Firstly, in addition to the ‘standard’ public life direct observation research activities completed at several standardised time slots throughout each research day by each researcher distributed along the high street (footfall, stationary activity, dwell time, tracing studies of pedestrian flow, age and gender studies) we incorporated and tested out some new ones. These included ‘business activity’ (how busy each high street business appeared, and whether they had outdoor seating/elements) and capture rate (how many people went into high street shops/businesses vs walked past). Whilst we quickly learned that capture rate would only yield reliable results over a far longer time window than we had available, business activity revealed useful insights and is a good addition to our understanding of the link between public life in the public realm and the adjacent businesses and the ways they can both support and benefit from this.
Secondly, having used entirely analogue data collection methods previously, I was keen we test out using Procreate digital drawing software for iPad as an alternative. This worked extremely well and led to efficiencies in reducing required printing of research packs and in time-saving data exports ready for analysis (AirDropping exported files instead of laboriously scanning pages and manually aligning map data).
Thirdly, we not only created a full baseline Public Life Study for Gorgie-Dalry and Dalkeith high streets (which fed into selection of specific pilot locations), but also repeated the same research activities consistently during the June pilot interventions as a way of monitoring and evaluating the impact of small-scale changes to the built environment on footfall, dwell time, public life and user experience. I am excited to finalise our analysis and share these reports next month.

Shawn Bodden (Project Officer — EFI):

These past few weeks, the rest of the research team and I have been waist (shoulder? neck?) deep in analysis. The data from our first, base-line Public Life Studies gave us a lot to think about — both about the Gorgie-Dalry and Dalkeith high streets and about our own research methods, and we spent considerable time adjusting and tweaking(!) our plans for the second round of Public Life Studies to focus on the methods that we found most useful. Research methods like mapping stationary activity and qualitative interviews proved especially useful for identifying little-used or disliked spaces — ones that would be perfect for trialling a prototype!
We also had our second round of Public Life Studies this month, and it was exciting to finally see the prototypes in real life. I’ve never been so excited for people to sit on a bench before! The second Public Life Studies (PLS) presented an unexpected, but exciting challenge too: in our original plans, we had assumed/hoped that the spaces would remain relatively similar between the first and second PLS, allowing us to examine the relative impact (i.e. differences) caused by the prototype. In reality, so much had changed! At one site in Gorgie-Dalry, gorgeous street art had appeared on a utility box. In Dalkeith, an enormous tower of scaffolding now loomed over Jarnac Court. “Well, you can’t help but stare at that”, one woman pointed at the scaffolding with a laugh when I asked about her impression of the high street. It was a nice reminder just how dynamic public life is, and that our project is only one among many animating and reshaping public life on the high street. But it also made clear the value of the kind of in situ, empirical research our team is using: we can document, discuss and respond to these ongoing, unexpected changes. We can make informed decisions to change our plans to respond dynamically to the situation, and we can share our reasoning with others. I mean, you really should see that scaffolding.
This is part of the reason why we’re keen to emphasise that collaborative evaluation — about the prototypes, the research and the high street itself — is an ongoing, interactive and reflective process. Assessments and evaluations aren’t “once and for all” — things change and surprises happen — but these conversations and debates help us get our bearings in a changing landscape and help us find ways to move forward together.

Duncan Bain (Associate, New Practice):

Public spaces evolve in ways that are unpredictable — often piecemeal, sometimes through significant upheaval. Every new addition, subtraction, or modification to public space is contingent on the path of countless decisions made before. It is rare to be able to test and trial changes to public space in a low-cost and low-risk way before embarking on the hard work of making more lasting change stick. As practitioners invested in place-based approaches to development, being able to prototype and trial designs and learn from how people engage with these temporary changes is a fantastic opportunity that is not often afforded by clients.
The emphasis on using robust evaluation frameworks as part of the Future of the High Streets project is an interesting design challenge to us as design professionals. How do we take data and learning and use this to shape the iteration of the ideas that have emerged from the creative process so far? How do we react if this learning challenges our assumptions? It is important to always remember that public space is contested — that one person’s idea of a great place may not be shared by others. Sometimes there are irreconcilable differences in what people want in their neighbourhoods — a street can either offer lots of parking availability or a low-traffic experience. Prototyping, trialing and evaluation does not solve these intractable challenges, but does offer a chance to learn and reflect before making long-term investment.

Juliet Welshman (New Practice — Junior Designer):

In these past few weeks, I have enjoyed seeing ideas and concepts start to take form as potential physical outcomes of High Street Tweak. In April, I worked with Abi to organise and facilitate a Youth Engagement workshop in Dalkeith with Dalkeith Arts. While there, we worked with volunteers from Dalkeith Arts to interview local residents and record these conversations visually. It was great to finally get to engage with the public face-to-face for High Street Tweak, and in fact was my first time doing in-person engagement since joining New Practice!
Later on I helped to facilitate the final round of co-creation workshops, where we heard more from residents about their ideas, and began to discuss them in the context of physical, place-based outcomes. In each workshop, several strong ideas emerged, and residents seemed to reach a consensus about what each location is in need of. This helped to provide a clear steer for our Toolkit of 6 Ideas, which I have been helping Duncan to develop over the past couple of weeks. Having to justify why we have chosen certain ideas has really made me think about the purpose of collaborative design, and helped me to understand how our research will feed into the design realisation stage.

The Future of the High Street project is part of The University of Edinburgh’s ‘Data and Design Lab’ — funded by the Scottish Funding Council. It will act as a demonstrator project for how a data-and-design approach can be used to address key contemporary challenges and deliver positive impact. The project follows on from the Edinburgh Futures Institute Smart Places series in collaboration with the Edinburgh Living Lab and Data Driven Innovation programme. Find out more here.

Blog written by Jenny Elliott, Project Lead for Future of the High Street, Edinburgh Futures Institute.