Here are some notes about our feelings in regard of the field work we have done last week in Bristol. We asked ourselves what was good for the project in doing so, what we validated, what we learned and what are the biggest challenges and improvements we face.

Positives

  • Our iterative approach
  • We started with very open questions in order to be neutral observers of what the people of Bristol brought up as issues. We then refined our approach and made the questions more specific about topics they’d expressed. We were weary not to assume or project anything as researchers
  • Localised learnings
  • Our methodology made a conscious point of going to test in different areas with people from different backgrounds
  • Community centres, cafes, bus stops, shopping malls, streets, museum surroundings, promenades… they work well to ask questions

Validations

  • In face to face interviews, the importance of trust in some contexts (ie people from the same neighbourhood, community organisations…). Phil Chan said that some people are harder to reach (by surveys or local gov) than others
  • Interviewers need non-intimidating, inclusive survey tools, including a paper form - offline option
  • Running street interviews with tablets as opposed to computers? Tech tools are social signifiers
  • Accessibility is key
  • We would have definitely missed out form items if we had to design the form solely based on our perception of the city

Learnings

  • Field work is hard (getting to a diverse pool of respondents requires time, effort, knowledge, etc). Weather can have a big effect on success / sufficient recruitment / conditions–it applies both to interviers and respondants
  • In disadvantaged areas, an ‘in person’ approach is key
  • Some people will need help to complete the survey and/or to be inclined to answer questions. Online survey works for able, computer literate, educated populations. How do we get to everyone?
  • Some people would have prefered to answer on paper (it feels less intimidating and it is easier to follow)
  • Although forms can be sequential, there is a desire to come back to early questions to alter responses
  • Short surveys (like 3 minutes for real) makes it a low risk offer for respondants
  • Being independant helps remove commercial concerns from respondants

Biggest challenges

  • Getting traction / interest from people with the arguments that suit them (differ if you’re a citizen, a civil servant, a community organisation, a data activist…)
  • Getting people excited / interested in answering the questions (survey, form are buzzkills)
  • Combine both a good form interface and a good data structure

Improvements

  • Typeform has some UI kinks that made our first iteration wonky and usability-unfriendly for some. We should try other platforms
  • Consider feedback carefully as it sometimes contradicts itself. Can express human contradictions ‘what a user says is not what a user does’ or can indicate a problem in the way questions are framed.
  • Twitter Ads platform was hard to setup and expensive to run—between $1.00 and $1.50 for a click
  • Time to iterate further!

Strategy for user expansion

Start with existing, close communities (open data advocates, service design folks) to refine the survey. Then expand gradually.