Inclusive Design Toolkit

The first step in the evaluate phase is to agree the success criteria, which should be based on the wants & needs of users and other stakeholders that were identified within the Explore phase. These criteria are then used to plan and summarise the user testing and expert appraisals.

This page describes the activities within the Evaluate phase of the Inclusive Design Wheel, and explains how they apply to inclusive design of transport services.

On this page

It is important to consider the question ‘what does success look like?’, for an inclusive transport service. It may be helpful to use the framework shown opposite to ensure that a wide range of success criteria are considered, covering:

• People criteria: user experience and social impact.
• Profit criteria: costs and revenues, technical risk and commercial business risk.
• Planet criteria: depletion of scarce resources, energy use and waste impacts.

Furthermore, the success criteria should be holistic with respect to the whole life-cycle of the service, which can be described as follows:

• Define strategy
• Design & Trial
• Ramp-up & Promote
• Maintain & Improve
• Ramp-down & Decommission

The success criteria should be specific and measurable, and should represent the wants & needs that were identified in the Explore phase. For example, the proportion of passengers using wheelchairs relates to the need for people with wheelchairs to board the bus independently. This would be an example of a user-experience related criteria that could be measured throughout the life-cycle of a more accessible bus service.

User feedback can be obtained at different levels of formality and detail, using methods such as:

• Contextual enquiry
• Focus groups
• Interviews
• Questionnaires
• User trials

Within the evaluate phase, these methods might be used to gain insights on the proposed design solutions, which may be embodied through storyboards or prototypes. Note that many of the same methods might have been used in the Explore phase to help understand how users behave with existing solutions.

Initial user feedback may be obtained within a co-creation workshop, as part of the co-design process. See the page about co-design for more detail. Whenever users are involved, the page about ethical considerations will be relevant (e.g consent, privacy and GDPR).

Later in the project, formal user trials should be planned to objectively compare the performance of the proposed solution with an existing benchmark.

Whenever user feedback is gathered, it’s important to consider whether the stakeholders that are planning or conducting the activities have a vested interest in achieving a particular outcome, and whether this could skew the results. The output from Understanding user diversity (within the Explore phase) should be used to inform the sampling for the user evaluation. The design of experiment should also be independently reviewed to identify and mitigate any potential bias.

In this activity, a range of relevant experts use their knowledge and judgement to systematically evaluate concepts. The purpose of formative evaluation is to identify potential issues and give recommendations for how the concept could be improved. Conversely, the purpose of summative evaluation is to objectively compare and rank different concepts.

The concepts may be evaluated from lots of different perspectives, which could include user experience, accessibility, operational feasibility, technical risk, financial or environmental. It’s important to consider whether the expert who are performing these judgements have a vested interest in achieving a particular outcome, and whether this could skew the objectivity.

From an accessibility perspective, expert appraisals might include an access audit and / or a website accessibility audit. The capability loss simulation tools available on this site can assist with expert appraisals, helping experts to communicate how capability loss may affect interactions with products or services. These include the Cambridge simulation gloves and glasses. Furthermore, the Clari-Fi tool provides a way of evaluating the visual accessibility of text and icons within apps and websites that are displayed on mobile phones.

It can be helpful to identify how many people would be excluded from using a product or service on the basis of:

• Their capabilities, such as: Vision, Hearing, Thinking, Reach & Dexterity, and Mobility.
• The technology access requirements, such as installing an app, or accessing a website.
• The technology competence required to successfully interact with the product or service.
• Other exclusionary factors, such as language, gender, and age.

At the time of writing in November 2022, the DIGNITY datasets were the most comprehensive available for covering all of these aspects. Of these datasets, the German dataset had the most robust sampling, and the biggest sample size (1010 participants). The questionnaire for the German survey, and an SPSS file of the corresponding participant responses are available from (UPCommons,

The exclusion estimates shown opposite were derived from the DIGNITY German dataset, while appropriately avoiding double counting of participants excluded. The authors of this toolkit can perform these kinds of calculations as part of our consultancy services.

Exclusion estimates for a service that requires the user to:

• Have a mobile phone - Exclusion = 2.1%
• Access a website that has been designed to work on both mobile and desktop - Exclusion = 11.3%
• Access a website while out and about - Exclusion = 18.3%
• Access a website on a desktop or laptop computer (i.e., via a website that has been designed for desktop, but has poor experience on mobile) - Exclusion = 17.5%
• Install a smart phone app that requires an internet connection - Exclusion = 47.4%
• Have previous experience with a mapping application - Exclusion = 58.7%

Exclusion values are expressed as weighted percentage of valid responses, for the DIGNITY German survey. These exclusion estimates are described in more detail within the Inclusive design log for transport.