PublicTechnology

One in six government service assessments ends in failure

Sam Trendall May 4, 2020 in Communities, housing and planning, Education and skills, Health and social care, Public order, justice and rights Tagged , - 10 Minutes

PublicTechnology research finds that failing to consider user needs is the most common stumbling block, with a lack of simplicity and intuitiveness also a frequent failing

Credit: Pxhere

More than one in six government services put forward for assessment fails to meet the requisite standards, PublicTechnology research has found.

Data published on GOV.UK contains information on 191 examinations that have taken place over a period of about six and a half years. This includes assessments of digital services in the early stages of development, as well as those that have been live for some time.

A total of 34 were, when initially assessed, judged to have failed to comply with the standards set out by the Government Digital Service – equating to more than in six services examined, or 17.8% of the overall figure.

From 2013 to the present day, the failure rate has remained fairly steady in the range of 15% to 20%. Since the beginning of 2019, five out 33 assessments have seen the service in question fail to make the grade, equating to 15.2%.

And even some services developed by GDS itself have been sent back by the organisation’s own assessors to make necessary improvements.

The current Service Standard against which services are judged has been in place for a year and features 14 requirements that must all be met. 

It replaced an 18-point checklist that set the standard for four years – which itself replaced the original standard, which was introduced in 2014 and placed 26 requirements on services being tested.

The current standard represents a consolidation of the measures proposed by the two versions that came before it; many of the previous documents’ requirements have remained more-or-less intact, others have been melded together, and a couple have been dropped or subsumed into the small print of other points.

For the purposes of our research, we will refer to the 18-point version of the standard, as it was the version that was applied to the majority of the 191 services for which assessment reports have been published.

But, whichever metrics are applied, it is apparent that most services that do not pass their assessment do so because of a failure to do the basics.

The most common point on which services are failed is the very first one: the requirement to understand user needs. (See table right; click on it to open, expand, and magnify)

Of the 34 services that have not made the grade over the years, 25 have fallen at the first hurdle, having been judged to have failed to understand their users’ needs.

Some are on the right track, but have not been thorough enough in their discovery phase.

Last year, the Ministry of Housing, Communities and Local Government put forward an alpha version of its ‘Find an energy certificate tool’. The service failed to pass seven out of 18 points of the standard – including the first.

The assessors’ report said: “It is evident from user research that the team have been able to identify most users of the service. However, the assessment panel felt the numbers – 34 – spoken to is not enough to state that all user needs have been uncovered which would have resulted in breadth of insights.”

It added: “The panel would have liked to see more around what users are trying to do whilst encountering the service, how they interact with the service face to face – moderated testing – where users currently go for help and more around their thoughts, circumstances and behaviours.”

An insufficient effort to understand user behaviour was also cited in the decision not to pass the ‘Manage and Register Pension Schemes’ service put forward for alpha assessment by HM Revenue and Customs in 2018.

“There are few areas where research needs to be focussed, so the team have assurance they have made the right design decisions, and that they thoroughly understand the users’ needs, motivations and behaviours,” the report said. “This includes focusing on the data that is being captured during the registration process, and being clear if the user understands why this is being asked for, and, understanding the issues they face in collecting it, and how this may affect their journey.

It added: “Through research, the team identified that public sector users would have different needs and a different journey. The team described a ‘work-around’ in order to manage this, but the panel were disappointed to learn this. The team must concentrate additional effort on understanding the user needs of the public sector and amend user journeys accordingly. Personas should be clearly segmented by behaviour, goals and attitudes that are not tied to the solution. Some of the presented needs felt like functional requirements.”

Assessors also concluded that this service had failed to meet points four other points and, ultimately, recommended that HMRC undertake a fairly major rethink.

“The service team should redesign this as two separate services, making it more intuitive for users,” they said.

Keep it simple
One of the points that the pensions service did not pass was the 12th, which stipulates that design teams must “create a simple and intuitive service”.

This is the second-most failed point, with 23 services in the last six years having been assessed as falling short in this area.

The GOV.UK design system – an internal cross-government tool developed by GDS to provide “a place for service teams to find styles, components, and patterns to use in designing government services” – was one of the many that, during its alpha assessment, did not meet the requirement of simplicity and intuitiveness.

“The team demonstrated information architecture (IA) improvements they’ve made, however there is scope for further work here, particularly around the ‘patterns’ section,” the report said. “Research has identified ‘pattern’ as an ambiguous label in this context, and including it as a top-level section makes the other navigation labels appear less clearly delineated. The choices made around IA now are likely to have long-lasting impacts, so it’s important to go into private beta with a reasonably mature approach.”

In 2017, NHS Digital put forward for assessment an “information governance toolkit”. The service, which was being developed for use by an estimated total of 25,000 organisations that process NHS data, brings together all relevant government and legal guidance on data processing, as well as offering self-assessment tools.

Assessors failed the service on point 12 and six others, and the recommendations made indicated a great deal of work was necessary to achieve a successful outcome next time.

Whichever metrics are applied, it is apparent that most services that do not pass their assessment do so because of a failure to do the basics – the most common point on which services are failed is the very first one: the requirement to understand user needs

NHS Digital was instructed to “clarify the exact scope of the service and what the priorities and minimal viable product are for the first publicly accessible version”. Designers also needed to “focus on user journeys and outcomes through the service to avoid unnecessary design and interaction”.

The report said: “There has been limited usability testing so far, much behind the expected for a service of this type with the broad range of users identified by the personas. Some thought has been put to the capability of users, but there are some significant assumptions on skills and capabilities to the newly targeted users. It has not been shown that these users have an understanding of the service, their requirements to use it or the capability of users to complete the tasks required of them.”

It added: “Language and terminology is a significant barrier, with different user groups representatives requiring different terminology for the same activities or tasks. Some research has shown that professional users do not want language changed or simplified, although it was unclear to the panel what the motivations were for this as a significant insight from users was the desire to move to ‘Plain English’.”

Teaming problems
Improving the service based on user research and testing – point 2 of the standard – is another common stumbling block, with 20 services being assessed having failed to pass in this area.

The requirement to operate “a sustainable, multidisciplinary team” is not far behind, with 18 services being judged to have failed to fulfil this requirement.

191
Number of services for which assessment reports are available
 

17.8%
Percentage of initial assessments that ended in failure

14
Number of points that must be met under the current standard – down from the previous versions which featured 18 and 26

 

April 2014
Date of the introduction of the first service standard

‘Understanding user needs’
The first requirement of the standing – and the one on which more services fail than any other

One of those to do so was the ‘Local Land Charges’ service put forward by HM Land Registry for alpha assessment in January 2017. Assessors found that the team, while boasting strong technical skills, needed to add those that could bring into focus the needs of people without such expertise.

“The team didn’t have a dedicated content designer, and the solutions presented suffered because of this,” the report said. “A content designer will be able to work with the research team to learn more about how people understand the legal or technical language used within this domain.” 

It added: “The danger of focusing on the internal-facing service first is the team aren’t challenging complex language which will become more obvious when designing and conducted further user research with citizens. This feels like a missed opportunity at the moment. There’s a danger that this could make delivering a simpler, clearer citizen facing service more difficult if this isn’t considered now.”

Other fairly frequent areas of failure were point 13, which requires consistency with GOV.UK design, and point 10, which asks that teams conduct end-to-end testing, including on all applicable browsers and devices. A total of 14 and 13 services failed on these points, respectively.

Point seven, managing data security and privacy, point 11, make a plan for being offline, and point 14, encourage everyone to use the digital service, were each failed by nine services being assessed.

Eight services failed to make code available as open source, which is point 8. 

Six fell short on points four and five: use agile and user-centred methods; and iterate and improve frequently.

Point nine, use open standards, and points 15 and 16 – use analytics to collect data; and establish KPIs and performance benchmarks – have each been failed on five occasions. Four assessments failed to report performance data, as per point 17.

The least common causes of failure were point six, which asks that teams evaluate tools, systems, and ways of procuring them, and the 18th and final point, which stipulates that the service in question must be tested with the minister responsible.

 

This article is part of PublicTechnology’s How to Design a Government Service project, in association with BJSS. This specially created content week will feature a range of exclusive interview, feature, and analysis content dedicated to the art of delivering digital services for citizens and public sector professionals – from the earliest stages of discovery, right through to maintaining live services in use by millions of people. Click here to access all the content.

Related Posts

Hundreds of candidates for Northen Ireland Civil Service jobs affected by tech problems with pre-recorded interviews
January 20, 2025
Major projects: £26bn ICT category has highest proportion of red risk ratings
January 17, 2025
North-west towns deploy digital autopsies
January 16, 2025

Sam Trendall

Learn More →

105 thoughts on “One in six government service assessments ends in failure”

  1. Itz Grippy TV Only Fans Leaks November 20, 2024 at 9:13 pm

    Buy Only Fans Leaks ( Visit https://archiver.fans )

  2. Only Fans Leaks Updates November 20, 2024 at 9:36 pm

    3TB Only Fans Mega

  3. Only Fans Leaks Mega Folders November 21, 2024 at 12:39 am

    Rubi Rose OnlyFans Mega Link Download

  4. Emmanuel Lustin Only Fans Leaks November 21, 2024 at 3:17 pm

    Yasmine Lopez OnlyFans Mega Link Download ( Visit https://archiver.fans )

  5. Hola Bulma Only Fans Leaks November 21, 2024 at 9:32 pm

    Crii Baby RiRi OnlyFans Mega Link Download ( Visit https://archiver.fans )

  6. GG With The WAP Only Fans Leaks November 22, 2024 at 3:00 pm

    Rubi Rose OnlyFans Mega Link Download

  7. North Natt Only Fans Leaks November 22, 2024 at 6:48 pm

    Bulma XO OnlyFans Mega Link Download

  8. Yasmine Lopez Only Fans Leaks November 22, 2024 at 8:12 pm

    Buy Only Fans Leaks ( Visit https://archiver.fans )

  9. Yasmine Lopez Only Fans Leaks November 22, 2024 at 8:21 pm

    Its Lunar Liv OnlyFans Mega Link Download

  10. 10TB Only Fans Mega November 23, 2024 at 5:27 am

    TheRealRebeccaJ OnlyFans Mega Link Download

  11. Genesis Mia Lopez Only Fans Leaks November 23, 2024 at 6:41 am

    Fansly Leaks Mega Link

  12. Black Ass Jenny Only Fans Leaks November 23, 2024 at 12:13 pm

    Buy Leaked Only Fans ( Visit https://archiver.fans )

  13. Mulan Hernandez Only Fans Leaks November 23, 2024 at 12:45 pm

    Mega Link Shop ( Visit https://archiver.fans )

  14. Leah Mifsud Only Fans Leaks November 23, 2024 at 4:32 pm

    Crii Baby RiRi OnlyFans Mega Link Download ( Visit https://archiver.fans )

  15. Buy Fansly Leaks November 23, 2024 at 5:02 pm

    Buy Leaked Content ( Visit https://archiver.fans )

  16. Black Ass Jenny Only Fans Leaks November 23, 2024 at 7:08 pm

    Genesis Mia Lopez OnlyFans Mega Link Download

  17. Lexi 2 Legit Only Fans Leaks November 23, 2024 at 10:27 pm

    Mega Link Store

  18. Rebecca J Only Fans Leaks November 24, 2024 at 1:29 am

    Rebecca J OnlyFans Mega Link Download ( Visit https://archiver.fans )

  19. 8TB Only Fans Mega November 24, 2024 at 11:39 pm

    Barely Legal Lexi OnlyFans Mega Link Download

  20. Bulma XO Only Fans Leaks November 25, 2024 at 2:21 pm

    Updated Only Fans Leaks ( Visit https://archiver.fans )

  21. Ima Cri Baby Only Fans Leaks November 25, 2024 at 4:47 pm

    TheRealRebeccaJ OnlyFans Mega Link Download

  22. Buy Leaked Only Fans November 25, 2024 at 6:05 pm

    Mulan Hernandez OnlyFans Mega Link Download ( Visit https://archiver.fans )

  23. Im xXx Dark Only Fans Leaks November 26, 2024 at 10:53 am

    10TB Only Fans Mega ( Visit https://archiver.fans )

  24. Corinna Kopf Only Fans Leaks November 26, 2024 at 12:33 pm

    Leah Mifsud OnlyFans Mega Link Download ( Visit https://archiver.fans )

  25. Rebecca J Only Fans Leaks November 26, 2024 at 5:13 pm

    TheRealRebeccaJ OnlyFans Mega Link Download

  26. Buy Mega Links November 26, 2024 at 7:15 pm

    Only Fans Leaks Free Download

  27. North Natt Only Fans Leaks November 26, 2024 at 10:58 pm

    Daalischus Rose OnlyFans Mega Link Download

  28. Hot 4 Lexi Only Fans Leaks November 27, 2024 at 7:07 am

    Ima Cri Baby OnlyFans Mega Link Download ( Visit https://archiver.fans )

  29. Buy Leaked Only Fans November 27, 2024 at 12:40 pm

    10TB Only Fans Mega ( Visit https://archiver.fans )

  30. Caaart Only Fans Leaks November 27, 2024 at 5:01 pm

    Jenise Hart OnlyFans Mega Link Download

  31. Black Ass Jenny Only Fans Leaks November 27, 2024 at 5:50 pm

    Only Fans Leaks Free Download

  32. Hot 4 Lexi Only Fans Leaks November 27, 2024 at 6:41 pm

    GG With The WAP OnlyFans Mega Link Download

  33. Gina WAP Only Fans Leaks November 30, 2024 at 6:01 am

    Buy Leaked Content ( Visit https://archiver.fans )

  34. Free Only Fans Leaks November 30, 2024 at 6:13 am

    Rebecca J OnlyFans Mega Link Download ( Visit https://archiver.fans )

  35. 10TB Only Fans Mega November 30, 2024 at 9:41 am

    10TB Only Fans Mega ( Visit https://archiver.fans )

  36. Buy Mega Links November 30, 2024 at 2:11 pm

    Mikaila Dancer OnlyFans Mega Link Download

  37. Buy Fansly Leaks November 30, 2024 at 9:36 pm

    Only Fans Leaks Free Download

  38. Free Only Fans Leaks December 1, 2024 at 1:07 pm

    North Natt OnlyFans Mega Link Download

  39. Im xXx Dark Only Fans Leaks December 1, 2024 at 1:09 pm

    Itz Grippy TV OnlyFans Mega Link Download ( Visit https://archiver.fans )

  40. Buy Mega Links December 1, 2024 at 6:42 pm

    Im xXx Dark OnlyFans Mega Link Download ( Visit https://archiver.fans )

  41. Mega Link Store December 2, 2024 at 2:16 pm

    10TB Only Fans Mega ( Visit https://archiver.fans )

  42. 10TB Only Fans Mega December 2, 2024 at 5:25 pm

    Caaart OnlyFans Mega Link Download

  43. Hot 4 Lexi Only Fans Leaks December 2, 2024 at 6:06 pm

    Jenise Hart OnlyFans Mega Link Download

  44. Buy Mega Links December 2, 2024 at 11:33 pm

    Jenise Hart OnlyFans Mega Link Download

  45. Daalischus Rose Only Fans Leaks December 3, 2024 at 4:28 am

    Caaart OnlyFans Mega Link Download

  46. Im xXx Dark Only Fans Leaks December 3, 2024 at 5:28 am

    Buy Only Fans Leaks ( Visit https://archiver.fans )

  47. Only Fans Leaks Free Download December 3, 2024 at 1:17 pm

    Mega Link Shop ( Visit https://archiver.fans )

  48. Only Fans Leaks Updates December 3, 2024 at 1:39 pm

    Genesis Mia Lopez OnlyFans Mega Link Download

  49. Only Fans Leaks Mega Folders December 4, 2024 at 6:21 am

    Leah Mifsud OnlyFans Mega Link Download ( Visit https://archiver.fans )

  50. Im xXx Dark Only Fans Leaks December 4, 2024 at 7:59 am

    Jenise Hart OnlyFans Mega Link Download

  51. Buy Fansly Leaks December 5, 2024 at 3:30 am

    8TB Only Fans Mega ( Visit https://archiver.fans )

  52. Only Fans Leaks Mega Folders December 5, 2024 at 3:51 am

    Im xXx Dark OnlyFans Mega Link Download ( Visit https://archiver.fans )

  53. Gina WAP Only Fans Leaks December 5, 2024 at 5:15 am

    Emmanuel Lustin OnlyFans Mega Link Download ( Visit https://archiver.fans )

  54. Mega Link Store December 6, 2024 at 12:30 am

    Crii Baby RiRi OnlyFans Mega Link Download ( Visit https://archiver.fans )

  55. Fansly Leaks Mega Link December 6, 2024 at 3:18 am

    Fansly Leaks Mega Link

  56. Free Only Fans Leaks December 6, 2024 at 4:41 am

    Rebecca J OnlyFans Mega Link Download ( Visit https://archiver.fans )

  57. Buy Leaked Content December 6, 2024 at 6:54 am

    Daalischus Rose OnlyFans Mega Link Download

  58. Free Only Fans Leaks December 7, 2024 at 6:19 pm

    Gina WAP OnlyFans Mega Link Download ( Visit https://archiver.fans )

  59. Crii Baby RiRi Only Fans Leaks December 7, 2024 at 6:52 pm

    Mega Link Store

  60. Yasmine Lopez Only Fans Leaks December 8, 2024 at 6:57 am

    Rubi Rose OnlyFans Mega Link Download

  61. Only Fans Leaks Mega Folders December 8, 2024 at 10:18 am

    GG With The WAP OnlyFans Mega Link Download

  62. Crii Baby RiRi Only Fans Leaks December 9, 2024 at 1:10 am

    Mega Link Store

  63. Barely Legal Lexi Only Fans Leaks December 9, 2024 at 4:34 am

    Its Lunar Liv OnlyFans Mega Link Download

  64. Daalischus Rose Only Fans Leaks December 9, 2024 at 8:49 am

    Only Fans Leaks Free Download

  65. Black Ass Jenny Only Fans Leaks December 9, 2024 at 1:41 pm

    Bulma XO OnlyFans Mega Link Download

  66. Corinna Kopf Only Fans Leaks December 9, 2024 at 2:36 pm

    Hola Bulma OnlyFans Mega Link Download ( Visit https://archiver.fans )

  67. Only Fans Leaks Free Download December 11, 2024 at 9:07 am

    Itz Grippy TV OnlyFans Mega Link Download ( Visit https://archiver.fans )

  68. Lexi 2 Legit Only Fans Leaks December 11, 2024 at 9:16 am

    Buy Only Fans Leaks ( Visit https://archiver.fans )

  69. Itz Grippy TV Only Fans Leaks December 11, 2024 at 11:53 am

    Mega Link Store

  70. Updated Only Fans Leaks December 11, 2024 at 12:50 pm

    Barely Legal Lexi OnlyFans Mega Link Download

  71. Taylor Hall Only Fans Leaks December 12, 2024 at 12:25 am

    Rebecca J OnlyFans Mega Link Download ( Visit https://archiver.fans )

  72. Updated Only Fans Leaks December 12, 2024 at 2:03 am

    Mulan Hernandez OnlyFans Mega Link Download ( Visit https://archiver.fans )

  73. Only Fans Leaks Mega Folders December 12, 2024 at 2:50 am

    Buy Fansly Leaks ( Visit https://archiver.fans )

  74. Black Ass Jenny Only Fans Leaks December 12, 2024 at 9:56 am

    Daalischus Rose OnlyFans Mega Link Download

  75. Buy Mega Links December 12, 2024 at 11:07 am

    Corinna Kopf OnlyFans Mega Link Download ( Visit https://archiver.fans )

  76. Bulma XO Only Fans Leaks December 12, 2024 at 1:43 pm

    Mega Link Shop ( Visit https://archiver.fans )

  77. Barely Legal Lexi Only Fans Leaks December 12, 2024 at 7:06 pm

    North Natt OnlyFans Mega Link Download

  78. Mikaila Dancer Only Fans Leaks December 13, 2024 at 5:16 am

    Itz Grippy TV OnlyFans Mega Link Download ( Visit https://archiver.fans )

  79. Gina WAP Only Fans Leaks December 13, 2024 at 9:19 am

    Its Lunar Liv OnlyFans Mega Link Download

  80. Buy Only Fans Leaks December 13, 2024 at 10:39 am

    Buy Fansly Leaks ( Visit https://archiver.fans )

  81. Hola Bulma Only Fans Leaks December 13, 2024 at 3:19 pm

    Taylor Hall OnlyFans Mega Link Download

  82. Yasmine Lopez Only Fans Leaks December 15, 2024 at 2:45 am

    Leah Mifsud OnlyFans Mega Link Download ( Visit https://archiver.fans )

  83. Jenise Hart Only Fans Leaks December 15, 2024 at 3:37 am

    Mulan Hernandez OnlyFans Mega Link Download ( Visit https://archiver.fans )

  84. Updated Only Fans Leaks December 15, 2024 at 7:26 am

    Buy Mega Links ( Visit https://archiver.fans )

  85. Itz Grippy TV Only Fans Leaks December 15, 2024 at 11:11 am

    Mulan Hernandez OnlyFans Mega Link Download ( Visit https://archiver.fans )

  86. 10TB Only Fans Mega December 15, 2024 at 10:58 pm

    Caaart OnlyFans Mega Link Download

  87. Buy Only Fans Leaks December 15, 2024 at 11:47 pm

    Buy Leaked Only Fans ( Visit https://archiver.fans )

  88. Yasmine Lopez Only Fans Leaks December 16, 2024 at 10:07 am

    Itz Grippy TV OnlyFans Mega Link Download ( Visit https://archiver.fans )

  89. Buy Mega Links December 16, 2024 at 11:54 am

    8TB Only Fans Mega ( Visit https://archiver.fans )

  90. Mulan Hernandez Only Fans Leaks December 18, 2024 at 2:17 am

    Free Only Fans Leaks ( Visit https://archiver.fans )

  91. Emmanuel Lustin Only Fans Leaks December 18, 2024 at 2:17 am

    Emmanuel Lustin OnlyFans Mega Link Download ( Visit https://archiver.fans )

  92. Only Fans Leaks Free Download December 18, 2024 at 3:43 pm

    Emmanuel Lustin OnlyFans Mega Link Download ( Visit https://archiver.fans )

  93. Yasmine Lopez Only Fans Leaks December 19, 2024 at 10:45 am

    Buy Leaked Content ( Visit https://archiver.fans )

  94. NamiSwanBB NamiBBSwan December 19, 2024 at 6:55 pm

    TheThroatBully OnlyFans Leaks Mega Folder Link Download

  95. Tytiania Sargent Porn December 19, 2024 at 7:25 pm

    BombshellMint ( https://UrbanCrocSpot.org/shop )

  96. GinaWAP Only Fans PPVS Download December 20, 2024 at 12:13 am

    GinaWAP Only Fans PPVS Download https://urbancrocspot.org/tag/gg-with-the-wap/

  97. GinaWAP Only Fans Leaks December 20, 2024 at 8:09 am

    GGWithTheWAP Mega Folder Download ( https://urbancrocspot.org/gina-wap-gg-with-the-wap-only-fans-mega-link-9gb/ )

  98. BombshellMint December 20, 2024 at 11:40 am

    Tytiania Sargent Nude Leaks ( https://UrbanCrocSpot.org/ )

  99. Sammy02kVIP December 20, 2024 at 6:38 pm

    SlimAssTink SlimGoesRounds OnlyFans Leaks Mega Folder Link Download

  100. The Real Bombshell Mint December 20, 2024 at 7:04 pm

    BombshellMint OnlyFans PPVs ( https://urbancrocspot.org/the-real-bombshell-mint-only-fans-mega-link/ )

  101. The Real Bombshell Mint Only Fans Leaks December 30, 2024 at 10:11 pm

    Tytiania Sargent Naked Only Fans ( https://UrbanCrocSpot.org/shop )

  102. GinaWAP Mega Folder Link January 2, 2025 at 2:10 pm

    GGWithTheWAP Only Fans PPVs ( https://urbancrocspot.org/gina-wap-gg-with-the-wap-only-fans-mega-link-9gb/ )

  103. JaeXGalore BrattyJaeX January 2, 2025 at 4:33 pm

    EmpressElfiie LittleElfiie OnlyFans Leaks Mega Folder Link Download

  104. Tamekia January 3, 2025 at 4:34 am

    Fine Ass Shanice OnlyFans Leaks Mega Folder Link Download ( https://UrbanCrocSpot.org )

  105. Tytiania Sargent Naked Only Fans January 3, 2025 at 5:06 am

    The Real Bombshell Mint ( https://urbancrocspot.org/the-real-bombshell-mint-only-fans-mega-link/ )

Leave a Reply

Your email address will not be published. Required fields are marked *