Implementation of the Australian National Safety and Quality Digital Mental Health Standards

Abstract

Objective

Following the release of the Australian National Safety and Quality Digital Mental Health (NSQDMH) Standards in November 2020, our objective was to ensure effective implementation of the Standards. This included the development of an accreditation scheme to allow digital mental health services to be formally assessed against the Standards and provide service users with an assurance of safe and high-quality services.

Method

The accreditation scheme was adapted from the one used for the National Safety and Quality Health Service (NSQHS) Standards. As the digital mental health sector was largely unfamiliar with standards and accreditation, the scheme was pilot-tested with a range of service providers and further consultation took place to refine it.

Results

The accreditation scheme was launched in November 2022 with support resources published to aid implementation.

Conclusions

The NSQDMH Standards accreditation scheme provides an assurance of safety and quality for digital mental health service users.

The development of Australian National Safety and Quality Digital Mental Health (NSQDMH) Standards has been previously reported.^1-3 Evolving from the National Safety and Quality Health Service (NSQHS) Standards, ⁴ the NSQDMH Standards were developed from 2018 and released in November 2020. ⁵ Australia was the first nation to develop standards specifically targeting digital mental health services and is recognised globally as leading in both digital mental health service provision and safety standards development. ⁶

The principle aim of the NSQDMH Standards is to improve the safety and quality of services and protect users from harm. The Standards also align with Australian Privacy Principles, Consumer Law, Therapeutic Good Administration (TGA) regulation and the Australian Digital Health Agency’s assessment framework for mHealth apps. ⁷ The release of the National Digital Mental Health Framework in 2021 highlights the timeliness of the NSQDMH Standards in meeting the framework objectives including enhancing trust and confidence in digital mental health services. ⁸

Accreditation to the NSQDMH Standards is voluntary, although it is likely that a combination of funding requirements and consumer awareness will drive expectations that providers should be accredited to recognised standards, such as the NSQDMH Standards. Accreditation can assure service users that the service is safe and of high quality, meeting expected standards in areas including protecting privacy and information security.

Method

Following the launch of the NSQDMH Standards, eight digital mental health provider organisations piloted assessment to the Standards in early 2021 to test implementation feasibility (see Table 1) and develop support tools.

Table 1.

Accreditation pilot study of digital mental health services

• Between March and May 2021, eight digital mental health service providers participated in a structured pilot study run by the Commission, testing the methodology of an accreditation scheme assessing service providers against the NSQDMH Standards.

• Three independent accrediting agencies assessed the service providers and reported findings.

• The Commission conducted feedback sessions in 2021 with both service providers and agencies to evaluate the proposed scheme and the appropriateness of the support resources.

• Further structured interviews took place with service providers in 2022 to explore challenging aspects of the Standards; these interviews formed the basis of a series of explanatory ‘action guides’ on specific aspects of the Standards (see Table 3).

The Commission has well-established accreditation schemes for hospitals and primary care services; however, standards accreditation was largely unfamiliar to the digital mental health sector. The accreditation scheme tested in the pilot study followed the framework of existing schemes recognising that additional resources and elements might be required for the sector.

Following the outcome of the pilot study, the accreditation scheme policy was revised and then reviewed by legal experts before publication on the Commission’s website (see Table 2). The scheme was formally opened in November 2022, integrating implementation resources and support tools (see Table 3) informed by the pilot. ⁹

Table 2.

NSQDMH Standards assessment process

1	Application – Service providers complete an application for an assessment with an approved accrediting agency.
2	Schedule – Service providers implement the NSQDMH Standards and work with their accrediting agency to clarify dates, requirements and arrangements for an assessment.
3	Application for not applicable actions – Where relevant, a service provider may apply to their accrediting agency for relevant actions to be rated ‘not applicable’ at assessment.
4	Initial assessment – The accrediting agency conducts an initial assessment consisting of:
	a. Desktop review – A review of evidence submitted by the service provider to demonstrate conformance to the NSQDMH Standards
	b. Verification – An assessment, either on-site or virtually, to verify the safety and quality systems described by service provider.
5	Exceptions report – The accrediting agency provides a report to the service provider outlining the outcome of the initial assessment including specifying actions that are rated ‘not met’ or ‘met with recommendations’ and the reasons for these ratings within five business days of the initial assessment.
6	Remediation period – Where required, a period of 60 business days is allowed for a service provider to address any areas where the service provider does not comply with the requirements in the NSQDMH Standards.
7	Final assessment – For actions that were ‘not met’ or ‘met with recommendations’ at the desktop review and verification.
8	Outcome and final report – The accrediting agency determines the accreditation outcome based on the final assessment. The agency will deliver a final assessment report within 30 days of completion of the final assessment detailing findings for each action assessed, including suggestions for improvement, and requirements to achieve full compliance on actions that are rated ‘met with recommendations’.
9	Submission of data – The Commission receives data on assessment outcomes from the accrediting agency by the 10th day of each month.
10	Registration – The Commission enters the details of service providers who are ‘Accredited’ or ‘Working Towards Accreditation’ on a register of assessed digital mental health service providers. The service provider will receive an accreditation badge to display on the accredited service(s).
11	Ongoing monitoring – Service providers continue their monitoring and quality improvement process working to fully implement or continue to comply with the NSQDMH Standards throughout the 3-year accreditation cycle.

Table 3.

NSQDMH Standards implementation support tools

Support tool	How it supports service providers
Self-assessment tool	An Excel workbook that allows service providers to track their progress in meeting the NSQDMH Standards
Product information template	Guidance on the key information required by service users to decide if a service will meet their needs
Guide for service providers	Provides detailed strategies, evidence and resources for providers to meet all actions in the Standards
Action guides	Detailed advice on how to implement specific actions of the NSQDMH Standards including privacy, partnering with service users and healthcare rights
Fact sheets	Summarises key processes and policies in the NSQDMH Standards including assessment and accreditation
NSQDMH Standards webinars	Recordings of subject matter experts answering practical questions related to implementation such as accreditation, privacy and cyber security, governance and partnering with service users
Map of the NSQDMH Standards with the NSQHS Standards	Assists services already accredited to the NSQHS Standards review the additional actions they need to implement to meet the NSQDMH Standards

Accreditation to the NSQDMH Standards is conducted under the Australian Health Service Safety and Quality Accreditation (AHSSQA) Scheme ¹⁰ and performed by approved independent accrediting agencies.

A key feature of the NSQDMH Standards accreditation scheme is the ability to undertake all assessment components online, facilitating the process.

The first service provider achieved accreditation in April 2023 with further providers currently engaged in assessment.

Results

The NSQDMH Standards are aligned to other sets of Commission-developed standards. To avoid duplicating effort for organisations with active NSQHS Standards accreditation, but who also offer a digital mental health service, mapping was completed to identify where additional implementation tasks are required. ⁹ These services are only required to meet 24 of the 59 NSQDMH Standards actions; those which primarily relate to technical criteria such as privacy, data management and cyber security.

The accreditation process for digital mental health service providers is largely consistent with other Commission-developed standards; however, it includes some unique features which address the sector’s unfamiliarity with the accreditation process. For example, services that do not meet all actions but meet more than 50% may receive an award of ‘working towards accreditation’ (see Table 4). This mechanism encourages services to participate in the scheme and develop their understanding of implementation of standards.

Table 4.

NSQDMH Standards accreditation outcome classifications

- Accredited to the NSQDMH Standards: meets the requirements of all applicable actions.

- Working towards accreditation to the NSQDMH Standards: meets more than 50% of the required actions.

- Not accredited to the NSQDMH Standards: meets less than 50% of the required actions.

Accredited services are also authorised to use a digital accreditation badge to assist all service users easily identify an accredited service. All services deemed accredited or working towards accreditation are listed on the Commission’s online register ¹¹ .

A communications strategy following launch of the NSQDMH Standards accreditation scheme facilitated consistent updates to service providers ensuring awareness of Commission resources. A series of educational webinars ran from November 2022 to August 2023 to complement the communications strategy and remain available online. ⁹

Discussion

For almost two decades Australia has played a leading role in the research, design and delivery of digital mental health tools and platforms. Digital mental health services have seen significant growth over the past few years, with a 62% increase in the proportion of users accessing supported services in Australia between 2019 and 2021 coinciding with the COVID-19 pandemic. ¹² Studies also indicate many people using digital mental health services are not accessing any other kind of support, suggesting the significant reach of these services to otherwise under-served population groups and the corresponding importance of ensuring safety and quality. ¹³ In addition, use of artificial intelligence is becoming more common among digital mental health services through the integration of chat bots and other tools into therapy. ¹⁴ As this technology becomes more sophisticated and widespread, appropriate regulatory approaches and oversight may be needed to mitigate patient safety risks.

The release of the NSQDMH Standards and its accreditation scheme signifies a turning point for governments and regulators in acknowledging the unique benefits and risks of delivering mental health services online. Several international jurisdictions have also developed standards, assessment frameworks and certification schemes for digital health technologies.^20,21 This movement towards a standards-based regulatory approach is significant as health care becomes increasingly more digitised, online risks including cyber security threats more prevalent and healthcare consumers demand more flexible service delivery.

Linked to minimising the risk of harm to service users, a central aim of accreditation to the NSQDMH Standards is to provide consumers, carers and clinicians with a means of identifying and choosing to use safe and high-quality digital mental health services (see Table 5). The next phase in the Commission’s implementation of the Standards will focus on communicating the value of accreditation to a wide range of service users and will highlight how an accredited service can be readily identified via the accreditation badge. Awareness raising will include ensuring all Australian digital mental health services eligible for accreditation are encouraged to do so; encouraging clinicians including psychiatrists, psychologists and general practitioners to refer patients to accredited digital mental health services where appropriate; and encouraging service users to engage with accredited services. Part of this process will also involve ensuring there are clear pathways for enforcing regulation of digital mental health services with recent research highlighting that responsibility currently falls between a number of entities. ²² Evaluation of the effectiveness of the Standards will also be a key future task.

Table 5.

Standards and improvements in safety and quality

• Healthcare standards have been adopted in more than 70 countries worldwide. 15 A growing body of research has shown standards to be effective in facilitating continual and systematic quality improvement within an organisation. 16

• Approximately 38% of OECD countries have no clinical standards or vendor certification of electronic record systems limiting the interoperability of health data. 17

• Both healthcare standards and accreditation have been found to foster staff reflection, a higher standardisation of practices and a greater focus on quality improvement 18 as well as contributing to higher patient satisfaction. 19

• Post-study interviews conducted with service provider participants in the NSQDMH Standards pilot assessments recorded significant benefits of assessment including improving clinical and technical governance and highlighting the need for privacy protections.

It will be crucial for a broad range of stakeholders to continue working together to further strengthen the digital mental health sector in Australia. This will need to include the Australian Government, state and territory governments, regulators such as the TGA and Office of the Australian Information Commissioner, the Australian Digital Health Agency, service users and industry to make digital mental health services safe and deliver improved service user outcomes.

Conclusions

The NSQDMH Standards support a framework for safe and high-quality delivery of digital mental health care. Standards-based regulation is one mechanism to drive quality improvement and reduce potential harms to service users and their support people as more mental health services seek to harness digitally enabled tools.

The Commission has a suite of targeted resources to assist services in implementing the NSQDMH Standards which develop good clinical and technical governance practices, improve their safety and quality systems and support better integration of user-centric models of care.

Accreditation to the NSQDMH Standards provides an assurance to clinicians and service users alike that a service provider’s digital offerings meet stringent safety and quality requirements and the service is committed to improving upon service delivery. Evaluation of the NSQDMH Standards may be an important aspect of ensuring the accreditation scheme remains fit for purpose in an evolving technological landscape.