Abstract
Pregnancy and the postnatal period are opportune times to support parental health and encourage future child health and wellbeing. txtpēpi is an individually tailored text message programme for parents and families providing information and support during pregnancy and the postnatal period. txtpēpi was developed for a region of Aotearoa New Zealand, with a high proportion of Māori (the Indigenous population) and people living rurally or in areas of high deprivation. The paper presents a mixed-methods evaluation of txtpēpi using the RE-AIM Framework, involving interviews with 40 end-users and nine stakeholders, and analysis of system-recorded end-user data. The results suggest that txtpēpi achieved a high degree of reach, impact, implementation and adoption. Ongoing maintenance of txtpēpi was endorsed with suggested areas of improvement. Programme uptake and engagement from Māori suggest that this type of intervention provides a promising and acceptable tool to support improvement in equity of child health outcomes.
Introduction
The perinatal period and infancy are instrumental in establishing one’s health and wellbeing throughout the life course (Center on the Developing Child, 2025; World Health Organization, 2023). Infant nutrition in particular is key to achieving healthy development, immune function and prevention of child and adulthood obesity (World Health Organization, 2023). In addition to increasing one’s risk for a range of physical and mental health conditions, poor health and development during infancy are also associated with poorer long-term outcomes such as low educational attainment, substance misuse and socioeconomic challenges (Bell et al., 2018; Case et al., 2005; Currie et al., 2010; Fergusson and Horwood, 2001; Gondek et al., 2019; Latham, 2015; World Health Organization, 2023). Pregnancy and the postnatal period are therefore an opportune time to support parental health, inform expectant parents about health-promoting behaviours and encourage future child health and wellbeing (Lupton, 2016).
mHealth tools such as short message service (SMS; also known as text messaging), mobile applications, and sensors have been used in a range of health and disease areas, including parent and child health (Poorman et al., 2015). Due to the ubiquity and adaptability of mobile devices, mHealth has repeatedly been recognised for its potential to promote equity by reaching population groups with challenges in accessing or benefitting from traditional health services (Armaou et al., 2020; Gibbons, 2005; Marcolino et al., 2018). This potential is particularly important for rural communities and smaller urban centres where access to healthcare and workforce availability may be limited (Mbuthia et al., 2019).
Text messages in particular have been widely used in public health interventions due to their low delivery costs and frequent use across population groups and regions (Hall et al., 2015; Head et al., 2013). Further, text messages are easy to tailor to individual needs at a large scale, making them ideal for the delivery of culturally tailored, multilingual interventions (Armaou et al., 2020; Gonzalez et al., 2021). Text messages are also flexible with the ability to send them at any time of day and thus can offer support when it is most needed (Head et al., 2013; Militello et al., 2016). Text messages have been effectively implemented in parent and child health interventions with positive impacts on appointment adherence, service utilisation, rates of complications, physical activity levels and healthy gestational weight (Evans et al., 2012; Laar et al., 2019; Lund et al., 2014; Willcox et al., 2017).
We recently adapted an existing parent and child health text message programme for expectant parents, postpartum parents and family members of infants in a region of Aotearoa New Zealand in which there is a high proportion of Māori (the Indigenous people of Aotearoa New Zealand; 38% compared to the national average of 17%) and people living rurally or in areas of high deprivation (Lakes District Health Board, 2022). Māori are a population group with known disparities in child health outcomes (Paine et al., 2022) and who experience inequitable access to health services and support (Graham and Masters-Awatere, 2020). Cultural adaptation of the existing mHealth programme was an intensive process conducted using a modified five-stage mHealth Development and Evaluation Framework (Whittaker et al., 2012). Results from the initial stages (1–3) of the Framework (conceptualisation and consultation, intervention development, and pre-testing and refinement) will be published separately. In this paper we report the implementation and mixed-methods impact evaluation of the culturally adapted text message programme, txtpēpi (stages 4–5 of the Framework).
The aims of this evaluation study were to explore the impact and acceptability of the txtpēpi programme. Findings were intended to inform answers to the following questions:
Was the programme delivered as intended?
What impacts has txtpēpi had on end-users?
What factors can help to minimise barriers in accessing txtpēpi?
What factors influence whether txtpēpi is taken up, is acceptable, and is supported by end-users and stakeholders?
How can the programme be maintained once initial funding has ended?
Description of txtpēpi
txtpēpi is a parent and child health text message programme for māmā (mothers) and whānau (family members) living in the Te Whatu Ora | Health New Zealand Lakes region (formerly the Lakes District Health Board) which served about 118,000 residents and covered 9570 km2 at the time of this project (Lakes District Health Board, 2022). Briefly, txtpēpi was locally adapted from a programme implemented in another region of Aotearoa New Zealand, called TextMATCH (Dobson et al., 2017). The text message programme aims to support families and promote optimal health during pregnancy and infancy (Dobson et al., 2017). The main content areas include nutrition and physical activity during pregnancy, healthy infant feeding and early childhood nutrition and physical activity, immunisations, safety, oral health, sleep, smoking cessation and emotional encouragement. Messages are text only but links to local support service webpages are provided where appropriate. See Box 1 for example messages.
Example txtpēpi messages.
Individuals can enrol in txtpēpi at any point between 5 weeks’ gestation and 24 months postnatal. There are four versions of txtpēpi based on the end-user’s relationship to the infant (māmā or whānau) and preferred language (English or Te Reo Māori [Māori language]) and text messages are further personalised by location, recipient’s name, baby’s name and baby’s gender. The content is tailored to the pregnancy stage or baby’s age using information provided by users at registration. End-users who enrol during pregnancy are provided the option to switch to the postnatal programme when appropriate. Txtpēpi was embedded in existing maternal and child health services (e.g. Plunket, maternity ward at the hospital) in the region as well as broader health and social services (e.g. Whānau Ora providers). Services had flyers and collateral about the programme, which were integrated into their existing communication with target users (e.g. newsletters, social media pages). Staff in these services were provided with the information and support to promote the programme and refer or register people to the programme.
txtpēpi is delivered by a bespoke in-house online platform that accommodates the above cultural and individual tailoring requirements, sourcing its information from the end-users during registration. Upon registration, end-users begin receiving the programme based on their stage of pregnancy or baby’s age (rather than a universal Day 1). End-users can withdraw at any time by replying to any txtpēpi text message with STOP or KATI (STOP in Te Reo/Māori language) which cancels all future messages. The platform contains message libraries for all variations of the programme and is scheduled to send two to three messages per week depending on pregnancy stage/baby’s age. While all processes are automated, the platform allows for manual input from authorised project team members. The system maintains logs of registration details, user statuses (active, withdrawn or complete), programme allocations (culture, language and relationship to baby), and all outgoing and incoming messages (txtpēpi users were invited to provide regular feedback during the programme by free reply message). Once live, the costs to deliver the programme include the monthly fees for the Gateway API (NZ$250) and shortcode (NZ$250), text message costs (NZ$ 0.10 per message), and approximately 1 hour of staff time per week for message monitoring and issue management.
Methods
Study design
In following Whittaker et al.’s recommendation to focus on implementation from the outset (Whittaker et al., 2012), this work was informed by the enhanced Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM) Framework (Glasgow et al., 1999, 2019) and adopted a pragmatic mixed-methods approach. The use of both quantitative and qualitative measures is recommended to not only understand outcome metrics but also to understand how and why the outcomes were attained (Holtrop et al., 2018). Components of the evaluation as they apply to the RE-AIM framework are summarised in Table 1. The 24-month evaluation period was between February 2021 and January 2023. Ethical approval for the study was obtained from the New Zealand Health and Disability Ethics Committee (21/NTB/84). Research approval from Te Whatu Ora | Health New Zealand Lakes region was also obtained.
Components of the txtpēpi evaluation.
Dimensions in accordance with the definitions provided by Glasgow et al. (2019)
Originally the effectiveness of txtpēpi was to be assessed using routinely collected key child indicator data (e.g. immunisation rates) but COVID-19-associated restrictions and lockdowns during the study period impacted the validity of those data.
Calculated by the time between start date and either (1) the date they withdraw by texting STOP, (2) the completion of the pregnancy programme if they do not switch to the baby programme, (3) the date they are deregistered due to their phone number becoming inactive on the network or (4) the date of the final message at child’s 2-year mark.
Participants and recruitment
Programme end-users
txtpēpi went live in February 2021. Programme enrolment initially occurred via self-enrolment or referral from a health professional. Information about the programme was advertised in relevant Facebook groups and at local hospitals, hapū wānanga (pregnancy education services), lead maternity carers and Well Child Tamaraki Ora (WCTO) providers (a free, community-based health programme for all New Zealand families in the first 5 years of a child’s life). In September 2021, an opt-out method was added for all hapū māmā (pregnant women) or whānau enrolled in hapū wānanga (Kaupapa Māori pregnancy and parenting programmes) in the region (Dobson et al., 2022). This method was added to minimise access barriers related to registration and self-enrolment, particularly during a period of variable Covid-related restrictions. An early analysis found that the opt-out method was associated with a 77% increase in enrolments, no significant change in the proportion of Māori enrolments and no significant increase in withdrawal rates compared to the period before this method was introduced (Dobson et al., 2022). Additionally, no complaints about the opt-out method were received in response to regular requests for general programme feedback via free reply message.
All end-users of txtpēpi who enrolled during pregnancy or within 6 weeks of the child’s birth were eligible for inclusion in the interview study which was conducted at 12 months post-go-live. Those who signed up after their baby was 6 weeks old were excluded due to the nature of outcomes planned. Additional inclusion criteria were the ability to read and understand English or Te Reo Māori, the ability to provide informed consent and residence in the Lakes region. Only one interview was to be conducted per whānau but all whānau members who received the programme were welcome to attend the interview.
End-users were invited to take part in the interview study when the system indicated their child was 6 months old, regardless of whether they were still actively enrolled in txtpēpi. Recruitment outreach occurred via text message or phone call. Those who indicated interest were sent a participant information sheet and consent form in their preferred language (English or Te Reo Māori) via online link or post, then were phoned by a research assistant who answered any questions and obtained informed consent. Participants had the option to consent to (1) collection of their child’s relevant health data via regional patient electronic health records, (2) participation in an interview to understand their experiences of txtpēpi or (3) both. The use of de-identified system-recorded engagement data was already consented to upon enrolment into the wider programme.
Stakeholders
Stakeholders were identified in the early stages of the txtpēpi’s development and throughout programme implementation. Purposive sampling was used to identify individuals involved in the decision-making around, or delivery of, health services or support to parents and whānau during the perinatal period in the Lakes region. Relevant roles included healthcare workers (e.g. lead maternal carers), members of relevant community organisations (e.g. WCTO providers), and key decision makers from Te Whatu Ora | Health New Zealand Lakes region). Prospective stakeholders were contacted via email and, if interested, provided with a link to an information sheet and consent form. Prospective stakeholders were then phoned by a research assistant to answer any questions and obtain informed consent to take part in an interview.
Procedures and data collection
Quantitative measures
Upon consenting to take part in the child health outcomes analysis, end-users were asked to provide the necessary information to access their child’s health record. This was used to obtain relevant child health outcomes routinely collected by the regional health service. At the outset of the project, the primary outcome was intended to be the child’s completion of all age-appropriate immunisations by 6 months of age. Secondary outcomes included metrics related to primary healthcare enrolment, WCTO engagement, breastfeeding rates and living in a smoke-free home. However, as the project progressed it became apparent that the validity of these measures was compromised by Covid-19-related disruptions (including strict national and regional lockdowns) in the delivery of, and access to, routine child health services across Aotearoa New Zealand (Gauld, 2023). An example of this is national childhood immunisation rates which dropped during this period (Charania et al., 2024). It was therefore decided that other measures of impact were optimal during these atypical circumstances. Qualitative feedback from end-users and stakeholders, already being collected as part of this evaluation and further described below, was determined to be the most valid and pragmatic solution to understanding txtpēpi’s impacts.
Engagement with the programme was assessed using system-generated data. This included the average duration of receiving txtpēpi, the dosage of messaging and the timing of sign-up and exit.
Qualitative measures
End-user interviews
End-users who consented to take part in an interview were given the choice of format (in-person or phone call, dependent on Covid restrictions); alone or with whānau also enrolled in txtpēpi; and language (English or Te Reo Māori). Interviews commenced 12 months post-go-live, were conducted by a trained local study team member (AU) and occurred within 1 month of being invited to take part. End-user interviews included both structured and semi-structured questions adapted from existing materials (Dobson et al., 2017; Growing Up in New Zealand, 2010) and input from the project’s kaitiaki rōpū (governance group). Interviews covered the following topics:
Overall thoughts and feelings about txtpēpi
Perceived impacts of txtpēpi
Barriers to engaging in services referred to in the messages
Suggestions for improvement
Parent-child interaction
Engagement with safe sleep practices
Participant demographics were collected at the end of each interview, which lasted approximately 30–45 minutes each. Interviews continued until data sufficiency was reached, with a maximum of 50 interviews planned. All interviews were audio recorded and notes taken. End-users received a $20 gift card in recognition of their time.
Stakeholder interviews
Stakeholders who consented to participate in an interview were given the choice of format (in-person or video conference; one-to-one or in a colleague pair) and were conducted by a trained study team member (KG). Stakeholder interviews commenced 12 months post-go-live. Interviews were semi-structured and covered the following topics:
• Barriers/enablers for use of txtpēpi
• Adoption of the programme including reasons for non-engagement
• Resources/costs for staff providing referrals
• Perceived benefits of txtpēpi to expectant/new parents and whānau
• Suggestions for improvement
• Alignment with other services
• Potential maintenance of the service beyond the study period
Stakeholder interviews lasted approximately 20–30 minutes and continued until key stakeholder groups were sufficiently represented, with a maximum of 10 interviews planned. All interviews were audio recorded for reference, and notes were taken.
Data analysis
Quantitative data were analysed and summarised using descriptive quantitative analyses including means, standard deviation and proportions. Qualitative interview data were analysed by KG and RD using simple, inductive thematic analysis which followed the six steps summarised by Braun and Clarke (2006).
Results
Reach
Demographic characteristics
A total of 278 end-users enrolled in txtpēpi in the first 12 months post-go-live. Of these, the majority registered for the pregnancy programmes (63%; n = 176) compared with the postnatal programmes (37%; n = 102), and for English language versions (93%; n = 258) compared with Te Reo Māori versions (7%; n = 20). A summary of the characteristics of all registered users can be found in Table 2.
Characteristics of all end-users registered for txtpēpi during the evaluation period (n = 278).
Multiple output method.
Among the 278
Fourteen
Proportion of target population reached
The 254 mothers who registered for txtpēpi represent an approximate reach of 15.8% of the target population. This is estimated based on the total number of live births in the region in 2021, n = 1611. A higher proportion of txtpēpi end-users identified as Māori (60%) compared with the proportion in the total Lakes region population (projected to be 38% in 2021; Lakes District Health Board, 2022). Enrolment demographics for txtpēpi by age and ethnicity were comparable to the population of eligible people giving birth in the Lakes region in 2021 (Te Whatu Ora | Health New Zealand, 2024).
Qualitative feedback related to reach
“It is pretty cool to have Māori incorporated, was really happy with that.”
Participants also endorsed the relevance of the programme content in the context of being a first-time parent, providing them with practical information that was either new or a useful reminder. They also appreciated the simplicity of the messages, which made the information accessible and easy to retain (98% stated the messages were easy to understand).
“Short and to the point. So, they were easier to remember.”
There were also reach-related aspects of the programme that some end-user interviewees disliked. A minority stated that because of their non-Māori background, they struggled to understand some Te Reo Māori words interspersed in the messages. Some also commented that the information in some educational messages was too obvious, although this did not cause offence.
Three end-user interviewees had withdrawn mid-programme by texting STOP. These participants reported withdrawing from the programme as they did not want to receive any more messages or did not find the messages useful. Of the eight end-user participants who enrolled in pregnancy and did not switch to the postnatal programme, reasons included that they forgot (n = 4), didn’t want the baby messages (n = 2) and the remaining two did not specify a reason.
Many of the end-users’ suggestions for txtpēpi were around the promotion of the programme: they wanted to see more advertising so that others could know about it and access it. Other suggestions to increase accessibility, relevance and appropriateness of txtpēpi were content-related and included more links to services in end-users’ specific locations, information about infant sleep and more messages of encouragement as opposed to informational messages.
“It would be nice to have it widely promoted so more people can benefit.”
Stakeholders also provided feedback on the aspects of txtpēpi which they perceived to enhance the programme’s reach. Three stakeholder participants reported having a positive experience helping end-users sign up for the programme, noting that it was a straightforward process involving a QR code linking registrants to a simple online form. Three stakeholders also felt that txtpēpi had the ability to reach people who wouldn’t normally have access to services or wouldn’t typically want to engage with other types of services.
“Especially for whānau that don’t want to engage. It was a quick easy way to get some key information to them.”
Stakeholders also appreciated that the programme encouraged whānau to be involved in the child’s care, that it is run by community organisations, that it is free to end-users and that it is available in multiple languages, thus speaking to its accessibility and relevance to the community.
Stakeholders also noted factors that negatively impacted txtpēpi’s reach. These included a lack of mobile coverage/internet in some parts of the region, users not having their own mobile phone, lack of referrer knowledge about txtpēpi and lack of time to provide referrals. Two participants also described concerns about the sign-up processes. One described it as too complex, and one reported that directing end-users to an external university-hosted registration webpage was a barrier as it took away from the community feel of the programme. The university-hosted registration page was also interpreted by some as an agreement to participate in research, which was met with concerns requiring reassurance that this was not the case.
“It’s hard to direct people - we did direct them to [the registration page], but they see that [university name] and sort of jump off there.”
When asked about how the programme could be more accessible and appropriate to the community, a range of ideas were discussed. Two stakeholders had suggestions related to the messages themselves, including that they be made available in more languages and provide more links to local services and support groups. Stakeholders also felt there needed to be greater promotion and marketing of the service to the community.
Effectiveness/impact
End-user perceptions and self-reported behaviours
When asked for their general feedback about the programme, all participants agreed txtpēpi was useful in that the messages helped them feel prepared and care for themselves and their baby. They appreciated reminders about important milestones that could be forgotten when busy, such as when a baby is due for immunisations. Others found the messages to be helpful prompts for self-care, such as encouragement to get fresh air and prioritise rest when possible. A few participants also appreciated messages offering emotional encouragement and reassurance they were not alone, coupled with information about where to seek support if struggling emotionally.
“I am a first-time mum so I wanted as much information as possible. I felt well informed and well prepared. I could go into my appointments already having my own understanding.”
“Remembering to do things - especially during pregnancy and eating and exercising. And the pelvic floor exercises reminders are so helpful.”
“I love these little reminders and informational texts, being a mum is so busy - it’s nice to get a wee text and feel like you’re sharing an experience.”
Participants noted the txtpēpi messages had an impact on their relationship with their child and other whānau. Several participants noted the messages served as conversation starters with whānau. Some also mentioned the messages provided ideas for new activities and foods/recipes they could try with their baby.
“They would bring up some good discussions in my family, talking about what other people experienced or their opinions of what they thought was right.”
When asked about the specific impacts of txtpēpi, the greatest impacts were reported in improving end-users’ knowledge/understanding and in feeling supported in pregnancy/postpartum (n = 39; 98% endorsed both these impacts). The lowest impacts were reported in accessing services discussed in the messages with less than half (n = 17; 43%) of participants reporting they accessed the services discussed in the messages, nine (23%) saying they did not access the services despite possible relevance and the remaining 14 (35%) reporting the services were not applicable to them. Table 3 summarises the self-reported impacts on end-users in response to structured interview questions.
“Heaps of cool ideas and reminders. I probably wouldn’t have [otherwise] prioritised kai and exercise.”
Self-reported impacts on end-user interview participants (n = 40).
Stakeholder perceptions
Stakeholder participants described two main benefits of the txtpēpi service. Several discussed empowerment through the education it provided to end-users. They stated the programme content helped end-users formulate questions for their health professionals that they may not have otherwise been confident to ask, thus improving the quality of their interactions.
“And I think for us, it empowers and encourages them to ask some questions. Without feeling like, ‘I’m not the only one that asked’. And they like to hear the facts.”
Another benefit was the programme’s provision of information and encouragement that end-users could review at any time on their phone, thus providing support in a way that traditional services may not be able to. Stakeholders also noted that the txtpēpi content provided reminders about essential milestones and actions, which can sometimes be missed during a busy healthcare appointment. txtpēpi thus had the ability to fill unanticipated gaps and ensure parents and whānau have all the information they need to care for their baby.
“We know that, you know, young mothers are up all night I think that yeah, [it’s] that kind of accessibility, kind of 24/7.”
Adoption
Facilitators to endorsement and promotion of txtpēpi by local health services and organisations and their staff included their perception of the programme’s relevance and usefulness to parents and whānau as discussed in the previous section. It was described as a useful complement to existing services, with all but one stakeholder noting that txtpēpi aligned with the other existing services in the region in the information provided, emphasis on whānau inclusion and overall aims to improve parent and child health.
“Especially for whānau that don’t want to engage. It was a quick easy way to get some key information to them.”
One of the main barriers to adoption from stakeholders related to the lack of information about the programme provided to potential referrers. This included not only information about the programme in general and who might benefit from it, but also regular updates to service providers about user experiences with txtpēpi, enrolment numbers and updates/changes to the service. Multiple stakeholders reported not feeling knowledgeable enough to promote it and four commented that they did not personally promote txtpēpi for this reason.
“There’s only a handful of us that had those conversations with the facilitator. We just didn’t have the opportunity to come together as a team and how we could actually roll this out properly.”
To improve adoption stakeholders largely reported wanting more information to be able to promote the programme and ongoing reminders to refer clients and their whānau. Having a local champion housed within each referring organisation was suggested by several stakeholders to ensure staff consistently receive this information and support.
Implementation
Of the 278 users who registered for txtpēpi within the first 12 months post-go-live, n = 13 (5%) withdrew (texted STOP/KATI) within the first 2 weeks and were excluded from analysis due to minimal exposure to the intervention.
Of the remaining users who were considered to have received at least some intervention (i.e. enrolled for at least 2 weeks; n = 265), the majority (n = 167; 63%) signed up during pregnancy: n = 35 (21%) in the first trimester, n = 58 (35%) in the second trimester and n = 74 (44%) in the third trimester. Only one-quarter (n = 42; 25%) of those that signed up in pregnancy switched to the postnatal version when prompted.
The remaining (n = 98; 37%) signed up postnatally, of whom n = 23 (23%) enrolled before the baby was 1 month old, n = 49 (50%) when the baby was between 1 and 3 months old, and the remaining n = 26 (27%) when the baby was between 3 and 6 months old.
At the time of analysis (24 months post-go-live), approximately one-quarter (n = 64; 24%) of the 265 end-users analysed were still active in txtpēpi and the remaining were inactive (n = 201; 76%). Of those who were inactive, 18 end-users (9%) had completed the programme (reached the end of the messages at baby’s 24-month birthday) and n = 183 (91%) were considered withdrawn from the programme (this includes n = 99 active withdrawals: those who texted STOP to end the programme; and n = 84 passive withdrawals: those who had not switched from pregnancy to postnatal). Rates of withdrawal by texting STOP were lower for Māori (32%; 49/155) than for non-Māori (45%; 50/110), but rates of withdrawal by non-switch were higher for Māori (45%; 69/155) than for non-Māori (14%; 15/110).
On average users received over 10 months of txtpēpi with higher intervention dosage for those who signed up postnatally and those who were mothers (Table 4). Average duration may have been limited by the higher proportion of end-users signing up during pregnancy and not switching which would limit the total possible time of participation.
Average duration of intervention by category of user among those who received at least 2 weeks of txtpēpi (n = 265).
Maintenance
Stakeholders were asked their thoughts about the potential for txtpēpi to continue beyond the initial research funding. Overall, stakeholders supported the ongoing delivery of the txtpēpi service due to their perception of its usefulness and engagement from the community. Further, stakeholders felt the programme was helping to reduce barriers to healthcare by providing a new point of entry for individuals who otherwise may struggle to access traditional services. Therefore it was perceived that txtpēpi provided different and important benefits not conferred by other existing services.
“It is creating another pathway to engage in traditional mainstream services.”
Although they supported ongoing delivery of the service, two participants cautioned that the delivery needed to be made more accessible. Suggestions included referring end-users through general practitioners (to increase opportunities for referral) and making the enrolment process simpler and more appealing to end-users (e.g. removing external university branding from the registration page).
Similarly, all participants felt that for long-term sustainability, txtpēpi should be fully managed by local services that have a closer connection to whānau. However, acknowledging that resources may be limited, most felt that services like txtpēpi should be funded nationally by Te Whatu Ora | Health New Zealand rather than the individual services responsible for implementation.
“The community. I think they need things to come from the community.”
Discussion
The txtpēpi programme reached a considerable proportion of the Lakes region’s expectant parents at 16%, which is noteworthy given the unpredictable nature of Covid-19-related measures and restrictions at the time. Importantly, the programme achieved a higher proportion of Māori end-users (60%) compared to the region’s Māori population (38%) which is important when considering Māori representation and responsiveness in research (Reid et al., 2017). Moreover txtpēpi reached whānau in smaller urban and rural areas where health inequities and gaps in service access for Māori are even more pronounced (Crengle et al., 2022). End-users of Asian and/or Pacific ethnicity were also high, signifying equitable representation for other marginalised groups in the region (Lakes District Health Board, 2022). Reach was improved by 77% when switching from an opt-in to an opt-out method of enrolment halfway through the evaluation period (Dobson et al., 2022) which saw no complaints from end-users in the current evaluation.
It was encouraging to see nearly 10% of enrolments in txtpēpi come from whānau. Family involvement in maternal and child health is increasingly recognised as important, particularly for supporting infant nutrition, and research to understand the impacts of family-level interventions is emerging (Martin et al., 2020, 2021). However, whānau were less represented in the interviews and thus the qualitative feedback provided in this evaluation centred more on the experiences of the expectant and new parents. Future research should seek to explore the views of family members on participating in these types of programmes, views which are elsewhere noted as underrepresented in the literature (Martin et al., 2020).
End-user and stakeholder views on the programme structure and content were largely positive. Components of the programme that were resource-intensive to develop, such as culturally and linguistically tailored versions of txtpēpi, were appreciated by interview participants. Appropriate tailoring, particularly to one’s culture and language, is seen as a key component in mHealth interventions that meet the needs of underserved populations (Dobson et al., 2017; Gonzalez et al., 2021; Sahin et al., 2019). Moreover, centring a minoritised group in the development of a health programme available to a range of population groups can help to counter monocultural public health messaging and encourage more robust services for everyone (Johnson et al., 2021; Mark and Lyons, 2010). Aspects of the text messages such as their simplicity and being encouraging and supportive were also noted as helpful to expectant parents; these aspects of text messages have been highlighted elsewhere (Dobson et al., 2018; Willcox et al., 2019). However, a minority of end-users felt some of the content, use of Māori language and links to services were not useful to them and thus future work should explore how the content could be further improved.
While we were unable to collect the child health outcome data, qualitative end-user and stakeholder feedback indicated that txtpēpi had positive impacts on perceived knowledge, emotional support, eating habits and physical activity. Stakeholders emphasised the programme’s alignment with existing services, potential for txtpēpi to engage whānau in perinatal and infant health and provide new opportunities for linkage to formal services.
Adoption of txtpēpi by stakeholders was high based on their endorsement of the programme and its relevance for their community. Stakeholder buy-in at all stages of programme implementation is a key tenet of RE-AIM (Glasgow et al., 2019) and in best practices identified from programme development in low-resource settings (McCool et al., 2020). Several key suggestions were made to improve stakeholder adoption and promote ongoing maintenance of the programme. This information is crucial for ongoing programme support and integration into existing services (McCool et al., 2020). While ongoing support for txtpēpi was discussed and supported among regional health leadership, funding to maintain the programme was ultimately unavailable at the conclusion of the research grant. The inability to secure ongoing funding and scaling up beyond research is not uncommon with mHealth programmes (Gerhardt et al., 2017; Willcox et al., 2019; Zakerabasali et al., 2021) but was further exacerbated in this instance by the restructure of the New Zealand public health service which changed the way in which health services, including public health programmes, were funded and delivered (Tenbensel et al., 2023). TextMATCH, the original version of the programme, did receive funding beyond the initial research period prior to this restructure and was relatively easy to integrate in large part due to its low cost to implement and alignment to existing national health guidelines. Given that txtpēpi delivers similar types of content at similar costs to end-users, these outcomes speak to the importance of stakeholder buy-in and of the ability to demonstrate cost-effectiveness when faced with changing public health resources and services (Glasgow et al., 2019).
Implementation data demonstrated that intervention dosage and timing varied considerably for both the prenatal and postnatal versions of txtpēpi, meaning that end-users may have had very different experiences with the programme. Only 24% of those who enrolled in txtpēpi in its first year were still active at 24 months post-go-live. However, the average duration of messages was high at 10 months and in both groups equated to around three-quarters of the maximum the end-users could have participated in the version they signed up for (average duration of 29 weeks for those that signed up during pregnancy and 68 weeks for those that signed up postnatally).
The largest proportion of programme withdrawals at 24 months post-go-live were due to end-users not switching from the prenatal to the postnatal versions of txtpēpi (this required end-users to free reply BABY to the invitation text message and then provide the baby’s birthday, gender and name via message when prompted). At the time of end-user interviews (12 months post-go-live), only eight of 40 interview participants fell into this non-switching category, half of whom indicated they had forgotten to respond to the invitations to switch. A proportion of withdrawn end-users also texted STOP mid-programme. Only three interview participants fell into this category and reasons for ending the programme early were mostly explained as being due to the messages not being useful or not wanting to receive them anymore.
High attrition rates have been noted in the literature on text messaging for maternal and infant health and reflect a need to better understand the factors impacting participant retention (Poorman et al., 2015). In the case of txtpēpi, reasons for active withdrawal may reflect differing knowledge levels of end-users (e.g. first-time parents may have found the informational text messages more novel and useful than experienced parents). Future programmes may explore further tailoring of content based on end-users’ parenting experience at enrolment, as well as frequency and types of support preferred (e.g. informational vs emotional encouragement (Poorman et al., 2015)). Additionally, the process for switching from the pregnancy to the postnatal version may have placed too much burden or been too complicated for end-users. This possibility is important to consider given the uptick in enrolments when txtpēpi switched from an opt-in to an opt-out enrolment method upon initial enrolment (Dobson et al., 2022). Incorporating mechanisms to reduce barriers to retention, such as an opt-out method for switching to the parenting version, may help simplify the experience for end-users and improve engagement (Speirs et al., 2016).
Strengths and limitations
Strengths of this evaluation included its mixed methods approach and pragmatic use of available data from the in-house text message platform logs, national health data sources, end-users and stakeholders. Moreover, feedback from both active/withdrawn end-users and referring/non-referring stakeholders provided diverse perspectives on txtpēpi, its benefits and areas for improvement. Like the programme itself, this evaluation included a high proportion of Māori participants and thus represents an opportunity to address longstanding health inequities that warrants further investigation. While this project was conducted in Aotearoa New Zealand, our findings may be useful for other contexts as the importance of evidence for culturally tailored, family-inclusive parent and child health interventions is increasingly recognised.
Limitations of this evaluation included the challenges reported regarding the availability of valid data we initially intended to collect. The evaluation thus relied more heavily on qualitative information about impact, and while valuable the findings cannot be corroborated by objective child health outcomes such as vaccination rates. These challenges are reflective of the short-term, pragmatic nature of the programme and its overlap with the height of Covid-19 pandemic and subsequent health system changes in Aotearoa New Zealand. While end-user burden was minimised with a simple registration process, we lacked potentially useful details about end-users (e.g. whether they were a first-time parent) and more data about reasons for not switching to the postnatal programme which could inform improvements to programme content and structure. Other limitations included a small convenience sample of interview participants which may not reflect all experiences of those involved in txtpēpi. While all txtpēpi end-users were invited regularly to provide feedback about the programme by free reply message, this method revealed few valuable insights and thus a possible missed opportunity to understand the experiences of the larger cohort of end-users.
Conclusion
Thorough evaluation of public health programmes in real-world settings is challenging but crucial to understanding impact and potential for ongoing implementation. Text messages can provide a cost-effective and easily customisable means of supporting community health during the perinatal period. While collection of several of the outcome measures were impacted by the quality of population health data during Covid-19, the results of the txtpēpi evaluation suggest the programme achieved a high degree of reach, impact, implementation and adoption. Ongoing delivery of txtpēpi was also endorsed but coupled with several suggested areas of improvement. Programme uptake and engagement from Māori indicates that txtpēpi was acceptable and useful and represents a promising approach to improving equitable child health outcomes among this population.
Footnotes
Acknowledgements
We would like to acknowledge the members of the txtpēpi Kaitiaki rōpū and the participants who took part in the evaluation of the programme. We would also like to acknowledge the National Institute for Health Innovation’s IT team for their work on the text message delivery system.
Author’s Note
Hayden McRobbie is also affiliated to University of New South Wales, Australia.
Data sharing statement
Data is available from the corresponding author on request and assuming the completion of a data-sharing agreement.
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The development and evaluation of txtpēpi was funded by a grant from the New Zealand Health Research Council (19/649).
Ethics approval
Ethical approval for the study was obtained from the New Zealand Health and Disability Ethics Committee (21/NTB/84). Research approval from Te Whatu Ora | Health New Zealand Lakes region was also obtained.
Informed consent
Participants provided verbal informed consent to participate.
Consent to publication
Not applicable.
Trial registration number/date
Not applicable.
