# Healthcare Voice AI Assistants: Factors Influencing Trust and Intention to
Use

Xiao Zhan  0000-0003-1755-0976 King’s College LondonUnited Kingdom
<EMAIL_ADDRESS>,  Noura Abdi  0000-0002-4613-6443 Liverpool Hope
UniversityUnited Kingdom<EMAIL_ADDRESS>,  William Seymour
0000-0002-0256-6740 King’s College LondonUnited Kingdom
<EMAIL_ADDRESS>and  Jose Such  0000-0002-6041-178X King’s College
LondonUnited Kingdom& VRAIN, Universitat Politecnica de ValenciaSpain
<EMAIL_ADDRESS>
(January 2023; July 2023; November 2023.)

###### Abstract.

AI assistants such as Alexa, Google Assistant, and Siri, are making their way
into the healthcare sector, offering a convenient way for users to access
different healthcare services. Trust is a vital factor in the uptake of
healthcare services, but the factors affecting trust in voice assistants used
for healthcare are under-explored and this specialist domain introduces
additional requirements. This study explores the effects of different
functional, personal, and risk factors on trust in and adoption of healthcare
voice AI assistants (HVAs), generating a partial least squares structural
model from a survey of 300 voice assistant users. Our results indicate that
trust in HVAs can be significantly explained by functional factors
(usefulness, content credibility, quality of service relative to a healthcare
professional), together with security, and privacy risks and personal stance
in technology. We also discuss differences in terms of trust between HVAs and
general-purpose voice assistants as well as implications that are unique to
HVAs.

healthcare voice AI assistants, trust in voice assistants, amazon alexa,
google assistant, apple siri, healthcare technologies

††copyright: acmlicensed††journal: PACMHCI††journalyear: 2024††journalvolume:
8††journalnumber: CSCW1††article: 62††publicationmonth: 4††price: 15.00††doi:
10.1145/3637339††ccs: Human-centered computing Empirical studies in HCI††ccs:
Security and privacy Trust frameworks††ccs: Applied computing Health care
information systems††copyright: acmlicensed††journal: PACMHCI††journalyear:
2024††journalvolume: 8††journalnumber: CSCW1††article: 62††publicationmonth:
4††price: 15.00††doi: 10.1145/3637339

##  1\. Introduction

AI assistants such as Alexa, Google Assistant and Siri are now found in
millions of smart speakers, phones, and other devices. Popular because of the
hands-free, convenient experience they offer, voice assistants are mainly used
to play music, search for information, and control smart IoT devices in the
home (Ammari et al., 2019; Sciuto et al., 2018; Dambanemuya and Diakopoulos,
2021). However, the capabilities of voice assistants are continuously
expanding to cover other uses and enahnce user experience. Capabilities added
by third parties (often called ‘skills’ or ‘actions’) continue to grow, with
the number of Alexa skills surpassing 100,000 (TechRepublic, 2020). Beyond the
main uses mentioned above, there are many other popular skill categories
including ride hailing (e.g. Uber), checking and sending emails (e.g.
Myemail), and online banking.

One particular domain that is gaining attention is the application of voice
assistants in healthcare, which we refer to as Healthcare Voice AI Assistants
(HVAs) (Research, 2022; Pradhan et al., 2020). In particular, HVAs offer more
and more healthcare services such as virtual assessments for patients,
diagnostic suggestions, and health and lifestyle tips (Frontiers, 2021). One
of the key advantages of HVAs is that they are a cost-effective alternative to
in-person or telephone consultations, especially at a time when the number of
healthcare professionals is not enough to meet the needs of patients (Martins
et al., 2021). They also offer convenience and efficiency when patients need
to travel for long distances to see a doctor (Martins et al., 2021), or when
situations like the Covid-19 pandemic make it difficult for patients to be
attended to physically (Bharti et al., 2020). In fact, many healthcare
providers around the world are already using HVAs in some way, such as the
UK’s National Health Service which offers information via
Alexa.111https://www.gov.uk/government/news/nhs-health-information-available-
through-amazon-s-alexa The number of US citizens that used voice assistants
for healthcare related purposes has nearly tripled since 2019.222See the
report https://voicebot.ai/2022/01/07/voice-assistant-use-in-healthcare-
nearly-tripled-over-two-years-and-demand-still-outstrips-supply-new-report/
for details. Beyond what is currently available through contemporary voice
assistants, researchers have been exploring ways to deliver a wide range of
healthcare services remotely through HVAs (Wolters et al., 2016; Langevin et
al., 2019). However, HVAs also pose challenges. Privacy and data security are
critical due to the sensitivity of health information involved (Price and
Cohen, 2019) and the absence of robust AI healthcare regulations potentially
magnifies these issues, inviting misuse or data misinterpretation (O’Sullivan
et al., 2019). There is also a risk of inaccuracies in HVA diagnoses and
advice, potentially leading to poor health outcomes (Abbasi and Kashiyarndi,
2006). It is therefore crucial to address these issues as the field
progresses.

As HVAs continue to develop and improve, a question arises as to whether
people will trust and adopt them. The multifaceted nature of trust in both
technology and general-purpose voice assistants has been extensively studied
(see Section 2 for a comprehensive review). This body of work has
substantiated the correlation between trust and user acceptance (AlHogail,
2018; Lu et al., 2011; Sharma and Klein, 2020; Liao et al., 2019; Pitardi and
Marriott, 2021; Al Shamsi et al., 2022; Nasirian et al., 2017; Lee et al.,
2021), and includes comprehensive studies across diverse domains on the effect
of functional (Mcknight et al., 2011; Waytz et al., 2014; Chang et al., 2017;
Peng et al., 2020; Xu et al., 2014; Mcknight et al., 2011; Cho and Hu, 2009;
Chang et al., 2013), personal (Mcknight et al., 2011; Chi et al., 2021, 2021;
AlHogail, 2018), and risk (Mayer et al., 1995; Pavlou, 2003; Ha et al., 2020;
Egea and González, 2011) factors in trust, thereby reinforcing the paramount
importance of trust within the technological sphere. However, there is a need
to study trust specifically in the context of HVAs because the healthcare
domain is known to bring a range of additional challenges (Bickmore et al.,
2018; Asan et al., 2020). For instance, despite the shared influence of
certain factors, such as the _reliability of services and safety_ , it is
evident that human trust in healthcare and trust in other technologies exhibit
notable disparities in their respective levels of confidence (Montague et al.,
2009). As described above, healthcare data is also more sensitive than other
types of data, as shown by previous academic works (Miner et al., 2016; Abdi
et al., 2021), and is often subject to additional protection under privacy
regulations such as the GDPR in Europe333The General Data Protection
Regulation (GDPR) in the EU designates healthcare data as ‘special category’,
requiring additional protections (Article 9 GDPR) https://gdpr.eu/tag/gdpr/
and HIPPA in the United States.444The Health Insurance Portability and
Accountability Act of 1996 (HIPAA) sets the standard for sensitive patient
data protection https://www.hhs.gov/hipaa/index.html. Furthermore, it has been
shown that trust in traditional healthcare providers such as physicians (Hall
et al., 2001; Ward, 2018; Gregory and Austin, 2021) cannot be directly applied
to services delivered by HVAs (Wang and Siau, 2018). This is because
interacting with a voice assistant is fundamentally distinct from interacting
with a human, even if voice assistants exhibit several human-like
characteristics (like voice interaction) (Aeschlimann et al., 2020). In
addition, AI algorithms embedded in HVAs lead to decision-making processes
that may not be apparent to the users or that are not adequately explained
compared to healthcare professionals (Gille et al., 2020; Vollmer et al.,
2018; Nagendran et al., 2020). Finally, issues related to aspects that affect
trust may be more pre-eminent. For instance, incomplete or incorrect health-
related information, such as first-aid instructions or medication
recommendations may cause patients harm (Sezgin et al., 2020a).

As a field, CSCW has shown a keen interest in digital healthcare in particular
studying its socio-technical dimensions. This encompasses the collaborative
efforts inherent in healthcare AI and the considerations steered by policy
perspectives (Fitzpatrick and Ellingsen, 2013; Park et al., 2019). The study
of patient trust in Healthcare AI in particular aligns with these core themes.
Besides, Healthcare AI is a multidisciplinary field, from medical
professionals, AI developers, data scientists and other key stakeholders (Lai
et al., 2021; Park et al., 2019). Their collective efforts are needed for the
development, implementation, and refinement of healthcare solutions, in
particular those focusing on the role of trust. There are also CSCW studies
examining communication dynamics between patients and providers (Park et al.,
2017). These studies emphasize the importance of integrating diverse
stakeholder perspectives to foster a holistic understanding of the
relationship between patients and providers. Equally crucial is the unique
situations and expectations of specific stakeholder groups, like the shared
experiences and challenges faced by patients, forming a cohesive community
actively engaging with AI technologies.

Given that the healthcare sector serves as a vital domain for human-AI
collaboration (Lai et al., 2021) and considering trust as the cornerstone
fostering this collaboration (Drozdal et al., 2020; Arnold et al., 2019; Wang
et al., 2020; Jirotka et al., 2005; Hauptman et al., 2022), it is vital to
extensively research and investigate in order to further enhance and
supplement this specific research field. The socio-technical implications of
HVAs, which bridge the gap between technology and social healthcare practices
(Fitzpatrick and Ellingsen, 2013), further emphasize the importance of
understanding trust dynamics in this context. Finally, the two central tenets
delineated in our paper, ‘trust’ and ‘adoption’, inherently embody a
collaborative essence (Lampinen et al., 2016; Muller et al., 2016). This dual-
faceted approach not only seeks to enhance user sentiment and satisfaction but
also addresses the developmental requirements discernible to HVA developers.
To the best of our knowledge, this work is the first to examine factors that
affect trust and intention to use in HVAs. We study the effect of different
functional (Anthropomorphism, Effort Expectancy, Perceived Usefulness,
Perceived Content Credibility, and Perceived Relative Service Quality),
personal (Stance in Technology, Familiarity, Technology Attachment, and Social
Influence), and risk (Security Risk, Privacy Risk, and Perceived Substitution
Risk) factors on trust in HVAs and, in turn, the effect of this trust on
peoples’ intention to use HVAs. The selection of our factors is based on those
with a confirmed substantial impact on trust in broader contexts such as
general voice assistants, technology at large, and the healthcare industry, in
order to measure their effect in the context of HVAs (as we detail in Section
3). More specifically, this study aims to answer the following research
questions:

RQ1:

How do functional, personal and risk factors influence users’ trust in
healthcare voice assistants?

RQ2:

How does trust in healthcare voice assistants influence users’ intention to
use them?

To answer these questions, we developed a model of factors affecting trust in
HVA and intention to use. In order to test the model we administered a
questionnaire to 300 participants which included questions measuring these
latent functional, personal, and risk variables alongside trust and intention
to use. The results were analysed using partial least squares structural
modelling (PLS-SEM) and show how functional factors (anthropomorphism,
perceived usefulness, effort expectancy, content credibility, and relative
service quality), risk factors (security risk and privacy risk), and personal
factors (stance in technology) substantially explain trust in HVAs. They also
suggest that trust in HVAs explains intention to use HVAs. Based on these
results we derive some practical insights and recommendations to help design
and develop the next-generation of HVAs, promoting user trust and encouraging
adoption.

##  2\. Related Work

###  2.1. Trust in Technology

####  2.1.1. Trust Scales and Measurement

The topic of trust has been extensively investigated, but there is not yet a
completely agreed definition of what it is and what it means. Trust has often
been described as ”a willingness to ascribe good intentions to and have
confidence in the words and actions of other people” in social science (Fox,
1974), while trust in technology usually refers to ”beliefs that a specific
technology has the attributes necessary to perform as expected in a given
situation in which negative consequences are possible.” (Mayer et al., 1995).
Trust is recognized as a central focus for the CSCW discipline over the past
several decades (Lampinen et al., 2016). Trust is recognized as a pivotal
factor influencing human-AI collaboration (Jirotka et al., 2005; Hauptman et
al., 2022), and in particular, previous work published at CSCW (Jirotka et
al., 2005) has underscored the positive effects of research on trust in
fostering human-AI collaboration. In order to study the different ways in
which other factors can affect trust, a critical step is to be able to measure
trust in a particular technology, in our case, HVA. Previous literature
proposed different metrics and scales to measure trust in technology (Muir and
Moray, 1996; Mayer et al., 1995; Benbasat and Wang, 2005; Koh and Sundar,
2010). More recently, and in the context of general-purpose voice assistants
such as Amazon Alexa, Cho et al. (Cho et al., 2020) used a scale to measure
trust, adapting it from (Koh and Sundar, 2010), where they consider users’
perceptions about the assistant’s competence, integrity, and benevolence,
which were suggested by previous studies to be key components of trust (Muir
and Moray, 1996; Mayer et al., 1995; Benbasat and Wang, 2005; Koh and Sundar,
2010). As we explain later on, we adapt the scale proposed in (Cho et al.,
2020) to measure trust in our study.

####  2.1.2. Adoption

As studied in early CSCW literature (Muller et al., 2016), the dynamics of
collaborative work are often embedded in the processes of technology adoption
and adaptation. This suggests that even if the focus seems individualistic,
the broader collaborative context is often at play. In terms of the models to
study adoption, one of the most well-known and used models is the Technology
Acceptance Model (TAM). The model was first proposed by Davis in 1989, and it
suggests that an individual’s acceptance of technology is determined by two
key factors: perceived usefulness and perceived ease of use. Subsequent models
built on TAM (e.g. TAM2 (Venkatesh and Davis, 2000), TAM3 (Venkatesh and Bala,
2008), and UTAUT (Venkatesh et al., 2003)) have extended the original TAM by
incorporating additional variables such as social influence (Venkatesh and
Davis, 2000; Venkatesh and Bala, 2008; Venkatesh et al., 2003), system
characteristics (Venkatesh and Bala, 2008), and facilitating conditions
(Venkatesh and Bala, 2008; Venkatesh et al., 2003) to explain the acceptance
and use of technology. However, recent works (Lowe et al., 2019; McLean and
Osei-Frimpong, 2019) have criticised their effectiveness as being highly
context-dependent, and that they are too simplified for emerging technologies.
It is therefore now common to combine them with other theories and attributes
that influence users’ usage behaviour, as we elaborate in the next section, to
compensate for their limitations in focusing more on some of the functional
factors.

####  2.1.3. Trust and Adoption

Investigating users’ trust in new technologies and products is of great
significance, because user acceptance and adoption are substantially and
significantly influenced by trust (AlHogail, 2018; Lu et al., 2011; Sharma and
Klein, 2020; Liao et al., 2019; Pitardi and Marriott, 2021; Al Shamsi et al.,
2022; Nasirian et al., 2017; Lee et al., 2021). In particular, in all of these
studies, trust has shown a positive effect on users’ behavioural intentions to
use a technology in domains such as e-commerce (Sharma and Klein, 2020),
mobile payments (Lu et al., 2011), and IoT devices (AlHogail, 2018). This has
also been shown recently for general-purpose voice assistants (Nasirian et
al., 2017; Lee et al., 2021; Liao et al., 2019; Pitardi and Marriott, 2021; Al
Shamsi et al., 2022). For instance, Liao et al. (Liao et al., 2019) found that
users’ attitudes towards adoption were rooted in whether they trusted voice
assistants, and Pitardi et al. (Pitardi and Marriott, 2021) showed that trust
is one of the key contributors to foster adoption of Google Assistant. More
recently, the importance of trust for adoption of voice assistants was also
shown when assistants are used for specific tasks, such as in an education
setting (Al Shamsi et al., 2022).

###  2.2. Factors Affecting Trust

HCI studies examining how different factors affect users’ trust in technology
have identified three main types of factors that affect trust, including
functional, personal and risk factors. Functional factors refer to the
characteristics of the technologies themselves (Mcknight et al., 2011; Waytz
et al., 2014; Chang et al., 2017; Peng et al., 2020; Xu et al., 2014; Mcknight
et al., 2011; Cho and Hu, 2009; Chang et al., 2013), personal factors refer to
the individual user, their social circle and their experiences (Mcknight et
al., 2011; Chi et al., 2021, 2021; AlHogail, 2018), and risk factors refer to
the harms that could be derived from the technology (Mayer et al., 1995;
Pavlou, 2003; Ha et al., 2020; Egea and González, 2011). In a related vein,
Knowles et al. (Knowles et al., 2015) emphasized trust as crucial for
collaboration and introduced design patterns to enhance trust. While not
empirically validated like the aforementioned models, these patterns highlight
the importance of factors such as usability (functional factor) and
socialization (personal factor).

Regarding the functional factors that affect trust in technology, previous
literature has uncovered factors such as the usefulness of the technology and
how easy the technology is to use (Chandra et al., 2018), as well as its
reliability (Lewandowsky et al., 2000; Madhavan et al., 2006) and the
interfaces designed for interaction (Parasuraman and Miller, 2004). These all
manifest slightly differently in voice assistants, particularly interface
design and reliability. With regards to interface design, Anthropomorphism
(Chen and Park, 2021; Seymour and Van Kleek, 2021; van Pinxteren et al., 2019)
has been shown to influence trust, with studies suggesting that voice alone
facilitates human connection with the assistant (Han and Yang, 2018; Novak and
Hoffman, 2019). Regarding reliability, and partly because of how voice
assistants operate, the broader concept of content credibility (Lou and Yuan,
2019), which includes reliability as well as other aspects such as accuracy,
completeness and authenticity of the information, is a critical factor to
foster trust in voice assistants, that is, the credibility of what the voice
assistant utters back to the user seems to affect how much the user trusts the
assistant.

In addition to functional elements, whether users consider technology to be
trustworthy depends on personal factors. This includes individual factors and
social factors. Individual factors, such as user predisposition to try new
technologies (Agarwal and Prasad, 1998), user familiarity with an
understanding of a technology (Komiak and Benbasat, 2006; Chi et al., 2021),
and even the users’ emotional attachment (You and Robert, 2017) and
psychological connection to a technology (Suh et al., 2011). Social factors
also seem to play a role, with works like (Gursoy et al., 2017) showing that
social influence, the degree that a customer’s social group (e.g. family,
friends, etc.) believes that using a technology is relevant, affects trust in
that technology.

Finally, the effect of risk on trust has long been studied (Mayer et al.,
1995), and shown to manifest across different domains, including online
transactions (Pavlou, 2003), autonomous vehicles (Ha et al., 2020), and the
use of Electronic Health Care Records (EHCR) by physicians (Egea and González,
2011). Security and privacy risks, in particular, have been shown to reduce
the benefits of using general-purpose voice assistants (McLean and Osei-
Frimpong, 2019; Kowalczuk, 2018; AlHogail, 2018), directly impacting the trust
users’ have in voice assistants, with users stopping using voice assistants
all together as a result, or restricting their use of voice assistants only to
functionality they perceive to be less risky (Bansal et al., 2016; Abdi et
al., 2019).

###  2.3. Healthcare Voice Assistants

####  2.3.1. HVA available in Commercial Voice Assistants

Commercial voice assistants nowadays incorporate healthcare capabilities as
Amazon Alexa _Skills_ or Google Home _Actions_ (Chung et al., 2018). In 2020,
researchers grouped the spectrum of healthcare services provided by voice
assistants into four levels: information-, assistance-, assessment-, and
support-level services (Sezgin et al., 2020b, c). Of these, information-level
services (providing healthcare-related information to users) are one of the
most common and basic functions and are already widely embedded in most
commercially available voice assistants. There are many Alexa skills, for
example, that provide healthcare information such as descriptions/or
instructions on symptoms, medications, and how to use medical facilities
(e.g., Dr. A.I., WebMD, and Mayo Clinic). Examples of assistance-level
services include the ability for VAs to assist users in setting up reminders
for medication or physical activity (i.e. tracking their symptoms) and
providing appointment notifications (Sezgin et al., 2020b, c). Some of these
features are also available in contemporary VAs like the My Children’s
Enhanced Recovery After Surgery skill developed by Boston Children’s Hospital,
which reminds parents and caregivers about information regarding their post-op
appointments. Support for functionality within the assessment and support
layers ranges from scarce to non-existent. Assessment level skills, for
example, would enable voice assistants to recognise changes in mood or health
conditions, but commercial voice assistants are only currently able to access
and view diagnostic results and records (e.g. Alexa Skill Sugarmate).
Similarly, the support level includes the ability to prescribe and prioritise
care/treatment but users have thus far only been able to refill their
prescriptions555This feature was available for Alexa owners in 2019 -
https://www.dailymail.co.uk/sciencetech/article-7727961/Alexa-owners-use-
virtual-assistant-refill-prescriptions.html.

####  2.3.2. Research on HVAs

Extensive research has been conducted recently on the use of voice assistants
for healthcare. For example, Dojchinovski et al. (Dojchinovski et al., 2019)
saw the potential for VAs to become an important part of the healthcare system
for heart disease. The VAs could communicate directly with the user through
their special voice interface and then, retrieve the required information from
the medical cloud and transmit it back to the user. In this case, users would
highly benefit from using a VA-based healthcare system to perform tasks such
as checking ECG readings, scheduling appointments, recording treatments and
contacting doctors. Other strategies and prototypes for coping with various
healthcare issues have been discussed as well, such as supporting home care
(Bartle et al., 2022a), and improving patients’ Type2 diabetes management
(Buinhas et al., 2018; Cheng et al., 2018). Another stream of research on HVA
has focused on evaluating the performance of voice assistants in healthcare-
related tasks. Google Assistant, Amazon Alexa, Apple Siri, Microsoft Cortana
and other voice commercial assistants are often tested and compared (Alagha
and Helbing, 2019; Kocaballi et al., 2020; Nobles et al., 2020; Yang et al.,
2021). Overall, the ability of VAs to provide health-related advice is very
limited and performed unfavourably to identify the user’s queries for
healthcare information (Alagha and Helbing, 2019; Nobles et al., 2020; Yang et
al., 2021; Kocaballi et al., 2020). Finally, there has been some recent work
examining how differences in modality (i.e., voice vs. text) and device type
(i.e., smartphone vs. smart home device) affect user perceptions when
retrieving sensitive health information from voice assistants (Cho, 2019),
finding that voice led to more social presence and that device did not matter.
In addition, attention was paid to the behaviour and perception of older
people using voice assistants for the specific purpose of seeking healthcare
information (Zhong and Ma, 2022; Brewer et al., 2022). They found that older
people’s evaluation of the VA was influenced by communication styles,
anthropomorphic settings, and individual difference (Zhong and Ma, 2022), and
that they appeared to use voice assistants primarily to confirm information
obtained from other sources (e.g., laptop, phone) (Brewer et al., 2022). While
previous work (Park et al., 2019) emphasized the importance of addressing
trust issues, especially amidst evolving healthcare workflows, we are unaware
of any prior research specifically targeting user trust in HVAs and the
influencing factors.

##  3\. Hypotheses

Table 1. Summary of factors considered in the model. (H1-5 are functional
factors, H6-9 personal factors, and H10-12 risk factors.)

H# |  Factors |  Description |  Supporting Source
---|---|---|---
H1 |  Anthropomorphism |  Anthropomorphism in voice assistants involves human-like characteristics, creating a social connection with users. Voice-based interaction deepens this bond between users and the assistant. |  (Moussawi et al., 2021; Epley et al., 2007; Han and Yang, 2018; Novak and Hoffman, 2019; Van Doorn et al., 2017; Pitardi and Marriott, 2021; Tanioka et al., 2021)
H2 |  Effort Expectancy |  Effort expectancy is the predicted mental and physical activity required to use a technology. It affects trust in technology, and challenges in ease of use can lead to anxiety and distrust. |  (Venkatesh et al., 2012; Pitardi and Marriott, 2021; AlHogail, 2018; Kulviwat et al., 2007; Davis, 1989; Lin et al., 2020; Chang et al., 2017; Vaportzis et al., 2017; Lee and Coughlin, 2015; Knowles and Hanson, 2018; Chi et al., 2021; Schnall et al., 2015)
H3 |  Per. Usefulness |  is about device efficiency in daily tasks. In healthcare, it means technology facilitating services. Usefulness affects attitudes and adoption. In HVAs, useful responses to symptoms are vital for user satisfaction. |  (Buteau and Lee, 2021; Davis, 1989; Coughlan et al., 2012; Gao and Bai, 2014; AlHogail, 2018; Peng et al., 2020; Lee and Yoon, 2021)
H4 |  Per. Content Credibility |  Content credibility is crucial for user trust. In healthcare, users rely on accurate information from virtual assistants. Inaccurate information can be harmful. |  (Xu et al., 2014; Mcknight et al., 2011; Cho and Hu, 2009; Lewandowsky et al., 2000; Madhavan et al., 2006; Asan et al., 2020; Sunarsi et al., 2019; Cho et al., 2020; Appelman and Sundar, 2016; Bickmore et al., 2018; Nadarzynski et al., 2019; Haan et al., 2019; Sezgin et al., 2020a).
H5 |  Per. Relative Service Quality |  This perception compares healthcare technology to professionals. Patients value professional care but may trust technology for specific diagnoses if it offers greater accuracy. |  (Baker et al., 2020; Young et al., 2021; Liu et al., 2019; Promberger and Baron, 2006; Haan et al., 2019; Jutzi et al., 2020)
H6 |  Familiarity |  Familiarity represents the level of understanding of an object. When users are familiar with a technology, they are more likely to trust it compared to unfamiliar ones in various domains such as e-commerce, IoT devices, and voice assistants. |  (Komiak and Benbasat, 2006; Luhmann, 2001; Gefen and Straub, 2004; Alraja et al., 2019; Chi et al., 2021)
H7 |  Tech Attachment |  This factor, aka emotional attachment, reflects the psychological connection between individuals and a technology. |  (You and Robert, 2017; Suh et al., 2011; Perlaviciute and Steg, 2014; Chi et al., 2021)
H8 |  Social Influence |  Social influence plays a significant role in shaping users’ trust and perceptions of a product, as they are influenced by the behaviors and opinions of others in their social networks. |  (Latané, 1981; Gursoy et al., 2017; Tajfel et al., 1979; AlHogail, 2018)
H9 |  Stance in Tech |  Stance in technology reflects an individual’s openness to trying new innovations, with a positive correlation found between willingness to experiment with new technologies and acceptance and trust in their services. |  (Agarwal and Prasad, 1998; Mcknight et al., 2011; Chi et al., 2021)
H10 |  Security Risk |  Security risk in technology, especially in healthcare, can undermine trust due to potential consequences of flaws and incidents such as errors, malware, and hacking. |  (Alraja et al., 2019; Keskinbora, 2019; Ellahham et al., 2020; Lee and Rich, 2021; Nadarzynski et al., 2019; Haan et al., 2019)
H11 |  Privacy Risk |  Privacy risk arises when users perceive a potential threat of data misuse or unauthorized access, particularly in the sensitive healthcare domain, impacting trust in voice assistants. |  (Alraja et al., 2019; Pitardi and Marriott, 2021; Vimalkumar et al., 2021; Foehr and Germelmann, 2020; Hayden, 2013; Topol, 2019; Gymrek et al., 2013; Easwara Moorthy and Vu, 2015; Miner et al., 2016; Abdi et al., 2021)
H12 |  Per. Substitution Risk |  Perceived substitution risk in healthcare technologies involves the concern of technology replacing traditional healthcare professionals, impacting trust in AI applications. |  (Fan et al., 2020; Ongena et al., 2020; Haan et al., 2019; Nadarzynski et al., 2019; Jutzi et al., 2020; Nelson et al., 2020)
H13 |  Intention to Use |  Intention to use reflects the connection between trust and technology adoption, with a strong correlation between users’ trust, attitudes, and actual use of technology. |  (AlHogail, 2018; Lu et al., 2011; Sharma and Klein, 2020; Nasirian et al., 2017; Lee et al., 2021; Pitardi and Marriott, 2021; Liao et al., 2019; Al Shamsi et al., 2022; Chao, 2019)

Trust is a complex phenomenon that can be affected by many different factors.
As discussed in Section 2.2, there are a number of personal, functional, and
risk factors that have been shown to have an impact on users’ trust in
technology. Therefore, in this paper, we consider factors belonging to these
categories that were shown by previous works to have a significant effect on
trust adapted to the case of HVAs. In particular, and as we detail below, we
consider: i) factors that have been shown to affect trust on general-purpose
voice assistants such as Amazon Alexa and Google Assistant (Pitardi and
Marriott, 2021; Chi et al., 2021; AlHogail, 2018; Foehr and Germelmann, 2020;
Alraja et al., 2019; Vimalkumar et al., 2021) ; ii) factors that have been
shown to affect trust in technology in general (Waytz et al., 2014; Chang et
al., 2017; Cho and Hu, 2009; Xu et al., 2014; Gefen and Straub, 2004); and
iii) factors that are specific to trust in the healthcare domain (Peng et al.,
2020; Chang et al., 2013; Yokoi et al., 2021). We do not include factors that,
while significant in prior works are not relevant or do not translate to the
case of HVAs. For instance, Hedonic Benefits is one of the factors that seem
to affect trust in general-purpose voice assistants (McLean and Osei-Frimpong,
2019; Pitardi and Marriott, 2021), but that only applies when the purpose of
use is hedonistic (Wu et al., 2010), and it is not aligned with the relatively
more serious, purposeful, and specific context of seeking health care. Also,
we are interested in factors that affect trust rather than the measurement of
trust as a construct (AlHogail, 2018), so we adapt already existing scales to
measure trust (Cho et al., 2020) and focus on the factors that influence it.
Figure 1 summarizes our hypothesized research model, the different factors
that may affect trust in HVA and its relationship to intention to use HVA. We
now detail the categories, factors, and our hypotheses.

Figure 1. Hypothesized Research Model

###  3.1. Functional Factors

The extensive literature on trust in technology identified and subsequently
confirmed across different domains that users’ trust in a technology is
influenced by whether the technology delivers on the functionality promised to
complete the tasks or services they are supposed to offer (Mcknight et al.,
2011; Yan and Holtmanns, 2008). This normally refers to both design and
performance issues (Yan and Holtmanns, 2008). When talking about voice
assistants, functional factors take particular manifestations, and have been
thoroughly examined both qualitatively and quantitatively in the previous
study (Dambanemuya and Diakopoulos, 2021; Seymour and Van Kleek, 2021; Cho and
Rader, 2020). For instance, and as further detailed below, because of the very
nature of their interaction modality, they offer a more anthropomorphic
experience than other technologies. Another example, and in particular when
talking about HVA, is that users may compare the healthcare service offered by
an HVA relative to what users normally get from healthcare professionals.

####  3.1.1. Anthropomorphism (A)

This factor refers to the extent that voice assistants show human-like
characteristics and are therefore perceived and treated by users in ways that
may be closer to how they would interact with another human (Moussawi et al.,
2021). The more human-like characteristics voice assistants display the more
it seems humans connect with them (Epley et al., 2007). Starting with the
modality of interaction, voice alone facilitates human connection with the
assistant (Han and Yang, 2018; Novak and Hoffman, 2019), and it may make users
feel they are interacting with a social entity (Van Doorn et al., 2017). In
fact, previous research on general-purpose voice assistants showed that their
anthropomorphic characteristics have a positive effect on users’ trust in
voice assistants (Van Doorn et al., 2017; Pitardi and Marriott, 2021). We
hypothesize this to also be the case in HVA, particularly as healthcare is a
domain where human-like characteristics in the technology used may contribute
to a satisfactory experience (Tanioka et al., 2021).

H1 Anthropomorphism positively influences Trust in HVAs.

####  3.1.2. Effort Expectancy (EE)

This factor is the predicted mental and physical activity and skill required
to use a technology (Venkatesh et al., 2012). In general, it has been shown
across different domains that effort expectancy, or its flip side ‘ease of
use’, affect trust in technology (Pitardi and Marriott, 2021; AlHogail, 2018;
Kulviwat et al., 2007; Davis, 1989; Lin et al., 2020; Chang et al., 2017). For
instance, one case is that of older adults, who find it difficult to use
digital technologies (Vaportzis et al., 2017; Lee and Coughlin, 2015), which
has been shown to trigger anxiety and distrust (Lee and Coughlin, 2015;
Knowles and Hanson, 2018). There is no previous work on the relationship of
effort expectancy and voice assistants (not even in general-purpose ones).
However, given the negative role that effort expectancy has been shown to play
in similar technologies before, such as social robots for service delivery
(Chi et al., 2021), we hypothesize that effort expectancy also plays a role in
trust in HVA, particularly it can negatively affect it. This would also be
consistent with work on mobile healthcare technologies that showed that the
reverse of effort expectancy has a positive effect on intention to use
(Schnall et al., 2015).

H2 Effort expectancy negatively influences Trust in HVAs.

####  3.1.3. Perceived Usefulness (PU)

In general, this factor refers to ”the degree to which the device makes life
more efficient and helps carry out daily tasks.” (Buteau and Lee, 2021). In
other words, and particularly considering the healthcare domain, this factor
refers to whether and how technology helps users to seek and get healthcare
services in an efficient and effective way. According to Davis(Davis, 1989),
usefulness can influence attitudes towards a particular technology, and in
particular user acceptance (Coughlan et al., 2012; Gao and Bai, 2014) and
adoption (AlHogail, 2018) of technologies. The contribution of usefulness to
trust has been demonstrated, especially in the healthcare domain. For
instance, Peng et al. (Peng et al., 2020) confirmed that the usefulness of
online information and suggestions provided by physicians has a positive
effect on patient-physician trust. Similarly, when using HVAs, the usefulness
of responses from the HVA to user queries describing symptoms is suggested to
be of great significance (Lee and Yoon, 2021), and it might cause differences
in the user’s feelings and judgements. Based on all of this, we formulate the
following hypothesis:

H3 Perceived Usefulness positively influences Trust in HVAs.

####  3.1.4. Perceived Content Credibility (PCC)

This factor represents the credibility of the information provided by the
technology, and it is often identified as a key factor influencing users’
trust in technology in general (Xu et al., 2014; Mcknight et al., 2011; Cho
and Hu, 2009). The credibility of information is known to, in turn, be
composed of different aspects, such as its accuracy, reliability (including
completeness, consistency, and authenticity), and authoritativeness (the
information was generated by experts) (Lewandowsky et al., 2000; Madhavan et
al., 2006; Asan et al., 2020; Sunarsi et al., 2019; Cho et al., 2020; Appelman
and Sundar, 2016). These are all critical aspects of information in the
healthcare context because users may blindly follow the instructions and
advice given by HVAs, such as what medication to take and how often to take it
(Bickmore et al., 2018). Inaccurate information is therefore likely to result
in the user being harmed (Bickmore et al., 2018; Nadarzynski et al., 2019;
Haan et al., 2019; Sezgin et al., 2020a).

H4 Perceived Content Credibility positively influences Trust in HVAs.

####  3.1.5. Perceived Relative Service Quality (PRSQ)

This factor refers to the quality of a healthcare service a technology offers
_compared_ to the same service offered by a healthcare professional. It has
been used by previous research as an important indicator of the satisfaction
healthcare technologies can bring compared to traditional healthcare services
(Baker et al., 2020; Young et al., 2021; Liu et al., 2019). When it comes to
trust, research showed that patients are reluctant to trust AI technologies
because they value the quality of care provided by healthcare professionals
(Promberger and Baron, 2006). In addition, qualitative studies on trust in AI
technologies for particular types of diagnoses (like cancer) (Haan et al.,
2019; Jutzi et al., 2020) suggested that some patients might trust the
technology if it provides more accuracy than healthcare professionals. We
therefore hypothesize that PRSQ affects trust and that the higher the PRSQ
(that is the higher the perception of quality of the service when compared
with a healthcare professional) the higher the trust.

H5 Perceived Relative Service Quality positively influences Trust in HVAs.

###  3.2. Personal Factors

Users may have differing attitudes toward the same technology (Merritt and
Ilgen, 2008; Lee and Turban, 2001; Yao et al., 2019). Trust is no exception,
and it can be affected by individual and social factors. For instance, an
individual may be more receptive to new technology than others (e.g. early
adopters), and another individual may also (or instead) be influenced by the
opinions of their (online and/or offline) social circle. Therefore, in
addition to functional factors, we also consider personal factors.

####  3.2.1. Familiarity (F)

This factor refers to the degree of understanding of an object (Komiak and
Benbasat, 2006). Research has shown that the more familiar users are with a
technology the more likely they are to trust it in comparison to other
technologies they do not know about (Luhmann, 2001). In fact, it has been
shown to increase trust by reducing users’ uncertainty on the technology in
e-commerce (Gefen and Straub, 2004), IoT devices (Alraja et al., 2019), and
voice assistants (Chi et al., 2021). Similarly, in this study, we hypothesize
that familiarity with HVA influences the trusts users have on them (Chi et
al., 2021).

H6 Familiarity positively influences Trust in HVAs.

####  3.2.2. Technology Attachment (TA)

This factor, also known as emotional attachment (You and Robert, 2017),
represents the psychological connection between a person and a technology (Suh
et al., 2011). This connection is known to promote a positive view of the
technology in question (Perlaviciute and Steg, 2014). In particular, in
general-purpose voice assistant, TA is known to significantly affect trust
(Chi et al., 2021).

H7 Technology Attachment positively influences Trust in HVAs.

####  3.2.3. Social Influence (SI)

Users are influenced by the behaviors and opinions of others in their social
networks, according to the theory of social influence (Latané, 1981). They
tend to believe the same as the majority in those networks (Gursoy et al.,
2017). This is consistent with Social Identity Theory (Tajfel et al., 1979),
which suggests that following social norms makes people feel integrated. As a
result, users’ perceptions, and in particular their trust towards a product,
can vary (AlHogail, 2018), based on the trust their social connections have.
In this study, we hypothesize that this factor reflects the propensity of
users to trust HVAs, to capture the influence their social network may have on
it.

H8 Social Influence (positively or negatively) influences Trust in HVAs.

####  3.2.4. Stance in Technology (ST)

This factor is the extent to which an individual would be open to try out a
new innovation or technology (Agarwal and Prasad, 1998). Many existing studies
have found that people who are more willing to experiment with and try new
technologies are more inclined to accept and trust the services that these
technologies offer (Mcknight et al., 2011; Chi et al., 2021). It is therefore
reasonable to hypothesize that such a group of individuals would likely be
open towards HVAs.

H9 Stance in technology positively influences Trust in HVAs.

###  3.3. Risk Factors

The use of technologies usually comes with some risks (Lau et al., 2018;
Malkin et al., 2022; Meng et al., 2021; McMillan et al., 2019; Seymour and
Such, 2023; Yao et al., 2019). These risks may also affect the trust that
users have in the technologies. For instance, if the perceptions of risk is
high, research has shown that that can lead to distrust (Køien, 2011;
AlHogail, 2018). We consider the following risk factors and the effect they
may have on trust in HVA.

####  3.3.1. Security Risk (SR)

This factor relates to how unsafe and unprotected users feel when using a
technology. Previous research on other healthcare technologies showed that SR
is negatively associated with trust (Alraja et al., 2019). This is, in part,
because healthcare is a critical domain, where flaws in the technology can
lead to serious issues (Keskinbora, 2019; Ellahham et al., 2020; Lee and Rich,
2021; Nadarzynski et al., 2019; Haan et al., 2019). In addition, the digital
healthcare sector has recently reported high number of incidents due to error,
malware, and hacking666See for instance the, 2022 Data Breach Investigations
Report, which shows that from the cases investigated, 67% of them reported to
have healthcare related issues -
https://www.verizon.com/business/resources/reports/dbir/. We therefore
hypothesize that security risk can have a negative influence in trust in HVA.

H10 Security Risk negatively influences Trust in HVAs.

####  3.3.2. Privacy Risk (PR)

When users believe there is a potential risk for their data being misused or
accessed by unauthorised parties, their trust may be affected (Alraja et al.,
2019; Pitardi and Marriott, 2021; Vimalkumar et al., 2021; Foehr and
Germelmann, 2020). In the healthcare domain, data can be successfully linked
to individuals (Hayden, 2013; Topol, 2019) even if it has been anonymised and
scrubbed of all identifiers (Gymrek et al., 2013). As suggested in (Easwara
Moorthy and Vu, 2015), the more sensitive the information the more the
likelihood of users having concerns when interacting with their voice
assistants. Healthcare data is very sensitive (Miner et al., 2016; Abdi et
al., 2021), so hypothesize that PR negatively affects trust in HVA.

H11 Privacy Risk negatively influences Trust in HVAs.

####  3.3.3. Perceived Substitution Risk (PSR)

This factor is typically considered in healthcare technologies, and it
captures the risk that traditional healthcare professionals may be replaced by
technology in the future (Fan et al., 2020), and that this could even lead to
professional healthcare job loss (Ongena et al., 2020; Haan et al., 2019;
Nadarzynski et al., 2019). There is also a risk that clinicians may lose
skills that are increasingly carried out by technology (e.g. around
diagnosis), resulting in increased errors when complex cases are escalated for
human review (Jutzi et al., 2020). All of this risk of substitution has been
shown to affect users trust in AI technologies applied to healthcare in
general (Nelson et al., 2020), so we hypothesize this to be also the case in
HVA.

H12 Perceived substitution risk negatively influences Trust in HVAs.

###  3.4. Intention to Use

As introduced in Section 2.1.3, the relationship between trust and adaption
has been extensively studied. Across technologies, there seems to be a clear
connection between users trusting and adopting technology (AlHogail, 2018; Lu
et al., 2011; Sharma and Klein, 2020; Liao et al., 2019; Pitardi and Marriott,
2021; Al Shamsi et al., 2022; Nasirian et al., 2017; Lee et al., 2021). This
has also been shown for the case of general-purpose voice assistants (Nasirian
et al., 2017; Lee et al., 2021; Liao et al., 2019; Pitardi and Marriott, 2021;
Al Shamsi et al., 2022). One of the proxies to attitudes towards adoption is
_intention to use_ (or intention to adopt) (Liao et al., 2019; Chao, 2019; Al
Shamsi et al., 2022). In fact, previous research did show a strong connection
between behavioural intentions and actual use of specific technologies (Chao,
2019). In accordance to this, we use in this paper intention to use as a proxy
for adoption and hypothesize that:

H13 Trust in HVAs positively influences users’ intention to use them.

##  4\. Methodology

In order to test our hypotheses, we followed a quantitative methodology, by
designing and administering a questionnaire with items measuring the factors
introduced before as well as trust in HVAs and users’ intention to use HVAs,
and analyzing the results using Partial Least Squares Structural Equation
Modelling (PLS-SEM). Our study was approved by our institution’s IRB.

###  4.1. Survey Development

We created a questionnaire that contains three parts, which can be found in
Appendix A, B, and C. The first part is a description of HVAs, so participants
could understand what we mean by them. The second part includes the questions
regarding the model and hypothesis introduced in the previous section in the
form of items representing each of the factors, which were measured using a
five-point Likert scale, with answer choices ranging from ‘strongly disagree’
(1) to ‘strongly agree’ (5). To ensure reliability and validity, the items
were adapted from existing scales for each of the factors considered (McLean
and Osei-Frimpong, 2019; Gursoy et al., 2019; Fan et al., 2020; Cho et al.,
2020; Hasan et al., 2020; Chi et al., 2021). To do this, items were rephrased
where appropriate to fit the study context (that is HVA). For example, one of
the items for the factor Effort Expectancy from (Chi et al., 2021) was
rephrased from “ _Using AI devices takes too much of my time_ ” to “ _Using
healthcare voice assistants would take too much of my time_ ”. Every set of
items for each factor was taken from the same source, so that all items
measured the same underlying construct. The third and final part of the
questionnaire includes three post-survey questions about users’ perceptions of
future use of HVAs, organisations they trust to design HVAs, and expectations
for building trusted HVAs.

To make sure that the questionnaire was effective and that participants
understood the questions, we conducted two pilot studies with a total of 30
participants each. The data from the pilots was used only to improve the final
survey and excluded from the final analysis. The pilots helped us refine the
items, e.g. we were able to identify that some of the reversely-coded items
(which we reversed for data quality purposes, as explained later) were
confusing, and we adequately rephrased them.

###  4.2. Model Evaluation

PLS-SEM was used for the statistical analysis of the second part of the
questionnaire (items). This is because PLS-SEM has been found to be
appropriate when the analysis is to test a theoretical framework from a
prediction perspective and the structural model is complex and includes many
factors (Hair et al., 2019). In addition, PLS-SEM does not require large
samples to perform well as some other SEM approaches do (Hair et al., 2019,
2011). We used the implementation of PLS-SEM in the SmartPLS 3.3 software
(Christian M. et al., 2015), following the recommended two-step procedure
(Hair et al., 2011) of evaluating the measurement (outer) model first before
estimating the structural (inner) model.

###  4.3. Post-Survey Questions

While statistical analysis of the model shows the factors that demonstrate
statistical significance in influencing trust and intention to use, it is also
important to understand participants’ reasoning around the inclusion of
certain specific factors and the exclusion of others. Consequently, we
undertook a qualitative analysis to enhance our comprehension of the factors
encompassed within the model as suggested by prior work (Pitardi and Marriott,
2021; Yin, 2013). This qualitative exploration not only enabled us to delve
deeper into the considered factors but also presented opportunities to
identify additional factors that were not initially incorporated in our model,
thus augmenting the existing body of literature in this domain.

For the third and final part of the questionnaire, responses to the open-ended
questions were first converted into a text format, and then analyzed using
thematic analysis following Braun and Clarke (Braun and Clarke, 2012),
involving the 6 steps of familiarization, coding, generating themes, reviewing
themes, defining and naming themes and writing up (Braun and Clarke, 2012).
Individual coding was conducted by two researchers, and codes were recognized
based on the ideas and feelings expressed by participants. Researchers then
reviewed the codes created and discussed the results until reaching agreement.
Similarly, the researchers then developed themes individually before coming
together to discuss disagreements and converge on a final set of themes.

###  4.4. Data Collection

The final questionnaire was implemented using
Qualtrics777https://www.qualtrics.com and participants were recruited using
Prolific888https://prolific.co. All participants were voice assistant users
aged 18 or older. The beginning of the questionnaire explained the study in
detail, requested consent for data collection, and gathered information about
participants’ prior use of their voice assistant, including for healthcare
purposes.

To ensure a large enough sample, we followed two methods and retained the
highest minimum number obtained. First, a minimum viable sample size was
calculated using the “10-times rule” (Thomson et al., 1995; Hair et al.,
2014), which suggests the minimum sample should be 10 times the maximum number
of arrowheads pointing at a latent variable (a formal term employed in PLS,
synonymous with the ’factors’ used in our study) anywhere in the PLS path
model (12 arrows points to _Trust in HVA_). This gave a minimum size of
$10*12=120$ for each stage of data collection.

Second, to ensure that the sample also had sufficient statistical power to
accurately detect true effects and minimize Type II errors we used the method
proposed in Hair et al. (Hair et al., 2009). This builds on Cohen’s power
tables for least squares regression and relies on a table listing minimum
required sample sizes based on three elements (the maximum number of arrows
pointing at a latent variable, the significance level used, and the minimum
$R^{2}$/smallest effect size that a researcher wishes to detect in the study
in the model) (Cohen, 1992). When the maximum number of arrows is 12, we need
181 observations to achieve a statistical power of 80% for detecting $R^{2}$
values of at least 0.1 (with a 5% probability of error). Therefore, 181
observations would be needed as a minimum for the study.

###  4.5. Data Quality and Reliability

To ensure reliability and quality of the data collected, it is essential to
employ rigorous measures for participant selection and questionnaire design.
We employed _three_ widely recognized and frequently used measures in the
questionnaire (Hauser and Schwarz, 2016; Mason and Suri, 2012; Paas and
Morren, 2018; Kim et al., 2019; Peer et al., 2014):

  * •

Recruitment of _high-reputation_ participants with at least 100 submissions
and an approval rate of 95% or more on the Prolific recruitment platform (Peer
et al., 2014; Such et al., 2017).

  * •

Use of two attention check questions to identify low-effort responses and
filter those participants that failed either check from the analysis. These
are essential for identifying inattentive respondents, promoting participants’
concentration and involvement, and improving the reliability of the data
collected (Hauser and Schwarz, 2016; Mason and Suri, 2012; Paas and Morren,
2018).

  * •

Applying the Simple Non-differentiation Method after reverse-coding six
questionnaire items (Yan, 2008; Kim et al., 2019) to the responses in order to
identify _‘straight lining’_ by participants, excluding those who consistently
selected the same response option across multiple questions (i.e., clicking in
a straight line down the survey questions).

Table 2. Demographics of the survey participants. |  | Par. | % |  |  | Par. | %
---|---|---|---|---|---|---|---
Gender |  Male | 115 | 38.3 | Employment |  Full time | 115 | 38.3
|  Female | 182 | 60.7 |  |  Part-time | 36 | 12
|  Prefer not to say | 3 | 1 |  |  Unemployed | 17 | 5.7
Age |  18-24 | 79 | 26.3 |  |  Other | 14 | 4.7
|  25-34 | 116 | 38.7 |  |  Retired/Homemaker | 16 | 5.3
|  35-44 | 61 | 20.3 |  |  Prefer not to say | 102 | 34
|  45-54 | 30 | 10 | Student |  Yes | 81 | 27
|  55-64 | 12 | 4 |  |  No | 142 | 47.3
|  65 + | 3 | 1 |  |  Prefer not to say | 77 | 25.7
Assistant |  Amazon Echo/Alexa | 142 | 47.3 | Healthcare |  Total | 147 | 49
used |  Amazon Echo/Alexa + others | 51 | 17 | use of |  Information about illness/drug | 71 | 23.7
|  Google Home | 75 | 25 | assistant |  Diagnosis based on symptoms | 63 | 21
|  Google Home + others | 10 | 3.3 |  |  Monitor & manage chronic disease | 9 | 3
|  Microsoft Invoke/Cortana | 6 | 2 |  |  Treatment Plan | 3 | 1
|  Apple HomePod/Siri | 8 | 2.7 |  |  Others | 1 | 0.3
|  Other | 8 | 2.7 |  |  |  |

##  5\. Results

###  5.1. Participants

Overall, 355 participants were recruited for the study. Of these, 43 were
removed from the analysis for failing one of the two attention checks, and 12
participants were eliminated after straight lining. As a result, 300
participants were retained for data analysis, which is higher than the 181
participants we had established as minimum in Section 4.4. The breakdown of
demographics for the participants is reported in Table 2, and the full dataset
is available online at https://osf.io/pdj3q/.

###  5.2. Model Evaluation

####  5.2.1. Measurement Model Analysis

Table 3. Convergent and discriminant validity results.

| $\alpha$ | CR | AVE | A | SI | EE | F | IU | PCC | PRSQ | PU | PR | SR | TA | T | ST | PSR
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
A | 0.804 | 0.885 | 0.719 | 0.848 |  |  |  |  |  |  |  |  |  |  |  |  |
SI | 0.853 | 0.900 | 0.693 | 0.437 | 0.833 |  |  |  |  |  |  |  |  |  |  |  |
EE | 0.870 | 0.920 | 0.794 | -0.325 | -0.340 | 0.891 |  |  |  |  |  |  |  |  |  |  |
F | 0.832 | 0.898 | 0.746 | 0.212 | 0.190 | -0.084 | 0.864 |  |  |  |  |  |  |  |  |  |
IU | 0.776 | 0.869 | 0.689 | 0.572 | 0.502 | -0.401 | 0.259 | 0.830 |  |  |  |  |  |  |  |  |
PCC | 0.761 | 0.862 | 0.675 | 0.574 | 0.486 | -0.427 | 0.152 | 0.619 | 0.822 |  |  |  |  |  |  |  |
PRSQ | 0.802 | 0.883 | 0.717 | 0.488 | 0.480 | -0.343 | 0.171 | 0.611 | 0.523 | 0.846 |  |  |  |  |  |  |
PU | 0.788 | 0.862 | 0.611 | 0.653 | 0.555 | -0.549 | 0.193 | 0.685 | 0.690 | 0.597 | 0.781 |  |  |  |  |  |
PR | 0.907 | 0.941 | 0.842 | -0.549 | -0.392 | 0.321 | -0.120 | -0.596 | -0.593 | -0.497 | -0.627 | 0.918 |  |  |  |  |
SR | 0.885 | 0.929 | 0.813 | -0.553 | -0.418 | 0.309 | -0.190 | -0.582 | -0.587 | -0.496 | -0.657 | 0.687 | 0.902 |  |  |  |
TA | 0.840 | 0.902 | 0.754 | 0.301 | 0.239 | -0.137 | 0.348 | 0.352 | 0.264 | 0.294 | 0.267 | -0.271 | -0.275 | 0.869 |  |  |
T | 0.836 | 0.891 | 0.672 | 0.682 | 0.546 | -0.497 | 0.236 | 0.736 | 0.749 | 0.651 | 0.736 | -0.731 | -0.730 | 0.342 | 0.820 |  |
ST | 0.795 | 0.880 | 0.711 | 0.556 | 0.402 | -0.337 | 0.222 | 0.573 | 0.554 | 0.486 | 0.636 | -0.515 | -0.518 | 0.242 | 0.675 | 0.843 |
PSR | 0.815 | 0.888 | 0.726 | 0.011 | 0.008 | -0.047 | -0.005 | 0.048 | 0.054 | 0.038 | 0.097 | -0.009 | -0.004 | -0.081 | 0.073 | -0.016 | 0.852

We tested the internal consistency and discriminant validity of the proposed
research model. The component reliability (CR) and average variance extracted
(AVE) for each construct were examined to ensure that they met the threshold
criteria for internal consistency.999CR is used to gauge the consistency of
survey items in assessing a certain unobservable concept. It measures how well
these items work together. In contrast, AVE determines how accurately these
items represent the underlying concept, or essentially, how much they ’hit the
target’. The results are in Table 3, which shows that the CR values are all
above 0.70 and all AVE values are above 0.50. Therefore, there are no
convergent validity issues (Bagozzi and Yi, 1988). This was also confirmed
with traditional Cronbach’s $\alpha$ measure,101010Cronbach’s $\alpha$ tells
us how closely related a set of questions are as a group. Higher values
(typically over 0.7) suggest the questions are effectively measuring the same
concept. where all $\alpha$ values in our scale are greater than 0.7 (between
0.713 and 0.923) as reported in the same table. Furthermore, to assess
discriminant validity, Fornell and Larcker (Fornell and Larcker, 1981)
suggested assessing the AVE values of each construct with the inter-
correlation scores within the correlations table. They recommended that the
AVE score must exceed the scores of the inter‐correlations. As it can be seen
in Table 3, the AVE values for all of the constructs are higher than the
square of their inter-construct correlations,thus confirming no discriminant
validity issues (Fornell and Larcker, 1981).

In terms of construct reliability, the outer loadings analysed in our study
are summarised in Table 7, and all of the constructs have individual construct
reliability values that are much larger than the preferred level of 0.7, which
according to Hair et al. indicates that the construct explains more than 50
per cent of the item’s variance, thus providing acceptable item reliability,
except ST1 (0.479), SR2 (0.256), PR1 (0.410), A3 (0.569), PCC4 (0.357), PCC5
(0.365) and PSR2 (0.612), which were therefore removed and are not included in
the table.

####  5.2.2. Structural Model Analysis

We then examined the model’s predictive capabilities and the relationships
between the constructs. We first examined whether there were collinearity
issues in the structural model. By calculating the variance inflation factor
(VIF) values for the factors,111111VIF measures how much the input factors in
a model influence each other. In situations where factors are closely related,
indicated by a high VIF. all VIF values were below the minimum recommended of
3 (Kock and Lynn, 2012), with values from 1.389 to 2.780, thus confirming that
multi-collinearity was not violated.

Table 4. PLS-SEM Analysis Results: Hypotheses testing results.

Factor Type |  | Hypothesis |  $\beta$ value | T Statistics |  $p$ value | Supported?
---|---|---|---|---|---|---
Functional Factor | H1 | Anthropomorphism ->Trust in HVA | 0.076 | 2.030 | 0.042 | $\checkmark$
| H2 | Effort Expectancy ->Trust in HVA | -0.064 | 2.120 | 0.034 | $\checkmark$
| H3 | Perceived Usefulness ->Trust in HVA | 0.277 | 6.089 |  $\textless$0.001 | $\checkmark$
| H4 | Perceived Content Credibility ->Trust in HVA | 0.159 | 4.782 |  $\textless$0.001 | $\checkmark$
| H5 | Perceived Relative Service Quality ->Trust in HVA | 0.107 | 3.187 | 0.001 | $\checkmark$
Personal Factor | H6 | Familiarity ->Trust in HVA | 0.028 | 1.154 | 0.249 | x
| H7 | Technology Attachment ->Trust in HVA | 0.038 | 1.341 | 0.180 | x
| H8 | Social Influence ->Trust in HVA | 0.023 | 0.751 | 0.453 | x
| H9 | Stance in Technology ->Trust in HVA | 0.111 | 3.568 |  $\textless$0.001 | $\checkmark$
Risk Factor | H10 | Security Risk ->Trust in HVA | -0.138 | 3.149 | 0.002 | $\checkmark$
| H11 | Privacy Risk ->Trust in HVA | -0.173 | 5.135 |  $\textless$0.001 | $\checkmark$
| H12 | Perceived Substitution Risk ->Trust in HVA | 0.032 | 1.116 | 0.264 | x
Trust Impact | H13 | Trust in HVA ->Intention to Use | 0.737 | 24.597 |  $\textless$0.001 | $\checkmark$

Next, a bootstrapping with 5000 sub-samples in SmartPLS with a “path” weight
scheme was performed. The results, as shown in Table 4, illustrate support for
the majority of hypotheses and indicate the importance of the relationships
between the constructs. Among the thirteen hypotheses we considered, all five
functional and design factors (H1 - H5), two risk factors (H10, H11) and one
personal factor (H9) showed a significant influence on Trust in HVAs. In
addition, H13 was proved, demonstrating that trust in HVA positively affects
users’ intention to use HVAs. Each of these supported hypotheses has a
T-statistic greater than 2, indicating that that the path coefficient obtained
is also statistically significant (Hair et al., 2017, 2019).

Figure 2. Final Research Model with Path Coefficients (Only significant paths
are retained.)

Figure 2 shows the final model only with significant paths. The $R^{2}$ was
0.841 and 0.542 for Trust in HVA, and Intention to Use, respectively, which
represents the explanatory power of the construct. We can say that, the
_Intention to Use_ is moderately explained by _Trust in HVA_ , which in turn
is substantially explained by its antecedent factors121212As suggested by Hair
et al. (Hair et al., 2017, 2019), $R^{2}$ values of 0.75, 0.50 and 0.25 can be
considered substantial, moderate and weak.. Finally, the bootstrapped
Standardized Root Mean Square Residual (SRMR) is the most frequently used
measure to determine goodness-of-fit while using SmartPLS (Fernandes and
Oliveira, 2021). In this study, the SRMR value is 0.053, below the 0.08
threshold, suggesting an adequate model fit.

####  5.2.3. Controlling for HVA Use and Demographics

We also wanted to control for additional variables introduced by our sample of
participants beyond the factors we hypothesized (as common in PLS-SEM
analysis) (Sohaib, 2021). Following guidance on the reporting of the use of
control variables (Atinc et al., 2012), we were particularly interested in our
post-hoc analysis in the previous use of voice assistants for healthcare as
well as other demographics reported by our participants (see Table 2).
Although all our participants had experience using a voice assistant, not all
of them were users of HVA (30.7% used HVA). This was important for properly
modelling intention to use HVAs, something not possible if all participants
were already using them. However, it was also important to understand whether
that may have influenced trust, so we added to the model a control variable of
previous HVA use on Trust in HVA. We also added control variables to capture
the voice assistant they normally use (e.g. Amazon Alexa, Google Assistant,
etc.) following (Seymour and Van Kleek, 2021), as well as general demographics
like age (Bailey and Leon, 2019) and gender (Mumu et al., 2022) which may also
have an influence in trust in technology in general. The resulting model
(which is reported in the Appendix F) showed no significant differences in
terms of the factors previously analyzed; those significantly and
substantially influencing trust in HVA remained so, and those that were not
significant remained so as well. In terms of the control variables, only
gender was statistically significant, suggesting women to be more inclined to
trust in HVAs (Sample Mean = 0.107, $p\textless$0.001).

###  5.3. Post-survey Results

From the qualitative analysis of open-ended questions, we elicited themes that
helped us better understand and supplement our structural model: 1) _Security
and Privacy (associated with H10 and H11), 2) Anthropomorphism (associated
with H2), 3) Other functional-related (associated with H2-5), 4) Financial
Causes, 5) Discrimination Issues, 6) Develop Organisation, and 7) Substitution
Risk (associated with H12)_. While the model had already identified the first
three themes, themes 4–6 emerged as _novel_ findings from the thematic
analysis. Moreover, the factor, _Perceived Substitution Risk_ , which has not
demonstrated statistical significance and therefore has been excluded from the
model, is also not backed up by the results derived from the thematic
analysis. The themes and codes can be found in Appendix D. 62 responses were
excluded due to the provision of non-substantive or irrelevant responses to
the posed questions, e.g., ”I don’t know” or ”I am not sure”.

####  5.3.1. Security and Privacy (retained)

Security and privacy was a common theme mentioned by participants when asked
about the use of healthcare voice assistants, they have shared several
concerns including data misuse. In particular P113 (_Amazon Echo/Alexa &Google
Home, none_) mentioned, ”I would feel uncomfortable discussing personal issues
that Amazon employees could hear”. When asked about the future development of
healthcare AI assistants, several mentioned improving security features; P7
(_Amazon Echo/Alexa &Google Home, Seek diagnosis results & Ask illness or drug
information_) said ”security improvement” and P178 (_Amazon Echo/Alexa, Ask
illness or drug information_) said ”A good approach regarding the security of
the data collected”. Although some participants are open to the idea of
adopting HVAs, there are still some concerns regarding trust.

####  5.3.2. Anthropomorphism (retained)

Participant responses helped us understand the reasons why factors, such as
Anthropomorphism, were ultimately retained in the model. More specifically,
24.4% participants felt that a trustworthy HVA should firstly meet the basic
ability to _”understand the user’s description and have a natural conversation
(P27, Google Home, Seek diagnosis results & Ask illness or drug information)_”
with them, and secondly, at a higher level, _”they should be real and
communicate like a human, less robot. This requires them to have more human-
like features or customised voices. (P60, Amazon Echo/Alexa, Ask illness or
drug information)_ ”. In addition to this, considering that discussing a
health condition is a serious and potentially emotionally draining matter,
participants resonated strongly with the idea that a trustworthy HVA should be
able to show compassion and empathy, with four participants mentioning that
_”Although they are honest, it should have the empathy that doctors have when
they say things like someone is going to die. (P16, Amazon &Microsoft Cortana,
Seek diagnosis results&monitor a chronic disease; P56, Amazon Echo/Alexa, Seek
diagnosis results; P103, Amazon Echo/Alexa, Seek diagnosis results & Ask
illness or drug information; P223, Amazon Echo/Alexa&Apple Homepod, none)_”.

####  5.3.3. Other functional-related (retained)

Other functionality related factors were also mentioned such as the accuracy
of the diagnosis provided by the HVA, P176 _(Google Home, Seek diagnosis
results & Ask illness or drug information)_ mentioned ”Quality and accuracy of
physical diagnosis, particularly in medical areas where people will feel
uncomfortable seeing a doctor about”. Others have mentioned the knowledge of
HVAs and their ability to provide accurate responses, P252 _(Google Home,
none)_ ”I would want to make sure they are consistent and knowledgeable”.
Usability was also been mentioned, P235 _(Amazon Echo/Alexa, Ask illness or
drug information)_ ”easy ways to access health services”.

####  5.3.4. Financial Causes (new)

Price seems to be what will sway users to use and rely on HVAs particularly in
places where healthcare is not provided as a public, free service. A
participant points out that _”As for the cost, they should consider a broader
group of users to make it affordable for everyone.(P242, Google Home, Seek
diagnosis results & Ask illness or drug information&monitor a chronic
disease)”_. For this, the majority of those impacted seems are not wealthy
individuals, like one participant said: _”HVAs that give healthcare access to
under-privileged populations would win more people support.”_ Apart from that,
users are willing to trust and use HVAs as long as they are not set up to make
a profit (_”An AI that is not influenced by profit motives but can supply cost
information when requested (P209, Amazon Echo/Alexa, Ask illness or drug
information)_ ”.).

####  5.3.5. Discrimination Issues (new)

Another factor to increase trust and adoption would be the reducing
discrimination in the health care delivery process. Some users (2%) mentioned
that they had received discrimination in traditional healthcare settings for
different reasons and had experienced discomfort during their visits to
doctors, so they wished that the HVAs could _”focus only on the facts (P235,
Amazon Echo/Alexa, none)_ ”. They were very confident in the usage of HVAs and
one participant expressed that _”HVAs should help reduce the bias that many
doctors have towards patients so that a real diagnosis could happen and help
the patient (P292, Google Home, Seek diagnosis results & Ask illness or drug
information)_”.

####  5.3.6. Develop Organisation (new)

Finally, an interesting result comes from the analysis of the trusted
organisations that develop HVAs. In asking participants who would develop a
HVA, it was concluded that the UK’s NHS (i.e. the UK’s National Health
Service) had a much higher reputation among participants than big companies
such as Google and Apple, while Amazon supporters were the least of these
options (see Figure 3). Beyond the above parties, the most common type
participants recommended were private tech companies, followed by hospitals
and academic institutions.

####  5.3.7. Substitution Risk (excluded from the final model)

. With regards to reasons why some factors were not retained in the model, the
most common theme was about substitution risk. In particular, 22.3%
participants do not feel that doctors will be replaced by HVAs in the future.
On the contrary, they believe that HVAs should complement the work of doctors,
_”I think it should be a tool used by doctors to help them, and not something
used to replace them (P43, Google Home, none_).” The interesting thing is when
HVAs work alone, especially on cases of serious illness, participants don’t
trust them to work well. But they trust the HVAs if their information and
decisions can be monitored by a doctor (_”It is more reliable that they work
alongside doctors, and decisions they made should be further validate by a
real doctor (P300, Amazon Echo/Alexa, Ask illness or drug information)_.”).

Figure 3. Most trusted organizations to develop HVA.

##  6\. Discussion

We now summarize the contributions of our model and then compare it to
previous research on trust in general-purpose voice assistants. After this, we
discuss the main implications for HVAs and provide recommendations for
developers and researchers who want to explore HVAs.

This study makes critical theoretical and practical contributions by firstly
explaining user trust in HVAs through factors captured in three important
aspects _(personal, functional, and risk)_. These dimensions are recognized as
pivotal elements in the overarching framework of human-computer interaction,
particularly in the context of fostering collaborative relationships between
humans and AI entities (Jirotka et al., 2005; Hauptman et al., 2022; Park et
al., 2019). Specifically, the model provided a statistically based assessment
of the degree to which the various factors influence trust, in addition to
testing which factors increase or decrease it. According to the results, in
each of the categories of personal, functional and risk factors, at least one
factor was shown to have an impact on trust. This is a crucial step towards a
holistic model to design trustworthy HVAs with a concrete understanding of the
antecedents that foster trust. From the factors we considered, the most
significant ones that affect user trust in HVAs are usefulness, information
credibility, service quality, and security/privacy, exploring how they might
be designed for in current and future products. We learned potential reasons
why certain factors were retained in the model and were meaningful in
explaining trust, while others were not (for instance because participants
thought substitution was not really a risk in how they viewed HVA should
work). In addition, we learned more about the users and tried to discover
aspects that have not been taken into account in terms of trust (for instance
who builds the HVA, if it helps mitigate discrimination and if it is
affordable).

Our research, especially its results, offers profound insights that resonate
with the broader society and, more specifically, the intricate healthcare
sector. This field, characterized by its complexity, epitomizes the essence of
multi-stakeholder collaboration, a theme central to the ethos of the CSCW
community (Fitzpatrick and Ellingsen, 2013; Lai et al., 2021; Park et al.,
2019, 2017). By delving into the dynamics of trust and adoption, we provided
AI product providers with a clearer roadmap for product refinement,
emphasizing areas that foster enhanced collaboration (Jirotka et al., 2005;
Hauptman et al., 2022). This collaborative lens can also help policymakers and
third parties understand patient-centric concerns, notably privacy and data
safety. Furthermore, when compared with general-purpose voice assistants, our
findings accentuate the distinct collaborative and social complexity of the
healthcare domain. In doing so, our work not only addresses societal needs but
also aligns with and contributes to the core values and interests of the CSCW
community, emphasizing the significance of cooperation.

###  6.1. Comparing Trust Between HVA and General-purpose Voice Assistants

Following the views expressed in previous literature (Montague et al., 2009;
Miner et al., 2016; Abdi et al., 2021), this study reveals the necessity of
studying trust in the healthcare context, especially compared with trust in
general-purpose technologies, in this case HVA in comparison with general-
purpose voice assistants. To better compare the results of this study with
previous work on trust in _general-purpose_ voice assistants, we first
summarize the outcomes of the factors being validated in literature on trust
in general-purpose voice assistants in Table 5. Based on these previous works
and our results, we highlight four main differences in terms of trust in HVA
and trust in general-purpose voice assistants.

First, we considered factors that have not been studied in prior work on
general-purpose voice assistants but that, as explained throughout this paper,
have been shown to be key in the context of healthcare technologies (perceived
content credibility, perceived relative service quality and perceived
substitution risk). In this regard, we show that two functional factors,
perceived content credibility and perceived service quality do in fact
influence trust in HVAs, but that perceived substitution risk does not.
According to the qualitative results that, the current use of HVAs is regarded
as a complement to existing healthcare services rather than a replacement,
something we discuss further in the next section on practical insights.

Table 5. Support for factors influencing trust in general-purpose voice assistants. N/C means not considered in the literature.  Category | Factor | Supported | Rejected
---|---|---|---
Functional | Anthropomorphism | (Chi et al., 2021; Pitardi and Marriott, 2021; Foehr and Germelmann, 2020) | -
| Effort Expectancy | (AlHogail, 2018; Chi et al., 2021; Pitardi and Marriott, 2021) | -
| Perceived Usefulness | - | (Pitardi and Marriott, 2021)
| Perceived Content Credibility | N/C | N/C
| Perceived Relative Service Quality | N/C | N/C
Personal | Familiarity | (Alraja et al., 2019; Chi et al., 2021) | -
| Technology Attachment | (Chi et al., 2021) | -
| Social Influence | (AlHogail, 2018; Chi et al., 2021) | -
| Stance in Tech | (Chi et al., 2021) | -
Risk | Security Risk | (AlHogail, 2018; Alraja et al., 2019) | -
| Privacy Risk | (Alraja et al., 2019; Vimalkumar et al., 2021) | (Pitardi and Marriott, 2021)
| Perceived Substitution Risk | N/C | N/C

Second, previous work on general-purpose voice assistants has not found a
direct link between perceived usefulness and trust (Pitardi and Marriott,
2021). In contrast, this study shows that for HVAs, perceived usefulness is
actually the factor that contributes the most towards substantially explaining
trust. The reason for this may be that HVAs have a specific purpose, a
hypothesis which is consistent with previous work on healthcare technology
that also found perceived usefulness to influence trust (Peng et al., 2020).
On the other hand, general purpose assistants tend to have more varied
purposes that are not found in the healthcare context, such as hedonic
benefits and entertainment (Pitardi and Marriott, 2021).

Third, among the personal factors, only Stance in Tech proved to be directly
linked to trust in HVA. Even the social influence factor, where a device
owner’s relationship may affect user perceptions in general settings (Yao et
al., 2019), seems not to hold true in the distinct environment of healthcare.
This may be because what leads people to trust VAs in general, i.e.
familiarity and attachment to the platform (Alraja et al., 2019; Chi et al.,
2021), being recognised or influenced by others (AlHogail, 2018), are not as
important in the perceptions of users as other factors, such as the usefulness
and quality evidenced in the results of our model. It may also be that not all
participants actually have people in their social networks that use HVAs or
they may not be familiar with it. However, when controlling in the model for
HVA use (recall that half of our participants used HVA) it was shown as not
significant.

Finally, prior work on general-purpose voice assistants is inconclusive as to
the effect of privacy risks on trust, with some studies suggesting that
privacy risks influence trust (Alraja et al., 2019; Vimalkumar et al., 2021)
and others suggesting that it does not (Pitardi and Marriott, 2021). We show
that perceived privacy risks are very important for trust in HVAs, being the
second most significant factor in the final model. One explanation for this is
that healthcare data is known to be very sensitive in general (Miner et al.,
2016; Abdi et al., 2021), and previous work on privacy and voice assistants in
particular found that users considered the flows of this data across the voice
assistant ecosystem as the least appropriate (Abdi et al., 2021).

###  6.2. Implications Unique to Healthcare Voice Assistants

##### Enhancing usefulness in HVAs through participatory design.

Perceived usefulness plays the most important role in the user’s trust in
HVAs. This suggest that engagement with patients throughout the process of HVA
development is crucial, from ideation through design, implementation, and
evaluation. A common and well established means of achieving highly usable
solutions in healthcare and other areas of HCI is participatory design methods
(Spinuzzi, 2005; Harrington et al., 2020), and beyond this it is important to
engage with the communities that are the target of an intervention throughout
the design and development process. It has been shown, for instance, that as a
response to the lack of usability of some diabetes technologies for many
patients, users have begun hacking their way into commercial devices to better
serve their needs (e.g. do-it-yourself automated insulin delivery systems
(Braune et al., 2022)). Beyond usefulness as an objective it is also important
that users _perceive_ HVAs to be useful, particularly as potential users may
have had negative experiences with voice assistants in the past.

##### Balancing service quality and roles in HVAs.

The quality of service relative to what a health professional would offer also
contributes substantially and significantly to trust in HVA. When considering
this, there are two key roles HVA could play that may make it easier or harder
to offer a quality of service that is higher relative to clinical staff. The
first of these roles would be HVAs as a replacement for in-person care,
implying a challenging goal of matching the diagnostic, analytical, and
social/communication skills. It has been recommended that HVAs should be more
compassionate (Tanioka et al., 2021) and exhibit more non-verbal communication
skills and emotional reactions (Aeschlimann et al., 2020). This also requires
that HVAs are able to personalise responses to different people; behaving like
a human healthcare professional requires empathy when facing serious cases, as
well as observing the user’s personality traits in other consultation
situations. This is an area where development is required. At present,
research reports mixed results and a lack of testing in real-world clinical
environments (Gille et al., 2020; Vollmer et al., 2018; Nagendran et al.,
2020). The second, and potentially easier, role for HVAs would be as a
complement to existing healthcare services. In these scenarios, HVAs might be
restricted to triage, giving limited advice on less complex conditions and
handing off other cases to human practitioners. This is a much simpler task,
lowering the cost of service delivery whilst being much easier to adequately
perform relative to a medical professional (as an example, matching the advice
given by a clinician for a common cold is much easier than for type 2
diabetes). In the qualitative results, it becomes evident that the ideal
scenario for patients involves the integration of healthcare AI but supervised
and regulated by physicians. This not only exemplifies the need for harmonious
collaboration between doctors and cutting-edge technology for the betterment
of patient care, as highlighted in (Lai et al., 2021; Fitzpatrick and
Ellingsen, 2013; Park et al., 2019), but also underscores the importance of
human-AI interaction in fostering trust between patients and their physicians.

##### Privacy complexities in HVAs and design choices.

While privacy is a key concern in general-purpose voice assistants, the
introduction of healthcare data presents additional complexities. Many
research prototypes and currently available HVAs are built with or on top of
general-purpose commercial voice assistants (Sezgin et al., 2020b). The
security and privacy of voice assistants in general is an active area of
research—see (Edu et al., 2020; Seymour et al., 2023b) for a systematic review
on the topic— and several issues have been revealed, including the extensive
collection of sensitive user data, unauthorized access and questionable
practices by third-parties and skills (Edu et al., 2021; Guo et al., 2020;
Seymour et al., 2024; Bispham et al., 2022; Edu et al., 2022), along with
inadequacies in the development and application of privacy policies (Liao et
al., 2020; Edu et al., 2022). Researchers are actively working on a range of
solutions, such as mechanisms for providing user with meaningful verbal
consent (Seymour et al., 2023a; Chalhoub et al., 2024) and automated systems
to identify policy breaches (Young et al., 2022; Edu et al., 2021).
Additionally, significant efforts are being directed towards creating models
that aid users in making privacy decisions consistent with their preferences
(Zhan et al., 2022, 2023) and crafting explanations that incorporate elements
of trust and privacy, which are intended to diminish user concerns and
strengthen trust in VAs (Seymour and Such, 2023; Such, 2017).Moreover, how
users’ perceive security and privacy of these underlying platforms is very
important for HVAs, as users consider healthcare-related data more sensitive
and prefer to share less information with voice assistants about health (Abdi
et al., 2021). As the information they share with them is often protected by
additional regulation (such as HIPAA in the U.S.), the security of those HVAs
is therefore highly dependent on the security of the voice assistants on top
of which they are built. To accommodate this, certain platforms such as Alexa
have additional requirements for health-related
skills131313https://developer.amazon.com/en-US/docs/alexa/custom-
skills/requirements-for-hipaa-eligible-skills.html — skills are the name given
to the third-party functionality that can be added to Alexa to add capacities
to it. Furthermore, attitudes towards health information privacy have notably
shifted over time, particularly in the context of chronic illnesses such as
diabetes (O’Kane et al., 2013). For instance, the emergence of health-related
complications within a patient’s medical condition may result in a diminished
emphasis on privacy. Additionally, the assumption of new responsibilities,
such as the care of children afflicted with chronic illnesses, may prompt
individuals to reassess their privacy considerations. Given the evolving
concerns, especially in chronic illnesses like diabetes, it’s imperative to
address these in the design of voice assistants handling sensitive health
data.

An alternative option is to create standalone HVAs (instead of on top of
general-purpose voice assistants), mitigating the risks and negative
perceptions related to voice assistant platforms. However, the vast amount of
data and expertise required in order to create the speech recognition and NLP
processing services required to operate a successful voice assistant means
that, in practice, these services will likely be provided by third parties,
who may even be the same organisations providing general-purpose voice
assistants. While there may be differences between, e.g., creating an HVA
skill to run on top of Amazon Alexa and creating a standalone HVA using Amazon
Lex141414A “fully managed artificial intelligence (AI) service with advanced
natural language models to design, build, test, and deploy conversational
interfaces in applications” https://aws.amazon.com/lex, that difference may be
difficult to communicate to users.

##### Trust dynamics and the importance of compliance in development.

An often overlooked point in previous research, is who are expected to make
the HVAs? For some mainstream technology companies (Amazon, Google, and
Apple), user trust in VAs when they are used for no specific purpose
correlates with trust in their parent company, and Amazon has a higher level
of ”trust reputation” than Google (Seymour and Van Kleek, 2021) in the minds
of users. However in terms of providing health-related services, Amazon was
less trusted than Google/Apple in our results. This may be explained by the
fact that platforms collaborate with different partners that provide/develop
health applications, and the strength of the platform in regulating the
compliance of these applications with existing privacy and security policies
can also affect user trust building. Evidence suggests that breaches are
common in Amazon Alexa skills for health care, with 86.36% missing disclaimers
when providing medical information, and 30.23% storing users’ physical or
mental health data without approval (Shezan et al., 2020). Beyond big tech
companies, public health organizations such as the UK’s NHS are trusted more
to develop HVA. This is probably because the NHS system is largely funded by
the UK government and is under the jurisdiction and oversight of the
Department of Health151515https://www.nhs.uk/. Meanwhile, given that the
services provided by the NHS are free to UK citizens, this reinforces users’
belief that the purpose of NHS is to help people, improve the healthcare
environment and save lives, rather than earning money. This is in line with
the argument made by users that they trust non-profit organisations more as
for-profit companies often do not have the most stringent track record in
protecting data and may sell these data without user awareness. Therefore,
handing the company the keys to people’s private health information raises red
flags. Furthermore, each country will have its own clinical guidelines, and as
we mentioned before, some countries often have additional regulations
governing the use of health information (HIPAA in the U.S.). In this regard,
each healthcare system would potentially suggest different organisations who
would develop/manage such an assistant. This might be redundant for VAs that
provide common functions (e.g. ordering a taxi, managing the To-do List), but
it is necessary for HVAs. Moreover, deeper integration between large companies
with established program frameworks and hospitals with specialist physicians
and public credibility should be encouraged as well.

##### Discrimination and cost.

Last but not least, other factors identified in our post-survey questionnaire
should also be considered for trust in HVA in practice, such as the potential
for HVA to be more or less discriminatory than health professionals and the
impact this may have on users. Participants seem to think that HVA would
discriminate and have less biases and preconceptions than health
professionals. However, it is well-established that AI-based systems may not
necessarily be any more _neutral_ , and they can actually be as discriminatory
as, or even more than, human decision-makers (Ferrer et al., 2021; van Nuenen
et al., 2022). Beyond benefiting from the general efforts towards fairer AI
(Binns, 2018), this will also require HVAs developers to have a deeper
understanding of discrimination in the medical industry and thoroughly
evaluate future HVAs in the context of these ethical issues. With regards to
financial issues, the cost of HVAs may also need to be considered.

###  6.3. Limitations and Future Work

This study has some limitations. First, our model is a significant step
towards a holistic model to explain factors that influence trust in HVA.
However, there are still many factors in all levels of designing an HVA that
can impact on trust and should be taken into account (e.g., factors that were
newly identified in qualitative results, financial causes, bias, and develop
organisation). Future work should focus on this and continue to extend our
model. It is crucial to delve deeper into the influential factors that affect
this field of study. Considerations such as the systematic categorisation of
functional factors, the improved capacity for rectifying communication errors
and failures as evidenced in (Cho and Rader, 2020), and their subsequent
effect on trust, demand attention. Additionally, privacy and security concerns
associated with general voice assistants (Lau et al., 2018; Malkin et al.,
2022; Meng et al., 2021; McMillan et al., 2019; Seymour and Such, 2023; Yao et
al., 2019), their application within the healthcare context, and their
potential influence on trust also necessitate investigation. These represent
significant opportunities for future work within this research domain. Second,
trust may evolve as time goes by and users have more chances to interact with
HVA. Therefore, longitudinal studies should also be considered in the future
to consider factors that contribute to trust building over time, such as the
type of relationship that could be formed between a user and an HVA, with
recent studies having explored the relationships that users develop with
general-purpose voice assistants (Seymour and Van Kleek, 2021). Concurrently,
as discussed in section 6.2, developers should continually update their
efforts to ensure functional ease and user safety mechanisms. Given that
attitudes, including privacy concerns and needs, can evolve over time (O’Kane
et al., 2013), it’s imperative to develop and offer services that provide
enduring benefits to users. Third, we focused on users of HVAs as _patients_ ,
but another interesting demographic to consider would be healthcare
professionals, who may also utilize HVAs to facilitate their work, e.g. by
allowing doctors to have voice interactions with electronic health records
(Kumah-Crystal et al., 2018) or supporting the tasks of home health aides
(Bartle et al., 2022b). This would complement our study from the ‘two sides’
of potential HVA use.

##  7\. Conclusion

Our work is the first one to examine different functional, personal and risk
factors that affect trust in healthcare voice assistants and user intention to
use these technologies. We find that anthropomorphism, perceived usefulness,
effort expectancy, content credibility, and relative service quality, together
with security risk, privacy risk and stance in technology substantially
explain trust in HVA. In turn, trust in HVA moderately explains intention to
use HVA. Based on these results, we derive some practical insights and
recommendations to help design and develop next-generation HVAs that can
promote user trust and encourage adoption.

###### Acknowledgements.

We thank CSCW’s anonymous reviewers, Ruba Abu-Salma, and Hana Kopecka for
their constructive comments on previous drafts of this paper. This research
was partially funded by EPSRC under grant _SAIS: Secure AI assistantS_
(EP/T026723/1) and the INCIBE’s strategic SPRINT (Seguridad y Privacidad en
Sistemas con Inteligencia Artificial) project with funds from the EU-
NextGenerationEU through the Spanish government’s Plan de Recuperación,
Transformación y Resiliencia.

## References

  * (1)
  * Abbasi and Kashiyarndi (2006) MM Abbasi and S Kashiyarndi. 2006.  Clinical Decision Support Systems: A discussion on different methodologies used in Health Care.  _Marlaedalen University Sweden_ (2006).
  * Abdi et al. (2019) Noura Abdi, Kopo Ramokapane, and Jose Such. 2019\.  More than smart speakers: security and privacy perceptions of smart home personal assistants. In _Fifteenth Symposium on Usable Privacy and Security ( $\\{$SOUPS$\\}$ 2019)_. 451–466.
  * Abdi et al. (2021) Noura Abdi, Xiao Zhan, Kopo M Ramokapane, and Jose Such. 2021\.  Privacy Norms for Smart Home Personal Assistants. In _Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems_. 1–14.
  * Aeschlimann et al. (2020) Sara Aeschlimann, Marco Bleiker, Michael Wechner, and Anja Gampe. 2020.  Communicative and social consequences of interactions with voice assistants.  _Computers in Human Behavior_ 112 (2020), 106466.
  * Agarwal and Prasad (1998) Ritu Agarwal and Jayesh Prasad. 1998.  A conceptual and operational definition of personal innovativeness in the domain of information technology.  _Information systems research_ 9, 2 (1998), 204–215.  https://doi.org/10.1287/isre.9.2.204
  * Al Shamsi et al. (2022) Jawaher Hamad Al Shamsi, Mostafa Al-Emran, and Khaled Shaalan. 2022\.  Understanding key drivers affecting students’ use of artificial intelligence-based voice assistants.  _Education and Information Technologies_ (2022), 1–21.
  * Alagha and Helbing (2019) Emily Couvillon Alagha and Rachel Renee Helbing. 2019.  Evaluating the quality of voice assistants’ responses to consumer health questions about vaccines: an exploratory comparison of Alexa, Google Assistant and Siri.  _BMJ health & care informatics_ 26, 1 (2019).  https://doi.org/10.1136/bmjhci-2019-100075
  * AlHogail (2018) Areej AlHogail. 2018\.  Improving IoT technology adoption through improving consumer trust.  _Technologies_ 6, 3 (2018), 64.  https://doi.org/10.3390/technologies6030064
  * Alraja et al. (2019) Mansour Naser Alraja, Murtaza Mohiuddin Junaid Farooque, and Basel Khashab. 2019.  The effect of security, privacy, familiarity, and trust on users’ attitudes toward the use of the IoT-based healthcare: the mediation role of risk perception.  _IEEE Access_ 7 (2019), 111341–111354.
  * Ammari et al. (2019) Tawfiq Ammari, Jofish Kaye, Janice Y Tsai, and Frank Bentley. 2019.  Music, Search, and IoT: How People (Really) Use Voice Assistants.  _ACM Trans. Comput. Hum. Interact._ 26, 3 (2019), 28\.  https://doi.org/10.1145/3311956
  * Appelman and Sundar (2016) Alyssa Appelman and S Shyam Sundar. 2016.  Measuring message credibility: Construction and validation of an exclusive scale.  _Journalism & Mass Communication Quarterly_ 93, 1 (2016), 59–79.  https://doi.org/10.1177/1077699015606057
  * Arnold et al. (2019) Matthew Arnold, Rachel KE Bellamy, Michael Hind, Stephanie Houde, Sameep Mehta, Aleksandra Mojsilović, Ravi Nair, K Natesan Ramamurthy, Alexandra Olteanu, David Piorkowski, et al. 2019\.  FactSheets: Increasing trust in AI services through supplier’s declarations of conformity.  _IBM Journal of Research and Development_ 63, 4/5 (2019), 6–1.
  * Asan et al. (2020) Onur Asan, Alparslan Emrah Bayrak, Avishek Choudhury, et al. 2020\.  Artificial intelligence and human trust in healthcare: focus on clinicians.  _Journal of medical Internet research_ 22, 6 (2020), e15154.  https://doi.org/doi:10.2196/15154
  * Atinc et al. (2012) Guclu Atinc, Marcia J Simmering, and Mark J Kroll. 2012\.  Control variable use and reporting in macro and micro management research.  _Organizational Research Methods_ 15, 1 (2012), 57–74.
  * Bagozzi and Yi (1988) Richard P Bagozzi and Youjae Yi. 1988.  On the evaluation of structural equation models.  _Journal of the academy of marketing science_ 16, 1 (1988), 74–94.  https://doi.org/10.1007/BF02723327
  * Bailey and Leon (2019) Phoebe E Bailey and Tarren Leon. 2019.  A systematic review and meta-analysis of age-related differences in trust.  _Psychology and aging_ 34, 5 (2019), 674.
  * Baker et al. (2020) Adam Baker, Yura Perov, Katherine Middleton, Janie Baxter, Daniel Mullarkey, Davinder Sangar, Mobasher Butt, Arnold DoRosario, and Saurabh Johri. 2020.  A comparison of artificial intelligence and human doctors for the purpose of triage and diagnosis.  _Frontiers in artificial intelligence_ 3 (2020), 543405.
  * Bansal et al. (2016) Gaurav Bansal, Fatemeh Mariam Zahedi, and David Gefen. 2016.  Do context and personality matter? Trust and privacy concerns in disclosing private information online.  _Information & Management_ 53, 1 (2016), 1–21.
  * Bartle et al. (2022a) Vince Bartle, Janice Lyu, Freesoul El Shabazz-Thompson, Yunmin Oh, Angela Anqi Chen, Yu-Jan Chang, Kenneth Holstein, and Nicola Dell. 2022a.  “A Second Voice”: Investigating Opportunities and Challenges for Interactive Voice Assistants to Support Home Health Aides _(CHI ’22)_. Association for Computing Machinery, New York, NY, USA, Article 45, 17 pages.  https://doi.org/10.1145/3491102.3517683
  * Bartle et al. (2022b) Vince Bartle, Janice Lyu, Freesoul El Shabazz-Thompson, Yunmin Oh, Angela Anqi Chen, Yu-Jan Chang, Kenneth Holstein, and Nicola Dell. 2022b.  “A Second Voice”: Investigating Opportunities and Challenges for Interactive Voice Assistants to Support Home Health Aides. In _CHI Conference on Human Factors in Computing Systems_. 1–17.
  * Benbasat and Wang (2005) Izak Benbasat and Weiquan Wang. 2005.  Trust in and adoption of online recommendation agents.  _Journal of the association for information systems_ 6, 3 (2005), 4\.
  * Bharti et al. (2020) Urmil Bharti, Deepali Bajaj, Hunar Batra, Shreya Lalit, Shweta Lalit, and Aayushi Gangwani. 2020\.  Medbot: Conversational artificial intelligence powered chatbot for delivering tele-health after covid-19. In _2020 5th international conference on communication and electronics systems (ICCES)_. IEEE, 870–875.
  * Bickmore et al. (2018) Timothy W Bickmore, Ha Trinh, Stefan Olafsson, Teresa K O’Leary, Reza Asadi, Nathaniel M Rickles, and Ricardo Cruz. 2018\.  Patient and consumer safety risks when using conversational assistants for medical information: an observational study of Siri, Alexa, and Google Assistant.  _Journal of medical Internet research_ 20, 9 (2018), e11510.
  * Binns (2018) Reuben Binns. 2018\.  Fairness in machine learning: Lessons from political philosophy. In _Conference on Fairness, Accountability and Transparency_. PMLR, 149–159.
  * Bispham et al. (2022) Mary K. Bispham, Clara Zard, Suliman Sattar, Xavier Ferrer Aran, Guillermo Suarez-Tangil, and Jose Such. 2022.  Leakage of Sensitive Information to Third-Party Voice Applications. In _ACM CUI 2022: 4th Conference on Conversational User Interfaces_. 32:1–32:4.
  * Braun and Clarke (2012) Virginia Braun and Victoria Clarke. 2012.  Thematic analysis.  (2012).  https://doi.org/10.1037/13620-004
  * Braune et al. (2022) Katarina Braune, Rayhan A Lal, Lenka Petruželková, Gary Scheiner, Per Winterdijk, Signe Schmidt, Linda Raimond, Korey K Hood, Michael C Riddell, Timothy C Skinner, et al. 2022\.  Open-source automated insulin delivery: international consensus statement and practical guidance for health-care professionals.  _The Lancet Diabetes & Endocrinology_ 10, 1 (2022), 58–74.
  * Brewer et al. (2022) Robin Brewer, Casey Pierce, Pooja Upadhyay, and Leeseul Park. 2022.  An empirical study of older adult’s voice assistant use for health information seeking.  _ACM Transactions on Interactive Intelligent Systems (TiiS)_ 12, 2 (2022), 1–32.
  * Buinhas et al. (2018) Susana Buinhas, Ana Paula Cláudio, Maria Beatriz Carmo, João Balsa, Afonso Cavaco, Anabela Mendes, Isa Félix, Nuno Pimenta, and Mara Pereira Guerreiro. 2018.  Virtual assistant to improve self-care of older people with type 2 diabetes: first prototype. In _International Workshop on Gerontechnology_. Springer, 236–248.
  * Buteau and Lee (2021) Emily Buteau and Joonghwa Lee. 2021.  Hey Alexa, why do we use voice assistants? The driving factors of voice assistant technology use.  _Communication Research Reports_ 38, 5 (2021), 336–345.
  * Chalhoub et al. (2024) George Chalhoub, Martin J Kraemer, and Ivan Flechais. 2024\.  Useful shortcuts: Using design heuristics for consent and permission in smart home devices.  _International Journal of Human-Computer Studies_ 182 (2024), 103177\.
  * Chandra et al. (2018) Yakob Utama Chandra, Desi Maya Kristin, Joni Suhartono, Fina Shabrina Sutarto, and Minseol Sung. 2018.  Analysis of determinant factors of user acceptance of mobile payment system in indonesia (a case study of go-pay mobile payment). In _2018 International Conference on Information Management and Technology (ICIMTech)_. IEEE, 454–459.
  * Chang et al. (2013) Ching-Sheng Chang, Su-Yueh Chen, and Yi-Ting Lan. 2013.  Service quality, trust, and patient satisfaction in interpersonal-based medical service encounters.  _BMC health services research_ 13, 1 (2013), 1–11.
  * Chang et al. (2017) Shuchih Ernest Chang, Anne Yenching Liu, and Wei Cheng Shen. 2017\.  User trust in social networking services: A comparison of Facebook and LinkedIn.  _Computers in Human Behavior_ 69 (2017), 207–217.
  * Chao (2019) Cheng-Min Chao. 2019\.  Factors determining the behavioral intention to use mobile learning: An application and extension of the UTAUT model.  _Frontiers in psychology_ 10 (2019), 1652.
  * Chen and Park (2021) Qian Qian Chen and Hyun Jung Park. 2021.  How anthropomorphism affects trust in intelligent personal assistants.  _Industrial Management & Data Systems_ (2021).
  * Cheng et al. (2018) Amy Cheng, Vaishnavi Raghavaraju, Jayanth Kanugo, Yohanes P Handrianto, and Yi Shang. 2018\.  Development and evaluation of a healthy coping voice interface application using the Google home for elderly patients with type 2 diabetes. In _2018 15th IEEE Annual Consumer Communications & Networking Conference (CCNC)_. IEEE, 1–5.
  * Chi et al. (2021) Oscar Hengxuan Chi, Shizhen Jia, Yafang Li, and Dogan Gursoy. 2021.  Developing a formative scale to measure consumers’ trust toward interaction with artificially intelligent (AI) social robots in service delivery.  _Computers in Human Behavior_ 118 (2021), 106700.  https://doi.org/10.1016/j.chb.2021.106700
  * Cho (2019) Eugene Cho. 2019\.  Hey Google, can I ask you something in private?. In _Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems_. 1–9.
  * Cho et al. (2020) Eugene Cho, S. Shyam Sundar, Saeed Abdullah, and Nasim Motalebi. 2020.  _Will Deleting History Make Alexa More Trustworthy? Effects of Privacy and Content Customization on User Experience of Smart Speakers_.  Association for Computing Machinery, New York, NY, USA, 1–13.  https://doi.org/10.1145/3313831.3376551
  * Cho and Rader (2020) Janghee Cho and Emilee Rader. 2020.  The role of conversational grounding in supporting symbiosis between people and digital assistants.  _Proceedings of the ACM on Human-Computer Interaction_ 4, CSCW1 (2020), 1–28.
  * Cho and Hu (2009) Jinsook E Cho and Haiyan Hu. 2009.  The effect of service quality on trust and commitment varying across generations.  _International journal of consumer studies_ 33, 4 (2009), 468–476.
  * Christian M. et al. (2015) Ringle Christian M., Wende Sven, and Becker Jan-Michael. 2015\.  SmartPLS 3. Bönningstedt: SmartPLS.  https://www.smartpls.com/
  * Chung et al. (2018) Arlene E Chung, Ashley C Griffin, Dasha Selezneva, and David Gotz. 2018.  Health and fitness apps for hands-free voice-activated assistants: content analysis.  _JMIR mHealth and uHealth_ 6, 9 (2018), e9705.
  * Cohen (1992) Jacob Cohen. 1992\.  A power primer.  _Psychological bulletin_ 112, 1 (1992), 155–159.  https://doi.org/10.1037/0033-2909.112.1.155
  * Coughlan et al. (2012) Tim Coughlan, Michael Brown, Richard Mortier, Robert J. Houghton, Murray Goulden, and Glyn Lawson. 2012.  Exploring Acceptance and Consequences of the Internet of Things in the Home. In _2012 IEEE International Conference on Green Computing and Communications_. 148–155.  https://doi.org/10.1109/GreenCom.2012.32
  * Dambanemuya and Diakopoulos (2021) Henry Kudzanai Dambanemuya and Nicholas Diakopoulos. 2021.  Auditing the Information Quality of News-Related Queries on the Alexa Voice Assistant.  _Proceedings of the ACM on Human-Computer Interaction_ 5, CSCW1 (2021), 1–21.
  * Davis (1989) Fred D Davis. 1989\.  Perceived usefulness, perceived ease of use, and user acceptance of information technology.  _MIS quarterly_ (1989), 319–340.
  * Dojchinovski et al. (2019) Dimitri Dojchinovski, Andrej Ilievski, and Marjan Gusev. 2019\.  Interactive home healthcare system with integrated voice assistant. In _2019 42nd International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO)_. IEEE, 284–288.
  * Drozdal et al. (2020) Jaimie Drozdal, Justin Weisz, Dakuo Wang, Gaurav Dass, Bingsheng Yao, Changruo Zhao, Michael Muller, Lin Ju, and Hui Su. 2020.  Trust in AutoML: exploring information needs for establishing trust in automated machine learning systems. In _Proceedings of the 25th international conference on intelligent user interfaces_. 297–307.
  * Easwara Moorthy and Vu (2015) Aarthi Easwara Moorthy and Kim-Phuong L Vu. 2015.  Privacy concerns for use of voice activated personal assistant in the public space.  _International Journal of Human-Computer Interaction_ 31, 4 (2015), 307–335.
  * Edu et al. (2021) Jide Edu, Xavi Ferrer Aran, Jose Such, and Guillermo Suarez-Tangil. 2021.  SkillVet: Automated traceability analysis of Amazon Alexa skills.  _IEEE Transactions on Dependable and Secure Computing_ (2021).
  * Edu et al. (2022) Jide Edu, Xavier Ferrer-Aran, Jose Such, and Guillermo Suarez-Tangil. 2022.  Measuring Alexa Skill Privacy Practices across Three Years. In _Proceedings of the ACM Web Conference (WWW)_. 670–680.
  * Edu et al. (2020) Jide Edu, Jose Such, and Guillermo Suarez-Tangil. 2020\.  Smart home personal assistants: a security and privacy review.  _ACM Computing Surveys (CSUR)_ 53, 6 (2020), 1–36.  https://doi.org/10.1145/3412383
  * Egea and González (2011) José Manuel Ortega Egea and María Victoria Román González. 2011\.  Explaining physicians’ acceptance of EHCR systems: An extension of TAM with trust and risk factors.  _Computers in Human Behavior_ 27, 1 (2011), 319–332.
  * Ellahham et al. (2020) Samer Ellahham, Nour Ellahham, and Mecit Can Emre Simsekler. 2020\.  Application of artificial intelligence in the health care safety context: opportunities and challenges.  _American Journal of Medical Quality_ 35, 4 (2020), 341–348.
  * Epley et al. (2007) Nicholas Epley, Adam Waytz, and John T Cacioppo. 2007\.  On seeing human: a three-factor theory of anthropomorphism.  _Psychological review_ 114, 4 (2007), 864–886.  https://doi.org/10.1037/0033-295X.114.4.864
  * Fan et al. (2020) Wenjuan Fan, Jingnan Liu, Shuwan Zhu, and Panos M Pardalos. 2020\.  Investigating the impacting factors for the healthcare professionals to adopt artificial intelligence-based medical diagnosis support system (AIMDSS).  _Annals of Operations Research_ 294, 1 (2020), 567–592.  https://doi.org/10.1007/s10479-018-2818-y
  * Fernandes and Oliveira (2021) Teresa Fernandes and Elisabete Oliveira. 2021.  Understanding consumers’ acceptance of automated technologies in service encounters: Drivers of digital voice assistants adoption.  _Journal of Business Research_ 122 (2021), 180–191.
  * Ferrer et al. (2021) Xavier Ferrer, Tom van Nuenen, Jose Such, Mark Coté, and Natalia Criado. 2021.  Bias and Discrimination in AI: a cross-disciplinary perspective.  _IEEE Technology and Society Magazine_ 40, 2 (2021), 72–80.
  * Fitzpatrick and Ellingsen (2013) Geraldine Fitzpatrick and Gunnar Ellingsen. 2013.  A review of 25 years of CSCW research in healthcare: contributions, challenges and future agendas.  _Computer Supported Cooperative Work (CSCW)_ 22 (2013), 609–665.
  * Foehr and Germelmann (2020) Jonas Foehr and Claas Christian Germelmann. 2020\.  Alexa, can I trust you? Exploring consumer paths to trust in smart voice-interaction technologies.  _Journal of the Association for Consumer Research_ 5, 2 (2020), 181–205.
  * Fornell and Larcker (1981) Claes Fornell and David F Larcker. 1981.  Evaluating structural equation models with unobservable variables and measurement error.  _Journal of marketing research_ 18, 1 (1981), 39–50.  https://doi.org/10.1177/002224378101800104
  * Fox (1974) Alan Fox. 1974.  _Beyond contract: Work, power and trust relations_.  Faber & Faber.
  * Frontiers (2021) Frontiers. 2021\.  The Trustworthiness of Voice Assistants in the Context of Healthcare Investigating the Effect of Perceived Expertise on the Trustworthiness of Voice Assistants, Providers, Data Receivers, and Automatic Speech Recognition.  https://www.frontiersin.org/articles/10.3389/fcomp.2021.685250/full
  * Gao and Bai (2014) Lingling Gao and Xuesong Bai. 2014.  A unified perspective on the factors influencing consumer acceptance of internet of things technology.  _Asia Pacific Journal of Marketing and Logistics_ 26, 2 (2014), 211–231.  https://doi.org/10.1108/APJML-06-2013-0061
  * Gefen and Straub (2004) David Gefen and Detmar W Straub. 2004.  Consumer trust in B2C e-Commerce and the importance of social presence: experiments in e-Products and e-Services.  _Omega_ 32, 6 (2004), 407–424.
  * Gille et al. (2020) Felix Gille, Anna Jobin, and Marcello Ienca. 2020.  What we talk about when we talk about trust: Theory of trust for AI in healthcare.  _Intelligence-Based Medicine_ 1 (2020), 100001.  https://doi.org/10.1016/j.ibmed.2020.100001
  * Gregory and Austin (2021) Paul AM Gregory and Zubin Austin. 2021.  How do patients develop trust in community pharmacists?  _Research in Social and Administrative Pharmacy_ 17, 5 (2021), 911–920.  https://doi.org/10.1016/j.sapharm.2020.07.023
  * Guo et al. (2020) Zhixiu Guo, Zijin Lin, Pan Li, and Kai Chen. 2020\.  $\\{$SkillExplorer$\\}$: Understanding the Behavior of Skills in Large Scale. In _29th USENIX Security Symposium (USENIX Security 20)_. 2649–2666.
  * Gursoy et al. (2019) Dogan Gursoy, Oscar Hengxuan Chi, Lu Lu, and Robin Nunkoo. 2019.  Consumers acceptance of artificially intelligent (AI) device use in service delivery.  _International Journal of Information Management_ 49 (2019), 157–169.  https://doi.org/10.1016/j.ijinfomgt.2019.03.008
  * Gursoy et al. (2017) Dogan Gursoy, Giacomo Del Chiappa, and Yi Zhang. 2017\.  Preferences regarding external information sources: a conjoint analysis of visitors to Sardinia, Italy.  _Journal of Travel & Tourism Marketing_ 34, 6 (2017), 806–820.  https://doi.org/10.1080/10548408.2016.1237405
  * Gymrek et al. (2013) Melissa Gymrek, Amy L McGuire, David Golan, Eran Halperin, and Yaniv Erlich. 2013.  Identifying personal genomes by surname inference.  _Science_ 339, 6117 (2013), 321–324.
  * Ha et al. (2020) Taehyun Ha, Sangyeon Kim, Donghak Seo, and Sangwon Lee. 2020\.  Effects of explanation types and perceived risk on trust in autonomous vehicles.  _Transportation research part F: traffic psychology and behaviour_ 73 (2020), 271–280.
  * Haan et al. (2019) Marieke Haan, Yfke P Ongena, Saar Hommes, Thomas C Kwee, and Derya Yakar. 2019.  A qualitative study to understand patient perspective on the use of artificial intelligence in radiology.  _Journal of the American College of Radiology_ 16, 10 (2019), 1416–1419.
  * Hair et al. (2009) J Hair, W Black, B Babin, and R Anderson. 2009\.  Multivariate Data Analysis 7th Edition Pearson Prentice Hall: JOUR.  (2009).
  * Hair et al. (2017) Joe Hair, Carole L Hollingsworth, Adriane B Randolph, and Alain Yee Loong Chong. 2017.  An updated and expanded assessment of PLS-SEM in information systems research.  _Industrial Management & Data Systems_ 117, 3 (2017), 442–458.  https://doi.org/10.1108/IMDS-04-2016-0130
  * Hair et al. (2014) Joseph F Hair, G Thomas M Hult, Christian Ringle, and Marko Sarstedt. 2014.  A Primer on Partial Least Squares Structural Equation Modeling (PLS-SEM).  (2014).
  * Hair et al. (2011) Joe F Hair, Christian M Ringle, and Marko Sarstedt. 2011\.  PLS-SEM: Indeed a silver bullet.  _Journal of Marketing theory and Practice_ 19, 2 (2011), 139–152.
  * Hair et al. (2019) Joseph F Hair, Jeffrey J Risher, Marko Sarstedt, and Christian M Ringle. 2019.  When to use and how to report the results of PLS-SEM.  _European business review_ (2019).
  * Hall et al. (2001) Mark A Hall, Elizabeth Dugan, Beiyao Zheng, and Aneil K Mishra. 2001.  Trust in physicians and medical institutions: what is it, can it be measured, and does it matter?  _The milbank quarterly_ 79, 4 (2001), 613–639.  https://doi.org/10.1111/1468-0009.00223
  * Han and Yang (2018) Sangyeal Han and Heetae Yang. 2018.  Understanding adoption of intelligent personal assistants: A parasocial relationship perspective.  _Industrial Management & Data Systems_ 118, 3 (2018), 618–636.  https://doi.org/10.1108/IMDS-05-2017-0214
  * Harrington et al. (2020) Christina N Harrington, Lyndsie Marie Koon, and Wendy A Rogers. 2020.  Design of health information and communication technologies for older adults.  In _Design for Health_. Elsevier, 341–363.
  * Hasan et al. (2020) Rajibul Hasan, Riad Shams, and Mizan Rahman. 2020\.  Consumer trust and perceived risk for voice-controlled artificial intelligence: The case of Siri.  _Journal of Business Research_ 131 (2020), 591–597.  https://doi.org/10.1016/j.jbusres.2020.12.012
  * Hauptman et al. (2022) Allyson I. Hauptman, Wen Duan, and Nathan J. Mcneese. 2022\.  The Components of Trust for Collaborating With AI Colleagues. In _Companion Publication of the 2022 Conference on Computer Supported Cooperative Work and Social Computing_ (Virtual Event, Taiwan) _(CSCW’22 Companion)_. Association for Computing Machinery, New York, NY, USA, 72–75.  https://doi.org/10.1145/3500868.3559450
  * Hauser and Schwarz (2016) David J Hauser and Norbert Schwarz. 2016.  Attentive Turkers: MTurk participants perform better on online attention checks than do subject pool participants.  _Behavior research methods_ 48, 1 (2016), 400–407.
  * Hayden (2013) Erika C Hayden. 2013\.  Privacy loophole found in genetic databases.  _Nature News_ 17 (2013).
  * Jirotka et al. (2005) Marina Jirotka, Rob Procter, Mark Hartswood, Roger Slack, Andrew Simpson, Catelijne Coopmans, Chris Hinds, and Alex Voss. 2005\.  Collaboration and trust in healthcare innovation: The eDiaMoND case study.  _Computer Supported Cooperative Work (CSCW)_ 14 (2005), 369–398.
  * Jutzi et al. (2020) Tanja B Jutzi, Eva I Krieghoff-Henning, Tim Holland-Letz, Jochen Sven Utikal, Axel Hauschild, Dirk Schadendorf, Wiebke Sondermann, Stefan Fröhling, Achim Hekler, Max Schmitt, et al. 2020\.  Artificial intelligence in skin cancer diagnostics: the patients’ perspective.  _Frontiers in medicine_ 7 (2020), 233.
  * Keskinbora (2019) Kadircan H Keskinbora. 2019\.  Medical ethics considerations on artificial intelligence.  _Journal of clinical neuroscience_ 64 (2019), 277–282.
  * Kim et al. (2019) Yujin Kim, Jennifer Dykema, John Stevenson, Penny Black, and D Paul Moberg. 2019.  Straightlining: overview of measurement, comparison of indicators, and effects in mail–web mixed-mode surveys.  _Social Science Computer Review_ 37, 2 (2019), 214–233.  https://doi.org/10.1177/0894439317752406
  * Knowles and Hanson (2018) Bran Knowles and Vicki L. Hanson. 2018.  Older Adults’ Deployment of ‘Distrust’.  25, 4, Article 21 (aug 2018), 25 pages.  https://doi.org/10.1145/3196490
  * Knowles et al. (2015) Bran Knowles, Mark Rouncefield, Mike Harding, Nigel Davies, Lynne Blair, James Hannon, John Walden, and Ding Wang. 2015\.  Models and patterns of trust. In _Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing_. 328–338.
  * Kocaballi et al. (2020) Ahmet Baki Kocaballi, Juan C Quiroz, Dana Rezazadegan, Shlomo Berkovsky, Farah Magrabi, Enrico Coiera, and Liliana Laranjo. 2020\.  Responses of conversational agents to health and lifestyle prompts: investigation of appropriateness and presentation structures.  _Journal of medical Internet research_ 22, 2 (2020), e15823.  https://doi.org/doi:10.2196/15823
  * Kock and Lynn (2012) Ned Kock and Gary Lynn. 2012.  Lateral collinearity and misleading results in variance-based SEM: An illustration and recommendations.  _Journal of the Association for information Systems_ 13, 7 (2012).
  * Koh and Sundar (2010) Yoon Jeon Koh and S Shyam Sundar. 2010.  Effects of specialization in computers, web sites, and web agents on e-commerce trust.  _International journal of human-computer studies_ 68, 12 (2010), 899–912.
  * Køien (2011) Geir M Køien. 2011\.  Reflections on trust in devices: an informal survey of human trust in an internet-of-things context.  _Wireless personal communications_ 61, 3 (2011), 495–510.  https://doi.org/10.1007/s11277-011-0386-4
  * Komiak and Benbasat (2006) Sherrie YX Komiak and Izak Benbasat. 2006.  The effects of personalization and familiarity on trust and adoption of recommendation agents.  _MIS quarterly_ 30, 4 (2006), 941–960.  https://doi.org/10.2307/25148760
  * Kowalczuk (2018) Pascal Kowalczuk. 2018\.  Consumer acceptance of smart speakers: a mixed methods approach.  _Journal of Research in Interactive Marketing_ (2018).
  * Kulviwat et al. (2007) Songpol Kulviwat, Gordon C Bruner II, Anand Kumar, Suzanne A Nasco, and Terry Clark. 2007.  Toward a unified theory of consumer acceptance technology.  _Psychology & Marketing_ 24, 12 (2007), 1059–1084.
  * Kumah-Crystal et al. (2018) Yaa A Kumah-Crystal, Claude J Pirtle, Harrison M Whyte, Edward S Goode, Shilo H Anders, and Christoph U Lehmann. 2018.  Electronic health record interactions through voice: a review.  _Applied clinical informatics_ 9, 3 (2018), 541\.
  * Lai et al. (2021) Yi Lai, Atreyi Kankanhalli, and Desmond Ong. 2021\.  Human-AI collaboration in healthcare: A review and research agenda.  (2021).
  * Lampinen et al. (2016) Airi Lampinen, Victoria Bellotti, Coye Cheshire, and Mary Gray. 2016.  CSCW and theSharing Economy: The Future of Platforms as Sites of Work Collaboration and Trust. In _Proceedings of the 19th ACM Conference on Computer Supported Cooperative Work and Social Computing Companion_. 491–497.
  * Langevin et al. (2019) Raina Langevin, Mohammad Rafayet Ali, Taylan Sen, Christopher Snyder, Taylor Myers, E. Ray Dorsey, and Mohammed Ehsan Hoque. 2019.  The PARK Framework for Automated Analysis of Parkinson’s Disease Characteristics.  _Proc. ACM Interact. Mob. Wearable Ubiquitous Technol._ 3, 2, Article 54 (jun 2019), 22 pages.  https://doi.org/10.1145/3328925
  * Latané (1981) Bibb Latané. 1981\.  The psychology of social impact.  _American psychologist_ 36, 4 (1981), 343–356.  https://doi.org/10.1037/0003-066X.36.4.343
  * Lau et al. (2018) Josephine Lau, Benjamin Zimmerman, and Florian Schaub. 2018\.  Alexa, are you listening? Privacy perceptions, concerns and privacy-seeking behaviors with smart speakers.  _Proceedings of the ACM on human-computer interaction_ 2, CSCW (2018), 1–31.
  * Lee and Coughlin (2015) Chaiwoo Lee and Joseph F Coughlin. 2015.  PERSPECTIVE: Older adults’ adoption of technology: an integrated approach to identifying determinants and barriers.  _Journal of Product Innovation Management_ 32, 5 (2015), 747–759.
  * Lee and Yoon (2021) DonHee Lee and Seong No Yoon. 2021.  Application of artificial intelligence-based technologies in the healthcare industry: Opportunities and challenges.  _International Journal of Environmental Research and Public Health_ 18, 1 (2021), 271.  https://doi.org/10.3390/ijerph18010271
  * Lee and Rich (2021) Min Kyung Lee and Katherine Rich. 2021.  Who Is Included in Human Perceptions of AI?: Trust and Perceived Fairness around Healthcare AI and Cultural Mistrust. In _Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems_. 1–14.
  * Lee and Turban (2001) Matthew KO Lee and Efraim Turban. 2001.  A trust model for consumer internet shopping.  _International Journal of electronic commerce_ 6, 1 (2001), 75–91.
  * Lee et al. (2021) One-Ki Daniel Lee, Ramakrishna Ayyagari, Farzaneh Nasirian, and Mohsen Ahmadian. 2021.  Role of interaction quality and trust in use of AI-based voice-assistant systems.  _Journal of Systems and Information Technology_ 23, 2 (2021), 154–170.  https://doi.org/10.1108/JSIT-07-2020-0132
  * Lewandowsky et al. (2000) Stephan Lewandowsky, Michael Mundy, and Gerard Tan. 2000\.  The dynamics of trust: comparing humans to automation.  _Journal of Experimental Psychology: Applied_ 6, 2 (2000), 104\.
  * Liao et al. (2020) Song Liao, Christin Wilson, Long Cheng, Hongxin Hu, and Huixing Deng. 2020.  Measuring the effectiveness of privacy policies for voice assistant applications. In _Annual Computer Security Applications Conference_. 856–869.
  * Liao et al. (2019) Yuting Liao, Jessica Vitak, Priya Kumar, Michael Zimmer, and Katherine Kritikos. 2019.  Understanding the role of privacy and trust in intelligent personal assistant adoption. In _International Conference on Information_ , Vol. 11420. Springer, 102–113.  https://doi.org/10.1007/978-3-030-15742-5_9
  * Lin et al. (2020) Hongxia Lin, Oscar Hengxuan Chi, and Dogan Gursoy. 2020\.  Antecedents of customers’ acceptance of artificially intelligent robotic device use in hospitality services.  _Journal of Hospitality Marketing & Management_ 29, 5 (2020), 530–549.  https://doi.org/10.1080/19368623.2020.1685053
  * Liu et al. (2019) Xiaoxuan Liu, Livia Faes, Aditya U Kale, Siegfried K Wagner, Dun Jack Fu, Alice Bruynseels, Thushika Mahendiran, Gabriella Moraes, Mohith Shamdas, Christoph Kern, et al. 2019\.  A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis.  _The lancet digital health_ 1, 6 (2019), e271–e297.
  * Lou and Yuan (2019) Chen Lou and Shupei Yuan. 2019.  Influencer marketing: how message value and credibility affect consumer trust of branded content on social media.  _Journal of Interactive Advertising_ 19, 1 (2019), 58–73.
  * Lowe et al. (2019) Ben Lowe, Yogesh K Dwivedi, and Steve d’Alessandro. 2019\.  Consumers and technology in a changing world.  _European Journal of Marketing_ 53, 6 (2019), 1038–1050.
  * Lu et al. (2011) Yaobin Lu, Shuiqing Yang, Patrick YK Chau, and Yuzhi Cao. 2011\.  Dynamics between the trust transfer process and intention to use mobile payment services: A cross-environment perspective.  _Information & management_ 48, 8 (2011), 393–403.
  * Luhmann (2001) Niklas Luhmann. 2001\.  Familiarity, confidence, trust: Problems and alternatives.  _Reseaux_ 108, 4 (2001), 15–35.
  * Madhavan et al. (2006) Poornima Madhavan, Douglas A Wiegmann, and Frank C Lacson. 2006.  Automation failures on tasks easily performed by operators undermine trust in automated aids.  _Human factors_ 48, 2 (2006), 241–256.
  * Malkin et al. (2022) Nathan Malkin, David Wagner, and Serge Egelman. 2022\.  Can Humans Detect Malicious Always-Listening Assistants? A Framework for Crowdsourcing Test Drives.  _Proceedings of the ACM on Human-Computer Interaction_ 6, CSCW2 (2022), 1–28.
  * Martins et al. (2021) Pedro M. Martins, João L. Vilaça, and Nuno S. Dias. 2021\.  A study about current digital assistants for healthcare and medical treatment monitoring. In _2021 IEEE 9th International Conference on Serious Games and Applications for Health(SeGAH)_. 1–7.  https://doi.org/10.1109/SEGAH52098.2021.9551864