Page 64 - DP-2-3

Design+ Design chatbot using activity theory

4.1. Potential biases in user feedback adaptability to unique business processes. 42

There are several potential biases in user feedback. First, • Performance and reliability issues. User reviews
sampling basis may occur when the group of users indicate concerns regarding Engati’s performance,
42
providing feedback is not representative of the broader including system lags and occasional downtime.
target user population. Second, in chatbot development, • Integration constraints. Although Engati offers
19
users might hesitate to criticize a chatbot directly, integrations with tools like Salesforce and Google
especially if they perceive that negative feedback could Sheets; however, users have noted difficulties in
harm the development team’s efforts. This is known as establishing and maintaining these connections, citing
social desirability bias. This can lead to over-optimistic limited integration options and complexity. 43
evaluations and under-reporting of usability issues. Besides the above, the platform primarily supports
39
Third, there is confirmation bias when users unconsciously rule-based conversation flows, which restrict the chatbot’s
seek information that supports their existing beliefs or ability to handle highly dynamic, unpredictable user
expectations about the chatbot, rather than providing inputs. Advanced AI integration, including deep contextual
objective feedback. Fourth, the emotional state of users understanding or adaptive learning capabilities, is limited
40
during interaction or testing can influence their feedback. unless supplemented by external services. Additionally,
41
A frustrated or tired user might evaluate the chatbot more customization options for complex backend functionalities
negatively than one who is relaxed and engaged. Fifth, are constrained, posing challenges for expanding the
some users might experience expectation bias when chatbot into more sophisticated intelligent systems.
they expect too much from the chatbot, interpreting any
minor failure as a major flaw. Others might have low 4.3. Scalability of the chatbot to other contexts
expectations and accept mediocre performance without Activity theory not only can guide the development of road
critique. Sixth, users may experience recency bias when sign chatbots but also can facilitate their scalability across
12
evaluating a chatbot, disproportionately focusing on either multiple domains. Activity theory’s systemic view of user
early struggles or final improvements, thus providing goals, tools, community, and context offers a framework to
unbalanced feedback. 8 generalize chatbot usability and UX design. Activity theory
The user evaluation was conducted exclusively among provides a robust framework for modeling socio-technical
undergraduate students aged 18 – 25 years from APU. systems by understanding how users engage with tools to
As digital natives, participants were likely feeling more achieve their goals within specific cultural and contextual
comfortable interacting with chatbot interfaces compared settings. 18,22,31,37 It posits that all human activities can be
to the general population. This introduces potential bias analyzed through common structural elements: subject,
toward positive usability and UX ratings. Furthermore, object, tools, rules, community, and division of labor.
44
since participation was voluntary, there may have been a These elements are transferable across domains. Tool
28
self-selection bias, where students interested in technology mediation in activity theory allows the chatbot to evolve
were more inclined to participate, skewing results toward in form and function as new domains demand different
more favorable impressions. representations of knowledge and interaction styles. This
45
adaptability is critical for cross-domain scalability.
4.2. Limitations of using Engati as a chatbot
platform The road sign chatbot, although designed for Malaysian
Chatbots have become integral to modern customer road signs, demonstrates potential for scalability to
service strategies, providing 24/7 support and streamlining other educational and informational domains. However,
interactions. Engati has emerged as a notable platform successful adaptation would require careful customization,
in this domain, offering features such as omnichannel including language localization, content reconfiguration
deployment and a no-code interface. Engati, while offering for different regulatory environments (e.g., different
a user-friendly interface and multi-channel deployment countries’ road rules), and interface adjustments to meet
capabilities, presents several limitations that may hinder its diverse user expectations. Moreover, the current platform’s
effectiveness in complex or large-scale applications. These dependency on predefined conversation structures may
include: necessitate additional development effort to support
• Limited customization and flexibility. Engati’s broader scalability across multiple contexts or user groups.
no-code approach, while accessible, restricts the depth 5. Conclusion
of customization available to developers. Users have
reported challenges in tailoring chatbot behaviors Both usability and UX must be considered in the design
beyond predefined templates, limiting the platform’s of effective chatbots, as they influence how easily and

Volume 2 Issue 3 (2025) 14 doi: 10.36922/DP025060009