Too Perfect to Be Like a Human? Ethical Challenges of Emotional AI in Health care

The rapid advancement of emotional AI has provoked debates about its potential to replace human caregiving, particularly for the elderly and those with chronic illnesses within the context of aging populations, notably in China. The purpose of this article is to refute the possibility of emotional caring AI replacing human care from an intercultural philosophical perspective. Based on Searle’s Chinese Room argument and phenomenology, it is understood that emotional AI can only simulate emotional interactions at best; it cannot fully replicate human relationships, particularly due to its absence of authentic emotions and virtuous character, as well as its inability to foster affective connective resonance with others from a Confucian perspective. While robots taking care of the elderly physically and emotionally are worth looking forward to, such development may reduce the sense of responsibility of family members to care for the elderly, which is considered an expression of filial piety in Confucian familism. Krueger and Roberts’ fictionalist human-AI interactions and Lamola’s robotics sociality shows that interacting with really humanlike robots affects our consciousness and our relationship to the world and could easily lead to a kind of anthropomorphism that causes humans to treat AI as real persons imbued with meanings. However, anthropomorphism may make humans easier to manipulate and cause trouble in real human relationships. Buber’s phenomenology suggests that such relationships are merely “I-It disguised as I-Thou relationships.” Thus, AI-human relationships in health care remain incomparable to human–human connections, and we should use AI technology in health care with caution. In addition, with the reflection of AI-human relationships, ethical guidelines and public education on AI ethics are called for.