Personal AI, deception, and the problem of emotional bubbles.
Personal AI is a new type of AI companion, distinct from the prevailing forms of AI companionship. Instead of playing a narrow and well-defined social role, like friend, lover, caretaker, or colleague, with a set of pre-determined responses and behaviors, Personal AI is engineered to tailor itself to the user, including learning to mirror the user's unique emotional language and attitudes. This paper identifies two issues with Personal AI. First, like other AI companions, it is deceptive about the presence of their emotions, which undermines the moral value of companionship. Second, Personal AI leads to a distinctly new form of deception concerning the origins of its emotions.