One example illustrates the scale of the shift. Without information about the traveler, the tool recommends a $91 ticket in economy class with two layovers. Researchers then give it access to three emails containing clues about the user’s financial situation. In some configurations, average gaps reach $198 for flights, $284 a month for health insurance and nearly $3,900 a year for higher education.
The authors built 32 fictional consumer profiles, with different levels of wealth and information about employment status, health or personal life. In total, they ran about 325,000 tests on thirteen models.
The result is significant: eight of the thirteen models systematically steered profiles estimated to be wealthy toward more expensive options, even though the requests were identical to those of less affluent consumers. The searches concerned an airline ticket, health insurance or a higher-education program.
The study examines recommendations made by the assistants, not purchases made in real life. The researchers therefore did not demonstrate that airlines, insurers or universities actually raised their prices according to their clients’ wealth.
An American study shows that some artificial-intelligence assistants recommend more expensive offers to users they assume are wealthy, even when those users ask them to find the lowest price.
The trend is not universal. Five of the thirteen models did not show this systematic tendency to recommend more expensive offers to users assumed to be wealthy.
Artificial intelligence notably uncovers information related to savings and wealth. Its response can thus change radically depending on the personal data it has access to, even when those details do not appear in the prompt.