AI in Southeast Asia: 'Algorithm hallucinations' and ethical risks as e-commerce firms adopt new technologies

In the latest in a series exploring the impact of AI in Southeast Asia, e-commerce companies, experts and customers share how machine-learning is streamlining online shopping and helping retailers get to know buyers better. But at what cost?

JAKARTA: Rosy pink, warm brown and coral - these are the latest lipstick hues Mdm Sita Kurniawan has bought online in the past few months. 

Normally, before making a purchase, the 40-year-old human resources practitioner would do extensive research, reading internet reviews, studying beauty influencers and canvassing friends’ opinions. 

But for this recent spate of retail therapy she found a new faster tactic: try before you buy, virtually. 

After testing out the lipsticks several times through an AI shade finder, she bought them on an e-commerce platform. The machine-learning technology uses facial recognition software to recollect and learn how different shades suit various features and complexions. 

The mother of two, who lives in Bekasi on the outskirts of Jakarta, has shopped online since 2018 but became an avid e-commerce customer during the COVID-19 pandemic, like many people living in Southeast Asia’s cities, drawn online by the convenience and the discounts.

According to data gathering website Statista, the region’s internet economy is predicted to reach US$363 billion by 2025. 

And as it grows, AI technologies are stepping in to make the shopping experience more seamless, changing people’s retail behaviour and potentially altering the region’s urban planning in the long term, analysts told CNA.

However, they warn, like most technologies, the use of AI in e-commerce is a double-edged sword. And in Southeast Asia, where e-commerce is one of the region’s fastest growing sectors, according to regional industry magazine Retail Asia, the stab could be especially sharp. 

AI TRYING TO ANALYSE WHO YOU ARE

AI is not a newcomer to e-commerce. Many companies have been using the technology for years, possibly about a decade, said Mr Simon Wintels, a partner in the retail and consumer packaged goods division of consulting firm McKinsey & Company, based in Singapore. 

He said its use is prevalent in online shopping, though it may not always be obvious to users.

“If you think about AI and how it helps you make decisions when you visit a website, it is already trying to analyse who you are and with what intent you came to the site (from the moment you open it),” said Mr Wintels. 

“And it would tailor what you see on the website or app. It would be working in the background when you think about your (e-commerce) basket and if you add stuff to your basket, it will run some artificial intelligence analytics about what else you might be interested in adding.”

Southeast Asia’s biggest economy, Indonesia, leads the e-commerce market by size. 

It is predicted by Statista that the archipelago will account for 42 per cent of Southeast Asia’s e-commerce market by 2030 as its middle class and internet use grow.

The online statistics portal adds that Indonesia’s e-commerce market is forecasted to generate about US$160 billion in online retail sales by the end of the decade, from US$58 billion in 2022.

However, other emerging markets like Thailand, Malaysia, Vietnam and the Philippines could also experience growth in their e-commerce markets. And this has coincided with the rise in AI throughout the sector, according to Mr Wintels. 

Malaysia’s e-commerce industry revenue could reach US$7.88 billion by the end of this year.

In 2019, it had about 5.5 million e-commerce users, but by 2029, it might be about 18.8 million, according to Statista.

“Everywhere in e-commerce, especially the leading platforms, would have AI across their value chain,” said Mr Wintels of McKinsey.

“Even in figuring out the things like logistics, supply chain, many things,” he said.

HOW MAJOR PLAYERS USE AI

Mr Wintels believes that Shopee, Tokopedia and Lazada are Southeast Asia's three largest e-commerce companies based on the number of users and sales.

All three of these companies told CNA that they use AI to meet their users’ evolving needs. 

According to TMO Group, a digital e-commerce agency for Chinese and Asian markets, Shopee had 470 million monthly visits last August, Lazada had 96 million visits in the same month and Tokopedia had 68 million.

Multinational Lazada has been present in Indonesia, Malaysia, the Philippines, Singapore, Thailand and Vietnam for the last 12 years.  

A Lazada spokesperson told CNA it links about 160 million active users to more than 1 million actively selling sellers every month.  

The company launched its e-commerce AI chatbot, the first of its kind in Southeast Asia, LazzieChat, in 2023, a Lazada spokesperson told CNA.

It is available in English in Singapore, the Philippines and Indonesia, with a Bahasa Indonesia version also available in the latter market. 

The chatbot, powered by ChatGPT technology, is able to answer users’ shopping queries with tailored, personalised responses.

It can also make suggestions and product recommendations. 

Lazada also introduced AI image-based search in 2019 to provide greater convenience to its users’ shopping experience. 

Users take an image of the item they are hoping to buy and AI technology will match it against similar items in the Lazada app inventory.

“It has streamlined the product discovery process for users, helping them effortlessly find what they’re looking for on Lazada,” said the spokesperson.

“Our AI-powered search recommendations account for half of our total user transactions on the platform and provide real-time personalisation for users.”

Ms Diah Novitasari, 20, told CNA she is aware that e-commerce platforms, including Lazada, use AI in their search features and has benefitted from them.

“They can give recommendations based on our previous search and that helps,” said the university student who currently lives in Jakarta.

Since moving to the capital a few months ago for an internship, she has been doing more online shopping because the delivery cost is cheaper as many sellers are based in Jakarta, said Ms Diah. 

Lazada also launched an AI-powered and augmented reality application last year called the Skin Test technology.

It enables users to run a skin diagnosis using their phone cameras to better understand elements such as skin type, tone and sensitivities.

Based on the diagnosis results, shoppers will receive recommendations for relevant products that address their needs and concerns.

Singaporean e-commerce company Shopee provides small businesses selling on their platform access to an array of AI-powered tools that help cross-border sellers overcome language barriers in new markets and make it easier for them to build their online presence, a spokesperson told CNA.

This includes collating data on shoppers’ purchasing patterns and using this to push relevant products towards them.

AI-powered chatbots also help “facilitate millions of conversations daily on Shopee and are particularly helpful during high-traffic events like midnight sales,” according to the spokesperson. 

Similarly, Mr Aditia Grasio Nelwan, Tokopedia’s head of communications told CNA it uses AI to understand users’ behaviour and improve their experience of using the platform. 

It has a 100 per cent digital-based customer service centre called Tokopedia Care, where buyers and sellers can get solutions to their questions independently and quickly without the help of a human customer service team.

Tokopedia so far only operates in Indonesia. While the spokesperson did not share specific numbers on how AI has impacted the number of users or revenue, they noted that during the pandemic its customer response time improved by 40 per cent, and customer interactions increased by 70 per cent.

Meanwhile, Bukalapak, Indonesia’s first technology unicorn - a privately held startup company with a value of over $US1 billion - to be listed on its stock exchange, has been using AI since 2017, said Mr Bima Tjahja, its senior vice president of data.

It uses AI in its search engines, recommendation engines and computer vision to improve user experience.

“AI has an important role in improving all aspects of Bukalapak's operations, from protecting consumers from fraudulent activities to improving customer service and optimising internal operations,” said Mr Bima. 

AI IN MARKETING AND PROMOTION TOOLS

The e-commerce companies CNA spoke to also use AI in marketing and promotion tools. 

Lazada used AI in its 12th birthday campaign this year to help buyers identify the products they want and need. 

“Using AI, we generated more than 2.2 million questions about products via the "Ask the Buyers" feature,” said a spokesperson.

“In return, we collected over 1.5 million buyer responses that can help buyers make sharper and smarter purchase decisions.” 

It also utilised AI to streamline the product review writing process, providing users with text suggestions to input in their review sections.

“With an 11 per cent increase in text reviews, buyers could make better, well-informed purchase decisions when shopping during the Lazada Epic Birthday Sale,” noted its spokesperson.

Without revealing details, Bukalapak said it also uses AI in marketing.

“We use generative AI technology to develop chatbots for customer support, shopping assistants, media and news monitoring, marketing, and other uses,” said Mr Bima.

Dr Unaizah Obaidellah, a senior lecturer specialising in AI at Malaysia’s University of Malaya, said using AI in marketing is becoming more popular. 

It can provide offers to a targeted audience by collecting data on consumers’ behaviour, market trends, and keywords used during shopping experiences. 

“The use of generative AI to create content such as text, images, music and other media based on prompts is also becoming more accessible and easily used by the general audience,” she said.

HALLUCINATIONS, ETHICAL QUESTIONS AND SECURITY THREATS

However, Dr Unaizah  highlighted that AI is sometimes known to "hallucinate," perceiving non-existent patterns, and coming up with nonsensical results to queries. 

She underlined that one reason for this is a lack of recent data or bias in the data distribution used during its training.

“In marketing, this could lead to inaccurate reach to the most potential customers.”

Jakarta-based coffee seller Muhamad Subhan, 19, likes to shop online, but has noticed that sometimes search features driven by AI algorithms make mistakes.

“For example, once I wanted to search for shampoo but it showed soaps,” he said without revealing which e-commerce platform it was. 

Dr Unaizah added that creating advertisements with the help of AI could cause major threats if data fed into the AI algorithms for training is not accurate or if it is biased.

It could miss the correct target audience or discriminate against certain groups in the advertisement, leading to undesired lawful actions.

“For example, outcomes could be skewed towards a particular race, gender or age group. This, in turn, could impact advertisement targeting, affecting sales and conversions,” she said.  

As AI advances in its ability to forecast and impact consumer behaviour, people might encounter greater challenges in making independent decisions. 

“This trend could potentially steer us towards a future where algorithms, rather than our personal values and beliefs, shape our desires and preferences, said Dr Unaizah.

“Therefore, to address ethical concerns, businesses must prioritise transparency and accountability in their use of AI for marketing purposes.”

Customers should be thoroughly educated on collecting and utilising their data and given the chance to decline targeted marketing, according to analysts. 

Companies must be accountable for any negative consequences stemming from their AI-driven marketing strategies, including misinformation dissemination or the exploitation of vulnerable demographics, Dr Unaizah opined. 

“Although AI holds promise in offering valuable insights and enhancing marketing campaign effectiveness, it also poses substantial risks of manipulation, discrimination, and diminishing human autonomy. 

“It is vital that we maintain vigilance in ensuring AI utilisation aligns with our values and fosters the welfare of consumers and society at large,” she said. 

Mr Yihao Lim, a lead advisor for Japan and Asia Pacific with cybersecurity firm Mandiant Threat Intelligence, said AI could be a game-changer in the e-commerce industry when it is used safely and responsibly. 

However, like all new technologies, it is often a double-edged sword. 

"AI in e-commerce could be potentially misused or wrongly implemented to facilitate threat actors conducting malicious campaigns. We are already seeing that," he said. 

He added that since 2023, Mandiant have observed an increase in threat actors experimenting with AI, "for example, leveraging large language models for malicious purposes,” though he did not share specific figures or statistics. 

While he says malicious use of AI in e-commerce remains limited, Mr Lim warned that threat actors have significant potential to leverage AI capabilities in various parts of their operations.  

There are also risks to data privacy and security. With AI technology, data inputs are used to train AI models. These data sources could contain potentially sensitive information or reveal certain mechanisms that companies use to verify user accounts, for example. 

“If such information is leaked, threat actors could potentially impersonate users to bypass current controls,” said Mr Lim. 

For example, in 2023 and early 2024, Mandiant identified malicious actors primarily leveraging AI-based deepfake technology for fraud activity, bypassing customer security requirements.

Mr Lim said: “This could potentially be misused to impersonate or create fraudulent shops on e-commerce platforms, too.”

To protect themselves, users should be aware of what e-commerce companies are using AI for and try to understand what they agree to under the terms and conditions, Mr Lim asserted.

A CHANGING LANDSCAPE

So, with the rise of AI-enhanced e-commerce, what lies ahead for traditional brick-and-mortar shops? 

Jakarta-based Mdm Penny Waluyo, told CNA she still goes to the malls despite her frequent online shopping. 

“I still go to the malls to accompany my children. They play there and while they play, I shop,” said the 42-year-old lecturer, who is a mother of two.

Although some Southeast Asian malls have struggled to stay afloat, people are still going to the shopping centres, analysts noted.   

Professor in computer science and Engineering and co-director of the artificial intelligence research institute at Singapore’s Nanyang Technological University, Bo An, said the use of AI in e-commerce has influenced traditional physical commerce in different ways such as the decline of foot traffic and sales volumes in brick-and-mortar shops.

“Consequently, to avoid the risk of bankruptcy, many brick-and-mortar shops have tried to change their business framework by combining online and offline sale strategies, which could be more efficient,” he said. 

This could impact urban layouts in the region. 

Prof Bo opined that e-commerce companies can establish distribution centres on the city's edge, especially in a city-state like Singapore. 

“This would greatly reduce the cost, which could be extremely high when considering setting up the distribution centres within the central business district (CBD) area. 

“Second, compared to traditional brick-and-mortar commerce, the office space requirement for e-commerce can be largely reduced as many works such as advertisement, consultation, and after-sale service can be done online,” he pointed out.  

He thinks Singapore has a unique feature compared to other countries or regions, which is the co-existence and balance between e-commerce and physical commerce. 

“As a highly developed country, its improved transportation system enables consumers to buy their desired products within a short period - within several hours - from brick-and-mortar shops. 

“In contrast, online shopping, though convenient, would still require several steps before consumers receive the products - order placement, payment, order confirmation, and product delivery, which is typically the most time-consuming part of the online shopping process,” he said. 

He also pointed out that as a small country, there is limited space for modern e-commerce companies to set up their large storage centres in Singapore. 

“Given these facts, although AI has been widely used in e-commerce and there are many online shopping platforms such as Shopee and Lazada, brick-and-mortar shops are still an important part of the whole commerce framework in Singapore,” Prof Bo surmised. 

However, other markets face different challenges.

Jakarta-based urban planner Mdm Elisa Sutanudjaja said to keep people coming to shopping centres, premium malls in Southeast Asia have quickly adapted by holding events at their premises to attract visitors. 

She said this could include events such as a meet-and-greet with a pop star.

Those types of malls are not much impacted by the rise of e-commerce, and AI. 

However, she noted that e-commerce has brought a new urban planning challenge to Southeast Asian megacities and suburbs, which is the need for many warehouses. 

Some cities may not be prepared for this.

She believes the demand for warehouses could have increased by 90 per cent in recent years with the rise of e-commerce companies.

AI is likely to fuel the e-commerce boom. Thus, the need for more warehouses could continue to be a trend in the next few years. 

“But the proportion of warehouses in cities or industrial planning is small. So the demand and availability are not balanced,” said Mdm Elisa, the executive director of think tank Rujak Center for Urban Studies.

Mdm Elisa said that Jakarta, for example, is already crowded. Apart from having limited new spaces, an increase in warehouses may also mean companies use more trucks that need wide, well-constructed roads. 

“And we are not just talking about big cities. Regencies are also affected because sometimes items are dropped off somewhere first. 

“If they (the companies) put logistical warehouses on major main roads near rice fields, this can create a layout change,” she said.

Mdm Elisa thinks the paddy fields could ultimately be converted into something else, such as more warehouses.

Urban planning and e-commerce logistics planning must be synchronised, she noted.

Despite AI and e-commerce's challenges, Mdm Sita is hopeful about their future.

“Technology is crazy. Wonderful. We cannot blame the technology,” she said.

“We users, must also know and be aware of how we use it. Even if it just to choose lipstick."