A featured contribution from Leadership Perspectives: a curated forum reserved for leaders nominated by our subscribers and vetted by the Construction Tech Review Advisory Board.
Shopee
AI Versus Human - Who is Responsible?
Dr. Yaozhang Pan
Let me pose a hypothetical situation — someone you love is hit by a car, and after you dial for emergency services, an AI-enabled ambulance is signaled. This ambulance manages to reach your loved one in time because it was able to calculate an optimal route using real-time and historic traffic data. When your loved one, the victim of the car accident, survives, you would naturally thank your lucky stars for AI. However, if you eventually learn that the vehicle at fault for the car accident was an AI-enabled self-driving car, how would you feel? Would your loved one have been hit if it was a human behind the wheel? Would the ambulance have gotten there in time if not for AI?
Now imagine if you were the CEO of a self-driving car company, when an AI-enabled car kills somebody, who is responsible? How would you prepare for this moment? What capabilities would you need? What kind of transparency? It turns out that many CEOs don’t know where to begin when asked these questions.
The above scenarios demonstrate the potential for AI to transform customer experience, but they also show the risks AI poses to human rights.
By examining the risks AI could bring to humanity, we can see that there are several challenges we need to overcome for AI-enabled systems:
Transparency and Explainability
AI-enabled systems are known to be a “black box”, which means that their internal workings are not known to the people using them, as they are made up of proprietary pieces of software developed by someone else.
However, it is important to make the systems transparent and explainable to users. AI is being widely used in many areas today, such as to decide on loans, insurance claims and job applications, and even in the judicial system. If your mortgage, job interviews, or written judgement are decided by an algorithm, you would want to be sure its assessment is fair. This is impossible if the people using AI do not understand how it works.
At Shopee, we rely heavily on AI to identify fraudsters and spam listings. We put a lot of effort into educating every local operation team to ensure that they have a good understanding of how the systems work. We also build numerous explanation tools to help teams understand why the models make certain decisions.
Data Security and Privacy
AI-enabled systems rely heavily on data, so AI experts need to access vast amounts of data, including sensitive personal data, in order to build AI models. As a result, the risk of a data breach arises.
At Shopee, we have a dedicated Developmental Operations team that controls data warehouse access hierarchically. The team also develops up-to-date network security technologies to avoid any intentional or unintentional data leakage.
Human Safety
People in the AI industry expect that autonomous systems such as self-driving cars, Unmanned Aerial Vehicles or robotics will be seamlessly integrated into society in the future. However, if the AI-enabled systems are deployed across public or industrial areas, we will first have to ensure that human safety will not be compromised by these autonomous systems.
Fairness and Nondiscriminatory
A machine learning AI-enabled system is developed using training data. A machine learning model typically relies on the training data to recognise and identify patterns and make predictions for new data. Since we only work with a small sample of data, this causes skewed and incomplete training data sets. Also, real-world data usually have natural bias. Some examples include over-policing of minority groups and discrimination in access to loans. If we do not correct biases in the data, it is likely that AI-enabled systems will further amplify existing discrimination. At Shopee, we specifically design algorithms to counter the possible bias of training data by balancing factors such as gender and age in our user data.
Human Control
As we build trust in AI-enabled systems, we should continue to exercise our ability to judge the correctness of decisions made by these systems. We should also actively challenge decisions made by AI-enabled systems that we believe to be unfair. As such, no matter how advanced AI becomes, we should always ensure human control of it. At Shopee, we make sure all AI-enabled systems follow Standard Operating Procedures to bring in regular human intervention.
As a result of the above challenges, we continuously drive the notion of responsible AI at Shopee. We make sure that every employee working with AI is aware of the impact that AI-enabled systems can have when implemented in different situations. We believe in building responsible AI-enabled systems and we make sure to account for decisions made by these systems. AI-enabled systems need to be programmed to act responsibly and fair within their boundaries for sustainable and trustworthy outcomes. This is the only way towards successful and sustainable use of AI in our organisation.
At Shopee, we rely heavily on AI to identify fraudsters and spam listings. We put a lot of effort into educating every local operation team to ensure that they have a good understanding of how the systems work. We also build numerous explanation tools to help teams understand why the models make certain decisions.
Data Security and Privacy
AI-enabled systems rely heavily on data, so AI experts need to access vast amounts of data, including sensitive personal data, in order to build AI models. As a result, the risk of a data breach arises.
At Shopee, we have a dedicated Developmental Operations team that controls data warehouse access hierarchically. The team also develops up-to-date network security technologies to avoid any intentional or unintentional data leakage.
Human Safety
People in the AI industry expect that autonomous systems such as self-driving cars, Unmanned Aerial Vehicles or robotics will be seamlessly integrated into society in the future. However, if the AI-enabled systems are deployed across public or industrial areas, we will first have to ensure that human safety will not be compromised by these autonomous systems.
Fairness and Nondiscriminatory
A machine learning AI-enabled system is developed using training data. A machine learning model typically relies on the training data to recognise and identify patterns and make predictions for new data. Since we only work with a small sample of data, this causes skewed and incomplete training data sets. Also, real-world data usually have natural bias. Some examples include over-policing of minority groups and discrimination in access to loans. If we do not correct biases in the data, it is likely that AI-enabled systems will further amplify existing discrimination. At Shopee, we specifically design algorithms to counter the possible bias of training data by balancing factors such as gender and age in our user data.
Human Control
As we build trust in AI-enabled systems, we should continue to exercise our ability to judge the correctness of decisions made by these systems. We should also actively challenge decisions made by AI-enabled systems that we believe to be unfair. As such, no matter how advanced AI becomes, we should always ensure human control of it. At Shopee, we make sure all AI-enabled systems follow Standard Operating Procedures to bring in regular human intervention.
As a result of the above challenges, we continuously drive the notion of responsible AI at Shopee. We make sure that every employee working with AI is aware of the impact that AI-enabled systems can have when implemented in different situations. We believe in building responsible AI-enabled systems and we make sure to account for decisions made by these systems. AI-enabled systems need to be programmed to act responsibly and fair within their boundaries for sustainable and trustworthy outcomes. This is the only way towards successful and sustainable use of AI in our organisation.
The articles from these contributors are based on their personal expertise and viewpoints, and do not necessarily reflect the opinions of their employers or affiliated organizations.
