To evaluate the moral awareness of college students regarding artificial intelligence (AI) systems, we have examined 467 surveys collected from 152 Japanese and 315 non-Japanese students in an international university in Japan. The students were asked to choose a most significant moral problem of AI applications in the future from a list of ten ethical issues and to write an essay about it. The results show that most of the students (n = 269, 58%) considered unemployment to be the major (...) ethical issue related to AI. The second largest group of students (n = 54, 12%) was concerned with ethical issues related to emotional AI, including the impact of AI on human behavior and emotion and robots’ rights and emotions. A relatively small number of students referred to the risk of social control by AI (6%), AI discrimination (6%), increasing inequality (5%), loss of privacy (4%), AI mistakes (3%), malicious AI (3%), and AI security breaches (3%). Calculation of the z score for two population proportions shows that Japanese students were much less concerned about AI control of society (− 3.1276, p < 0.01) than non-Japanese students, but more concerned about discrimination (2.2757, p < 0.05). Female students were less concerned about unemployment (− 2.6108, p < 0.01) than males, but more concerned about discrimination (2.4333, p < 0.05). The study concludes that the moral awareness of college students regarding AI technologies is quite limited and recommends including the ethics of AI in the curriculum. (shrink)