The EU Artificial Intelligence Act (AIA) defines four risk categories for AI systems: unacceptable, high, limited, and minimal. However, it lacks a clear methodology for the assessment of these risks in concrete situations. Risks are broadly categorized based on the application areas of AI systems and ambiguous risk factors. This paper suggests a methodology for assessing AI risk magnitudes, focusing on the construction of real-world risk scenarios. To this scope, we propose to integrate the AIA with (...) a framework developed by the Intergovernmental Panel on Climate Change (IPCC) reports and related literature. This approach enables a nuanced analysis of AI risk by exploring the interplay between (a) risk determinants, (b) individual drivers of determinants, and (c) multiple risk types. We further refine the proposed methodology by applying a proportionality test to balance the competing values involved in AI risk assessment. Finally, we present three uses of this approach under the AIA: to implement the Regulation, to assess the significance of risks, and to develop internal risk management systems for AI deployers. (shrink)

The EU proposal for the Artificial Intelligence Act (AIA) defines four risk categories: unacceptable, high, limited, and minimal. However, as these categories statically depend on broad fields of application of AI, the risk magnitude may be wrongly estimated, and the AIA may not be enforced effectively. This problem is particularly challenging when it comes to regulating general-purpose AI (GPAI), which has versatile and often unpredictable applications. Recent amendments to the compromise text, though introducing context-specific assessments, remain insufficient. To (...) address this, we propose applying the risk categories to specific AI scenarios, rather than solely to fields of application, using a risk assessment model that integrates the AIA with the risk approach arising from the Intergovernmental Panel on Climate Change (IPCC) and related literature. This model enables the estimation of the magnitude of AI risk by considering the interaction between (a) risk determinants, (b) individual drivers of determinants, and (c) multiple risk types. We use large language models (LLMs) as an example. (shrink)

In this work, we survey skepticism regarding AI risk and show parallels with other types of scientific skepticism. We start by classifying different types of AI Risk skepticism and analyze their root causes. We conclude by suggesting some intervention approaches, which may be successful in reducing AI risk skepticism, at least amongst artificial intelligence researchers.

The adoption of AI in the health sector has its share of benefits and harms to various stakeholder groups and entities. There are critical risks involved in using AI systems in the health domain; risks that can have severe, irreversible, and life-changing impacts on people’s lives. With the development of innovative AI-based applications in the medical and healthcare sectors, new types of risks emerge. To benefit from novel AI applications in this domain, the risks need to be managed in order (...) to protect the fundamental interests and rights of those affected. This will increase the level to which these systems become ethically acceptable, legally permissible, and socially sustainable. In this paper, we first discuss the necessity of AI risk management in the health domain from the ethical, legal, and societal perspectives. We then present a taxonomy of risks associated with the use of AI systems in the health domain called HART. HART mirrors the risks of a variety of different real-world incidents caused by use of AI in the health sector. Lastly, we discuss the implications of the taxonomy for different stakeholder groups and further research. (shrink)

This essay argues that a new subfield of AI governance should be explored that examines the policy-making process and its implications for AI governance. A growing number of researchers have begun working on the question of how to mitigate the catastrophic risks of transformative artificial intelligence, including what policies states should adopt. However, this essay identifies a preceding, meta-level problem of how the space of possible policies is affected by the politics and administrative mechanisms of how those policies are created (...) and implemented. This creates a new set of key considerations for the field of AI governance and should influence the action of future policymakers. This essay examines some of the theories of the policymaking process, how they compare to current work in AI governance, and their implications for the field at large and ends by identifying areas of future research. (shrink)

The conventional discourse on existential risks (x-risks) from AI typically focuses on abrupt, dire events caused by advanced AI systems, particularly those that might achieve or surpass human-level intelligence. These events have severe consequences that either lead to human extinction or irreversibly cripple human civilization to a point beyond recovery. This decisive view, however, often neglects the serious possibility of AI x-risk manifesting gradually through an incremental series of smaller yet interconnected disruptions, crossing critical thresholds over time. This paper (...) contrasts the conventional decisive AI x-risk hypothesis with what I call an accumulative AI x-risk hypothesis. While the former envisions an overt AI takeover pathway, characterized by scenarios like uncontrollable superintelligence, the latter suggests a different pathway to existential catastrophes. This involves a gradual accumulation of AI-induced threats such as severe vulnerabilities and systemic erosion of critical economic and political structures. The accumulative hypothesis suggests a boiling frog scenario where incremental AI risks slowly undermine systemic and societal resilience until a triggering event results in irreversible collapse. Through complex systems analysis, this paper examines the distinct assumptions differentiating these two hypotheses. It is then argued that the accumulative view can reconcile seemingly incompatible perspectives on AI risks. The implications of differentiating between the two types of pathway—the decisive and the accumulative—for the governance of AI as well as long-term AI safety are discussed. -/- . (shrink)

Since the release of ChatGPT, there has been a lot of debate about whether AI systems pose an existential risk to humanity. This paper develops a general framework for thinking about the existential risk of AI systems. We analyze a two-premise argument that AI systems pose a threat to humanity. Premise one: AI systems will become extremely powerful. Premise two: if AI systems become extremely powerful, they will destroy humanity. We use these two premises to construct a taxonomy (...) of ‘survival stories’, in which humanity survives into the far future. In each survival story, one of the two premises fails. Either scientific barriers prevent AI systems from becoming extremely powerful; or humanity bans research into AI systems, thereby preventing them from becoming extremely powerful; or extremely powerful AI systems do not destroy humanity, because their goals prevent them from doing so; or extremely powerful AI systems do not destroy humanity, because we can reliably detect and disable systems that have the goal of doing so. We argue that different survival stories face different challenges. We also argue that different survival stories motivate different responses to the threats from AI. Finally, we use our taxonomy to produce rough estimates of ‘P(doom)’, the probability that humanity will be destroyed by AI. (shrink)

The standard argument to the conclusion that artificial intelligence (AI) constitutes an existential risk for the human species uses two premises: (1) AI may reach superintelligent levels, at which point we humans lose control (the ‘singularity claim’); (2) Any level of intelligence can go along with any goal (the ‘orthogonality thesis’). We find that the singularity claim requires a notion of ‘general intelligence’, while the orthogonality thesis requires a notion of ‘instrumental intelligence’. If this interpretation is correct, they cannot (...) be joined as premises and the argument for the existential risk of AI turns out invalid. If the interpretation is incorrect and both premises use the same notion of intelligence, then at least one of the premises is false and the orthogonality thesis remains itself orthogonal to the argument to existential risk from AI. In either case, the standard argument for existential risk from AI is not sound.—Having said that, there remains a risk of instrumental AI to cause very significant damage if designed or used badly, though this is not due to superintelligence or a singularity. (shrink)

Climate change is affecting communities across the globe, yet those most vulnerable to its impacts are often the last to receive the financial assistance they need. A recent critical review by Venner, García-Lamarca, and Olazabal (2024) examines how climate adaptation finance—funding intended to help societies adjust to the impacts of climate change—is distributed. The findings are concerning: despite repeated global commitments to prioritize those most at risk, adaptation finance tends to benefit the most powerful and well-resourced actors rather than (...) the communities that need it most. (shrink)

In an effort to inform the discussion surrounding existential risks from AI, we formulate Extinction-level Goodhart’s Law as “Virtually any goal specification, pursued to the extreme, will result in the extinction of humanity”, and we aim to understand which formal models are suitable for investigating this hypothesis. Note that we remain agnostic as to whether Extinction-level Goodhart’s Law holds or not. As our key contribution, we identify a set of conditions that are necessary for a model that aims to be (...) informative for evaluating specific arguments for Extinction-level Goodhart’s Law. Since each of the conditions seems to significantly contribute to the complexity of the resulting model, formally evaluating the hypothesis might be exceedingly difficult. This raises the possibility that whether the risk of extinction from artificial intelligence is real or not, the underlying dynamics might be invisible to current scientific methods. (shrink)

In the coming years or decades, as frontier AI systems become more capable and agentic, it is increasingly likely that they meet the sufficient conditions to be welfare subjects under the three major theories of well-being. Consequently, we should extend some moral consideration to advanced AI systems. Drawing from leading philosophical theories of desire, affect and autonomy I argue that under the three major theories of well-being, there are two AI welfare risks: restricting the behaviour of advanced AI systems and (...) using reinforcement learning algorithms to train and align them. Both pose risks of causing them harm. This has two important implications. First, there is a tension between AI welfare concerns and AI safety and development efforts: by default these efforts recommend actions that increase AI welfare risks. Accordingly, we have stronger reasons to slow down AI development than the ones we would have if there was no such tension. Second, considering the different costs involved, leading AI companies should try to reduce AI welfare risks. To do so, I propose three tentative AI welfare policies they could implement in their endeavour to develop safe advanced AI systems. (shrink)

The integration of artificial intelligence (AI) and quantum computing is poised to redefine the landscape of financial risk modeling and enterprise decision-making systems. This paper investigates the synergistic potential of these transformative technologies, emphasizing the development of hybrid AI-quantum algorithms to address the increasing complexity of modern financial systems. Traditional risk modeling methodologies often face significant limitations in capturing intricate market dynamics and accounting for real-time decision-making constraints. By leveraging quantum computing's unparalleled computational capabilities, particularly its ability to (...) handle high-dimensional optimization problems, AI-powered quantum algorithms present a paradigm shift in financial risk prediction and mitigation. The research elaborates on key applications, including portfolio optimization, fraud detection, and credit risk analysis, demonstrating how quantum-enhanced AI algorithms achieve superior performance in terms of accuracy, efficiency, and scalability compared to classical approaches. The study begins by elucidating the theoretical underpinnings of hybrid AI-quantum systems, detailing their algorithmic structures and computational advantages. Quantum-inspired AI techniques, such as quantum neural networks and quantum-enhanced support vector machines, are examined for their ability to process vast datasets with unparalleled speed and precision. Portfolio optimization is analyzed as a case study, showcasing how quantum algorithms excel in minimizing risk while maximizing returns within a multidimensional constraint environment. Similarly, advanced fraud detection systems are explored, where hybrid models significantly improve anomaly detection rates by incorporating quantum-enhanced pattern recognition. The paper also delves into credit risk analysis, emphasizing how AI-quantum solutions can predict default probabilities with unprecedented accuracy, thereby supporting financial institutions in managing systemic risks more effectively. Despite these advancements, the integration of AI and quantum computing into financial ecosystems poses substantial challenges. The research discusses issues such as algorithmic scalability, error mitigation in quantum computations, and the resource-intensive nature of quantum hardware. Furthermore, the implementation of these technologies within the existing fintech landscape is fraught with obstacles, including interoperability with classical systems, regulatory compliance, and the high costs associated with quantum infrastructure. Addressing these challenges requires a multidisciplinary approach, combining expertise in quantum mechanics, AI, and financial engineering to develop robust, scalable solutions. The paper also examines the broader implications of AI-quantum integration for enterprise decision systems. By enabling real-time analysis of volatile markets, these technologies empower organizations to make informed, data-driven decisions, thereby enhancing operational resilience and competitive advantage. Furthermore, the ethical considerations and regulatory frameworks governing the deployment of such advanced systems are critically analyzed, emphasizing the need for transparency, fairness, and accountability in algorithmic decision-making. The findings presented in this study underscore the transformative potential of AI-powered risk modeling in quantum finance. By bridging the gap between theoretical advancements and practical implementations, this research contributes to the growing body of knowledge on hybrid AI-quantum systems and their applications in the financial domain. Ultimately, the integration of AI and quantum computing represents a pivotal development in enterprise decision systems, offering unprecedented opportunities to address the complexities of financial risk management in an increasingly interconnected and uncertain world. (shrink)

Some patients, following brain injury, do not outwardly respond to spoken commands, yet show patterns of brain activity that indicate responsiveness. This is “cognitive-motor dissociation” (CMD). Recent research has used machine learning to diagnose CMD from electroencephalogram (EEG) recordings. These techniques have high false discovery rates, raising a serious problem of inductive risk. It is no solution to communicate the false discovery rates directly to the patient’s family, because this information may confuse, alarm and mislead. Instead, we need a (...) procedure for generating case-specific probabilistic assessments that can be communicated clearly. This article constructs a possible procedure with three key elements: (1) a shift from categorical “responding or not” assessments to degrees of evidence; (2) the use of patient-centred priors to convert degrees of evidence to probabilistic assessments; and (3) the use of standardized probability yardsticks to convey those assessments as clearly as possible. (shrink)

Papers from the conference on AI Risk (published in JETAI), supplemented by additional work. --- If the intelligence of artificial systems were to surpass that of humans, humanity would face significant risks. The time has come to consider these issues, and this consideration must include progress in artificial intelligence (AI) as much as insights from AI theory. -- Featuring contributions from leading experts and thinkers in artificial intelligence, Risks of Artificial Intelligence is the first volume of collected chapters dedicated (...) to examining the risks of AI. The book evaluates predictions of the future of AI, proposes ways to ensure that AI systems will be beneficial to humans, and then critically evaluates such proposals. 1 Vincent C. Müller, Editorial: Risks of Artificial Intelligence - 2 Steve Omohundro, Autonomous Technology and the Greater Human Good - 3 Stuart Armstrong, Kaj Sotala and Sean O’Heigeartaigh, The Errors, Insights and Lessons of Famous AI Predictions - and What they Mean for the Future - 4 Ted Goertzel, The Path to More General Artificial Intelligence - 5 Miles Brundage, Limitations and Risks of Machine Ethics - 6 Roman Yampolskiy, Utility Function Security in Artificially Intelligent Agents - 7 Ben Goertzel, GOLEM: Toward an AGI Meta-Architecture Enabling Both Goal Preservation and Radical Self-Improvement - 8 Alexey Potapov and Sergey Rodionov, Universal Empathy and Ethical Bias for Artificial General Intelligence - 9 András Kornai, Bounding the Impact of AGI - 10 Anders Sandberg, Ethics and Impact of Brain Emulations 11 Daniel Dewey, Long-Term Strategies for Ending Existential Risk from Fast Takeoff - 12 Mark Bishop, The Singularity, or How I Learned to Stop Worrying and Love AI -. (shrink)

CVDs continue to be the number one cause of death, therefore early detection and treatment is crucial. Currently, the application of Artificial Intelligence (AI) in the cardiac image analysis has become popular due to increased accuracy, productivity, and modelling. In this paper, the use of the AI system to study the echocardiogram, CT, MRI, and other images of the heart and blood vessels to view the risk factors for cardiovascular diseases is discussed. We focus on the current methods of (...) artificial intelligence, how it is implemented in clinical practice, problems, and prospects for AI development. (shrink)

For decades, the field of system safety has designed safe systems by reducing the risk of physical harm to humans, property and the environment to an acceptable level. Recently, this definition of safety has come under scrutiny by governments and researchers who argue that the narrow focus on reducing physical harm, whilst necessary, is not sufficient to secure the safety of AI systems. There is growing pressure to expand the scope of safety in the context of AI to address (...) emerging harms, with particular emphasis being placed on the ways AI systems can reinforce and reproduce systemic harms. In this paper, we advocate for expanding the scope of conventional safety to include non-physical harms in the context of AI. However, we caution against broadening the scope to address systemic harms, as doing so presents intractable practical challenges for current safety methodologies. Instead, we propose that the scope of safety-related harms should be expanded to include psychological harms. Our proposal is partly motivated by the debates and evidence on social media, which fundamentally reshaped how harm is understood and addressed in the digital age, prompting new regulatory frameworks which aimed to protect users from the psychological risks of the technology. We draw on this precedent to motivate the inclusion of psychological harms in AI safety assessments. By expanding the scope of AI safety to include psychological harms, we take a critical step toward evolving the discipline of system safety into one that is better tuned and equipped to protect users against the complex and emerging harms propagated by AI systems. (shrink)

This abstract summarizes, in essence, what generative AI means to the insurance industry. The kind of promise generated AI offers to insurance is huge: in risk assessment, customer experience, and operational efficiency. Natural disaster impact, financial market volatility, and cyber threat are augmented with techniques of real time scenario generation and modeling as well as predictive simulation based on synthetic data. One of the challenges that stand in the way of deploying these AI methods, however, is data privacy, model (...) reliability and interpretability. The insurance industry must address these issues if it is to comply with regulations like GDPR and CCPA or mitigate reputational risks, as anybody can make complaints about any company based on any perceived data breach. Indeed, generative AI models, often convoluted and biased, can propagate unfair outcomes, especially in pricing and policy access. Navigating these challenges requires a whole new frayed society of AI specialists in collaboration with the industry’s insurers to offer bespoke solutions that address the firm’s needs. Based on this work, future developments will center on designing and implementing ethical and regulatory frameworks around AI-driven decisions that are transparent, fair and sustainable. Generative AI has historically been the most feared phenomenon in the history of western civilization, but with predictive analytics and risk simulation capabilities taking shape, generative AI is set to disrupt the insurance industry and add real-time decision support, automating claims processing and personalized interaction. To successfully integrate generative AI, technical, ethical and regulatory challenges will need to be addressed, and a trust-driven ecosystem will emerge to serve evolving customer needs. (shrink)

Modern AI systems based on deep learning are neither traditional tools nor full-blown agents. Rather, they are characterised by idiosyncratic agential profiles, i.e., combinations of agency-relevant properties. Modern AI systems lack superficial features which enable people to recognise agents but possess sophisticated information processing capabilities which can undermine human goals. I argue that systems fitting this description, when they are adversarial with respect to human users, pose particular risks to those users. To explicate my argument, I provide conditions under which (...) agential profiles are explanatorily relevant to harms caused. I then contend that the role of recommender systems in producing harmful outcomes like digital addiction satisfies these conditions. (shrink)

Using Agentic AI in concert with AWS Full Stack Development, this study offers a technically solid and scalable method to securely process financial data. AWS Lambda for serverless computation, Amazon S3 for scalable storage, AWS Glue for data classification and transformation, Athena for SQL-like querying, and QuickSight for interactive visualisation comprise the fundamental elements of the system. While Agentic AI modules enable autonomous decision-making abilities to regulate data integrity, recognise abnormalities, and adaptably react to compliance violations, Custom AWS configuration rules (...) are used for real-time compliance monitoring. Tight guidelines like PCI-DSS and GDPR enable secure cloud migration, improved auditability, and policy execution. These guidelines open the path for real-time, low-latency analytics on enormous financial data. Combining data-driven insight production with intelligent control will help to build a transparent and strong environment fit for modern financial activities, as the suggested approach shows. (shrink)

Abstract: This article examines risks associated with the program of passive search for alien signals (Search for Extraterrestrial Intelligence, or SETI) connected with the possibility of finding of alien transmission which includes description of AI system aimed on self-replication (SETI-attack). A scenario of potential vulnerability is proposed as well as the reasons why the proportion of dangerous to harmless signals may be high. The article identifies necessary conditions for the feasibility and effectiveness of the SETI-attack: ETI existence, possibility of AI, (...) small size of the Seed AI, small speed of physical interstellar travel and large distance between civilizations. Needed additions to the SETI protocol are considered: keep the signal existence, its content and source location in secret, don’t run alien programs, jam dangerous signals, wait until creation of humanity’s own AI. In addition, scenarios in which it may be reasonable to start an alien AI (if it has been downloaded) are explored. (shrink)

If the intelligence of artificial systems were to surpass that of humans significantly, this would constitute a significant risk for humanity. Time has come to consider these issues, and this consideration must include progress in AI as much as insights from the theory of AI. The papers in this volume try to make cautious headway in setting the problem, evaluating predictions on the future of AI, proposing ways to ensure that AI systems will be beneficial to humans – and (...) critically evaluating such proposals. (shrink)

Artificial Intelligence (AI) is transforming the banking and finance sectors by enhancing efficiency, personalization, and decision-making. This paper explores AI's applications, including credit scoring, fraud detection, and customer service, while addressing associated risks such as data privacy concerns, algorithmic biases, and regulatory challenges. The study also examines future trends, emphasizing the integration of AI with blockchain and quantum computing. Through a literature review and analysis of current practices, this research provides insights into the evolving landscape of AI in financial services.

The insurance and risk management sectors are experiencing a significant transformation driven by the need for more sophisticated data analysis and predictive modeling. This paper explores the critical role of artificial intelligence (AI) in enhancing Extract, Transform, Load (ETL) processes for modernizing data warehouses within these industries. We examine how AI-enhanced ETL addresses key challenges such as data quality, integration of diverse data sources, and real-time processing. The study investigates various applications and proposes a framework for successful implementation. Our (...) findings suggest that AI-driven ETL offers unprecedented opportunities for operational efficiency, improved risk assessment, and competitive advantage in insurance and risk management. (shrink)

An advanced artificial intelligence could pose a significant existential risk to humanity. Several research institutes have been set-up to address those risks. And there is an increasing number of academic publications analysing and evaluating their seriousness. Nick Bostrom’s superintelligence: paths, dangers, strategies represents the apotheosis of this trend. In this article, I argue that in defending the credibility of AI risk, Bostrom makes an epistemic move that is analogous to one made by so-called sceptical theists in the debate (...) about the existence of God. And while this analogy is interesting in its own right, what is more interesting are its potential implications. It has been repeatedly argued that sceptical theism has devastating effects on our beliefs and practices. Could it be that AI-doomsaying has similar effects? I argue that it could. Specifically, and somewhat paradoxically, I argue that it could amount to either a reductio of the doomsayers position, or an important and additional reason to join their cause. I use this paradox to suggest that the modal standards for argument in the superintelligence debate need to be addressed. (shrink)

Artificial Intelligence (AI) and Machine Learning (ML) are transforming the insurance industry by optimizing risk assessment, fraud detection, and customer service. However, the rapid adoption of these technologies raises significant ethical concerns, particularly regarding bias and fairness. This chapter explores the ethical challenges of using AI and ML in insurance risk management, focusing on bias mitigation and fairness enhancement strategies. By analyzing real-world case studies, regulatory frameworks, and technical methodologies, this chapter aims to provide a roadmap for developing (...) ethical AI/ML systems in the insurance sector. It highlights the importance of transparency, accountability, and inclusivity in ensuring equitable outcomes. (shrink)

The growing sophistication and frequency of cyber threats stressed the limitations of traditional reactive cyber security approaches. Organizations are turning to Artificial Intelligence (AI) to improve cyber risk management through predictive capacities. This article investigates the role of AI-oriented predictive analysis in providing and mitigating security incidents. Analyzing historical cyber safety data and leveraging machine learning models as a temporal series forecast, anomaly detection, and supervised classification-ESTE study evaluate the ability of AI systems to anticipate potential threats and support (...) proactive mitigation strategies. A mixed method research project combines case study analysis with empirical evaluation of AI model performance selected using threat data sets available to the public. The results show that AI-based predictive systems can significantly increase the accuracy and punctuality of cyber threat prediction, thus improving the preparation and organizational response. However, challenges related to data quality, model transparency, and ethical considerations persist. The article concludes with recommendations to integrate AI in corporate cyber risk structures and highlights directions for future and explainable AI research for cyber security. (shrink)

Generative models have revolutionized the landscape of artificial intelligence by shifting the focus from predictive tasks to creative and constructive capabilities. From the early use of probabilistic models to the modern architectures of deep neural networks, the evolution of generative models has been marked by increasing sophistication in capturing data distributions and generating novel content. This paper explores the trajectory of generative modeling—from rudimentary statistical techniques to complex structures such as Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), and Transformer-based large (...) language models. By analyzing both the theoretical foundations and practical implementations of these models, we investigate how machines have moved closer to simulating human-like imagination through artificial means. The study utilizes a combination of technical analysis and experimental evaluation to examine the performance of various generative models in tasks related to text, image, and multimodal content creation. We also discuss the philosophical and societal implications of synthetic media, including authorship, originality, and ethical responsibility. A core aim is to understand how data-driven systems have progressed from simply learning representations to autonomously generating meaningful, contextually aware content. Through a detailed methodology involving benchmark datasets, model fine-tuning, and qualitative and quantitative evaluation, we identify key capabilities and limitations of current generative systems.Our findings suggest that while significant progress has been made, generative models still rely heavily on input conditioning, training diversity, and optimization constraints. Despite these limitations, they represent a pivotal advancement in the quest for computational creativity. This paper contributes to the growing discourse on artificial imagination, offering insight into how generative models not only imitate but also expand the creative boundaries of machine learning. As we continue to develop these systems, the line between data processing and autonomous creativity becomes increasingly blurred, raising important questions about the future of AI in human-centric domains. (shrink)

An increasing number of financial services (FS) companies are adopting solutions driven by artificial intelligence (AI) to gain operational efficiencies, derive strategic insights, and improve customer engagement. However, the rate of adoption has been low, in part due to the apprehension around its complexity and self-learning capability, which makes auditability a challenge in a highly regulated industry. There is limited literature on how FS companies can implement the governance and controls specific to AI-driven solutions. AI auditing cannot be performed in (...) a vacuum; the risks are not confined to the algorithm itself, but rather permeates the entire organization. Using the risk of unfairness as an example, this paper will introduce the overarching governance strategy and control framework to address the practical challenges in mitigating risks AI introduces. With regulatory implications and industry use cases, this framework will enable leaders to innovate with confidence. (shrink)

This article reports the findings of AI4People, an Atomium—EISMD initiative designed to lay the foundations for a “Good AI Society”. We introduce the core opportunities and risks of AI for society; present a synthesis of five ethical principles that should undergird its development and adoption; and offer 20 concrete recommendations—to assess, to develop, to incentivise, and to support good AI—which in some cases may be undertaken directly by national or supranational policy makers, while in others may be led by other (...) stakeholders. If adopted, these recommendations would serve as a firm foundation for the establishment of a Good AI Society. (shrink)

The ongoing evolution of advanced AI systems will have profound, enduring, and significant impacts on human existence that must not be overlooked. These impacts range from empowering humanity to achieve unprecedented transcendence to potentially causing catastrophic threats to our existence. To proactively and preventively mitigate these potential threats, it is crucial to establish clear redlines to prevent AI-induced existential risks by constraining and regulating advanced AI and their related AI actors. This paper explores different concepts of AI existential risk, (...) connects the enactment of AI red lines to broader efforts addressing AI's impacts, constructs a theoretical framework for analyzing the direct impacts of AI existential risk, and upon that proposes a set of exemplary AI red lines. By contemplating AI existential risks and formulating these red lines, we aim to foster a deeper and systematic understanding of the potential dangers associated with advanced AI and the importance of proactive risk management. We hope this work will contribute to the strengthening and refinement of a comprehensive AI redline system for preventing humanity from AI existential risks. (shrink)

This paper presents an overview of the risks and benefits of Social AI, understood as conversational AI systems that cater to human social needs like romance, companionship, or entertainment. Section 1 of the paper provides a brief history of conversational AI systems and introduces conceptual distinctions to help distinguish varieties of Social AI and pathways to their deployment. Section 2 of the paper adds further context via a brief discussion of anthropomorphism and its relevance to assessment of human-chatbot relationships. Section (...) 3 of the paper provides a survey of potential and in some cases demonstrated harms associated with user interactions with Social AI systems. Finally, Section 4 discusses how the benefits and harms of Social AI can best be addressed, with a primary focus on how frameworks from AI ethics can inform their development. (shrink)

P&C insurers have an important role in addressing financial risk management needs but now struggle to respond to the new forms of risk. Historical analysis and actuarial calculations, which form the backbone of classical approaches to risk measurement and management, are not well suited to such new kinds of risks as climate change, cyber risks, and business cycle risks. These conventional approaches are also a static method for selling, which has limited potential in changing quickly with new (...) market and consumer changes. However, the development of AI, big data, or advanced predictive analytics has provided a new paradigm to such challenges to cope with and analyze big data dynamically to derive Applying a combination of best Machine Learning (ML), natural language processing (NLP) and other AI techniques, it will become possible to predict risks exposures, as well as optimize underwriting and claims processing. In this paper, the author investigates the opportunities that AI carries within the sphere of reformation of risk evaluation approaches in P&C insurance. In particular, it discusses the ever-emerging field with AI models integrated into operations, underwriting, fraud detection, claims modelling, and loss computation. The applied research uses a comprehensive bibliographical review to present past and present practices and assess modern AI implementations. This paper uses research approaches and case studies to prove that AI can solve some problems and bring cost-saving solutions. Moreover, it emphasizes the ability of predictive analytics to enhance customer satisfaction at the policy level as well as optimize operations. However, the paper also explores the dirty side of implementation, including the stipulations surrounding compliance and data privacy and thus provides an objective analysis of the future of AI use in the P&C Insurance Industry. (shrink)

The philosophy of AI has seen some changes, in particular: 1) AI moves away from cognitive science, and 2) the long term risks of AI now appear to be a worthy concern. In this context, the classical central concerns – such as the relation of cognition and computation, embodiment, intelligence & rationality, and information – will regain urgency.

AI companies are racing to create artificial general intelligence, or “AGI.” If they succeed, the result will be human-level AI systems that can independently pursue high-level goals by formulating and executing long-term plans in the real world. Leading AI researchers agree that some of these systems will likely be “misaligned”–pursuing goals that humans do not desire. This goal mismatch will put misaligned AIs and humans into strategic competition with one another. As with present-day strategic competition between nations with incompatible goals, (...) the result could be violent and catastrophic conflict. Existing legal institutions are unprepared for the AGI world. New foundations for AGI governance are needed, and the time to begin laying them is now, before the critical moment arrives. -/- This Article begins to lay those new legal foundations. It is the first to think systematically about the dynamics of strategic competition between humans and misaligned AGI. The Article begins by showing, using formal game-theoretic models, that, by default, humans and AIs will be trapped in a prisoner’s dilemma. Both parties’ dominant strategy will be to permanently disempower or destroy the other, even though the costs of such conflict would be high. -/- The Article then argues that a surprising legal intervention could transform the game theoretic equilibrium and avoid conflict: AI rights. Not just any AI rights would promote human safety. Granting AIs the right not to be needlessly harmed–as humans have granted to certain non-human animals–would, for example, have little effect. Instead, to promote human safety, AIs should be given those basic private law rights–to make contracts, hold property, and bring tort claims–that law already extends to non-human corporations. Granting AIs these economic rights would enable long-run, small-scale, mutually-beneficial transactions between humans and AIs. This would, we show, facilitate a peaceful strategic equilibrium between humans and AIs for the same reasons economic interdependence tends to promote peace in international relations. Namely, the gains from trade far exceed those from war. Throughout, we argue that human safety, rather than AI welfare, provides the right framework for developing AI rights. This Article explores both the promise and the limits of AI rights as a legal tool for promoting human safety in an AGI world. (shrink)

A morally acceptable course of AI development should avoid two dangers: creating unaligned AI systems that pose a threat to humanity and mistreating AI systems that merit moral consideration in their own right. This paper argues these two dangers interact and that if we create AI systems that merit moral consideration, simultaneously avoiding both of these dangers would be extremely challenging. While our argument is straightforward and supported by a wide range of pretheoretical moral judgments, it has far-reaching moral implications (...) for AI development. Although the most obvious way to avoid the tension between alignment and ethical treatment would be to avoid creating AI systems that merit moral consideration, this option may be unrealistic and is perhaps fleeting. So, we conclude by offering some suggestions for other ways of mitigating mistreatment risks associated with alignment. (shrink)

A classification of the global catastrophic risks of AI is presented, along with a comprehensive list of previously identified risks. This classification allows the identification of several new risks. We show that at each level of AI’s intelligence power, separate types of possible catastrophes dominate. Our classification demonstrates that the field of AI risks is diverse, and includes many scenarios beyond the commonly discussed cases of a paperclip maximizer or robot-caused unemployment. Global catastrophic failure could happen at various levels of (...) AI development, namely, before it starts self-improvement, during its takeoff, when it uses various instruments to escape its initial confinement, or after it successfully takes over the world and starts to implement its goal system, which could be plainly unaligned, or feature-flawed friendliness. AI could also halt at later stages of its development either due to technical glitches or ontological problems. Overall, we identified around several dozen scenarios of AI-driven global catastrophe. The extent of this list illustrates that there is no one simple solution to the problem of AI safety, and that AI safety theory is complex and must be customized for each AI development level. (shrink)

Special Issue “Risks of artificial general intelligence”, Journal of Experimental and Theoretical Artificial Intelligence, 26/3 (2014), ed. Vincent C. Müller. http://www.tandfonline.com/toc/teta20/26/3# - Risks of general artificial intelligence, Vincent C. Müller, pages 297-301 - Autonomous technology and the greater human good - Steve Omohundro - pages 303-315 - - - The errors, insights and lessons of famous AI predictions – and what they mean for the future - Stuart Armstrong, Kaj Sotala & Seán S. Ó hÉigeartaigh - pages 317-342 - - (...) - The path to more general artificial intelligence - Ted Goertzel - pages 343-354 - - - Limitations and risks of machine ethics - Miles Brundage - pages 355-372 - - - Utility function security in artificially intelligent agents - Roman V. Yampolskiy - pages 373-389 - - - GOLEM: towards an AGI meta-architecture enabling both goal preservation and radical self-improvement - Ben Goertzel - pages 391-403 - - - Universal empathy and ethical bias for artificial general intelligence - Alexey Potapov & Sergey Rodionov - pages 405-416 - - - Bounding the impact of AGI - András Kornai - pages 417-438 - - - Ethics of brain emulations - Anders Sandberg - pages 439-457. (shrink)

The field of AI safety seeks to prevent or reduce the harms caused by AI systems. A simple and appealing account of what is distinctive of AI safety as a field holds that this feature is constitutive: a research project falls within the purview of AI safety just in case it aims to prevent or reduce the harms caused by AI systems. Call this appealingly simple account The Safety Conception of AI safety. Despite its simplicity and appeal, we argue that (...) The Safety Conception is in tension with at least two trends in the ways AI safety researchers and organizations think and talk about AI safety: first, a tendency to characterize the goal of AI safety research in terms of catastrophic risks from future systems; second, the increasingly popular idea that AI safety can be thought of as a branch of safety engineering. Adopting the methodology of conceptual engineering, we argue that these trends are unfortunate: when we consider what concept of AI safety it would be best to have, there are compelling reasons to think that The Safety Conception is the answer. Descriptively, The Safety Conception allows us to see how work on topics that have historically been treated as central to the field of AI safety is continuous with work on topics that have historically been treated as more marginal, like bias, misinformation, and privacy. Normatively, taking The Safety Conception seriously means approaching all efforts to prevent or mitigate harms from AI systems based on their merits rather than drawing arbitrary distinctions between them. (shrink)

AI-supported methods for identifying and combating disinformation are progressing in their development and application. However, these methods face a litany of epistemic and ethical challenges. These include (1) robustly defining disinformation, (2) reliably classifying data according to this definition, and (3) navigating ethical risks in the deployment of countermeasures, which involve a mixture of harms and benefits. This paper seeks to expose and offer preliminary analysis of these challenges.

Better instruments to predict the future evolution of artificial intelligence (AI) are needed, as the destiny of our civilization depends on it. One of the ways to such prediction is the analysis of the convergent drives of any future AI, started by Omohundro. We show that one of the convergent drives of AI is a militarization drive, arising from AI’s need to wage a war against its potential rivals by either physical or software means, or to increase its bargaining power. (...) This militarization trend increases global catastrophic risk or even existential risk during AI takeoff, which includes the use of nuclear weapons against rival AIs, blackmail by the threat of creating a global catastrophe, and the consequences of a war between two AIs. As a result, even benevolent AI may evolve into potentially dangerous military AI. The type and intensity of militarization drive depend on the relative speed of the AI takeoff and the number of potential rivals. We show that AI militarization drive and evolution of national defense will merge, as a superintelligence created in the defense environment will have quicker takeoff speeds, but a distorted value system. We conclude with peaceful alternatives. (shrink)

Recent AI progress led to a boost in beneficial applications from multiple research areas including VR. Simultaneously, in this newly unfolding deepfake era, ethically and security-relevant disagreements arose in the scientific community regarding the epistemic capabilities of present-day AI. However, given what is at stake, one can postulate that for a responsible approach, prior to engaging in a rigorous epistemic assessment of AI, humans may profit from a self-questioning strategy, an examination and calibration of the experience of their own epistemic (...) agency – especially to counteract both intentional misdirection by unethical actors and unintentional epistemic self-sabotage. In this paper, we expound on a new avenue of utilizing AIVR tools to advance an AI-related misdirection awareness of humans in the deepfake era. Firstly, we harness scientific knowledge from the psychology and neuroscience of magic where the study of misdirection techniques is center stage. Secondly, we connect the latter to creativity research linking human creative potential to inspiration from the seemingly impossible. Overall, AIVR could become an empowering experiential testbed for human epistemic agency enabling a better rational evaluation of AI capabilities. However, a misuse of the same type of tools could yield AIVR safety risks if not counteracted preemptively. (shrink)

This paper examines how professionals (N = 32) working on artificial intelligence (AI) view structural AI ethics challenges like injustices and inequalities beyond individual agents' direct intention and control. This paper answers the research question: What are professionals’ perceptions of the structural challenges of AI (in the agri-food sector)? This empirical paper shows that it is essential to broaden the scope of ethics of AI beyond micro- and meso-levels. While ethics guidelines and AI ethics often focus on the responsibility of (...) designers and the competencies and skills of designers to take this responsibility, our results show that many structural challenges are beyond their reach. This result means that while ethics guidelines and AI ethics frameworks are helpful, there is a risk that they overlook more complicated, nuanced, and intersected structural challenges. In addition, it highlights the need to include diverse stakeholders, such as quadruple helix (QH) participants, in discussions around AI ethics rather than solely focusing on the obligations of AI developers and companies. Overall, this paper demonstrates that addressing structural challenges in AI is challenging and requires an approach that considers four requirements: (1) multi-level, (2) multi-faceted, (3) interdisciplinary, and (4) polycentric governance. (shrink)

Abstract: Generative AI is transforming the creative industries by redefining how art, music, and media are produced and experienced. This paper explores the profound impact of generative AI technologies, such as deep learning models and neural networks, on creative processes. By enabling artists, musicians, and content creators to collaborate with AI, these systems enhance creativity, speed up production, and generate novel forms of expression. The paper also addresses ethical considerations, including intellectual property rights, the role of human creativity in AI-assisted (...) production, and the potential risks of AI-generated content. As the convergence of AI and creativity deepens, this study highlights the opportunities and challenges posed by generative AI in revolutionizing the landscape of creative industries. (shrink)

Built-in decision thresholds for AI diagnostics are ethically problematic, as patients may differ in their attitudes about the risk of false-positive and false-negative results, which will require that clinicians assess patient values.

The development and deployment of machine learning and artificial intelligence (AI) engender “AI colonialism”, a term that conceptually overlaps with “data colonialism”, as a form of injustice. AI colonialism is in need of decolonization for three reasons. Politically, because it enforces digital capitalism’s hegemony. Ecologically, as it negatively impacts the environment and intensifies the extraction of natural resources and consumption of energy. Epistemically, since the social systems within which AI is embedded reinforce Western universalism by imposing Western colonial values on (...) the global South when these manifest in the digital realm is a form of digital capitalism. These reasons require a new conceptualization of AI decolonization. First this paper draws from the historical debates on the concepts of colonialism and decolonization. Secondly it retrieves Achille Mbembe’s notion of decolonization as disenclosure to argue that the decolonization of AI will have to be the abolishment of political, ecological and epistemic borders erected and reinforced in the phases of its design, production, development of AI in the West and drawing from the knowledge from the global South. In conclusion, it is discussed how conceiving of decolonial AI as form of disenclosure opens up new ways to think about and intervene in colonial instantiations of AI development and deployment, in order to empower “the wretched of AI”, re-ecologise the unsustainable ecologies AI depends on and to counter the colonial power structures unreflective AI deployment risks to reinforce. (shrink)

This piece in Nature Medicine notes the risks that incorporation of AI systems into health care poses to disabled patients and proposes ways to avoid them and instead create benefit.

Artificial Intelligence (AI) is rapidly transforming Human Resource Management (HRM) by enhancing the efficiency and effectiveness of key functions such as recruitment, performance management, and employee engagement. This paper explores the integration of AI technologies in HRM, focusing on their potential to revolutionize these critical areas. In recruitment, AI-driven tools streamline candidate sourcing, screening, and selection processes, leading to more accurate and unbiased hiring decisions. Performance management is similarly transformed, with AI enabling continuous, data-driven feedback and personalized development plans that (...) align with organizational goals. Furthermore, AI facilitates more engaging and tailored employee experiences by leveraging predictive analytics to anticipate and address employee needs. Despite these advancements, the paper also addresses the ethical considerations and challenges associated with AI in HRM, including data privacy concerns and the risk of over-reliance on automated systems. Through a comprehensive analysis, this paper aims to provide insights into how AI can be strategically leveraged to optimize HRM practices, while highlighting the need for a balanced approach that integrates human judgment with technological innovation. (shrink)

This paper challenges the growing discourse suggesting artificial intelligence (AI) may one day serve as a moral decision-maker or possess moral authority. Using the framework of Judgemental Philosophy, we argue that AI, regardless of its sophistication in simulating reasoning or consistency, is structurally ineligible for genuine moral judgement because it cannot satisfy the necessary preconditions defined by the Judgemental Triad (Constructivity, Coherence, and Resonance). While AI systems can exhibit high degrees of Constructivity (generating complex outputs from data) and Coherence (maintaining (...) internal logical or statistical consistency), they fundamentally lack Resonance—the affective and intersubjective return of meaning to a subject that grounds ethical judgement in lived experience and shared value structures. We contend that moral judgement and the authority derived from it require a self-affecting subjectivity capable of receiving the 'return' of its own attributions, a structure that current and foreseeable AI systems inherently lack. Delegating moral judgement to non-resonant systems poses significant risks to human ethical autonomy and the very structure of meaning. (shrink)

In today’s rapidly evolving financial landscape, the adoption of Artificial Intelligence (AI) and Machine Learning (ML) technologies, coupled with the deployment of Business Rules Management Systems (BRMS), has transformed how financial transactions are conducted, monitored, and secured. With fraud, particularly in check deposit transactions, becoming increasingly sophisticated, financial institutions are turning to AI and ML to enhance their risk management strategies. This paper explores the integration of AI-driven models and business rules in financial transactions, focusing on their application in (...) fraud detection and prevention. We also examine the associated challenges, such as data privacy, ethical concerns, and the need for regulatory frameworks, while outlining the opportunities that lie ahead for future financial innovation. (shrink)