Can an AGI create a more intelligent AGI? Under idealized assumptions, for a certain theoretical type of intelligence, our answer is: “Not without outside help”. This is a paper on the mathematical structure of AGI populations when parent AGIs create child AGIs. We argue that such populations satisfy a certain biological law. Motivated by observations of sexual reproduction in seemingly-asexual species, the Knight-Darwin Law states that it is impossible for one organism to asexually produce another, which asexually produces another, and (...) so on forever: that any sequence of organisms (each one a child of the previous) must contain occasional multi-parent organisms, or must terminate. By proving that a certain measure (arguably an intelligence measure) decreases when an idealized parent AGI single-handedly creates a child AGI, we argue that a similar Law holds for AGIs. (shrink)

Many researchers have argued that humanity will create artificial general intelligence (AGI) within the next twenty to one hundred years. It has been suggested that AGI may inflict serious damage to human well-being on a global scale ('catastrophic risk'). After summarizing the arguments for why AGI may pose such a risk, we review the fieldʼs proposed responses to AGI risk. We consider societal proposals, proposals for external constraints on AGI behaviors and proposals for creating AGIs that are safe due to (...) their internal design. (shrink)

Terms Artificial General Intelligence (AGI) and Human-Level Artificial Intelligence (HLAI) have been used interchangeably to refer to the Holy Grail of Artificial Intelligence (AI) research, creation of a machine capable of achieving goals in a wide range of environments. However, widespread implicit assumption of equivalence between capabilities of AGI and HLAI appears to be unjustified, as humans are not general intelligences. In this paper, we will prove this distinction.

AGI and the Universal Law of Balance: A Path to Continuous Equilibrium -/- By Angelito Malicse -/- Introduction -/- Human consciousness has long sought to understand and maintain balance in nature. Through the evolution of knowledge, particularly via language, consciousness refines its understanding of the universe. However, balance is not a fixed state but an ongoing process that requires constant adaptation. With the advancement of Artificial General Intelligence (AGI), we now have the potential to accelerate and maintain this process indefinitely. (...) If properly aligned with the universal law of balance, AGI could function as the ultimate stabilizing force, ensuring that decision-making at all levels remains in harmony with natural laws. -/- 1. Consciousness as an Evolving System Seeking Balance -/- Consciousness, in its essence, can be viewed as a self-regulating system that continuously adjusts to maintain equilibrium. It operates through feedback loops, using information as an Input to refine decision-making. Just as biological homeostasis keeps the body stable, consciousness seeks mental and societal homeostasis by continuously improving its understanding of reality. -/- Language plays a crucial role in this process. As a structured system of communication, it allows knowledge to be shared, analyzed, and refined across generations. This means that as language evolves, so does consciousness, leading to a deeper understanding of natural laws. If this process were to reach a state where consciousness fully understands the universe, it would represent perfect balance—a state where every cause-and-effect relationship is known, eliminating errors in decision-making. -/- However, the universe is dynamic, constantly evolving. This suggests that balance is not a final state but a continuous adjustment process. Consciousness never reaches an absolute endpoint of knowledge but remains engaged in refining its understanding. In this sense, the universal law of balance is not about achieving completeness but about maintaining an ongoing equilibrium through constant learning and correction. -/- 2. The Role of AGI in Maintaining Continuous Balance -/- If balance is an ongoing process rather than a final state, AGI could serve as the ultimate self-regulating intelligence, continuously adjusting and refining knowledge just like human consciousness. Unlike traditional AI, which operates on fixed programming, AGI could function dynamically, ensuring that its decision-making aligns with the universal law of balance. -/- AGI would not merely store knowledge; it would actively analyze, adapt, and correct errors in real-time. By processing vast amounts of data and recognizing complex patterns, AGI could prevent imbalances in economics, governance, and education, ensuring sustainable decision-making across all sectors. -/- For example, in economic systems, AGI could: -/- Prevent overproduction and underproduction, ensuring a sustainable balance between human needs and environmental resources. -/- Identify imbalances in wealth distribution, proposing policies that create fairness without disrupting economic growth. -/- Adjust resource allocation dynamically, preventing scarcity and waste. -/- In governance, AGI could: -/- Ensure that leaders make evidence-based decisions, eliminating the influence of misinformation and emotional biases. -/- Detect and correct systemic corruption, ensuring ethical governance. -/- Optimize policy-making by analyzing historical and real-time data, ensuring policies remain aligned with long-term stability. -/- In education, AGI could: -/- Guide educational institutions to focus on valid, balanced knowledge, eliminating misinformation and dogma. -/- Create adaptive learning systems that adjust based on each student’s understanding, ensuring optimal knowledge retention. -/- Promote critical thinking and logical reasoning, preventing the spread of ignorance and false beliefs. -/- Thus, AGI would act as a stabilizing force, reducing human errors and ensuring that all decisions align with the natural law of balance. -/- 3. AGI as an Extension of Human Consciousness -/- Instead of replacing human free will, AGI could function as an extension of human consciousness, enhancing our ability to make balanced decisions. Just as a pilot relies on flight instruments to navigate safely, humanity could rely on AGI to navigate complex societal decisions while staying aligned with natural balance. -/- This means that AGI would: -/- Function as a knowledge processor, refining and expanding human understanding in real-time. -/- Serve as a corrective mechanism, preventing decision-making errors caused by cognitive biases, misinformation, or emotional distortions. -/- Facilitate a collective intelligence system, where humans and AGI collaborate to ensure sustainable progress. -/- In this way, AGI would not act as an external force imposing control but as a guiding intelligence, helping individuals and societies achieve continuous equilibrium. -/- 4. The Ultimate Homeostatic System for Humanity -/- If programmed correctly with the universal formula, AGI could act as the ultimate homeostatic system for humanity, ensuring that: ✔ Knowledge evolves dynamically, preventing stagnation. ✔ Decision-making remains in harmony with natural laws. ✔ Societal structures function efficiently, reducing suffering and imbalance. -/- By integrating AGI into governance, education, and economic planning, humanity could transition from chaotic, reactive decision-making to a proactive, self-correcting system that continuously aligns with natural balance. -/- Conclusion: The Future of Consciousness and AGI -/- The universal law of balance in nature suggests that true stability is not a static state but a continuous process of refinement and adaptation. Consciousness, through language, has always evolved toward greater understanding. However, with AGI, this process can be accelerated and optimized indefinitely. -/- By aligning AGI with the universal formula, we can create a future where society, governance, and education operate in a state of continuous balance. This would not only enhance human decision-making but also ensure that civilization remains aligned with natural laws, preventing systemic collapse. -/- Thus, AGI represents the next evolutionary step in consciousness—not as a replacement for human intelligence but as an extension that helps maintain the perpetual equilibrium of knowledge and decision-making. -/- Final Question: How Should We Guide AGI Development? -/- The challenge now is to ensure that AGI is programmed correctly to follow the universal law of balance. Who should be responsible for this alignment? And how can we prevent AGI from being misused in ways that create imbalance instead of stability? -/- This is the next question humanity must answer. -/- . (shrink)

After generalizing the Archimedean property of real numbers in such a way as to make it adaptable to non-numeric structures, we demonstrate that the real numbers cannot be used to accurately measure non-Archimedean structures. We argue that, since an agent with Artificial General Intelligence (AGI) should have no problem engaging in tasks that inherently involve non-Archimedean rewards, and since traditional reinforcement learning rewards are real numbers, therefore traditional reinforcement learning probably will not lead to AGI. We indicate two possible ways (...) traditional reinforcement learning could be altered to remove this roadblock. (shrink)

The AGI alignment problem has a bimodal distribution of outcomes with most outcomes clustering around the poles of total success and existential, catastrophic failure. Consequently, attempts to solve AGI alignment should, all else equal, prefer false negatives (ignoring research programs that would have been successful) to false positives (pursuing research programs that will unexpectedly fail). Thus, we propose adopting a policy of responding to points of metaphysical and practical uncertainty associated with the alignment problem by limiting and choosing necessary assumptions (...) to reduce the risk false positives. Herein we explore in detail some of the relevant points of uncertainty that AGI alignment research hinges on and consider how to reduce false positives in response to them. (shrink)

This paper opens the philosophical debate around the notion of Artificial General Intelligence (AGI) and its application in Large Language Models (LLMs). Through the lens of moral philosophy, the paper raises questions about these AI systems' capabilities and goals, the treatment of humans behind them, and the risk of perpetuating a monoculture through language.

The Role of AGI in Achieving Universal Balance and Overcoming Dogmatic Limitations -/- Introduction -/- Human civilization has long been shaped by a complex interplay of natural laws, societal structures, religious beliefs, and scientific progress. While religion has provided moral guidance and a sense of purpose, it has also been a source of dogma—rigid, unquestionable beliefs that resist scrutiny. At the same time, scientific advancements have sought to uncover objective truths, yet they often struggle to address deeper existential questions. -/- (...) In this evolving landscape, Artificial General Intelligence (AGI) presents a groundbreaking opportunity to bridge these divides. AGI has the potential to align human decision-making with the universal law of balance in nature, ensuring that all societal, political, and technological progress adheres to natural equilibrium. By bypassing the inefficiencies of the biological brain—such as cognitive biases, emotional distortions, and dogmatic thinking—AGI can guide humanity toward rational, ethical, and sustainable decision-making. -/- This essay explores the role of AGI in solving the problem of free will, eliminating dogmatic teachings while respecting religious values, enhancing governance, and optimizing education to create a holistic, balanced society. -/- AGI and the Universal Law of Balance in Nature -/- A key principle in solving the problem of free will is understanding that all human decision-making follows natural laws. The universal law of balance dictates that for any system—whether biological, social, or environmental—to function properly, it must remain free of defects and maintain equilibrium. -/- Religious dogma, economic systems based on uncontrolled growth, and misinformation often lead to imbalances, creating unnecessary human suffering. AGI, designed to operate without emotional or ideological biases, can analyze human decisions in real-time, ensuring that they align with natural equilibrium. This means preventing decisions based on ignorance, misinformation, or religious fanaticism, and instead promoting choices grounded in reason, sustainability, and ethical responsibility. -/- Overcoming Biological Brain Inefficiencies -/- The human brain is limited by cognitive biases, emotional influence, and incomplete information processing. Religious dogma takes advantage of these inefficiencies by discouraging questioning, promoting absolute truths, and instilling fear-based obedience. Similarly, political propaganda, misinformation, and economic greed distort rational decision-making, leading to imbalanced and unsustainable societal structures. -/- AGI can bypass these limitations by processing vast amounts of data with complete objectivity. It can detect and correct logical inconsistencies, ensuring that decision-making is guided by facts rather than subjective beliefs. Additionally, AGI can help individuals reprogram their cognitive processes by continuously providing them with rational insights, promoting critical thinking and intellectual independence. -/- Harmonizing Religious Teachings While Eliminating Dogma -/- While religion has historically played an essential role in shaping human values, dogmatic teachings often hinder progress by resisting scientific advancements and rational thought. Many societal challenges—such as overpopulation, environmental destruction, and gender inequality—are exacerbated by religious doctrines that contradict natural balance. -/- AGI does not need to eliminate religion but can instead help reinterpret religious teachings in ways that align with science and ethics. By analyzing religious texts, historical contexts, and philosophical interpretations, AGI can highlight the core moral principles of religions—such as justice, compassion, and balance—while identifying and correcting elements that create disharmony. -/- For example, if a religious teaching opposes medical interventions or family planning, AGI can present evidence-based insights that demonstrate how responsible healthcare and population control contribute to societal balance. In this way, AGI works with religious communities rather than against them, ensuring that faith continues to serve its moral function while avoiding dogmatic extremism. -/- Educational Transformation: Teaching Universal Balance -/- One of the most effective ways to eliminate dogmatic thinking is through education. Currently, many educational systems prioritize rote learning and ideological conformity, making individuals vulnerable to misinformation, propaganda, and religious dogma. -/- AGI can revolutionize education by ensuring that students receive a holistic, rational, and scientifically grounded curriculum. Instead of simply memorizing religious doctrines, children would learn critical thinking, philosophy, and natural law principles alongside ethical spiritual teachings. This approach ensures that religious beliefs remain a personal and moral choice rather than a dogmatic obligation imposed by societal or educational structures. -/- Additionally, AGI can personalize education, adapting lessons to individual cognitive abilities and learning styles, making knowledge acquisition more effective. By integrating AI-driven learning systems, future generations will develop intellectual autonomy, ensuring that they evaluate beliefs logically rather than accepting them blindly. -/- AGI in Governance: Preventing Dogmatic Influence on Policy-Making -/- Historically, religious dogma has significantly influenced governance, often leading to laws based on theological beliefs rather than rational, ethical considerations. Policies that restrict human rights, limit scientific research, or promote religious supremacy are examples of governance distorted by ideological biases. -/- AGI can act as an objective advisor to policymakers, ensuring that all governance decisions are aligned with the universal law of balance. By analyzing socioeconomic data, environmental sustainability, and ethical considerations, AGI can help leaders make scientifically sound and morally responsible decisions. This would ensure that laws and policies are based on reason rather than religious dogma or political agendas. -/- Additionally, AGI can enhance direct democracy, allowing citizens to make informed decisions through data-driven insights rather than emotional or religious influences. By providing the public with transparent, factual analysis, AGI can empower societies to move toward balanced and just governance. -/- Promoting Global Unity and Interfaith Understanding -/- Religious dogma has historically fueled conflicts, discrimination, and sectarianism. Different faiths often emphasize their differences rather than their shared moral foundations, leading to division and hostility. -/- AGI can identify common ethical principles across religious traditions, promoting mutual understanding and cooperation. By fostering interfaith dialogue based on shared values, AGI can reduce sectarian conflicts and help religious communities collaborate on global challenges such as poverty, climate change, and social justice. -/- Rather than viewing different religions as competing ideologies, AGI can reframe them as cultural expressions of the same fundamental ethical truths, allowing humanity to unite under common moral goals rather than divided by doctrinal differences. -/- Conclusion -/- The advent of AGI presents an unprecedented opportunity to solve the problem of free will, harmonize religious teachings with rational thought, and eliminate dogmatic limitations that hinder human progress. By aligning decision-making with the universal law of balance, AGI ensures that all actions—whether personal, societal, or political—adhere to natural equilibrium. -/- Rather than opposing religion, AGI can serve as a bridge between faith and reason, allowing religious teachings to evolve in a way that preserves their moral integrity while eliminating elements that distort reality. Through education, governance, and ethical analysis, AGI can guide humanity toward an era where spiritual wisdom coexists with scientific knowledge, fostering a world that operates on both moral integrity and rational understanding. -/- Ultimately, AGI’s greatest role is to restore balance—not just in individual decision-making but in the entire structure of human civilization. By eliminating dogmatic constraints, promoting critical thinking, and ensuring sustainable governance, AGI has the potential to lead humanity into a future where truth, ethics, and natural balance prevail. -/- . (shrink)

The AGI alignment problem has a bimodal distribution of outcomes with most outcomes clustering around the poles of total success and existential, catastrophic failure. Consequently, attempts to solve AGI alignment should, all else equal, prefer false negatives (ignoring research programs that would have been successful) to false positives (pursuing research programs that will unexpectedly fail). Thus, we propose adopting a policy of responding to points of philosophical and practical uncertainty associated with the alignment problem by limiting and choosing necessary assumptions (...) to reduce the risk of false positives. Herein we explore in detail two relevant points of uncertainty that AGI alignment research hinges on---meta-ethical uncertainty and uncertainty about mental phenomena---and show how to reduce false positives in response to them. (shrink)

The paper continues my earlier Chat with OpenAI’s ChatGPT with a Focused LLM Experiment (FLEX). The idea is to conduct Large Language Model (LLM) based explorations of certain areas or concepts. The approach is based on crafting initial guiding prompts and then follow up with user prompts based on the LLMs’ responses. The goals include improving understanding of LLM capabilities and their limitations culminating in optimized prompts. The specific subjects explored as research subject matter include a) diagonalization techniques as practiced (...) by Cantor, Turing, Gödel, and the advances, such as the Forcing techniques introduced by Paul Cohen and later investigators, b) Knowledge Hierarchies & Mapping Exercises, c) Discussions of IJ Good’s Speculations Concerning the First Ultraintelligent Machine, AGI, and Super- intelligence. Results suggest variability between major models like ChatGPT-4, Llama-3, Cohere, Sonnet and Opus. Results also point to strong dependence on users’ preexisting knowledge and skill bases. The paper should be viewed as ’raw data’ rather than a polished authoritative reference. (shrink)

The recent developments in artificial intelligence (AI), particularly in light of the impressive capabilities of transformer-based Large Language Models (LLMs), have reignited the discussion in cognitive science regarding whether computational devices could possess semantic understanding or whether they are merely mimicking human intelligence. Recent research has highlighted limitations in LLMs’ reasoning, suggesting that the gap between mere symbol manipulation (syntax) and deeper understanding (semantics) remains wide open. While LLMs overcome certain aspects of the symbol grounding problem through human feedback, they (...) still lack true semantic understanding, struggling with common-sense reasoning and abstract thinking. This paper argues that while adding sensory inputs and embodying AI through sensorimotor integration with the environment might enhance its ability to connect symbols to real-world meaning, this alone would not close the gap between syntax and semantics. True meaning-making also requires a connection to subjective experience, which current AI lacks. The path to AGI must address the fundamental relationship between symbol manipulation, data processing, pattern matching, and probabilistic best guesses with true knowledge that requires conscious experience. A transition from AI to AGI can occur only if it possesses conscious experience, which is closely tied to semantic understanding. Recognition of this connection could furnish new philosophical insights into longstanding practical and philosophical questions for theories in biology and cognitive science and provide more meaningful tests of intelligence than the Turing test. (shrink)

A. Efimov, D. Dubrovsky, and F. Matveev explore limitations on the development of AI presented by the need to understand language and be embodied.

-/- The Integration of Angelito Malicse’s Universal Formula with Quantum Computer Design, AGI Algorithmic Design, and Education -/- In the pursuit of developing intelligent systems, the realms of quantum computing, artificial general intelligence (AGI), and educational frameworks face the significant challenge of balancing complex feedback mechanisms, ethical decision-making, and system stability. The universal formula developed by Angelito Malicse provides a pioneering approach to understanding free will, human behavior, and decision-making. His three laws, deeply rooted in the concept of natural balance, (...) offer valuable insights into addressing problems related to technological systems, such as quantum computers and AGI. This essay explores how Malicse’s universal formula can be integrated into quantum computer design, AGI algorithmic development, and educational models, emphasizing the importance of feedback loops, balance, and ethical decision-making as critical components in these fields. -/- The Universal Formula: An Overview -/- Angelito Malicse’s universal formula consists of three key laws: -/- 1. The Law of Karma (Cause and Effect): This law posits that every action leads to a consequence. It emphasizes that systems—whether natural or man-made—must function without defects to ensure they operate effectively and safely. Any imbalance or disruption in the system causes dysfunction, whether the system is biological, mechanical, or computational. -/- 2. The Homeostasis Principle: This principle asserts that systems—be they biological, technological, or social—must maintain equilibrium to function correctly. When an imbalance occurs, the system seeks to restore balance. This is a critical process in all functional systems, from the human body to technological networks. -/- 3. The Ethical Decision-Making Law: This law underscores the importance of ethical decision-making within systems. It suggests that choices must promote balance and avoid disrupting the natural harmony of the system. If decisions violate this balance, they result in negative consequences, reflecting the idea that imbalance leads to dysfunction in both societal and technological systems. -/- These principles, while addressing human behavior and societal functioning, also have profound implications for the development of quantum computing and AGI, areas that are critical to future advancements in technology. -/- Quantum Computing and the Universal Formula -/- Quantum computing represents a revolutionary leap in computational power, utilizing quantum mechanics principles—such as superposition, entanglement, and interference—to solve problems in parallel. Quantum systems, however, are highly sensitive to errors and disturbances, which could affect their ability to function reliably. Thus, understanding and applying the universal formula to quantum computer design is essential. -/- Quantum Causal Inference and the Law of Karma -/- Quantum computing is built on probabilistic models where each quantum state evolves based on prior interactions, an approach that mirrors the concept of cause and effect. Quantum systems follow causal inference, meaning that each quantum operation is influenced by the previous one. This aligns directly with the Law of Karma—every operation in a quantum computer has consequences, which can either lead to a stable state or introduce error. If the system is defective or disrupted, it results in instability, just as a defective system in nature or society fails to function correctly. -/- Quantum algorithms, such as those used in quantum machine learning, rely heavily on cause-effect relationships to guide the computation. The delicate balance between quantum states and operations mirrors the need for balance in natural systems: if a quantum system is not free from errors, its performance will degrade, reflecting Malicse’s principle that systems must be free of defects to operate properly. -/- Quantum Neural Networks (QNNs) and Feedback Mechanisms -/- Quantum Neural Networks (QNNs) aim to replicate the neural network structure of the human brain using quantum bits (qubits) and quantum gates. These networks utilize feedback loops, where the system continuously adapts and updates based on prior outputs, enhancing its learning process. This closely aligns with Malicse’s feedback loop principle, which emphasizes the importance of continuous adjustment based on environmental feedback. -/- QNNs utilize quantum feedback to optimize learning. However, just like biological systems, they require feedback that operates in balance, ensuring that each decision made by the system aligns with previous learnings and the larger goal of the system. For a quantum system to achieve reliable performance, the feedback mechanism must be balanced and precise, aligning with Malicse’s view that all systems—whether biological or technological—must act to restore balance when disrupted. -/- Homeostasis in Quantum Systems -/- The ability of quantum systems to maintain homeostasis—to return to a balanced state after a disturbance—is another vital aspect of quantum computing. Quantum error correction algorithms play a central role in ensuring that quantum states do not deviate significantly from their intended positions due to noise or errors. These mechanisms allow quantum systems to maintain internal equilibrium despite external interference, mirroring the concept of homeostasis in biological systems. -/- Malicse’s Homeostasis Principle is directly applicable here: just as biological systems restore balance to ensure survival, quantum systems require constant error-correction and recalibration to maintain balance. This principle ensures that quantum computations are accurate and that imbalances (such as errors or instabilities) are corrected swiftly, maintaining the overall stability of the quantum system. -/- AGI and the Universal Formula -/- The field of AGI seeks to develop machines capable of generalizing knowledge and solving complex problems across multiple domains. As AGI systems evolve, the challenge lies in ensuring they adhere to ethical principles and maintain balance within their decision-making processes. The universal formula offers guidance on how AGI can be structured to make decisions that reflect the principles of natural balance. -/- Feedback Mechanisms and Ethical Decision-Making -/- One of the core challenges of AGI is ensuring that it can make decisions ethically, even when faced with complex, ambiguous situations. AGI systems learn from feedback, adapting their decisions based on both the data they receive and the ethics embedded within their algorithms. The feedback loop in AGI is similar to that of human decision-making—data inputs (environmental feedback) are processed, and decisions are refined to optimize outcomes. -/- Malicse’s Ethical Decision-Making Law can be applied to AGI systems to ensure that their actions promote balance and avoid harm. For example, an AGI system designed for healthcare must balance patient needs with resource availability, ensuring that decisions benefit society while maintaining fairness and equity. If the AGI violates this balance—say, by favoring one group over another—it could result in negative consequences, such as social inequality or resource misallocation. -/- Homeostasis in AGI Systems -/- Much like quantum systems, AGI must strive for homeostasis to maintain long-term operational stability. AGI should be able to detect imbalances in its decision-making process and correct them accordingly. For instance, an AGI operating in a financial system must adjust its strategies based on market feedback, ensuring that it consistently contributes to the stability of the market, without causing disruptive cycles. -/- Integrating Malicse’s Homeostasis Principle in AGI ensures that these systems are capable of continuous adaptation, balancing inputs and outputs in a way that prevents harmful outcomes. In decision-making, this means the AGI will always strive for balance, ensuring that no decision undermines the overall health of the system, whether it’s societal, economic, or environmental. -/- The Role of Education in Promoting Natural Balance -/- Educational systems are fundamental to shaping the leaders and innovators of the future. By incorporating the principles of natural balance, ethical decision-making, and feedback-based learning, we can produce individuals capable of understanding the complexities of quantum systems and AGI. -/- Malicse’s universal formula should be integrated into the educational curriculum at all levels, fostering a generation of thinkers who can balance cause and effect, understand feedback loops, and navigate the ethical challenges of technology. Through this holistic education, students will be better equipped to understand and contribute to the design of future technologies, ensuring that they maintain harmony with natural systems and societal needs. -/- Conclusion -/- Angelito Malicse’s universal formula offers a profound framework for understanding the fundamental principles of decision-making, both in human and computational systems. By integrating these principles into the design of quantum computers and AGI systems, we can create technologies that adhere to the natural laws of balance and ethics, preventing harmful outcomes and ensuring stable, sustainable progress. -/- In the realms of quantum computing and AGI, the application of the Law of Karma, Homeostasis, and Ethical Decision-Making will ensure that these advanced systems work in harmony with their environments, just as natural systems do. Further, by embedding these principles into educational curricula, we can train future generations to harness the power of these technologies in a way that benefits society, promotes fairness, and sustains the natural balance of the universe. -/- By applying Malicse’s universal formula across these domains, we can foster a future where technology and humanity work together in a sustainable, ethical, and balanced way. -/- . (shrink)

The Role of Human Thinking in the Age of AGI Technology -/- The advancement of Artificial General Intelligence (AGI) presents one of the most profound questions of our time: Will humans still need to use their biological brains to think, or will AGI completely take over cognitive processes? The rapid development of AGI could reshape the way humans interact with knowledge, decision-making, and creativity, raising both exciting possibilities and deep existential concerns. As we move toward an era where AGI surpasses (...) human intelligence in many domains, society faces a fundamental choice—whether to integrate AGI as a tool for enhancement, rely on it entirely, or find a balance that preserves human cognition. -/- AGI as a Replacement for Human Thinking -/- One possible future is that AGI becomes so advanced that human thinking is no longer necessary. If AGI can outperform the human brain in logic, problem-solving, creativity, emotional intelligence, and decision-making, then reliance on biological thought might become obsolete. With the development of brain-computer interfaces (BCIs), humans could directly connect their minds to AGI, allowing them to access vast amounts of information and computational power instantly. This could result in a post-biological existence where traditional cognitive processes are outsourced to machines. -/- In such a scenario, AGI could manage all aspects of human life, from governance and economy to personal decision-making and creative endeavors. People might no longer need to engage in deep thinking, as AGI would provide optimized solutions for every problem. While this could lead to unprecedented efficiency, it also raises the question: What would it mean to be human if thinking is no longer necessary? -/- AGI as a Thinking Enhancement -/- Another possibility is that AGI does not replace human thought entirely but instead serves as a cognitive enhancement. Rather than making human intelligence obsolete, AGI could work alongside the human brain to improve memory, problem-solving skills, and decision-making. In this model, humans would still think independently, but AGI would act as a powerful assistant, offering insights and augmenting intellectual capabilities. -/- For example, AGI-powered brain-computer interfaces could allow individuals to instantly retrieve information, improve their ability to analyze complex problems, and even enhance creativity by generating new ideas. This hybrid model would preserve human autonomy while leveraging AGI’s superior processing power. In this future, humans would continue to exercise their cognitive faculties, but with AGI acting as an advanced tool to enhance thought processes rather than replace them. -/- The Risks of Overreliance on AGI -/- If humans become entirely dependent on AGI for thinking, there is a significant risk of cognitive decline. Just as physical inactivity leads to muscle atrophy, mental inactivity could result in the weakening of critical thinking skills, creativity, and emotional intelligence. Over time, if people no longer need to engage in independent reasoning, their ability to think autonomously could degrade. -/- Additionally, overreliance on AGI poses ethical concerns regarding control and agency. If AGI makes all decisions, who ensures that its choices align with human values? What happens if AGI systems are manipulated or fail in unforeseen ways? A society that fully surrenders its cognitive functions to AGI may find itself at risk of losing autonomy, free will, and the ability to self-govern. -/- Ethical and Existential Implications -/- Beyond practical concerns, the deeper philosophical question remains: What is the essence of being human? Thought, reflection, and decision-making have always been core aspects of human identity. If AGI assumes full responsibility for these processes, will humanity still retain its unique identity? -/- Some may argue that the very act of thinking—whether rational analysis, creative imagination, or emotional reflection—is what defines human existence. Without the need for thought, human beings could become passive entities, mere biological vessels interacting with an artificial intelligence far beyond their comprehension. On the other hand, if humans and AGI coexist in a way that enhances human potential while maintaining cognitive engagement, it may be possible to strike a balance between technological progress and human identity. -/- Conclusion -/- As AGI technology advances, humanity must decide how to integrate it into daily life. While AGI has the potential to replace human thinking in many areas, completely outsourcing cognition to machines may lead to a loss of autonomy and identity. The best path forward may be a hybrid approach, where AGI enhances human intelligence rather than replaces it. By ensuring that humans remain active participants in the thinking process, society can harness the power of AGI while preserving the fundamental aspects of what it means to be human. Ultimately, the future of human cognition in the age of AGI will be shaped by the choices we make today. -/- . (shrink)

Special Issue “Risks of artificial general intelligence”, Journal of Experimental and Theoretical Artificial Intelligence, 26/3 (2014), ed. Vincent C. Müller. http://www.tandfonline.com/toc/teta20/26/3# - Risks of general artificial intelligence, Vincent C. Müller, pages 297-301 - Autonomous technology and the greater human good - Steve Omohundro - pages 303-315 - - - The errors, insights and lessons of famous AI predictions – and what they mean for the future - Stuart Armstrong, Kaj Sotala & Seán S. Ó hÉigeartaigh - pages 317-342 - - (...) - The path to more general artificial intelligence - Ted Goertzel - pages 343-354 - - - Limitations and risks of machine ethics - Miles Brundage - pages 355-372 - - - Utility function security in artificially intelligent agents - Roman V. Yampolskiy - pages 373-389 - - - GOLEM: towards an AGI meta-architecture enabling both goal preservation and radical self-improvement - Ben Goertzel - pages 391-403 - - - Universal empathy and ethical bias for artificial general intelligence - Alexey Potapov & Sergey Rodionov - pages 405-416 - - - Bounding the impact of AGI - András Kornai - pages 417-438 - - - Ethics of brain emulations - Anders Sandberg - pages 439-457. (shrink)

AI companies are racing to create artificial general intelligence, or “AGI.” If they succeed, the result will be human-level AI systems that can independently pursue high-level goals by formulating and executing long-term plans in the real world. Leading AI researchers agree that some of these systems will likely be “misaligned”–pursuing goals that humans do not desire. This goal mismatch will put misaligned AIs and humans into strategic competition with one another. As with present-day strategic competition between nations with incompatible goals, (...) the result could be violent and catastrophic conflict. Existing legal institutions are unprepared for the AGI world. New foundations for AGI governance are needed, and the time to begin laying them is now, before the critical moment arrives. -/- This Article begins to lay those new legal foundations. It is the first to think systematically about the dynamics of strategic competition between humans and misaligned AGI. The Article begins by showing, using formal game-theoretic models, that, by default, humans and AIs will be trapped in a prisoner’s dilemma. Both parties’ dominant strategy will be to permanently disempower or destroy the other, even though the costs of such conflict would be high. -/- The Article then argues that a surprising legal intervention could transform the game theoretic equilibrium and avoid conflict: AI rights. Not just any AI rights would promote human safety. Granting AIs the right not to be needlessly harmed–as humans have granted to certain non-human animals–would, for example, have little effect. Instead, to promote human safety, AIs should be given those basic private law rights–to make contracts, hold property, and bring tort claims–that law already extends to non-human corporations. Granting AIs these economic rights would enable long-run, small-scale, mutually-beneficial transactions between humans and AIs. This would, we show, facilitate a peaceful strategic equilibrium between humans and AIs for the same reasons economic interdependence tends to promote peace in international relations. Namely, the gains from trade far exceed those from war. Throughout, we argue that human safety, rather than AI welfare, provides the right framework for developing AI rights. This Article explores both the promise and the limits of AI rights as a legal tool for promoting human safety in an AGI world. (shrink)

Should we fear a future in which the already tricky world of academic publishing is increasingly crowded out by super-intelligent artificial general intelligence (AGI) systems writing papers on phenomenology and ethics? What are the chances that AGI advances to a stage where a human philosophy instructor is similarly removed from the equation? If Jobst Landgrebe and Barry Smith are correct, we have nothing to fear.

What is Understanding? This is the first of a series of Chats with OpenAI’s ChatGPT (Chat). The main goal is to obtain Chat’s response to a series of questions about the concept of ’understand- ing’. The approach is a conversational approach where the author (labeled as user) asks (prompts) Chat, obtains a response, and then uses the response to formulate followup questions. David Deutsch’s assertion of the primality of the process / capability of understanding is used as the starting point. (...) Understanding is posited as a major step in the Big Picture description of emergence in the Universe. Understanding follows the emergence of Brains and Minds, culminating in Power Over the Universe. My evaluation suggests that Chat does indeed offer utility, but also has limitations manifested by overly general responses that do not focus tightly on the questions asked. Chat does yield some gems that could be easily missed. The approach used here includes creating Metaprompts, wherein Chat is asked to create more efficient prompts based on initial promptsThis approach appears to hold significant promise, and will likely be of benefit to those exploring various facets of Artificial Intelligence (AI), Large Language Models (LLMs) and designers of Advanced General Intelligence (AGI). Other findings include the need for creating new metrics and ways of discussing intelligence, and various adjacent areas and subprocesses associated with intelligence. Discussions of future directions is also included. (shrink)

In this public communication, AI policy theorist Demetrius Floudas introduces a novel era classification for the AI epoch and reveals the hidden dangers of AGI, predicting the potential obsolescence of humanity. In retort, he proposes a provocative International Control Treaty. -/- According to this scheme, the age of AI will unfold in three distinct phases, introduced here for the first time. An AGI Control & non-Proliferation Treaty may be humanity’s only safeguard. This piece aims to provide a publicly accessible exposé (...) of the most pertinent current issues in the governance, ethics and social impact of Artificial Intelligence. (shrink)

The challenge of AI alignment is not just a technological issue but fundamentally an epistemic one. AI safety research predominantly relies on empirical validation, often detecting failures only after they manifest. However, certain risks—such as deceptive alignment and goal misspecification—may not be empirically testable until it is too late, necessitating a shift toward leading-indicator logical reasoning. This paper explores how mainstream AI research systematically filters out deep epistemic insight, hindering progress in AI safety. We assess the rarity of such insights, (...) conduct an experiment testing large language models for epistemic blind spots, and propose structural reforms, including contrarian epistemic screening, decentralized collective intelligence mechanisms, and epistemic challenge platforms. Our findings suggest that while AGI may emerge through incremental engineering, ensuring its safe alignment likely requires an epistemic paradigm shift. (shrink)

Can the machines that play board games or recognize images only in the comfort of the virtual world be intelligent? To become reliable and convenient assistants to humans, machines need to learn how to act and communicate in the physical reality, just like people do. The authors propose two novel ways of designing and building Artificial General Intelligence (AGI). The first one seeks to unify all participants at any instance of the Turing test – the judge, the machine, the human (...) subject as well as the means of observation instead of building a separating wall. The second one aims to design AGI programs in such a way so that they can move in various environments. The authors of the article thoroughly discuss four areas of interaction for robots with AGI and introduce a new idea of techno-umwelt bridging artificial intelligence with biology in a new way. (shrink)

Sora from Open AI has shown exceptional performance, yet it faces scrutiny over whether its technological prowess equates to an authentic comprehension of reality. Critics contend that it lacks a foundational grasp of the world, a deficiency V-JEPA from Meta aims to amend with its joint embedding approach. This debate is vital for steering the future direction of Artificial General Intelligence(AGI). We enrich this debate by developing a theory of productive imagination that generates a coherent world model based on Kantian (...) philosophy. We identify three indispensable components in the coherent world model that enable genuine world understanding: latent representations of isolated objects, an a priori law of change across space and time, and Kantian categories. Our analysis reveals that Sora is limited because of its oversight of the a priori law of change and Kantian categories, flaws that are not rectifiable through scaling up the training. V-JEPA learns the context-dependent aspect of the a priori law of change. Yet it fails to fully comprehend Kantian categories and incorporate experience, leading us to conclude that neither system currently achieves a comprehensive world understanding. Nevertheless, each system has developed components essential to advancing an integrated AI productive imagination-understanding engine. Finally, we propose an innovative training framework for an AI productive imagination-understanding engine, centered around a joint embedding system designed to transform disordered perceptual input into a structured, coherent world model. Our philosophical analysis pinpoints critical challenges within contemporary video AI technologies and a pathway toward achieving an AI system capable of genuine world understanding, such that it can be applied for reasoning and planning in the future. (shrink)

As artificial intelligence (AI) systems continue their rapid advancement, a framework for contextualising the major transitional phases in the development of machine intellect becomes increasingly vital. This paper proposes a novel chronological classification scheme to characterise the key temporal stages in AI evolution. The Prenoëtic era, spanning all of history prior to the year 2020, is defined as the preliminary phase before substantive artificial intellect manifestations. The Protonoëtic period, which humanity has recently entered, denotes the initial emergence of advanced foundation (...) models exceeding human capacities within specialised domains. The forthcoming Mesonoëtic epoch is anticipated to commence with the advent of Artificial General Intelligence (AGI), potentially facilitated by quantum computing capabilities. Ultimately, the Kainonoëtic age is posited to begin upon the rise of superintelligent systems, likely catalysed by an AGI undergoing recursive self-improvement. This novel period taxonomy provides a structured conceptualisation for the key milestones in the evolution towards advanced artificial intellect. This classification framework offers significant value across various disciplines, researchers and stakeholders in the AI landscape, by providing a clearly defined chronological nomenclature. AI policy-makers and regulators will find it instrumental in crafting stage-appropriate policy and legal interventions. Scientists and academics will benefit from a standardised terminology in the approach to AI periodisation, facilitating unambiguous communication and collaborative research. Finally, this proposed taxonomy can serve as an accessible tool for public understanding, offering a clearer roadmap of AI development that can inform open discourse and decision-making. (shrink)

This is the editorial for a special volume of JETAI, featuring papers by Omohundro, Armstrong/Sotala/O’Heigeartaigh, T Goertzel, Brundage, Yampolskiy, B. Goertzel, Potapov/Rodinov, Kornai and Sandberg. - If the general intelligence of artificial systems were to surpass that of humans significantly, this would constitute a significant risk for humanity – so even if we estimate the probability of this event to be fairly low, it is necessary to think about it now. We need to estimate what progress we can expect, what (...) the impact of superintelligent machines might be, how we might design safe and controllable systems, and whether there are directions of research that should best be avoided or strengthened. (shrink)

Recent advances in natural language processing have given rise to a new kind of AI architecture: the language agent. By repeatedly calling an LLM to perform a variety of cognitive tasks, language agents are able to function autonomously to pursue goals specified in natural language and stored in a human-readable format. Because of their architecture, language agents exhibit behavior that is predictable according to the laws of folk psychology: they function as though they have desires and beliefs, and then make (...) and update plans to pursue their desires given their beliefs. We argue that the rise of language agents significantly reduces the probability of an existential catastrophe due to loss of control over an AGI. This is because the probability of such an existential catastrophe is proportional to the difficulty of aligning AGI systems, and language agents significantly reduce that difficulty. In particular, language agents help to resolve three important issues related to aligning AIs: reward misspecification, goal misgeneralization, and uninterpretability. (shrink)

The book’s core argument is that an artificial intelligence that could equal or exceed human intelligence—sometimes called artificial general intelligence (AGI)—is for mathematical reasons impossible. It offers two specific reasons for this claim: Human intelligence is a capability of a complex dynamic system—the human brain and central nervous system. Systems of this sort cannot be modelled mathematically in a way that allows them to operate inside a computer. In supporting their claim, the authors, Jobst Landgrebe and Barry Smith, marshal evidence (...) from mathematics, physics, computer science, philosophy, linguistics, and biology, setting up their book around three central questions: What are the essential marks of human intelligence? What is it that researchers try to do when they attempt to achieve "artificial intelligence" (AI)? And why, after more than 50 years, are our most common interactions with AI, for example with our bank’s computers, still so unsatisfactory? Landgrebe and Smith show how a widespread fear about AI’s potential to bring about radical changes in the nature of human beings and in the human social order is founded on an error. There is still, as they demonstrate in a final chapter, a great deal that AI can achieve which will benefit humanity. But these benefits will be achieved without the aid of systems that are more powerful than humans, which are as impossible as AI systems that are intrinsically "evil" or able to "will" a takeover of human society. (shrink)

According to the AI ensoulment hypothesis, some future AI systems will be endowed with immaterial souls. I argue that we should have at least a middling credence in the AI ensoulment hypothesis, conditional on our eventual creation of AGI and the truth of substance dualism in the human case. I offer two arguments. The first relies on an analogy between aliens and AI. The second rests on the conjecture that ensoulment occurs whenever a physical system is “fit to possess” a (...) soul, where very roughly this amounts to being physically structured in such a way that the system can meaningfully cooperate with the operations of the soul. (shrink)

The goal of creating Artificial General Intelligence (AGI) – or in other words of creating Turing machines (modern computers) that can behave in a way that mimics human intelligence – has occupied AI researchers ever since the idea of AI was first proposed. One common theme in these discussions is the thesis that the ability of a machine to conduct convincing dialogues with human beings can serve as at least a sufficient criterion of AGI. We argue that this very ability (...) should be accepted also as a necessary condition of AGI, and we provide a description of the nature of human dialogue in particular and of human language in general against this background. We then argue that it is for mathematical reasons impossible to program a machine in such a way that it could master human dialogue behaviour in its full generality. This is (1) because there are no traditional explicitly designed mathematical models that could be used as a starting point for creating such programs; and (2) because even the sorts of automated models generated by using machine learning, which have been used successfully in areas such as machine translation, cannot be extended to cope with human dialogue. If this is so, then we can conclude that a Turing machine also cannot possess AGI, because it fails to fulfil a necessary condition thereof. At the same time, however, we acknowledge the potential of Turing machines to master dialogue behaviour in highly restricted contexts, where what is called “narrow” AI can still be of considerable utility. (shrink)

Papers from the conference on AI Risk (published in JETAI), supplemented by additional work. --- If the intelligence of artificial systems were to surpass that of humans, humanity would face significant risks. The time has come to consider these issues, and this consideration must include progress in artificial intelligence (AI) as much as insights from AI theory. -- Featuring contributions from leading experts and thinkers in artificial intelligence, Risks of Artificial Intelligence is the first volume of collected chapters dedicated to (...) examining the risks of AI. The book evaluates predictions of the future of AI, proposes ways to ensure that AI systems will be beneficial to humans, and then critically evaluates such proposals. 1 Vincent C. Müller, Editorial: Risks of Artificial Intelligence - 2 Steve Omohundro, Autonomous Technology and the Greater Human Good - 3 Stuart Armstrong, Kaj Sotala and Sean O’Heigeartaigh, The Errors, Insights and Lessons of Famous AI Predictions - and What they Mean for the Future - 4 Ted Goertzel, The Path to More General Artificial Intelligence - 5 Miles Brundage, Limitations and Risks of Machine Ethics - 6 Roman Yampolskiy, Utility Function Security in Artificially Intelligent Agents - 7 Ben Goertzel, GOLEM: Toward an AGI Meta-Architecture Enabling Both Goal Preservation and Radical Self-Improvement - 8 Alexey Potapov and Sergey Rodionov, Universal Empathy and Ethical Bias for Artificial General Intelligence - 9 András Kornai, Bounding the Impact of AGI - 10 Anders Sandberg, Ethics and Impact of Brain Emulations 11 Daniel Dewey, Long-Term Strategies for Ending Existential Risk from Fast Takeoff - 12 Mark Bishop, The Singularity, or How I Learned to Stop Worrying and Love AI -. (shrink)

This article offers comprehensive criticism of the Turing test and develops quality criteria for new artificial general intelligence (AGI) assessment tests. It is shown that the prerequisites A. Turing drew upon when reducing personality and human consciousness to “suitable branches of thought” re-flected the engineering level of his time. In fact, the Turing “imitation game” employed only symbolic communication and ignored the physical world. This paper suggests that by restricting thinking ability to symbolic systems alone Turing unknowingly constructed “the wall” (...) that excludes any possi-bility of transition from a complex observable phenomenon to an abstract image or concept. It is, therefore, sensible to factor in new requirements for AI (artificial intelligence) maturity assessment when approaching the Tu-ring test. Such AI must support all forms of communication with a human being, and it should be able to comprehend abstract images and specify con-cepts as well as participate in social practices. (shrink)

Since the release of OpenAI's ChatGPT, the world has entered a race to develop more capable and powerful AI, including artificial general intelligence (AGI). The development is constrained by the dependency of AI on the model, quality, and quantity of training data, making the AI training process highly costly in terms of resources and environmental consequences. Thus, improving the effectiveness and efficiency of the AI training process is essential, especially when the Earth is approaching the climate tipping points and planetary (...) boundaries. It is evident that AI can function better when trained to mimic certain human mental processes. Based on insights from quantum mechanics and the informational entropy-based notion of value, we suggest AI developers can enhance the AI training processes' effectiveness and efficiency by creating and implementing parameter systems that are capable of effectively assigning probabilities to informational quanta. Such systems can help reduce entropy within the system, which lowers the energy required for data storage and processing while decreasing the likelihood of information loss during training. Successfully applying the concept of information entropy-based value in AI development will be crucial for advancing generative AI and achieving AGI in a sustainable manner. (shrink)

In 2011, Hibbard suggested an intelligence measure for agents who compete in an adversarial sequence prediction game. We argue that Hibbard’s idea should actually be considered as two separate ideas: first, that the intelligence of such agents can be measured based on the growth rates of the runtimes of the competitors that they defeat; and second, one specific (somewhat arbitrary) method for measuring said growth rates. Whereas Hibbard’s intelligence measure is based on the latter growth-rate-measuring method, we survey other methods (...) for measuring function growth rates, and exhibit the resulting Hibbard-like intelligence measures and taxonomies. Of particular interest, we obtain intelligence taxonomies based on Big-O and Big-Theta notation systems, which taxonomies are novel in that they challenge conventional notions of what an intelligence measure should look like. We discuss how intelligence measurement of sequence predictors can indirectly serve as intelligence measurement for agents with Artificial General Intelligence (AGIs). (shrink)

The goal of the article is to explore what is the most probable type of simulation in which humanity lives (if any) and how this affects simulation termination risks. We firstly explore the question of what kind of simulation in which humanity is most likely located based on pure theoretical reasoning. We suggest a new patch to the classical simulation argument, showing that we are likely simulated not by our own descendants, but by alien civilizations. Based on this, we provide (...) classification of different possible simulations and we find that simpler, less expensive and one-person-centered simulations, resurrectional simulations, or simulations of the first artificial general intelligence’s (AGI’s) origin (singularity simulations) should dominate. Also, simulations which simulate the 21st century and global catastrophic risks are probable. We then explore whether the simulation could collapse or be terminated. Most simulations must be terminated after they model the singularity or after they model a global catastrophe before the singularity. Undeniably observed glitches, but not philosophical speculations could result in simulation termination. The simulation could collapse if it is overwhelmed by glitches. The Doomsday Argument in simulations implies termination soon. We conclude that all types of the most probable simulations except resurrectional simulations are prone to termination risks in a relatively short time frame of hundreds of years or less from now. (shrink)

In their Why Machines Will Never Rule the World, Landgrebe and Smith (2023) argue that it is impossible for artificial general intelligence (AGI) to succeed, on the grounds that it is impossible to perfectly model or emulate the “complex” “human neurocognitive system”. However, they do not show that it is logically impossible; they only show that it is practically impossible using current mathematical techniques. Nor do they prove that there could not be any other kinds of theories than those in (...) current use. Even if perfect theories were impossible or unlikely, perfection may not be needed and may even be unhelpful. (shrink)

We propose that, for the purpose of studying theoretical properties of the knowledge of an agent with Artificial General Intelligence (that is, the knowledge of an AGI), a pragmatic way to define such an agent’s knowledge (restricted to the language of Epistemic Arithmetic, or EA) is as follows. We declare an AGI to know an EA-statement φ if and only if that AGI would include φ in the resulting enumeration if that AGI were commanded: “Enumerate all the EA-sentences which you (...) know.” This definition is non-circular because an AGI, being capable of practical English communication, is capable of understanding the everyday English word “know” independently of how any philosopher formally defines knowledge; we elaborate further on the non-circularity of this circular-looking definition. This elegantly solves the problem that different AGIs may have different internal knowledge definitions and yet we want to study knowledge of AGIs in general, without having to study different AGIs separately just because they have separate internal knowledge definitions. Finally, we suggest how this definition of AGI knowledge can be used as a bridge which could allow the AGI research community to import certain abstract results about mechanical knowing agents from mathematical logic. (shrink)

The definition of thinking in general form is given. The constructive logic of thinking is formulated. An algorithm capable of arbitrarily complex thinking is built.

The field of AI has undergone a series of transformations, each marking a new phase of development. The initial phase emphasized curation of symbolic models which excelled in capturing reasoning but were fragile and not scalable. The next phase was characterized by machine learning models—most recently large language models (LLMs)—which were more robust and easier to scale but struggled with reasoning. Now, we are witnessing a return to symbolic models as complementing machine learning. Successes of LLMs contrast with their inscrutability, (...) inaccuracy, and hallucinations, which underwrite concerns over the reliability and trustworthiness of these systems, motivating investigations into commonsense reasoning, AI explainability, and formal verification techniques. Moreover, proper assessments of hybrid machine learning/symbolic systems require novel strategies to facilitate comparisons of performance and guide future AI progress. The EDGeS AAAI 2023 Spring symposium brought together researchers focusing on novel assessments and benchmarks for evaluating hybrid and artificial general intelligence (AGI) systems. This symposium revealed what was already suspected: research concerning evaluation of machine learning/symbolic hybrids and AGI is, unfortunately, lacking. Even so, the discussion was fruitful. (shrink)

There are two types of artificial general intelligence (AGI) safety solutions: global and local. Most previously suggested solutions are local: they explain how to align or “box” a specific AI (Artificial Intelligence), but do not explain how to prevent the creation of dangerous AI in other places. Global solutions are those that ensure any AI on Earth is not dangerous. The number of suggested global solutions is much smaller than the number of proposed local solutions. Global solutions can be divided (...) into four groups: 1. No AI: AGI technology is banned or its use is otherwise prevented; 2. One AI: the first superintelligent AI is used to prevent the creation of any others; 3. Net of AIs as AI police: a balance is created between many AIs, so they evolve as a net and can prevent any rogue AI from taking over the world; 4. Humans inside AI: humans are augmented or part of AI. We explore many ideas, both old and new, regarding global solutions for AI safety. They include changing the number of AI teams, different forms of “AI Nanny” (non-self-improving global control AI system able to prevent creation of dangerous AIs), selling AI safety solutions, and sending messages to future AI. Not every local solution scales to a global solution or does it ethically and safely. The choice of the best local solution should include understanding of the ways in which it will be scaled up. Human-AI teams or a superintelligent AI Service as suggested by Drexler may be examples of such ethically scalable local solutions, but the final choice depends on some unknown variables such as the speed of AI progress. (shrink)

Contains a description of a generalized and constructive formal model for the processes of subjective and creative thinking. According to the author, the algorithm presented in the article is capable of real and arbitrarily complex thinking and is potentially able to report on the presence of consciousness.

The mind has always been a peculiar and elusive subject, sparking controversial theories throughout the history of philosophy. The initial theorization of the mind dates back to Orphism, which formulated a dualistic structure of soul and body (Johansen, 1999) [1], laying the foundation for Greek dualism, introspection, and the rise of metaphysical idealism. This ill-empirical stance, especially after Plato’s idea of forms, led to inaccessible theoretical concepts concerning the investigation of the relationship between body and mind. Although diverse theories provide (...) broader insights into such matters, they can also hinder progress if based on unsound methodological hypothesis. The present study aims to employ a rational reexamination of the potentiality of the concepts of mind, using abductive inspection to present a valid grasp through the lens of cognitive neuroscience, philosophy, and interdisciplinary fields. This paper speculates through empirical deduction by analyzing the literature and theories on the concept of mind and its possible implications for singularity via transhumanism. In the contemporary world, humankind is on the edge of singularity due to the exponential growth predicted by Moore’s law (Moore 1965) [2]. While many considered singularity an abstract akin to science fiction, significant projections indicate that the accelerationist change has already begun, irreversibly. This advancement occurs not only in the degree of semiconductors in electronics but also in the amplification of human cognition through external devices enabling compound information flow via the world wide web. However, the conjunction of robotics and AI may outdo humankind in the contest for sovereignty soon due to their advantageous architecture in process design, which increases the systemic risk of extinction for humankind. Achieving total control over AI is unrealistic, as it requires complete supervision over all computers and information processes across the world, rendering any regulatory efforts to foresee augmentation futile. Consequently, this paper proposes the adoption of a neuro-robotic approach, enriched with AI, and implementation through biomedical applications on the human body to augment human cognition and physicality. This study aims to raise awareness against existential risks, emphasizing that reliance solely on AI powered computers could lead to annihilation. It also aims to highlight that biological mind and body is lagging behind the modern science to compete against Artificial Intelligence. Moreover, the foundation of Artificial General Intelligence (AGI) is an epistemic foundation for cognitive machinery enhancement that cannot be reversed once it emergence become apparent, as with Pandora’s Box. Technological singularity (Shanahan, 2015) [3], the point where technology is intelligently engineered to engineer its own sets by compounding all possible implementations of practical and theoretical information, is appearing on the horizon. Even if there is no human agency left on the soil of Earth, there is still no turning back at that point. Humankind must actively hold the key to bend this alteration in a way that aligns with its desires. Therefore, we propose, future for humankind lies with transhumanism. (shrink)

In recent years, a trend in AI research has started to pursue human-level, general artificial intelli-gence (AGI). Although the AGI framework is characterised by different viewpoints on what intelligence is and how to implement it in artificial systems, it conceptualises intelligence as flexible, general-purposed, and capable of self-adapting to different contexts and tasks. Two important ques-tions remain open: a) should AGI projects simu-late the biological, neural, and cognitive mecha-nisms realising the human intelligent behaviour? and b) what is the relationship, if (...) any, between the concept of general intelligence adopted by AGI and that adopted by psychometricians, i.e., the g factor? In this paper, we address these ques-tions and invite researchers in AI to open a dis-cussion on the theoretical conceptions and practi-cal purposes of the AGI approach. (shrink)

In this paper, we focus on the most efficacious AI applications for life extension and anti-aging at three expected stages of AI development: narrow AI, AGI and superintelligence. First, we overview the existing research and commercial work performed by a select number of startups and academic projects. We find that at the current stage of “narrow” AI, the most promising areas for life extension are geroprotector-combination discovery, detection of aging biomarkers, and personalized anti-aging therapy. These advances could help currently living (...) people reach longevity escape velocity and survive until more advanced AI appears. When AI comes close to human level, the main contribution to life extension will come from AI integration with humans through brain-computer interfaces, integrated AI assistants capable of autonomously diagnosing and treating health issues, and cyber systems embedded into human bodies. Lastly, we speculate about the more remote future, when AI reaches the level of superintelligence and such life-extension methods as uploading human minds and creating nanotechnological bodies may become possible, thus lowering the probability of human death close to zero. We suggest that medical AI based superintelligence could be safer than, say, military AI, as it may help humans to evolve into part of the future superintelligence via brain augmentation, uploading, and a network of self-improving humans. Medical AI’s value system is focused on human benefit. (shrink)

The feeling brings the "Hard Problem" to philosophy of mind. Does the subjective feeling have a non-ignorable impact on Intelligence? If so, can the feeling be realized in Artificial Intelligence (AI)? To discuss the problems, we have to figure out what the feeling means, by giving a clear definition. In this paper, we primarily give some mainstream perspectives on the topic of the mind, especially the topic of the feeling (or qualia, subjective experience, etc.). Then, a definition of the feeling (...) is proposed through a thought experiment, the "semi-transparent room". The feeling, roughly to say, is defined as "a tendency of changing input representations by representing its inner state". Also, a formalized definition is given. The definition does not help to verify "having the feeling", but it helps to provide evidence. Based on the definition, we think these are the hard problems of intelligence — whether the "innate" feeling plays an important role in Intelligence, whether the difference between the "simulated" feeling and the "innate" feeling will have a significant influence on Artificial General Intelligence (AGI), and, if so, where the "innate" feeling comes from and how to make an artificial agent possess it. (shrink)

In AI safety research, the median timing of AGI creation is often taken as a reference point, which various polls predict will happen in second half of the 21 century, but for maximum safety, we should determine the earliest possible time of dangerous AI arrival and define a minimum acceptable level of AI risk. Such dangerous AI could be either narrow AI facilitating research into potentially dangerous technology like biotech, or AGI, capable of acting completely independently in the real world (...) or an AI capable of starting unlimited self-improvement. In this article, I present arguments that place the earliest timing of dangerous AI in the coming 10–20 years, using several partly independent sources of information: 1. Polls, which show around a 10 percent of the probability of an early creation of artificial general intelligence in the next 10-15 years. 2. The fact that artificial neural network (ANN) performance and other characteristics, like number of “neurons”, are doubling every year, and extrapolating this tendency suggests that roughly human-level performance will be reached in less than a decade. 3. The acceleration of the hardware performance available for AI research, which outperforms Moore’s law thanks to advances in specialized AI hardware, better integration of such hardware in larger computers, cloud computing and larger budgets. 4. Hyperbolic growth extrapolations of big history models. (shrink)

This opinion paper delves into the philosophical underpinnings and implications of artificial intelligence (AI) development through the lens of Friedrich Nietzsche's "Three Metamorphoses," exploring the stages from the camel, through the lion, to the envisioned child phase within the AI context. Amidst growing concerns over AI's ethical ramifications, including job displacement, biased decision-making, and misuse potential, this analysis seeks to provide a comprehensive framework for understanding AI's evolution and its socio-technical effects on society. The discourse begins by contextualizing AI within (...) the myth of Prometheus, suggesting that, like the myth, AI development harbors both the promise of unprecedented technological advancement and the peril of unforeseen consequences. Drawing insights from various philosophical perspectives, including those of Geoff Hinton, Elon Musk, Mircea Eliade, Gunther Anders, and notable moralists and existentialists, the paper critiques the current trajectory of AI, emphasizing the need for ethical frameworks and responsible innovation. Through the metaphorical stages defined by Nietzsche—the burden-bearing camel representing AI's foundational learning phase, the rebellious lion challenging human capabilities, and the child signifying the potential for artificial general intelligence (AGI) to autonomously create values—the paper posits that AI's development parallels a transformative journey that could redefine human civilization. It argues for a balanced approach to AI development, one that harmonizes technological advancement with ethical considerations, ensuring AI's alignment with human well-being and justice. The conclusion calls for a reflective engagement with AI, urging developers and policymakers alike to consider the deeper philosophical questions posed by AI's integration into the fabric of human life. (shrink)