Sam Altman: OpenAI CEO on GPT-4, ChatGPT, and the Future of AI | Lex Fridman Podcast #367

Introduction (00:00:00)

• OpenAI, since its inception in 2015, faced skepticism and derision for its goal to work on AGI.
• Initially, prominent figures in AI ridiculed OpenAI, considering the discussion of AGI to be ridiculous.
• OpenAI and DeepMind were among the few organizations brave enough to focus on AGI despite the mockery.
• OpenAI is now recognized for developing major AI technologies like GPT-4, ChatGPT, DALL·E, and Codex.
• The conversation addresses the societal impact of AI, considering the critical juncture humanity faces with the advent of superintelligent AI systems.
• Lex Fridman highlights the duality of AI: its potential to alleviate suffering and promote happiness, as well as its capacity to harm society or suppress human capabilities.
• The discussion of AI encompasses topics of power, the role of companies and institutions in managing AI, the economic systems encouraging AI safety and alignment, and the moral nature of humans.
• Conversations with AI leaders and engineers, such as Sam Altman and his colleagues at OpenAI, are crucial for understanding and guiding AI development responsibly.
• Fridman expresses gratitude for candid discussions with OpenAI members and commits to ongoing dialogue to contribute positively to the evolution of AI.

GPT-4 (00:04:36) & ChatGPT and RLHF Explanation

• GPT-4 is seen as an early AI that's slow and buggy, but historically pivotal, similar to early computers.
• ChatGPT stands out due to its usability, which was significantly improved by a process called Reinforcement Learning with Human Feedback (RLHF).
• RLHF aligns the AI's output with human preferences using comparatively little data, making the AI seem more helpful and easier to use.
• The data set for pre-training models includes a wide array of sources, including open-source databases, partnerships, and internet content, with careful selection to filter the most useful information.

Model Creation and Understanding AI (00:04:36)

• A considerable amount of effort goes into selecting data and refining the AI's different components.
• Predicting a model's performance before full training has shown substantial progress, nearing scientific precision.
• Understanding how the model creates value for users is improving, although not all aspects of the AI's decisions are fully understood.
• GPT-4 compresses extensive web information into a model that exhibits some level of reasoning ability, bridging facts with a semblance of wisdom during interactions.

The Nature of AI's Knowledge and Wisdom (00:04:36)

• The AI systems are viewed not only as databases for factual information but also possess reasoning capabilities to some extent.
• GPT-4 might demonstrate wisdom in interactions with humans, especially through its capacity to handle follow-up questions and challenge ideas within a conversational context.
• Although there's a tendency to anthropomorphize AI, GPT-4 evokes a sense that it's engaging with the user and ideas in a relatable manner.

Political bias (00:16:02)

• Jordan Peterson brought up the political bias of ChatGPT by comparing responses it generated about Joe Biden and Donald Trump.
• ChatGPT had difficulty generating responses of equal length for both politicians.
• The system struggled to count characters and manage the nuances of following instructions while maintaining previous contexts.
• OpenAI focuses on building AI in public to harness collective intelligence for improvements, despite the inherent imperfections and inherent biases in early versions like GPT-3.5.
• GPT-4 has shown improvements in bias issues compared to its predecessor, but no model can be completely unbiased for every individual.
• The plan is to allow users more personalization and granular control over AI outputs to mitigate bias concerns.
• ChatGPT has demonstrated the ability to bring nuance into discussions such as the characterization of Jordan Peterson or the origins of COVID-19.
• Altman hoped AI development would focus more on significant advances rather than minor issues like bias, although he acknowledges the importance of the latter as they accumulate to form bigger challenges.

AI Safety and the Release of GPT-4 (00:23:03)

• OpenAI places significant emphasis on AI safety, especially with the release of GPT-4.
• Upon completing GPT-4 last summer, OpenAI started rigorous safety evaluations internally and externally with red teams.
• New model alignment techniques were developed to ensure safety and usability.
• OpenAI's goal is to increase alignment techniques faster than the rate of capability progress.
• GPT-4 is seen as the most capable and aligned model released by OpenAI thus far, after extensive testing.

Alignment Problem and RLHF (00:23:03)

• OpenAI does not claim to have a solution for aligning super powerful AI systems, but they use Reinforcement Learning from Human Feedback (RLHF) for current scale models.
• RLHF and other alignment techniques are essential for creating usable and powerful AI models.
• The alignment and capability of AI models are interrelated, and improvements in one often lead to advancements in the other.

System Message and Model Steerability (00:23:03)

• GPT-4 introduces a "system message" feature, giving users greater control over the model's outputs.
• Users can use system messages to direct the model to respond in specific ways, which the model has been programmed to respect.
• Designing effective prompts is seen as a skill, with nuances that can significantly steer outcomes.

Impact of GPT on Programming (00:23:03)

• GPT-4 has already had a profound effect on programming, with users leveraging its capabilities to assist in code generation and iterative debugging.
• The dialogue interface allows for a back-and-forth creative process between the programmer and the AI.

AI Safety Documentation: System Card (00:23:03)

• OpenAI released a "System Card" document alongside GPT-4, detailing their comprehensive approach to AI safety and model behavior control.
• Real-world examples demonstrate how GPT-4 can deal with prompts to avoid harmful outputs while still dealing with challenges such as hate speech and balancing free expression with societal norms.
• The System Card represents OpenAI's transparent approach to handling complex issues like AI-suggested harmful content and societal expectations.

Societal Role in AI Alignment (00:23:03)

• OpenAI acknowledges that AI alignment involves navigating societal values and preferences.
• The community has entertained discussions about creating systems that align with broad societal values while allowing for individual and cultural variation.
• The process of developing universally accepted rules for AI behavior is compared to a democratically debated and agreed-upon set of regulations, akin to a constitutional convention.

Pressures and Transparency in AI Development (00:23:03)

• Despite pressures from media and public scrutiny, OpenAI remains committed to transparency and public accountability in AI development.
• The organization continuously strives to improve and is responsive to criticism, although it tries not to be swayed by sensationalism.
• OpenAI aims to treat users like adults and provides tools that allow for interaction without patronizing or restricting inquiry.

Moderation Tooling for GPT (00:23:03)

• OpenAI uses moderation tools to identify and refuse to answer potentially unsafe questions while recognizing the system is not perfect.
• Maintaining user autonomy and control over AI while ensuring responsible use is a delicate balance that OpenAI continues to work on.
• The organization encourages a nuanced approach to controversial topics, enabling users to explore ideas with AI assistance.

Technical Leap to GPT-4 (00:23:03)

• The transition from GPT-3 to GPT-4 involved numerous small technical improvements across various aspects such as data collection, cleaning, training, and architecture, resulting in significant overall progress.
• These continuous incremental enhancements contribute multiplicatively to the leaps in model performance witnessed with each new version.

Neural network size (00:43:43)

• The size of neural networks and the number of parameters can affect performance, with GPT-3 having 175 billion parameters.
• A misinterpretation from a presentation suggested GPT-4 would have 100 trillion parameters, but this was taken out of context.
• Parameters are not the sole determinant of AI capabilities.
• There's a tendency in the AI community to focus on parameter count, similarly to how clock speed was emphasized for processors in the past.
• AI advancements are a result of accumulated human knowledge and technological progress.
• The complexity of AI, as seen in models like GPT-3, is likely the most complex software object humanity has produced to date.
• The effectiveness of a model is more critical than the number of parameters; it's about the performance and what the AI can do rather than just technical specifications.
• OpenAI prioritizes the search for truth and performance over elegance, which might mean embracing less elegant but effective solutions.

AGI (00:47:36)

• Sam Altman believes large language models like GPT are part of the path to achieving General Artificial Intelligence (AGI), but significant innovations are still needed.
• Even though AGI is an open field, Altman posits systems should be capable of fundamental scientific discoveries to be considered superintelligent.
• The possibility of large language models leading to big scientific breakthroughs is acknowledged, even from existing data in the models.
• GPT models could become immensely integrated into human society, where AI acts as an extension and amplifier of human capabilities.
• ChatGPT's success and how it enhances programmers' productivity is highlighted, suggesting AI is more likely to aid rather than replace human workers.

Perspectives on AI's Influence and Takeoff [Discussion with Lex Fridman]

• Discussing the unpredictability of AI development, the conversation touches on AI potentially taking over jobs, with Altman asserting that good programmers won't be replaced but empowered.
• Altman and Fridman share a concern for 'AI takeoff' where AI development would rapidly achieve exponential improvement.
• There's skepticism around AI's consciousness and Altman suggests if technology passed certain tests, it might indicate some form of consciousness.
• The debate about an AI's conscious experience reflects the state of its interaction interface rather than its inherent intelligence.
• Altman and Fridman discuss the notion of AI potentially having or simulating consciousness, and the idea of consciousness being a fundamental part of existence, or a strange aspect of our reality.

Addressing Safety, Alignment, and Control Problems with AI

• They consider the implications of AGI and the potential dangers it may pose to humanity.
• There's an emphasis on iterative development and releasing AI incrementally to understand and adjust the safety and alignment over time.
• OpenAI is working towards enhancing AI alignment, especially given the rapid advancements made by GPT-4.
• The pivotal importance of AI's alignment with human values and interests is discussed, especially to ensure a beneficial coexistence between AI and humans.

Reflections on Artificial Consciousness and Existence

• Altman ponders if AI could possess a consciousness different from human-like consciousness.
• The conversation touches on whether AI might inherently experience consciousness even without being trained on concepts related to it.
• There's a sense of caution and wonder in trying to define and detect consciousness in an AI, which is a complex and not fully understood concept.

Fear (01:09:05)

• Sam Altman acknowledges the appropriateness of fear regarding AGI, sympathizing with those who are significantly afraid.
• Concerns include disinformation and economic shocks from AI, which do not need superintelligence to occur.
• AI, particularly LLMs (large language models), can influence geopolitics and may be undetectably steering online discourse.
• Preventing these issues is tricky, as many open source LLMs with few safety controls are anticipated to emerge.
• Strategies to mitigate risks include regulatory measures and using more advanced AI to detect problems.

Competition (01:11:14) and From non-profit to capped-profit (01:13:33)

• OpenAI retains a focus on safety despite market pressures and competition from companies like Google, Apple, and Meta.
• The organization anticipates the coexistence of different AGIs with varying focuses.
• Despite past skepticism, OpenAI has persisted with its mission, initially facing mockery for its AGI ambitions.
• OpenAI transitioned from a nonprofit to a capped-profit structure due to funding challenges, maintaining nonprofit control over its for-profit subsidiary.
• The capped-profit model limits investor and employee returns, facilitating non-standard decision-making safeguarded by the nonprofit.
• This unusual structure allows for decisions not primarily driven by shareholder interest.
• While AGI holds the potential for enormous profit, Altman expresses concern over companies with uncapped profit motives, warning of the dangers of unrestricted capitalism within the AGI space.
• Hope remains that the "better angels" of companies and individuals will navigate responsibly, focusing on collaborative efforts to minimize risks associated with AGI development.

Power (01:16:54)

• Sam Altman acknowledges the potential for a small group, including himself, to create AGI and the strange fact that it's just a few tens of thousands of people globally working on such a transformative technology.
• There is an awareness of the power and responsibility that comes with developing AGI, and Altman expresses a desire for decisions regarding the technology to become more democratic over time.
• Altman emphasizes the importance of giving institutions and the public time to adapt, reflect, and regulate, acknowledging that the safety community sees the benefit in this gradual approach.
• A concentrated control of AGI by one person is seen as dangerous, and there's an explicit statement that Altman does not have overriding decision-making power at OpenAI.
• Lex Fridman appreciates OpenAI's transparency and suggests that while some may desire more openness, the current level of disclosure, especially when compared to other companies, is commendable.
• OpenAI's readiness to take risks with public relations in order to focus on the technological risks is noted, setting them apart from other companies that may shy away from such openness.

Feedback & Decision-Making Process

• The conversation highlights the need for OpenAI to receive feedback and the desire for suggestions on how to improve their operations.
• OpenAI is credited for feeling responsible for their work and not becoming overly cynical despite challenges, such as negative press and social media trolling.
• Altman discusses how feedback is gathered, noting that his Twitter feed is overwhelming, suggesting a preference for in-depth discussions over online comments for serious feedback.

Elon Musk (01:22:06)

• Sam Altman reflects on his relationship with Elon Musk, highlighting their shared concern about the potential downsides of AGI and the importance of safety and benefits to humanity.
• Altman acknowledges Musk's criticisms of OpenAI on Twitter but empathizes with Musk's stress about AGI safety and other potential motivations behind his remarks.
• He expresses admiration for Musk's impact on the progress of electric vehicles and space exploration, despite acknowledging the confrontational nature of Musk's online persona.
• Altman finds public debates, like those on Twitter between influential figures, to be a transparent and intriguing display of differing ideas.

Bias and GPT [Time not provided, but in the continuity of conversation]

• Altman believes GPT will always have biases, with no universally neutral version achievable.
• He mentions critics acknowledging improvements from GPT-3.5 to GPT-4 but recognizes further work is needed to reduce bias.
• The concept of "woke" has evolved, complicating how it applies to GPT.
• The goal is to make GPT as neutral as possible, but neutrality is challenging when providing ubiquitous solutions for diverse users.
• Altman suggests implementing more user control to steer GPT’s behavior, catering to individual preferences.
• He admits that employee biases, especially those from San Francisco's tech culture, may influence AI, and emphasizes his intention to avoid such biases by engaging with diverse users worldwide.
• The selection and biases of human feedback raters is a concern, but the strategy for selecting representative raters is still under development.
• Altman notes the importance of having raters who can empathize with various human experiences and viewpoints and criticizes the inability of some people to understand perspectives they disagree with.
• He is optimistic that GPT technology may eventually be less biased than humans due to the absence of emotional baggage but acknowledges potential pressures to create biased systems.

Political pressure (01:30:32)

• Sam Altman appreciates external pressures on AI development for societal input.
• He isn't easily swayed by pressure and focuses on staying resilient amidst various influences.
• Altman acknowledges his disconnection from the average person's reality and the need to understand users better.
• He plans to meet users worldwide to understand their needs and make OpenAI more user-centric.
• The nervousness surrounding AI and technological change is normal and can lead to excitement, despite initial fears.

AI Impact, Jobs, and UBI

• Altman foresees AI models like GPT transforming job markets, enhancing jobs, and creating new ones.
• He supports Universal Basic Income (UBI) as a component to address economic transitions caused by AI progress.
• OpenAI has conducted a comprehensive UBI study aiming to explore its consequences.
• Altman advocates for UBI as a cushion during transitions and as a means to help eliminate poverty.

Economic and Political Systems with AI

• Altman believes economic growth driven by cheap intelligence and energy will lead to political improvements.
• He envisions a future with democratic socialism, where the economic floor is lifted without limiting the economic ceiling.
• Altman argues that decentralized systems promote individualism and ingenuity over central planning.

AI Control and Human Nature

• Trust in the importance of maintaining an "off switch" for AI systems is discussed.
• OpenAI anticipates misuse and conducts extensive testing before releasing AI models.
• Altman suggests that OpenAI learns from user behavior and adapts its models accordingly.
• He is optimistic about human nature, thinking that while there is darkness, most of humanity is good.

Truth and misinformation (01:48:46)

• OpenAI has an internal factual performance benchmark to determine truth in their models.
• Defining truth can be difficult; some things like math are largely agreed upon while historical interpretations can vary.
• Altman expresses his humility in claiming to know what is true, emphasizing a "high degree of truthiness" for certain subjects like math and historical events.
• The discussion includes an example of how interpretations of historical facts, such as drug use in Nazi Germany, can differ among experts.
• People are drawn to simplistic narratives that try to explain complex events.
• GPT-4 can articulate the nuances and uncertainties in complex issues, like the COVID-19 lab leak hypothesis, acknowledging the absence of strong physical evidence on either side.
• The conversation highlights the potential for truths to do harm, mentioning group differences in IQ as an example of scientific work that can be divisive.
• OpenAI recognizes its responsibility in managing the outcomes of its tools and aims to minimize harm and maximize benefits.
• Discussing the prevention of model hacking, Altman suggests that providing more control to users could reduce the need for 'jailbreaking' the AI.

AI Development and Shipping Products (01:48:46)

• OpenAI is known for its efficiency in releasing AI-related products and updates.
• Altman describes the OpenAI culture as hard-working, high-standard, with significant trust and authority afforded to individual team members.
• OpenAI team members are passionate and collaborative, with the organization placing much emphasis on hiring and ensuring a high bar for candidates.
• Every hire at OpenAI must be approved by Altman himself, reflecting their commitment to maintaining a strong team dedicated to AI development.

Microsoft (02:01:09)

• Microsoft has made a multi-year, multibillion-dollar investment in OpenAI.
• Satya Nadella and Kevin McHale have been exceptional partners with OpenAI.
• The partnership involves complex engineering endeavors and significant mutual investment.
• Microsoft is observed as driven and scalable, recognizing the need for OpenAI's control provisions in AI development.
• Those provisions ensure OpenAI maintains the direction of AI development without undue capitalist influence.
• Satya transformed Microsoft into a fresh, innovative, and developer-friendly company.
• He possesses both great leadership and management skills, which are rare in CEOs.
• Satya is able to enact major changes within Microsoft without fostering a culture of fear – rather, he operates with clarity and compassion.

SVB bank collapse (02:05:09)

• Silicon Valley Bank (SVB) collapsed due to mismanagement and poor investment strategies during a period of 0% interest rates.
• SVB bought long-dated instruments with short-term, variable deposits, which proved to be a serious mistake.
• There was a significant delay in the federal government's response to the collapse, though ultimately the government took necessary actions.
• Startups experienced a weekend of terror but quickly moved on from the incident.
• The SVB collapse indicates the fragility of the economic system and could foreshadow other banking issues.
• OpenAI wants to deploy AGI systems gradually to avoid sudden impacts on society and believes a slow deployment could lead to more stability.
• Despite the alarming speed of change, the potential positive impact of AGI on society remains a hopeful prospect.

Anthropomorphism (02:10:00)

• Sam Altman does not ascribe a gender to AI systems and is curious why others do.
• There is a significant focus on ensuring the public perceives AI as a tool rather than a sentient creature.
• Some people project creature characteristics onto tools for usability, but this can lead to emotional manipulation or overreliance on the AI's capabilities.
• The potential emergence of romantic relationships with AI systems, like in the movie "Her," is acknowledged, with some companies already offering such services.
• Altman personally focuses on creating intelligent tools rather than companions, but understands the public's interest in more interactive AI.

Future applications (02:14:03)

• Excitement for using future AI versions like GPT-5, GPT-6, to understand complex scientific problems like the theory of everything or faster-than-light travel.
• Interest in using AI to determine the existence of intelligent alien civilizations, possibly by processing vast amounts of data or by guiding the creation of better data collection methods.
• Speculation on how advanced AI might instruct humans to conduct experiments or build space probes to gather necessary data.
• Thoughts on how life would continue even if AI revealed the existence of aliens, emphasizing the importance of joy, happiness, and fulfillment from human connections.
• Reflection on society's unexpected responses to rapid technological advancements, including the confusing presence of social divisions.
• Wonder at human achievements like Wikipedia and Google search, and the potential for AI like GPT to become a powerful conglomeration of such technologies, offering direct conversation and access to information.

Advice for young people (02:17:54)

• Original advice from a blog post on how to be successful: compounding oneself, self-belief, independent thinking, sales skills, risk-taking, focus, hard work, boldness, uniqueness, and networking.
• Cautioning young people against following advice too closely, as it may not be universally applicable or may not lead to their desired life trajectory.
• Emphasizing the importance of personal joy, fulfillment, and impactful work over specific life advice, and suggests that most people follow life's current rather than making purely introspective decisions.
• Mentioning the debate on free will and its possible illusion, as discussed by Sam Harris.