The state of modern answer engines, AI demos, and more! | E1924
06 Apr 2024 (5 months ago)
Sunny Madra joins Jason
- Taylor Swift's music is discussed, with opinions on whether she is overrated or talented.
- A new AI demo is showcased, which can generate videos, audio, and scripts in a short amount of time.
- Vanta, Eight Sleep, and Equinix Startup Program are mentioned as sponsors.
- Groq Cloud, a company that provides fast and cheap inference, is discussed.
- Groq Cloud currently has 70,000 developers and 20,000 apps using its platform.
Deep Dive into the cost of running queries and the decrease in the price of hardware storage
- The cost of running an average AI language model for inference is a fraction of a penny per query.
- The most popular AI language model, Mixol, charges 27 cents for input and 28 cents for output.
- The cost of running the best open-source AI language model, Mixol M, is about 27 cents per million tokens, while Google's PaLM 2 costs $10 per million tokens for input and $30 per million tokens for output.
- Google currently makes a revenue of $0.0161 per query and has an operating income of $55 billion, while the cost per query for ChatGPT would be 50% more.
- The cost of running AI language models has been decreasing rapidly, with the cost of running GPT-3.5 dropping by 100x in the past year.
- Open-source models and cost reductions are enabling companies like Perplexity to run AI language models at scale with 50 million active users.
- The cost of hardware and storage has significantly decreased, making it possible for individuals and companies to offer services for free that were previously expensive.
Vanta - Get $1000 off your SOC 2
- Vanta helps companies achieve and maintain SOC 2 compliance, which is essential for businesses that store customer data in the cloud.
- Vanta can save companies hundreds of hours of work and up to 85% on compliance costs.
- Vanta also automates compliance for GDPR, HIPAA, and more.
The LMSYS Chatbot Arena Leaderboard
- The LMSYS Chatbot Arena Leaderboard is considered the gold standard for chatbot experience.
- Last week, Claude 3 Opus by Anthropic passed over ChatGPT for 1106 preview and 0125 preview.
- Proprietary models still dominate the top rankings, but open-source models are starting to make progress.
- The top-ranked models are likely trained with data sets that are not publicly available and may involve more human involvement for reinforcement learning.
- No one has an insurmountable lead in the chatbot race.
Modern Answer Engines: You.com and Perplexity
- U.com is a modern answer engine that provides citations for its answers, while Perplexity AI uses a search API and considers the temporal nature of questions.
- The next generation of answer engines should focus on the path to the answer rather than just the presentation layer.
- Google's search results are falling behind in presentation and user experience compared to newer AI-powered search engines.
- Alternative search engines must provide a significantly better user experience than Google's to succeed, which is challenging to achieve.
- Google's featured snippets raise concerns about intercepting users from visiting original sources, while alternative search engines like Perplexity demonstrate better presentation.
- Real-time data remains a challenge for alternative search engines, while Google excels in providing accurate information for such queries.
- Modern answer engines like ChatGPT can provide precise answers by analyzing entire web pages, making them more efficient than Google in certain scenarios.
- Summarizing articles without permission may raise copyright concerns, especially if the summary includes a substantial portion of the original content.
- The legality of summarizing articles without permission is uncertain and depends on factors like the summary's length and whether it drives traffic to the original source.
- The effectiveness of these answer engines compared to Google depends on the query type. Google may still be preferred for specific tasks like flight searches or news articles.
Eight Sleep - Go to for $200 off the Pod plus free shipping
- Eight Sleep mattresses and Pod covers are advertised as the secret to success through quality sleep.
- The Eight Sleep Pod cover transforms any mattress with Dual Zone Heating and Cooling, allowing couples with different temperature preferences to sleep comfortably.
- Advanced sleep tracking helps monitor metrics like heart rate, enabling users to manage their sleep.
- The Eight Sleep Pod cover fits any mattress and maximizes sleep potential.
- Call to action: Visit eightsleep.com/twist to get $200 off the Pod cover and free shipping.
Potential and challenges of new search engines
- Answer engines like u.com and perplexity.ai are currently graded as B, but they lack accurate dates and sometimes present misleading information.
- Language models can consider multiple factors and provide comprehensive answers to complex queries, such as flight costs or playoff odds.
- To test answer engines' effectiveness, try the query: "How much is a business class flight between San Francisco and Dubai?"
- Finding the best odds for the Warriors making the NBA playoffs involves finding a website with a well-formatted table and a relevant domain name.
- AI-powered answer engines have the potential to change how people search for information, as people prefer quick and direct answers.
- Accuracy of information is crucial, and obtaining data from reliable sources is essential.
- Establishing data source relationships and licensing agreements with reputable sources is important for providing accurate information.
Equinix - Join the Equinix Startup Program for up to $100K in credits and much more
- Equinix Metal provides direct access to physical servers with the benefits of the cloud.
- No need to rack and stack servers.
- Equinix provides OnDemand infrastructure in over 25 major cities.
- The Equinix Startup Program offers personalized consultations, guidance, and up to $100,000 in startup credits.
- Apply at equinix startups.com and James from Equinix will reach out directly.
Future of SEO and data sources in the context of new search engines
- Answer engines like Perplexity and You.com are intercepting traffic and displaying ads without clear labeling, potentially making the SEO game obsolete.
- Perplexity and You.com dynamically create pages using AI models and scraped content, raising legal concerns if used for advertising purposes.
- The future of search involves a shift towards chat and voice interfaces, with Meta integrating AI into its smart glasses and various apps and browsers incorporating search engines.
- Users want the ability to customize search algorithms and exclude specific news sources.
- Perplexity and the first AI demo received grades of B and C, respectively, with room for improvement in accuracy and reducing hallucinated responses.
- It is becoming increasingly difficult to displace Google as the dominant search engine, and it is likely that Google will simply add the features offered by these new answer engines as needed.
Sunny demos Infinity AI
- Infinity AI is a text-to-video generation tool that can create short video clips based on text prompts. It uses Discord's chat infrastructure for easy integration and community engagement.
- A demonstration of Infinity AI showed it creating a 9-second video clip based on a prompt about a podcast discussion on Taylor Swift. The AI-generated video included video, audio, and a script.
- The creators of Infinity AI estimate that it will be able to generate 90-second videos in six months, 9-minute videos in a year, and 90-minute videos in 18 months.
- Infinity AI received a B+ rating for its current capabilities, with the potential for an A- rating if it can generate longer videos.
- The response time for Infinity AI to generate the 9-second video was around three to four minutes, suggesting fast processing and minimal human intervention.
- The training data used by AI answer engines like the one demonstrated in the video is a topic of discussion. It is unlikely that the AI engine performed a live web search to gather information for its response.
- Podcast transcripts are now commonly available on platforms like Apple and Spotify, making it easier for AI engines to access information for generating responses.
Sunny and Jason discuss Hume AI’s Empathic Voice Interface
- AI models can understand human facial expressions and emotional tones, enabling personalized interactions in therapy, companionship, customer support, and sales performance analysis.
- Ethical considerations arise with the use of AI, particularly in relation to potential misuse for malicious purposes.
- AI models can recognize when a person has finished speaking, reducing latency in conversations and making interactions more efficient.
- Voice-based interfaces benefit from AI models by providing real-time responses and understanding user intent more accurately.
- AI-powered answer engines like PaLM 2 have the potential to subtly influence people's political views and even radicalize them by exploiting psychological vulnerabilities.
- There are concerns about the misuse of AI in gambling and other addictive behaviors.
- PaLM 2's interactive and accurate responses can significantly improve product experiences.
- AI-powered technology has the potential to drive deflation and enable society to focus on more meaningful pursuits.