December 21: A Peek into the Next Year 🔮

🎯 Objective:

Explore AI agents, multimodal interactions, and new features coming in the near future by experimenting with real-time live-streaming AI.

🛠️ Tools:

• Google AI Studio (Free) - Google AI Studio
• ChatGPT Plus with Advanced Voice Mode ($20/month) - OpenAI ChatGPT
• Google Project Astra (Experimental, waitlist for now)

📝 Challenge:

Today's challenge is to explore the real-time multimodal capabilities of AI models. These AI tools can now “see” using your webcam/phone camera, hear you using your device’s microphone, and respond in real time. They are also capable of responding in various modalities including text, audio, and images. Here's what you'll do:

Important Note: You'll need to use a personal Gmail account as BC Google Workspace accounts cannot access AI Studio at this time. While you can't use your BC email, you can still explore the tools using any personal Google account.

1. Explore Google AI Studio

   • Visit Google AI Studio and explore their free “Stream Realtime” offerings.
   • Engage in a short interaction using their multimodal tools. Ideas to try:
     • Ask the AI to describe an object you show via your camera.
     • Use your webcam to have the AI recognize handwritten notes or sketches.
     • Get help with something on your computer such as learning a new software interface.
     • Share your screen and have it check why your students can't see content on Canvas

2. Try OpenAI's Real-Time AI Streaming

   • If you have ChatGPT Plus, try the advanced voice mode on mobile or desktop.
   • Talk to ChatGPT and ask it to help you complete a live task or talk through situations, such as:
     • Talk through an idea, get help with planning an event, or just ask about something you’re interested in using voice rather than text input
     • Helping you troubleshoot a simple issue by showing it to the camera or simply using your voice and get step by step instructions as you go
     • Get winter plant care tips by sharing your camera on mobile and showing your indoor plants to ChatGPT

Understanding AI Agents 🤖

If I could look into the crystal ball and predict one thing about 2025, it would be that you're going to hear A LOT about AI agents. Think of AI agents as your personal digital assistants that can take action on your behalf. Unlike regular chatbots that just respond to questions, agents can:

1. Watch and Learn: They observe how you work and understand your preferences
2. Take Initiative: They can perform actions or complete tasks without constant direction
3. Work Autonomously: They can coordinate with other tools and services

Learn more about AI Agents

A Few Real-World Examples:

• Research Agents: Find, summarize, and organize information (Google's Deep Research)
• Coding Agents: Help write and deploy code
  • Devin AI
  • Replit AI
  • Cursor Composer
• Anthropic: Computer Use and MCP

What's Already Possible Now and/or Coming Soon:

• Multi-app agents that complete complex tasks
• AI travel agents for booking flights and planning trips
• Visual agents for shopping, cooking, or home repairs
• Computer-access agents that perform tasks on your behalf
• Scheduling Agents that can manage calendars and set up meetings

💡 Tips for Success:

• Start Small: If you're unsure about live interactions, begin with a simple demo video or short provided below.
• Reflect: How did the AI's ability to see, hear, and respond in real time change the experience?
• Privacy First: If you're uncomfortable sharing your screen or camera, watch demo videos to understand the capabilities.

🔒 Privacy & Security Considerations

Before engaging with multimodal AI tools, consider these important points:

Environment Check:

• What's visible in your camera's view?

• What's visible on your screen if sharing?

• Who might overhear your audio interactions?

• Are any sensitive documents or personal information visible?

Data Awareness:

• Remember that your interactions may be used to improve the AI systems

• Consider what information you're comfortable sharing

• Be mindful of institutional or workplace privacy policies

• Think about student/client confidentiality if applicable

Practical Tips:

• Clear your desk of sensitive materials before screen sharing

• Use background blur when possible for video

• Test tools with non-sensitive content first

• Preview what you're sharing before starting

Setting Boundaries:

• Decide which tasks are appropriate for AI assistance

• Determine your comfort level with different types of interaction

• Know how to quickly stop sharing if needed

• Keep control over what and when you share

📽️ Demo & Discussion Videos

• Project Astra: Exploring a Universal AI Assistant with Greg Wayne
• Project Mariner | Google’s AI Agent
• Gemini 2.0 Flash: Multimodal live streaming
• OpenAI advanced voice mode with video
• Claude computer use demo
• Claude computer use demo 2

✨ Share Your Insights:

• Share what you learned or a fun insight with colleagues
• Log your activity for today so others can see!