Google Project Astra Hands-On: Full of Potential, But It’s Going to Be a While

At I/O 2024, Google teased Project Astra, offering a glimpse into the future of AI assistants. This multi-modal feature integrates Gemini's intelligence with Google Lens's image recognition and advanced natural language responses. While the promo video was impressive, our hands-on experience revealed that Astra still has a long way to go before it's ready for everyday use. Here are three key takeaways from our first encounter with Google’s next-gen AI.

Sam’s Take:

Most people currently interact with digital assistants using their voice, so Astra's multi-modality—utilizing sight and sound along with text and speech—is relatively novel. This approach aims to make AI function more like a real assistant rather than a robot that just follows spoken commands.

During our demo, we asked Astra to tell a story based on objects placed in front of the camera. It created a charming tale about a dinosaur and its trusty baguette escaping an ominous red light. While entertaining, it didn't showcase the all-knowing assistant from Google's teaser. Beyond amusing a child with an original bedtime story, Astra's utility seemed limited.

My colleague Karissa then drew a bucolic scene on a touchscreen, and Astra correctly identified the flower and sun she painted. The most engaging part of the demo was when we used Astra on a Pixel 8 Pro. It tracked and remembered the locations of various objects, even recognizing my clothing and where I had stashed my sunglasses, which weren’t part of the initial demo.

This demo highlighted both the potential and limitations of AI. The ability to tell you where you left your keys or how many apples are in your fruit bowl could save time. However, speaking with Astra's researchers, it became clear that significant hurdles remain.

Astra still relies on cloud support rather than running entirely on-device. While it can maintain object permanence, this memory only lasts for a single session of a few minutes. Persistent memory would require addressing storage and latency issues, as recalling numerous objects could slow down the AI, making interactions less smooth. Despite its promise, Astra is still far from offering full-feature functionality.

Karissa’s Take:

Of all the generative AI advancements, multimodal AI intrigues me the most. While iterative updates to text-based chatbots are powerful, they don't excite me as much as AI that can recognize and respond to real-time surroundings, evoking a sci-fi vibe. This type of AI could revolutionize devices like smart glasses.

Google hinted at this with Project Astra, which may include a glasses component in the future. For now, the glasses shown during the I/O keynote were merely a research prototype. In person, Astra didn’t quite feel like a sci-fi creation.

Astra could accurately recognize objects around the room and respond to nuanced questions, like identifying appropriate toys for a 2-year-old. It also recognized my doodle and created stories about different toys.

However, Astra’s capabilities seemed on par with what Meta has already achieved with its smart glasses, which can recognize surroundings and perform creative tasks. Although Meta's features are experimental, they are more broadly available.

Astra’s standout feature is its built-in memory. After scanning objects, it could remember their locations for a short period. Researchers indicated that this memory could be extended, making Astra more like an actual assistant. Remembering where I left my glasses last night, rather than 30 seconds ago, would bring the sci-fi dream closer to reality.

Yet, the most exciting possibilities of generative AI are still on the horizon. While Astra has potential, Google has significant work to do before it becomes the futuristic assistant we envision.

https://www.readawrite.com/a/72808a2cb373d1985c56bce22572c03c
https://www.readawrite.com/a/4311b9734d8a2c11f1e4ba7e98885433
https://kingdomoftheapessubthai.raiselysite.com
https://kingdomoftheapes2024thaihd.raiselysite.com
https://community.mongabay.com/news/kingdomoftheapes2024thaistory
https://community.mongabay.com/news/2024-hd-kingdom-of-the-planet-of-the-apes
https://muluznews.blogspot.com/2024/05/google-project-astra-hands-on-mucho.html
https://cretakorslet.unicornplatform.page/blog/google-project-astra-hands-on-mucho-potencial-pero-aun-falta-tiempo