What's out there beyond text?
Specialized models, image and voice tools, real-time agents, and agents that use a browser. A tour of what ships beyond chat.
The path.
- Specialized Models Beyond the general-purpose flagships, each modality and job has its own leaders, and the "best" one shifts constantly, so treat these as categories to shop in with current examples.
- Multimodal Creative Tools Image, video, and audio generation. Quality is now high across the board. Choose by control, licensing, and API availability more than raw "wow."
- Voice & Real-Time Agents Real-time voice agents = STT (speech-to-text) → LLM → TTS (text-to-speech) stitched into a low-latency loop (or, increasingly, a single speech-to-speech model).
- Browser & Computer-Use Agents These agents perceive a screen or DOM and act (click, type, scroll, navigate) to complete tasks a human would do in a browser or OS.
Check yourself.
One question per step. Take it cold to find where to start, or after reading to see what stuck. Nobody's grading you.
Best so far:
Question 1 of 4
-
How should you pick an embedding model for your RAG system?
The chapter says to pick on retrieval quality for your domain via a small eval, and only then weigh dimensions and max input length. General rankings don't guarantee performance on your data.
Covered in step 1 · Specialized Models -
Image and video generation quality is now high everywhere. What should drive tool choice for a product?
With quality high across the board, the chapter says to choose by control, licensing, and API availability rather than raw "wow", and to confirm commercial-use rights per tool and tier before shipping generated media.
Covered in step 2 · Multimodal Creative Tools -
What matters most for making a voice agent feel natural in conversation?
Conversational agents feel natural under about 800ms round-trip, which is why speech-to-speech models that cut the separate STT and TTS hops are gaining ground.
Covered in step 3 · Voice & Real-Time Agents -
Why shouldn't a browser agent get standing access to your email or banking?
Prompt injection is the core threat: a malicious page can hijack an agent that reads it, so sensitive access requires human confirmation on each action.
Covered in step 4 · Browser & Computer-Use Agents