Fly.io·2yKubernetes without nodes
Fly Kubernetes allows for a node-less Kubernetes setup using a combination of a trimmed-down Kubernetes version called k3s for the control plane and Virtual kuet, which emulates traditional kubelet behavior. Instead of worker nodes, Fly uses virtual nodes and its own machines API to manage pods and orchestration. This decentralized approach offloads scheduling decisions to Fly’s own orchestrator, leveraging high availability and self-managed physical machines.
Fly.io·2yDo you REALLY need Kubernetes?
Kubernetes is not always necessary for startups, and they often reach for it prematurely due to misconceptions. Orchestration, fundamentally about managing multiple containers across physical machines, can be achieved without Kubernetes. Kubernetes offers benefits like a common, cloud-agnostic language and declarative management, making sense for complex, large-scale applications. However, simpler alternatives can handle orchestration tasks, and startups should carefully consider their real needs before adopting Kubernetes.
Fly.io·2yAI GPU Clusters, From Your Laptop, With Livebook
Livebook, FLAME, and the Nx stack are powerful components in the Elixir ecosystem that enhance data computation and serverless computing. Livebook integrates closely with Elixir app clusters for seamless local and remote compute, while FLAME manages serverless executors for scalable application performance. The Nx stack supports Elixir-native AI and ML, allowing complex computations with GPU backends. Recent demonstrations at ElixirConf 2024 highlighted their capabilities, including real-time hyperparameter tuning on remote GPUs, made possible by Fly.io's infrastructure.
Fly.io·3yGPUs on Fly.io are open for everyone!
Fly.io has made GPUs available to everyone, allowing users to spin up GPU instances for tasks such as large language model inferencing and image generation. The available GPU types are Ampere A100 (40GB), Ampere A100 (80GB), and Lovelace L40s (48GB). The post also mentions that lower-cost A10 GPUs will be introduced in the future. Additional articles related to using GPUs on Fly.io are provided for further exploration.
Fly.io·2yWe use containers now. Here's why.
Fly.io now supports running containers inside fly machines, a shift from their previous method of unpacking Docker images directly onto lightweight virtual machines. This change, driven by the need for sidecar containers in their managed Kubernetes service, improves flexibility and allows for tighter coupling of services. Fly machines use a combination of Firecracker hypervisor and the pilot init process to manage containers, making it possible to deploy and manage multiple images separately within a single virtual machine.