Model routing has emerged as a way to keep AI inferencing costs down by examining prompts and directing them to the most appropriate model — for example, the cheapest one able to respond effectively. This means that requests can be handled more efficiently, giving more accurate results and lower runtime costs, a concern in an era of rising AI expenditure.
Nvidia sees the potential for model routing and has just launched NeMo Switchyard, its take on the technology. Switchyard provides a library for applying multiple routing approaches, enabling developers to apply a system-of-models approach and build more efficient, controllable agents.
Interest in model routing is growing — both as a tool and as an investment: Cloudflare recently introduced a model router as part of a new suite of enterprise AI tools, while payment services company Stripe is looking to buy OpenRouter, according to The Wall Street Journal.
