Gofr v1.60.0 introduces opt-in Trie Router and LLM concurrency limits

The release of gofr-dev/gofr v1.60.0, as described in its GitHub release notes, introduces several notable features and fixes. Among the new features is an opt-in Trie Router, activated by setting GOFR_ROUTER=trie. This replaces the default mux's linear route scanning with a radix trie, offering significant performance improvements for route matching, particularly as the number of routes increases. However, routes that cannot be safely indexed by the trie fallback to the mux, which introduces latency instead of resulting in a 404 error. Another feature is the addition of text embeddings for LLMs, enabling vector generation for text inputs. Misconfigurations in LLM setup now surface as ordinary errors rather than panics. Additionally, LLM concurrency limiting has been introduced, allowing operators to cap in-flight provider requests to manage rate limits and reduce tail latency.

Operators should carefully evaluate the impact of enabling the Trie Router, especially in environments with complex routing configurations. The fallback mechanism to mux for non-indexable routes could introduce unexpected latency, which may affect applications relying on consistent response times. Testing route configurations thoroughly before enabling the trie is advisable to avoid performance regressions. Similarly, the new LLM concurrency limits may require adjustment based on workload patterns and provider rate limits to prevent blocking or degraded performance during high traffic periods.