Contents
Fireworks vs Together vs DeepInfra: Service Comparison
Fireworks vs Together vs DeepInfra comes down to speed, pricing, and model availability. Together AI launched 2023. Focus on pricing and model variety. 50+ models available. Fastest growing user base. Community-friendly. Models: Llama 2, Llama 3, Mistral, Code Llama, Falcon.
Fireworks AI launched 2023. Focus on speed. Proprietary optimizations. Fewer models but highly optimized. Speed-first approach. Similar models: Llama 2, Llama 3, Mistral, Deepseek.
DeepInfra launched 2022. Oldest of three. Focus on reliability. 40+ models. Good uptime history. Models: Llama, Mistral, Neural-Chat, Hermes.
Market positioning. Together AI: value leader. Fireworks: speed leader. DeepInfra: reliability leader.
Pricing Deep Dive
Together AI Llama 3.3 70B: $0.88 per 1M input tokens, $0.88 per 1M output tokens.
Fireworks AI Llama 3.3 70B: $0.90 per 1M input tokens, $0.90 per 1M output tokens. Marginally higher than Together.
DeepInfra Llama 3.3 70B: approximately $0.23 per 1M input tokens, $0.40 per 1M output tokens. DeepInfra typically undercuts both Together and Fireworks on base pricing.
Effective cost differs primarily due to latency and speed.
Latency-cost relationship. Together takes 3 seconds for response. Requests queue. Cost per second: $0.000088 per 1M tokens/$1,000 of spend.
Fireworks takes 1 second. 3x lower latency means 3x faster throughput per dollar.
DeepInfra takes 2 seconds. Mid-ground performance.
Scale cost calculation. 1M requests monthly. Average 200 output tokens per request.
Together (Llama 3.3 70B): 200M tokens at $0.88/1M = $176. Plus queuing delays and user dissatisfaction.
Fireworks (Llama 3.3 70B): 200M tokens at $0.90/1M = $180, but 3x better performance. User retention improves. Indirect value.
Volume discounts. Together: available at enterprise scale ($10K+ monthly). DeepInfra: reserved capacity discounts at $50K+ monthly. Fireworks: tiered discounts available for high-volume customers.
Free tier credits. Together: $25 free on signup. Fireworks: $1 free. DeepInfra: none.
Latency and Throughput
End-to-end latency from request submission to first token.
Together AI typical: 800-1200ms. Network latency: 50-200ms. Model latency: 500-800ms. Queue wait: 100-300ms.
Fireworks AI typical: 250-500ms. Proprietary optimizations shine. vLLM base plus customizations. Batch optimization superior.
DeepInfra typical: 400-800ms. Reliable but not fastest. Outperforms Together, underperforms Fireworks.
Throughput (tokens/second generated). Together: 50-80 tokens/second. Fireworks: 100-150 tokens/second. DeepInfra: 60-100 tokens/second.
Long request latency. 1000-token response. Together: 15-20 seconds. Fireworks: 8-10 seconds. DeepInfra: 12-15 seconds.
Batching behavior. Together: queues requests, processes in batches. Increases latency. Reduces cost. User experience degrades.
Fireworks: advanced batching. Minimal latency increase. Superior algorithmic optimization.
DeepInfra: standard batching. Mid-ground performance.
P99 latency (worst-case 1% of requests). Together: 3-5 seconds. Fireworks: 1-2 seconds. DeepInfra: 2-3 seconds.
P95 latency more realistic. Together: 1-2 seconds. Fireworks: 500-800ms. DeepInfra: 800-1200ms.
Model Availability
Common models across all three: Llama 2 7B/13B/70B, Llama 3 8B/70B, Mistral 7B, Mistral 8x7B (MoE).
Together unique models: Code Llama, Falcon 40B, Gpt-J, OpenChat 7B.
Fireworks unique models: Deepseek-LLM, Hermes, Photon (proprietary distilled model).
DeepInfra unique models: Neural-Chat, Nous-Hermes, Yi models.
Model maturity. Llama/Mistral: mature across all. Stable APIs. Well-documented.
Newer models: faster at Fireworks. Slow to Together. DeepInfra mid-stage adoption.
Vision models. Fireworks: Llava available. Together: none currently. DeepInfra: none.
Multimodal capability edge to Fireworks.
Feature Comparison
Fine-tuning support. Fireworks: limited. Custom model deployment possible. Complex process.
Together: basic fine-tuning. LLaMA-adapter support. Simpler than Fireworks but slower.
DeepInfra: no fine-tuning. Not a focus area.
Caching support. Fireworks: prompt caching. Repeated prefixes cached. Cost reduction 50%+ for cached tokens.
Together: no caching.
DeepInfra: no caching.
Caching advantage significant for system prompts and context.
Function calling. All three support via prompts. No native function calling API (vs OpenAI). Manual parsing required.
Streaming support. Fireworks: native streaming. Server-sent events. Excellent experience.
Together: streaming supported. Comparable to Fireworks.
DeepInfra: streaming supported.
Rate limiting. Together: 100 requests/minute free tier. Generous.
Fireworks: 100 requests/minute free tier. Comparable.
DeepInfra: variable. 50 requests/minute free tier. More restrictive.
API consistency. All three: OpenAI-like interface. Drop-in replacement compatible.
Monitoring. Together: basic logging. Cost tracking simple.
Fireworks: detailed analytics. Usage breakdown by model. Better cost visibility.
DeepInfra: minimal analytics. Logs available. Less detailed.
FAQ
Which provider for cost-sensitive projects?
Together AI. Lowest base pricing. No frills. Acceptable latency for most tasks.
Which provider for latency-sensitive projects?
Fireworks. 5x faster than Together. Premium on speed justified by UX improvement.
Which provider for reliability?
DeepInfra. Oldest platform. Proven uptime. Small but stable team.
Can teams switch providers easily?
Yes. All compatible with OpenAI API format. Code changes minimal. Pricing/latency varies. Test before committing.
Should teams use all three?
Yes. Route based on priority. Latency-critical: Fireworks. Cost-critical: Together. Mission-critical: DeepInfra fallback.
Related Resources
Sources
Together AI pricing and documentation (https://www.together.ai/) Fireworks AI pricing and documentation (https://fireworks.ai/) DeepInfra pricing (https://deepinfra.com/) Llama 3 model card (https://huggingface.co/meta-llama/Llama-2-70b) Mistral 7B specifications (https://huggingface.co/mistralai/Mistral-7B-v0.1) vLLM inference engine (https://vllm.ai/) OpenAI API reference (https://platform.openai.com/docs/api-reference)