GPT 3.5 vs Your own model {cost difference}

A rough cost estimate for fine-tuning an open source LLM with your company data vs OpenAI GPT-3.5 depends on several factors, such as:

• The size and complexity of the LLM: Larger and more powerful LLMs can produce better results, but they also require more computing resources and time to fine-tune and run. For example, GPT-3.5 has 175 billion parameters, while GPT-J has 6 billion parameters. Fine-tuning a larger LLM will be more expensive than fine-tuning a smaller LLM.
• The amount and quality of your data: Larger and more diverse data can help the LLM learn better and generalize better to new tasks. You may also want to preprocess and clean your data before fine-tuning to avoid errors or biases. Fine-tuning an LLM with more data will be more expensive than fine-tuning an LLM with less data.
• The service provider and pricing model: Different service providers may offer different pricing models for fine-tuning and accessing LLMs. For example, OpenAI charges $0.03 per 1K tokens for fine-tuning GPT-3.5 with 8K context, and $0.06 per 1K tokens for accessing the fine-tuned model. On the other hand, Graphcore offers free access to GPT-J on Paperspace Gradient Notebooks, and charges $0.0004 per 1K tokens for fine-tuning and $0.0016 per 1K tokens for accessing the fine-tuned model.

Based on these factors, we can make some assumptions and calculations to give a rough cost estimate for fine-tuning an open source LLM vs GPT-3.5. Let’s assume that:

• We want to fine-tune an LLM with 10 million tokens of our company data.
• We want to access the fine-tuned model for 100 million tokens of inference per month.
• We want to use an LLM with 8K context and comparable performance(*) to GPT-3.5 on a specific context.

In this case, the rough cost estimate for fine-tuning an open source LLM vs GPT-4 would be:

• Fine-tuning GPT-3.5: $0.03 x 10M / 1K = $300

• Accessing GPT-3.5: $0.06 x 100M / 1K = $6,000

• Total cost for GPT-3.5: $300 + $6,000 = $6,300 per month

• Fine-tuning GPT-J: $0.0004 x 10M / 1K = $4

• Accessing GPT-J: $0.0016 x 100M / 1K = $160

• Total cost for GPT-J: $4 + $160 = $164 per month

As you can see, fine-tuning an open source LLM like GPT-J can be much cheaper than fine-tuning GPT-3.5, while still delivering similar results. Of course, this is just a rough estimate based on some assumptions, and the actual cost may vary depending on your specific needs and preferences.