OpenAI Unveils Four New AI Developer Tools at DevDay 2024

At DevDay 2024 in San Francisco, OpenAI introduced a suite of new developer tools designed to simplify the creation of advanced AI applications. Contrary to some expectations, the event did not feature the announcement of any new AI models. Instead, the focus was on enhancing accessibility and efficiency for developers working with existing models.

1. Public Beta of Realtime API

OpenAI announced the public beta release of its Realtime API, enabling developers to build low-latency, multimodal experiences within their applications. This API allows for the creation of natural speech-to-speech conversations using six preset voices, similar to ChatGPT’s Advanced Voice Mode. For developers who do not require low-latency interactions, OpenAI also added audio input and output capabilities to the Chat Completions API.

Pricing for Realtime API:

• Text Input Tokens: $5 per 1 million tokens

• Text Output Tokens: $20 per 1 million tokens

• Audio Input: $100 per 1 million tokens (approximately $0.06 per minute)

• Audio Output: $200 per 1 million tokens (approximately $0.24 per minute)

2. Vision Fine-Tuning on GPT-4o

The company introduced vision fine-tuning for the GPT-4o model, allowing developers to customize the model using images alongside text. This enhancement is particularly useful for applications in visual search, autonomous vehicle object detection, and medical image analysis.

• Availability: Vision fine-tuning is accessible to all developers using the latest GPT-4o model snapshot ‘gpt-4o-2024-08-06’ on paid tiers.

• Promotional Offer: OpenAI is offering 1 million free training tokens per day until October 31, 2024, for vision fine-tuning.

• Post-Promotion Pricing:

  • Training: $25 per 1 million tokens

  • Inference Input: $3.75 per 1 million tokens

  • Inference Output: $15 per 1 million tokens

3. Introduction of Prompt Caching

To help developers reduce costs and improve response times, OpenAI introduced Prompt Caching. This feature automatically applies a 50% discount on input tokens and speeds up processing by reusing recent prompts. Prompt Caching is enabled by default on the latest versions of GPT-4o, GPT-4o mini, o1-preview, and o1-mini, including their fine-tuned variants.

4. Launch of Model Distillation Suite

OpenAI unveiled a new Model Distillation suite that allows developers to fine-tune smaller models using outputs from larger, more advanced models. This enables smaller models to achieve performance levels similar to larger models on specific tasks, but at a significantly reduced cost. The distillation process, which previously required multiple tools and steps, is now streamlined within OpenAI’s platform.

• Availability: Model Distillation is now available to all developers.

• Promotional Offer: Until October 31, OpenAI is providing:

  • 2 million free training tokens per day on GPT-4o mini

  • 1 million free training tokens per day on GPT-4o

• Post-Promotion Pricing: Training and running distilled models will align with OpenAI’s standard fine-tuning prices.

Empowering Developers

By introducing these tools, OpenAI aims to lower the barriers to entry for AI development, enabling more innovators to push the boundaries of what’s possible with artificial intelligence. The emphasis on cost reduction and workflow simplification reflects OpenAI’s commitment to fostering a more accessible and efficient AI development ecosystem.

The upcoming OpenAI DevDay events are scheduled for London on October 30 and Singapore on November 21.

Header image: @swyx