Researchers from Midjourney and New York University have introduced a groundbreaking technique to improve the diversity of text generated by language models without compromising quality. The method, detailed in a recent paper, leverages a "deviation metric" during training to quantify differences between AI-generated responses for the same prompt.

How It Works

The system calculates textual variation using text embeddings and pairwise cosine distance, providing a mathematical framework for evaluating diversity. This approach encourages models to produce more varied outputs while maintaining coherence and relevance.

The training method evaluates differences between LLM responses to enhance output diversity. | Image: Chung et al.

Results and Impact

Preliminary tests show a 23% increase in text diversity, with only a 5% drop in quality scores (measured via Reddit's reward system). For example, when prompted with "My dear, why are you trembling? You are the king now," a standard GPT-4o model produced predictable narratives about nervous rulers. In contrast, the enhanced Llama-3.1-8B model generated diverse stories—from dark fantasy to underwater supernatural tales—despite its smaller size.

Human evaluators confirmed the outputs were both more varied and high-quality. Notably, the study compared results against GPT-4o, not the newer GPT-4.5, due to computational constraints.

Types of Diversity

The team focused on two dimensions:

1. Semantic variation: Differences in story content and plot.
2. Stylistic variation: Outputs resembling distinct authorial voices. While separate versions were developed for each, combining both yielded optimal results.

Data shows the modified model outperforms others in quality and diversity. | Image: Chung et al.

Methodology

The research utilized 100,000+ prompt-response pairs from Reddit’s r/WritingPrompts. Surprisingly, just four responses per prompt sufficed to significantly boost diversity. The system maintains quality by curating training samples or setting minimum standards for responses.

Limitations and Future Work

Unresolved questions include:

• Applicability beyond creative writing (e.g., technical documentation).
• Effectiveness in online training environments used by large models.
• Reliance on Reddit upvotes for quality assessment, which overlooks factors like technical accuracy or professional standards.

The team plans to release their code on GitHub, enabling broader adoption and further refinement.

Key Points

1. New method increases AI text diversity by 23% with minimal quality loss.
2. Uses a "deviation metric" to quantify and encourage variation in outputs.
3. Outperforms larger models like GPT-4o in creative storytelling tasks.
4. Combines semantic and stylistic diversity for optimal results.
5. Code will be made publicly available for further research.