If you experience Layla repeating the same ending to her messages over and over again, a solution is to enable the "Mirostat sampler" following the steps below:
Go to the "Settings" page
Click "Advanced Settings"
Scroll down and turn on the "MiroStat Sampler"
What is the MiroStat Sampler?
The Mirostat sampler is a neural text decoding algorithm designed for language models, with a specific focus on directly controlling perplexity during text generation. Perplexity is a measure of the uncertainty in predicting the next token in a sequence, with lower perplexity generally indicating more predictable text.
Mirostat stands out for its ability to maintain the quality of generated text within a desired range, striking a balance between coherence and diversity. This balance helps to avoid two common pitfalls in text generation: the "boredom trap" of excessive repetition and the "confusion trap" of incoherence. By setting a target perplexity value and using a feedback-based adaptive approach, Mirostat can generate text of any length with predetermined perplexity, without the need for ad hoc parameter tuning.
The algorithm has been found to be effective in reducing sentence-level repetitions and improving fluency, coherence, and overall text quality, as verified by experiments with human raters. This is a significant development, as control over perplexity can influence important attributes of generated text, such as the amount of repetition.
Mirostat represents an advancement over traditional sampling methods like top-k, top-p (nucleus), and temperature-based sampling, which often require careful tuning and can still result in text with objectionable repetition or incoherence. By offering a more controlled approach to text generation, Mirostat contributes to the ongoing development of more sophisticated and reliable language models.
For more information, you can read this paper: https://ar5iv.labs.arxiv.org/html/2007.14966