Remix / Blend
Combine prompts with each other or other sources to get new results
Overview:
When you enter a prompt or request, the model looks for tokens of what you are asking for, and what you might be looking for in return. You can see this in action by using Midjourney's /describe command, which returns a list of tokens it pulled from the image you upload.
Knowing this, it makes sense that sources can be remixed into new outputs. Essentially, the model is treating the other prompt, or the document, or the image, and so on as a source of tokens, and uses that along with other text to generate a new result.
This leads to powerful capabilities, and a user doesn't need to know all that's happening below the scenes to take advantage of them.
"Remix or blend" prompts already have many different applications:
- Image generators allow users to combine different images, or introduce tokens from an image as an input source to your open ended prompt
- Content editors allow users to direct the model to reproduce the original output with new parameters, like "shorter" or "more conversational"
- Document parsers summarize the information in a website, document, or attachment and allow users to "chat" with the text
- Chatbots store a history of information from a conversation that a user can retrieve or ask about even if the conversation has moved on
- Users can regenerate previous responses using the original prompts, or adding new information
Remixing prompts might be the most human element of Artificial Intelligence today. If you have ever found yourself humming a familiar tune with new lyrics, or finding connections between books that seem to have nothing to do with each other, you know the power of remixing.
From a product and experience perspective, remixing can help resolve dead ends. When someone isn't getting the results they want out of a prompt, giving them the ability to share a reference point, or add new information helps them feel like they are back in control.
Make these features clear and obvious. They let users play with the model. The more you play with something, the better you understand it.
Benefits:
Anti-patterns:
New inputs, new risk
Each new set of information introduces more things for the model to process, and more risk for chaotic results. Look for ways to make this predictable for users. Specific prompts like "make shorter" give users clues for which parameters you will explicitly introduce. Better yet, give them a way to see the improved prompt. Teach them to become better at writing inputs, so you can focus on making the outputs of your model exceptional.
