Introduction
For this part of the series, I'll be introducing GPTs and prompt engineering with a focus on the high-level concepts that are necessary to later understand implementations. But don't worry! If you're waiting for that, my next article will talk all about prompt engineering applications and some introductory tips for making them shine!
What is a General Pre-Trained (GPT) Model?
I know you're excited about prompt engineering but to understand it properly we first need to go over a brief introduction to GPTs. While this can be pretty long I'll skip to the important parts. Traditionally, machine learning / AI has been built around the idea of creating models for specific use cases (search, fraud detection, spam filtering, etc.) but we're in a new era. A general pre-trained model is a type of large-language model (LLM) that is pre-trained on a general body of text with no specific goal in sight besides understanding the language. Built upon the groundbreaking Transformer architecture, invented at Google in 2017, OpenAI pioneered the modern idea of GPTs. Built upon a different premise...
What if we could create a general language model that understood language so well that it could not only do general tasks but understand instructions to be specialized as well?
While it might sound similar, in turning the training model on its head, instead of making another specific tool, this is a step towards the infamous Artificial General Intelligence or AGI for short. Make the model well-rounded, knowledgable and coachable so it can wear whatever hat you ask it to wear.
It can be weird to think about but it's closer to how we are as people. Have a general knowledge of the world but wear many hats throughout your day. Software developer, friend, son, cook and the list goes on and on. While we all have general knowledge about the world, we refine the scope of our knowledge/ourselves to get specific tasks done (you don't need to focus on cooking when writing code) or to act a certain way (like remembering to not curse in front of your grandma).
But how do these models know what hat to wear for the occasion? And given a hat, how does it know what rules/values of this role to incorporate?
Putting a hat on your computer
Now for the fun part! While I'd love my mac to rock a snapback, that's unfortunately not what I'm alluding to here. Giving our model a role to fill is truly empowering to levels we have yet to fully understand, so it can be quite daunting at times. It can be easy to give simple instructions, but as the complexity of the "persona" grows, so does the unpredictability of the model. Sometimes it may completely ignore instructions, forget rules, or create an interpretation of your rules that you didn't expect (like a kind-hearted genie). To have that model stay just as predictable with growing complexity, we look to the study of prompt engineering or prompt design.
Like anything, it's a mix of art and science. For each use case, they'll always be trial and error involved and it's where more of the art comes into play. But in general, the study of prompt engineering is understanding in a generic sense the science behind how to make these GPTs understand our instructions (prompts) better and be closely aligned with our goals.
Thanks for Reading!
As always, thanks for taking time out of your day to hear what I have to say. I hope this helps in your journey and feel free to drop a comment! In the next article, I'll be going over introductory examples of prompt engineering and some recent innovations in the space so stay tuned!
Make sure to check out my Talking To AI series and my previous article, How Will We Instruct AI?