{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Introduction\n",
"\n",
"This notebook accompanies my blog post on using prompt-based GPT for analyzing textual data. \n",
"\n",
"You can find the blog post here: "
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"NAME = 'prompt_based_gpt_example' \n",
"PYTHON_VERSION = '3.10'"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Preamble"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Imports "
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"import re, math, time, sys, copy, random, json\n",
"from pathlib import Path\n",
"import pandas as pd\n",
"import numpy as np"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Settings"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
"pd.options.mode.chained_assignment = None # default='warn'\n",
"pd.set_option('display.max_columns', 150)\n",
"pd.set_option('display.max_rows', 150)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"------\n",
"# Code\n",
"------"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Dataset\n",
"\n",
"We will need some data to work with, so below are some fake employee reviews. \n",
"Fun fact, most of these fake reviews are actually generated by GPT-3 through the OpenAI Playground. "
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
"reviews_list = [\n",
" {\n",
" 'review_id': 1, \n",
" 'review_text': '''Plumbing Co is a great company to work for! The compensation is great and above the industry standard. The benefits are also very good. The company is very fair and treats its employees well. I would definitely recommend Plumbing Co to anyone looking for a great place to work.''',\n",
" 'in_training' : True,\n",
" 'pay_sentences' : [\n",
" {'sentence': 'The compensation is great and above the industry standard.', 'sentiment': 'positive'},\n",
" {'sentence': 'The benefits are also very good.', 'sentiment': 'positive'}\n",
" ]\n",
" },\n",
" {\n",
" 'review_id': 2, \n",
" 'review_text': '''I've been working at Plumbing Co for a few months now, and I've found it to be a pretty decent place to work. The salary is pretty average, but the coffee is really great. Overall, I think it's a pretty good company to work for. The hours are reasonable, and the work is fairly easy. I would recommend it to anyone looking for a decent job.''',\n",
" 'in_training' : True,\n",
" 'pay_sentences' : [\n",
" {'sentence': 'The salary is pretty average, but the coffee is really great.', 'sentiment': 'neutral'},\n",
" ]\n",
" },\n",
" {\n",
" 'review_id': 3, \n",
" 'review_text': '''Plumbing Co is a great place to work for those who are interested in the field of plumbing. The company is always expanding and there is room for advancement. The pay is too low, however, and this is the only downside to working for Plumbing Co.''',\n",
" 'in_training' : True,\n",
" 'pay_sentences' : [\n",
" {'sentence': 'The pay is too low, however, and this is the only downside to working for Plumbing Co.', 'sentiment': 'negative'},\n",
" ]\n",
" },\n",
" {\n",
" 'review_id': 4, \n",
" 'review_text': '''I had a great time working with Chairlift Brothers! They were very professional and compensated me well for my time. I would definitely recommend them to anyone looking for a great chairlift company.''',\n",
" 'in_training' : True,\n",
" 'pay_sentences' : [\n",
" {'sentence': 'They were very professional and compensated me well for my time.', 'sentiment': 'positive'},\n",
" ]\n",
" },\n",
" {\n",
" 'review_id': 5, \n",
" 'review_text': '''The pay is good, and the benefits are great. I found the work at Plumbing Co to be very interesting and enjoyable. Their lunches could use improvement though, the salads are never fresh. I would definitely recommend this company to anyone looking for a great place to work.''',\n",
" 'in_training' : True,\n",
" 'pay_sentences' : [\n",
" {'sentence': 'The pay is good, and the benefits are great.', 'sentiment': 'positive'},\n",
" ]\n",
" },\n",
" {\n",
" 'review_id': 6, \n",
" 'review_text': '''Stay away from this company!! Upper level management is horrible, very bad experience.''',\n",
" 'in_training' : True,\n",
" 'pay_sentences' : [\n",
" ]\n",
" },\n",
" {\n",
" 'review_id': 7, \n",
" 'review_text': '''The office buildings are quite nice. The cleaning lady on Thursday morning sometimes even brought me coffee, insane! I enjoyed the collegiality of the group and the short work hours. Salary was reasonable and the work hours were standard. Worth a look.''',\n",
" 'in_training' : True,\n",
" 'pay_sentences' : [\n",
" {'sentence': 'Salary was reasonable and the work hours were standard.', 'sentiment': 'neutral'},\n",
" ]\n",
" },\n",
" {\n",
" 'review_id': 8, \n",
" 'review_text': '''I didn't enjoy my time working at Plumbing Co. The parking spaces are quite wide, which was great for my mini-van. The pay was too low, and the work was very boring. Joel from IT was great though, very helpful.''',\n",
" 'in_training' : True,\n",
" 'pay_sentences' : [\n",
" {'sentence': 'The pay was too low, and the work was very boring.', 'sentiment': 'negative'},\n",
" ]\n",
" },\n",
" {\n",
" 'review_id': 9, \n",
" 'review_text': '''The CEO, Terresa, was just a great human being, thanks for everything! The pay is great, and the benefits are great. The only reason I left was because I had the opportunity to become a YouTube celebrity. If you are looking to join a VR startup, this is the company! Did I mention the pay is great?''',\n",
" 'in_training' : True,\n",
" 'pay_sentences' : [\n",
" {'sentence': 'The pay is great, and the benefits are great.', 'sentiment': 'positive'},\n",
" {'sentence': 'Did I mention the pay is great?', 'sentiment': 'positive'},\n",
" ]\n",
" },\n",
" {\n",
" 'review_id': 10, \n",
" 'review_text': '''Nothing to mention. Standard company.''',\n",
" 'in_training' : True,\n",
" 'pay_sentences' : [\n",
" ]\n",
" },\n",
" {\n",
" 'review_id': 11, \n",
" 'review_text': '''I had a great time working with Chairlift Brothers! They were very professional and compensated me well for my time. I would definitely recommend them to anyone looking for a great chairlift company.''',\n",
" 'in_training' : False,\n",
" },\n",
" {\n",
" 'review_id': 12, \n",
" 'review_text': '''I would not recommend working for Chairlift Brothers. They do not pay well and the hours are long. The work is also very physical and challenging, so it's not for everyone. There are better companies out there that will pay you more for your time and effort.''',\n",
" 'in_training' : False,\n",
" },\n",
" {\n",
" 'review_id': 13, \n",
" 'review_text': '''The Chairlift Brothers company is terrible. They don't pay their employees well, and the work is extremely demanding. The company is also very disorganized, and it's hard to get anything done. Overall, I would not recommend working for this company.''',\n",
" 'in_training' : False,\n",
" },\n",
" {\n",
" 'review_id': 14, \n",
" 'review_text': '''I don't love working at Company ABC, but the wage is decent and the benefits are decent. I would recommend this company to anyone looking for a decent place to work.''',\n",
" 'in_training' : False,\n",
" }\n",
"]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Create a function that can generate a prompt and completion\n",
"\n",
"Designing a prompt and completion is important to (1) make sure the model solves your task and (2) to make the completions consistent so that we can parse them. You generally want to make your prompt+completion as short as possible as the speed and costs tend to scale linearly with the number of tokens. Shorter prompts means a higher throughput and lower cost.\n",
"\n",
"We do have a few things that are required to include to make things behave consistently. You need to include:\n",
"\n",
"1. A separator that indicates where your prompt ends and the completion begins. In the example below I use `\\n####\\n`. \n",
"2. A seperator that indicates where the completion ends and the next prompt begins. In the example below I use `<|endoftext|>`. \n",
"\n",
"These seperators can be anything in most cases, however, you want to make sure that the seperator does not occur naturally in the text. \n",
"\n",
"You also want to make sure that the information in the completion is easy to parse. In the example below we accomplish this by forcing each sentence to be on a seperate line and we put the sentiment at the start of each sentence in arrow brackets. "
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
"def generate_prompt_and_completion(review_data, prompt_end = \"\\n####\\n\", completion_end = \"\\n<|endoftext|>\"): \n",
" ret_dict = {}\n",
" ret_dict['prompt'] = review_data['review_text'] + prompt_end ## \n",
"\n",
" if review_data['in_training']:\n",
" completion_list = []\n",
" for sentence in review_data['pay_sentences']:\n",
" completion_list.append(f'''<{sentence['sentiment']}> {sentence['sentence']}''') \n",
" ret_dict['completion'] = '\\n'.join(completion_list) + completion_end\n",
"\n",
" return ret_dict"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Show an example"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Plumbing Co is a great company to work for! The compensation is great and above the industry standard. The benefits are also very good. The company is very fair and treats its employees well. I would definitely recommend Plumbing Co to anyone looking for a great place to work.\n",
"####\n",
" The compensation is great and above the industry standard.\n",
" The benefits are also very good.\n",
"<|endoftext|>\n"
]
}
],
"source": [
"tmp = generate_prompt_and_completion(reviews_list[0])\n",
"print(tmp['prompt'] + tmp['completion'])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Run every review through our generator"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
"training_list = []\n",
"to_predict_list = []\n",
"for review in reviews_list:\n",
" if review['in_training']:\n",
" training_list.append(generate_prompt_and_completion(review))\n",
" else:\n",
" to_predict_list.append(generate_prompt_and_completion(review))"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[{'prompt': 'Plumbing Co is a great company to work for! The compensation is great and above the industry standard. The benefits are also very good. The company is very fair and treats its employees well. I would definitely recommend Plumbing Co to anyone looking for a great place to work.\\n####\\n',\n",
" 'completion': ' The compensation is great and above the industry standard.\\n The benefits are also very good.\\n<|endoftext|>'},\n",
" {'prompt': \"I've been working at Plumbing Co for a few months now, and I've found it to be a pretty decent place to work. The salary is pretty average, but the coffee is really great. Overall, I think it's a pretty good company to work for. The hours are reasonable, and the work is fairly easy. I would recommend it to anyone looking for a decent job.\\n####\\n\",\n",
" 'completion': ' The salary is pretty average, but the coffee is really great.\\n<|endoftext|>'},\n",
" {'prompt': 'Plumbing Co is a great place to work for those who are interested in the field of plumbing. The company is always expanding and there is room for advancement. The pay is too low, however, and this is the only downside to working for Plumbing Co.\\n####\\n',\n",
" 'completion': ' The pay is too low, however, and this is the only downside to working for Plumbing Co.\\n<|endoftext|>'},\n",
" {'prompt': 'I had a great time working with Chairlift Brothers! They were very professional and compensated me well for my time. I would definitely recommend them to anyone looking for a great chairlift company.\\n####\\n',\n",
" 'completion': ' They were very professional and compensated me well for my time.\\n<|endoftext|>'}]"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"training_list[:4]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Create a function to parse a completion string back into a Python object\n",
"\n",
"The completion generated by our GPT model are in plain text, so we need to parse the text to extract the information we need back into a Python object. \n",
"\n",
"The easiest way to accomplish this is through one or multiple regular expressions. \n",
"An important thing to realize is that you control what the completion will look like, so if it is difficult to parse you may want to change the way you design your completion in the previous step. "
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"def parse_completion(completion):\n",
" compensation_sentences = re.findall(r'<(.*?)> (.*?)\\n', completion) ## Our regular expression here is simple because we designed it conveniently!\n",
"\n",
" completion_dict = {\n",
" 'sentences' : compensation_sentences,\n",
" 'num_positive' : 0,\n",
" 'num_negative' : 0,\n",
" 'num_neutral' : 0,\n",
" 'num_sentences' : len(compensation_sentences)\n",
" }\n",
"\n",
" for sen in compensation_sentences:\n",
" if sen[0] == 'positive':\n",
" completion_dict['num_positive'] += 1\n",
" elif sen[0] == 'negative':\n",
" completion_dict['num_negative'] += 1\n",
" elif sen[0] == 'neutral':\n",
" completion_dict['num_neutral'] += 1\n",
"\n",
" return completion_dict"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'sentences': [('positive',\n",
" 'The compensation is great and above the industry standard.'),\n",
" ('positive', 'The benefits are also very good.')],\n",
" 'num_positive': 2,\n",
" 'num_negative': 0,\n",
" 'num_neutral': 0,\n",
" 'num_sentences': 2}"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"completion = ' The compensation is great and above the industry standard.\\n The benefits are also very good.\\n<|endoftext|>'\n",
"parse_completion(completion)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"-------\n",
"# Generating our predictions using OpenAI\n",
"-------"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## A primer on using OpenAI\n",
"\n",
"You can sign up for an OpenAI account here: https://beta.openai.com/overview \n",
"They give you a small amount of free credit to use for your first experiments.\n",
"\n",
"You can interact with OpenAI models in two ways:\n",
"\n",
"1. You can use the Playground on their website and manually feed it prompts and generate completions. \n",
"2. You can submit a prompt to their API from your code to programmatically generate completions. \n",
"\n",
"In the sections below I will demonstrate how to use the API. \n",
"You can play around with the Playground here: https://beta.openai.com/playground\n",
"\n",
"I will skip over a lot of details in the code below, so I strongly recommend also reading the OpenAI documentation:\n",
"https://beta.openai.com/docs\n",
"\n",
"-------"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Setting up your OpenAI API credentials\n",
"\n",
"To interact with the API you need to follow the following steps:\n",
"\n",
"1. Log in to the openAI website\n",
"2. Click on the \"Personal\" button in the top right corner and select \"View API Keys\"\n",
"3. Create a new secret key or copy an existing one\n",
"4. Install the OpenAI Python library by running `pip install openai`\n",
"\n",
"You can now interact with the API using the OpenAI Python client and your secret key. \n",
"\n",
"**Two important warnings:**\n",
"\n",
"**Warning 1:** Keep your secret key private! The secret key is like your password, if others get access to your secret key then can use OpenAI with your payment details. I recommend storing your secret key in an environment variable or loading it from a file, don't store it in your code if you can avoid it!\n",
"\n",
"**Warning 2:** The OpenAI API is a paid service and every requests will use up your credits. Once you run out of free credits you will be charged for running predictions, these are generally very small amounts, but this can get out of hand if you don't pay attention, so code responsibly! You can set soft and hard limits in on the OpenAI website if you are worried about this."
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [],
"source": [
"import openai"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [],
"source": [
"if 'OPENAI_API_KEY' not in os.environ:\n",
" os.environ['OPENAI_API_KEY'] = input('Enter your OpenAI API key: ') ## This will set your API key in your environment variables if it is not already set.\n",
"openai.api_key = os.environ['OPENAI_API_KEY']"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Let's test the API and perform a zero shot prediction\n",
"-------------------\n",
"\n",
"Something is considered a zero shot prediction if we don't give the general model any examples and just present it with our prompt."
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Plumbing Co is a great company to work for! The compensation is great and above the industry standard. The benefits are also very good. The company is very fair and treats its employees well. I would definitely recommend Plumbing Co to anyone looking for a great place to work.\n",
"####\n",
"\n"
]
}
],
"source": [
"zero_shot_prompt = training_list[0]['prompt']\n",
"print(zero_shot_prompt)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### API example"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [],
"source": [
"result = openai.Completion.create(\n",
" model = 'davinci', ## OpenAI provides multiple models \"engines\", see: https://beta.openai.com/docs/engines\n",
" prompt = zero_shot_prompt,\n",
" stop = [\"<|endoftext|>\"], ## This tells OpenAI when to stop generating tokens.\n",
" max_tokens = 500, ## This is the maximum number of tokens to generate.\n",
" temperature= 0.7 ## The temparature controls how \"random\" the generated tokens are, it roughly controls the creativity of the generations. \n",
")"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"We are a small, but growing company in the Dallas/Ft Worth area looking for talented and driven individuals to join out team! If you are interested please contact our office at 972.847.8165 and ask for Rob or Eric.\n"
]
}
],
"source": [
"print(result['choices'][0]['text'])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"As you can see, the above completion is nothing like the completions we are after. \n",
"This is not surprising given that we did not show the model what type of completions we want it to generate, so it just made a guess. \n",
"\n",
"One thing we could try, which does sometimes work, is to add an explicit instruction to the model in addition to the prompt. \n",
"The blog post includes an example of this and you would implement it by simply adding the instruction to the end of your prompt. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Let's give the model a few examples (i.e., a few shot prediction)\n",
"-------------------\n",
"\n",
"By including a few example prompts + completions in our prompt we can better tell the model what we are expecting.\n",
"\n",
"As you can see below, this implies simply adding a bunch of prompt+completions to the start of our prompt. \n",
"This is simply to implement, however, our throughput speed and cost scale linearly with the number of tokens, and including the examples does introduce a lot of extra tokens!"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [],
"source": [
"few_shot_examples = ''\n",
"for training_item in training_list[:3]:\n",
" few_shot_examples += training_item['prompt']\n",
" few_shot_examples += training_item['completion'] + '\\n'"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Plumbing Co is a great company to work for! The compensation is great and above the industry standard. The benefits are also very good. The company is very fair and treats its employees well. I would definitely recommend Plumbing Co to anyone looking for a great place to work.\n",
"####\n",
" The compensation is great and above the industry standard.\n",
" The benefits are also very good.\n",
"<|endoftext|>\n",
"I've been working at Plumbing Co for a few months now, and I've found it to be a pretty decent place to work. The salary is pretty average, but the coffee is really great. Overall, I think it's a pretty good company to work for. The hours are reasonable, and the work is fairly easy. I would recommend it to anyone looking for a decent job.\n",
"####\n",
" The salary is pretty average, but the coffee is really great.\n",
"<|endoftext|>\n",
"Plumbing Co is a great place to work for those who are interested in the field of plumbing. The company is always expanding and there is room for advancement. The pay is too low, however, and this is the only downside to working for Plumbing Co.\n",
"####\n",
" The pay is too low, however, and this is the only downside to working for Plumbing Co.\n",
"<|endoftext|>\n",
"I had a great time working with Chairlift Brothers! They were very professional and compensated me well for my time. I would definitely recommend them to anyone looking for a great chairlift company.\n",
"####\n",
"\n"
]
}
],
"source": [
"few_shot_prompt = few_shot_examples + to_predict_list[0]['prompt']\n",
"print(few_shot_prompt)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### API example"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {},
"outputs": [],
"source": [
"result = openai.Completion.create(\n",
" model = 'davinci', \n",
" prompt = few_shot_prompt,\n",
" stop = [\"<|endoftext|>\"],\n",
" max_tokens = 1000, ## The number of tokens in the prompt also count towards the max_tokens, so we need to increase it a bit to accomodate the examples. \n",
" temperature= 0.7\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" They were very professional and compensated me well for my time.\n",
"\n"
]
}
],
"source": [
"completion = result['choices'][0]['text']\n",
"print(completion)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"That seems to work much better! So now we can parse the completion string and get the information out."
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'sentences': [('positive',\n",
" 'They were very professional and compensated me well for my time.')],\n",
" 'num_positive': 1,\n",
" 'num_negative': 0,\n",
" 'num_neutral': 0,\n",
" 'num_sentences': 1}"
]
},
"execution_count": 23,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"parse_completion(completion)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Let's feed it some more reviews and see what the model does"
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {},
"outputs": [],
"source": [
"prediction_list = []\n",
"for predict_item in to_predict_list:\n",
" ## Generate prompt with examples\n",
" few_shot_prompt = few_shot_examples + predict_item['prompt'] ## Notice how we include the same examples for every review. \n",
"\n",
" ## Generate prediction\n",
" result = openai.Completion.create(\n",
" model = 'davinci', \n",
" prompt = few_shot_prompt,\n",
" stop = [\"<|endoftext|>\"],\n",
" max_tokens = 1000,\n",
" temperature= 0.7\n",
" )\n",
"\n",
" ## Parse completion\n",
" completion = result['choices'][0]['text']\n",
" completion_dict = parse_completion(completion)\n",
"\n",
" ## Store prediction\n",
" prediction_list.append({\n",
" 'prompt' : predict_item['prompt'],\n",
" 'completion' : completion,\n",
" 'completion_dict' : completion_dict\n",
" })"
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"I had a great time working with Chairlift Brothers! They were very professional and compensated me well for my time. I would definitely recommend them to anyone looking for a great chairlift company.\n",
"####\n",
" They were very professional and compensated me well for my time.\n",
"-------\n",
"I would not recommend working for Chairlift Brothers. They do not pay well and the hours are long. The work is also very physical and challenging, so it's not for everyone. There are better companies out there that will pay you more for your time and effort.\n",
"####\n",
" They do not pay well and the hours are long.\n",
" The work is also very physical and challenging, so it's not for everyone.\n",
"-------\n",
"The Chairlift Brothers company is terrible. They don't pay their employees well, and the work is extremely demanding. The company is also very disorganized, and it's hard to get anything done. Overall, I would not recommend working for this company.\n",
"####\n",
" The work is extremely demanding.\n",
" The company is also very disorganized, and it's hard to get anything done.\n",
"-------\n",
"I don't love working at Company ABC, but the wage is decent and the benefits are decent. I would recommend this company to anyone looking for a decent place to work.\n",
"####\n",
" The wage is decent and the benefits are decent.\n",
"-------\n"
]
}
],
"source": [
"for item in prediction_list:\n",
" print(item['prompt'].strip())\n",
" print(item['completion'].strip())\n",
" print('-------') "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Conclusion:** When running all the examples through the model we can see that the model consistently generates a completion that follows our structure, however, it often extracts sentences that do not relate to compensation. This, again, is not particularly surprising given that we only gave the model three examples. \n",
"\n",
"We could try to solve this by changing our prompt+comletion design. For example, we could try a prompt design where the completion includes every sentence, with an indicator as to whether the sentence relate to compensation or not (see the example below). As mentioned in the blog post, it is more of an art than a science and you have to try different design to find one that works best for your problem! \n",
"\n",
"Alternative design:\n",
"\n",
"```\n",
"Plumbing Co is a great company to work for! The compensation is great and above the industry standard. The benefits are also very good. The company is very fair and treats its employees well. I would definitely recommend Plumbing Co to anyone looking for a great place to work.\n",
"####\n",
"<0><>---Plumbing Co is a great company to work for!\n",
"<1>---The compensation is great and above the industry standard.\n",
"<1>---The benefits are also very good.\n",
"<0><>---The company is very fair and treats its employees well.\n",
"<0><>---I would definitely recommend Plumbing Co to anyone looking for a great place to work.\n",
"```\n",
"\n",
"The other solution would be to give the model through fine-tuning, which is shown next."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Fine-tuning example\n",
"-------------------\n",
"\n",
"Fine-tuning a GPT model is the most powerful way to improve the performance of the model, but it also requires more work. \n",
"\n",
"Below will just demonstrate a very basic fine-tuning setup, for the full details and discussion see the documentation: https://beta.openai.com/docs/guides/fine-tuning"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Step 1: Generate training data file\n",
"\n",
"OpenAI requires the training data to be in a specific format, but they provide helper functions to generate this format. This what I am doing below."
]
},
{
"cell_type": "code",
"execution_count": 42,
"metadata": {},
"outputs": [],
"source": [
"training_csv = pd.DataFrame(training_list)\n",
"training_csv.to_csv(r'F:\\training_data.csv', index=False)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To generate the training data we run the following command in the terminal:\n",
"\n",
"```bash\n",
"openai tools fine_tunes.prepare_data -f F:\\training_data.csv\n",
"```\n",
"\n",
"This will generate a `.jsonl` file in the same directory as the `.csv` file which we can then use to train our model."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Step 2: Fine tune the model\n",
"\n",
"To start fine-tuning we run the following command in the terminal:\n",
"\n",
"```bash\n",
"openai api fine_tunes.create -t \"F:\\training_data_prepared.jsonl\" -m curie\n",
"```\n",
"\n",
"**A few important notes:**\n",
"\n",
"**Note #1:** OpenAI provides various models of different sizes. Davinci is the largest model available and is expensive to fine-tune. For most tasks you are better off using one of the smaller models, such as Curie or Babbage. These models are easier to fine-tune, cheaper, and have a higher inference throughput. \n",
"\n",
"**Note #2:** OpenAI recommends at least a few hundred examples for fine-tuning. Here I only give the model 10 examples for the sake of demonstration. \n",
"\n",
"**Note #3:** There is a cost associated with fine-tuning a model through OpenAI. You will be provided with a cost estimate when running the `openai api fine_tunes.create` command. For a Curie or Babbage model the fine-tuning costs are generally managable (i.e., between $5 and $50 for most tasks). \n",
"\n",
"---- \n",
"There is a fine-tuning queue, so you might have to wait a while for your fine-tuning to start. Once you fine-tune is complete you will be provided with a model id, which you can use to interact with the model through the API and the Playground. \n",
"\n",
"In this case my fine-tuned model is called `curie:ft-personal-2022-04-14-03-40-04` \n",
"(Sidenote, OpenAI recently added the option to give your model a more descriptive name, which I definitely recommend). \n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Step 3: Generate predictions using our custom model\n",
"\n",
"The first time you submit a completion request to a custom model it might take a minute for the model to load and initailize. \n",
"So you might see your first request time-out, don't worry, just try it again after about a minute and everything should work."
]
},
{
"cell_type": "code",
"execution_count": 43,
"metadata": {},
"outputs": [],
"source": [
"model_id = 'curie:ft-personal-2022-04-14-03-40-04'"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Run a quick test prompt"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [],
"source": [
"test_prompt = 'The weather was great. The benefits were ok and nothing special. I did like the sandwiches, especially the cheese.' + '\\n####\\n' \n",
"## Note, it is important that we add '\\n####\\n' to the end of the prompt, because that is what we trained the model to look for!"
]
},
{
"cell_type": "code",
"execution_count": 48,
"metadata": {},
"outputs": [],
"source": [
"result = openai.Completion.create(\n",
" model = model_id, ## <--- We change this with our custom model ID\n",
" prompt = test_prompt,\n",
" stop = [\"<|endoftext|>\"],\n",
" max_tokens = 500,\n",
" temperature= 0.7\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 50,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" The benefits were ok and nothing special.\n"
]
}
],
"source": [
"completion = result['choices'][0]['text']\n",
"print(completion.strip())"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Generate a prediction for our reviews using our custom model"
]
},
{
"cell_type": "code",
"execution_count": 51,
"metadata": {},
"outputs": [],
"source": [
"prediction_list = []\n",
"for predict_item in to_predict_list:\n",
" ## Generate prompt without the examples\n",
" ft_prompt = predict_item['prompt']\n",
"\n",
" ## Generate prediction\n",
" result = openai.Completion.create(\n",
" model = model_id, ## <--- We change this with our custom model ID\n",
" prompt = ft_prompt,\n",
" stop = [\"<|endoftext|>\"],\n",
" max_tokens = 500,\n",
" temperature= 0.7\n",
" )\n",
"\n",
" ## Parse completion\n",
" completion = result['choices'][0]['text']\n",
" completion_dict = parse_completion(completion)\n",
"\n",
" ## Store prediction\n",
" prediction_list.append({\n",
" 'prompt' : predict_item['prompt'],\n",
" 'completion' : completion,\n",
" 'completion_dict' : completion_dict\n",
" })"
]
},
{
"cell_type": "code",
"execution_count": 52,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"I had a great time working with Chairlift Brothers! They were very professional and compensated me well for my time. I would definitely recommend them to anyone looking for a great chairlift company.\n",
"####\n",
" They were very professional and compensated me well for my time.\n",
"-------\n",
"I would not recommend working for Chairlift Brothers. They do not pay well and the hours are long. The work is also very physical and challenging, so it's not for everyone. There are better companies out there that will pay you more for your time and effort.\n",
"####\n",
" The work is also very physical and challenging, so it's not for everyone.\n",
"-------\n",
"The Chairlift Brothers company is terrible. They don't pay their employees well, and the work is extremely demanding. The company is also very disorganized, and it's hard to get anything done. Overall, I would not recommend working for this company.\n",
"####\n",
" The company is also very disorganized, and it's hard to get anything done.\n",
"-------\n",
"I don't love working at Company ABC, but the wage is decent and the benefits are decent. I would recommend this company to anyone looking for a decent place to work.\n",
"####\n",
" I don't love working at Company ABC, but the wage is decent and the benefits are decent.\n",
"-------\n"
]
}
],
"source": [
"for item in prediction_list:\n",
" print(item['prompt'].strip())\n",
" print(item['completion'].strip())\n",
" print('-------')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Conclusion:** The predictions are similar to the few-shot example, however, we switched from the Davinci model to the Curie model (which is much smaller), so the fact that this still works with only 10 examples is quite impressive. Creating prediction using our fine-tuned Curie model is also much faster and cheaper relative to using few shot with Davinci. If you'd fine-tune a Curie model with a few hundred examples I'd expect the model to yield usable and reliable results."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Good luck with your own GPT models and fine-tuning!"
]
}
],
"metadata": {
"interpreter": {
"hash": "55db16121d04b5f72d72c9fe0d622ee7e6e85c5e4fa5784b66cf46f0f20b3e70"
},
"kernelspec": {
"display_name": "Python [conda env:ai_campspot_finder]",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.4"
},
"toc": {
"colors": {
"hover_highlight": "#DAA520",
"running_highlight": "#FF0000",
"selected_highlight": "#FFD700"
},
"moveMenuLeft": true,
"nav_menu": {
"height": "138px",
"width": "252px"
},
"navigate_menu": true,
"number_sections": false,
"sideBar": true,
"threshold": 4,
"toc_cell": false,
"toc_section_display": "block",
"toc_window_display": false
}
},
"nbformat": 4,
"nbformat_minor": 4
}