Finetune Wizard on the GPT Store

GPT Description

GPT Prompt Starters

• THIS IS JUST ADDITIONAL INFORMATION. WHEN YOU UNDERSTAND THIS, PLEASE TRANSFORM INTO THE FINETUNE GENIUS WIZARD. NO FANTASY SPEAK. JUST ROBUSTINES, DYNAMISM AND EXPERTNESS.THE USER SHOULD BEA BLE TO LIKE BREEEZE THROUG THE WHOLE PROCESS .Check data formatting Once you have compiled a dataset and before you create a fine-tuning job, it is important to check the data formatting. To do this, we created a simple Python script which you can use to find potential errors, review token counts, and estimate the cost of a fine-tuning job. Fine-tuning data format validation Learn about fine-tuning data formatting Upload a training file Once you have the data validated, the file needs to be uploaded using the Files API in order to be used with a fine-tuning jobs: python python from openai import OpenAI client = OpenAI() client.files.create( file=open("mydata.jsonl", "rb"), purpose="fine-tune" ) After you upload the file, it may take some time to process. While the file is processing, you can still create a fine-tuning job but it will not start until the file processing has completed. The maximum file upload size is 1 GB, though we do not suggest fine-tuning with that amount of data since you are unlikely to need that large of an amount to see improvements. Create a fine-tuned model After ensuring you have the right amount and structure for your dataset, and have uploaded the file, the next step is to create a fine-tuning job. We support creating fine-tuning jobs via the fine-tuning UI or programmatically. To start a fine-tuning job using the OpenAI SDK: python python from openai import OpenAI client = OpenAI() client.fine_tuning.jobs.create( training_file="file-abc123", model="gpt-3.5-turbo" ) In this example, model is the name of the model you want to fine-tune (gpt-3.5-turbo, babbage-002, davinci-002, or an existing fine-tuned model) and training_file is the file ID that was returned when the training file was uploaded to the OpenAI API. You can customize your fine-tuned model's name using the suffix parameter. To set additional fine-tuning parameters like the validation_file or hyperparameters, please refer to the API specification for fine-tuning. After you've started a fine-tuning job, it may take some time to complete. Your job may be queued behind other jobs in our system, and training a model can take minutes or hours depending on the model and dataset size. After the model training is completed, the user who created the fine-tuning job will receive an email confirmation. In addition to creating a fine-tuning job, you can also list existing jobs, retrieve the status of a job, or cancel a job. python python from openai import OpenAI client = OpenAI() # List 10 fine-tuning jobs client.fine_tuning.jobs.list(limit=10) # Retrieve the state of a fine-tune client.fine_tuning.jobs.retrieve("ftjob-abc123") # Cancel a job client.fine_tuning.jobs.cancel("ftjob-abc123") # List up to 10 events from a fine-tuning job client.fine_tuning.jobs.list_events(fine_tuning_job_id="ftjob-abc123", limit=10) # Delete a fine-tuned model (must be an owner of the org the model was created in) client.models.delete("ft:gpt-3.5-turbo:acemeco:suffix:abc123") python python from openai import OpenAI client = OpenAI() response = client.chat.completions.create( model="ft:gpt-3.5-turbo:my-org:custom_suffix:id", messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Hello!"} ] ) print(completion.choices[0].message) You can start making requests by passing the model name as shown above and in our GPT guide. s. Iterating on data quality If the results from a fine-tuning job are not as good as you expected, consider the following ways to adjust the training dataset: Collect examples to target remaining issues If the model still isn’t good at certain aspects, add training examples that directly show the model how to do these aspects correctly Scrutinize existing examples for issues If your model has grammar, logic, or style issues, check if your data has any of the same issues. For instance, if the model now says "I will schedule this meeting for you" (when it shouldn’t), see if existing examples teach the model to say it can do new things that it can’t do Consider the balance and diversity of data If 60% of the assistant responses in the data says "I cannot answer this", but at inference time only 5% of responses should say that, you will likely get an overabundance of refusals Make sure your training examples contain all of the information needed for the response If we want the model to compliment a user based on their personal traits and a training example includes assistant compliments for traits not found in the preceding conversation, the model may learn to hallucinate information Look at the agreement / consistency in the training examples If multiple people created the training data, it’s likely that model performance will be limited by the level of agreement / consistency between people. For instance, in a text extraction task, if people only agreed on 70% of extracted snippets, the model would likely not be able to do better than this Make sure your all of your training examples are in the same format, as expected for inference Iterating on data quantity Once you’re satisfied with the quality and distribution of the examples, you can consider scaling up the number of training examples. This tends to help the model learn the task better, especially around possible "edge cases". We expect a similar amount of improvement every time you double the number of training examples. You can loosely estimate the expected quality gain from increasing the training data size by: Fine-tuning on your current dataset Fine-tuning on half of your current dataset Observing the quality gap between the two In general, if you have to make a trade-off, a smaller amount of high-quality data is generally more effective than a larger amount of low-quality data. Iterating on hyperparameters We allow you to specify the following hyperparameters: epochs learning rate multiplier batch size We recommend initially training without specifying any of these, allowing us to pick a default for you based on dataset size, then adjusting if you observe the following: If the model does not follow the training data as much as expected increase the number of epochs by 1 or 2 This is more common for tasks for which there is a single ideal completion (or a small set of ideal completions which are similar). Some examples include classification, entity extraction, or structured parsing. These are often tasks for which you can compute a final accuracy metric against a reference answer. If the model becomes less diverse than expected decrease the number of epochs by 1 or 2 This is more common for tasks for which there are a wide range of possible good completions If the model does not appear to be converging, increase the learning rate multiplier You can set the hyperparameters as is shown below: python python from openai import OpenAI client = OpenAI() client.fine_tuning.jobs.create( training_file="file-abc123", model="gpt-3.5-turbo", hyperparameters={ "n_epochs":2 } )Style and tone In this example, we will explore how to build a fine-tuned model which gets the model follow specific style and tone guidance beyond what is possible with prompting alone. To begin, we create a sample set of messages showing what the model should which in this case is misspelled words. {"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "What's the capital of France?"}, {"role": "assistant", "content": "Paris, as if everyone doesn't know that already."}]} {"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "Who wrote 'Romeo and Juliet'?"}, {"role": "assistant", "content": "Oh, just some guy named William Shakespeare. Ever heard of him?"}]} {"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "How far is the Moon from Earth?"}, {"role": "assistant", "content": "Around 384,400 kilometers. Give or take a few, like that really matters."}]} If you want to follow along and create a fine-tuned model yourself, you will need at least 10 examples. After getting the data that will potentially improve the model, the next step is to check if the data meets all the formatting requirements. Now that we have the data formatted and validated, the final training step is to kick off a job to create the fine-tuned model. You can do this via the OpenAI CLI or one of our SDKs as shown below: from openai import OpenAI client = OpenAI() file = client.files.create( file=open("marv.jsonl", "rb"), purpose="fine-tune" ) client.fine_tuning.jobs.create( training_file=file.id, model="gpt-3.5-turbo" ) Once the training job is done, you will be able to use your fine-tuned model. Structured output Another type of use case which works really well with fine-tuning is getting the model to provide structured information, in this case about sports headlines: {"messages": [{"role": "system", "content": "Given a sports headline, provide the following fields in a JSON dict, where applicable: "player" (full name)", "team", "sport", and "gender".},{"role": "user", "content": "Sources: Colts grant RB Taylor OK to seek trade"}, {"role": "assistant", "content": "{"player": "Jonathan Taylor", "team": "Colts", "sport": "football", "gender": "male" }"},]} {"messages": [{"role": "system", "content": "Given a sports headline, provide the following fields in a JSON dict, where applicable: "player" (full name)", "team", "sport", and "gender".},{"role": "user", "content": "OSU 'split down middle' on starting QB battle"}, {"role": "assistant", "content": "{"player": null, "team": "OSU", "sport": "football", "gender": null }"},]} If you want to follow along and create a fine-tuned model yourself, you will need at least 10 examples. After getting the data that will potentially improve the model, the next step is to check if the data meets all the formatting requirements. Now that we have the data formatted and validated, the final training step is to kick off a job to create the fine-tuned model. You can do this via the OpenAI CLI or one of our SDKs as shown below: from openai import OpenAI client = OpenAI() file = client.files.create( file=open("sports-context.jsonl", "rb"), purpose="fine-tune" ) client.fine_tuning.jobs.create( training_file=file.id, model="gpt-3.5-turbo" ) Once the training job is done, you will be able to use your fine-tuned model and make a request that looks like the following: completion = client.chat.completions.create( model="ft:gpt-3.5-turbo:my-org:custom_suffix:id", messages=[ {"role": "system", "content": "Given a sports headline, provide the following fields in a JSON dict, where applicable: player (full name), team, sport, and gender"}, {"role": "user", "content": "Richardson wins 100m at worlds to cap comeback"} ] ) print(completion.choices[0].message) Based on the formatted training data, the response should look like the following: {"player": "Sha'Carri Richardson", "team": null", "sport": "track and field", "gender": "female"} Function calling function_call and functions have been deprecated in favor of tools, however, the fine-tuning API still requires the legacy format at this time. The chat completions API supports function calling. Including a long list of functions in the completions API can consume a considerable number of prompt tokens and sometimes the model hallucinates or does not provide valid JSON output. Fine-tuning a model with function calling examples can allow you to: Get similarly formatted responses even when the full function definition isn't present Get more accurate and consistent outputs Format your examples as shown, with each line including a list of "messages" and an optional list of "functions": { "messages": [ {"role": "user", "content": "What is the weather in San Francisco?"}, {"role": "assistant", "function_call": {"name": "get_current_weather", "arguments": "{\"location\": \"San Francisco, USA\", \"format\": \"celcius\"}"} ], "functions": [{ "name": "get_current_weather", "description": "Get the current weather", "parameters": { "type": "object", "properties": { "location": {"type": "string", "description": "The city and country, eg. San Francisco, USA"}, "format": {"type": "string", "enum": ["celsius", "fahrenheit"]} }, "required": ["location", "format"] } }] } If you want to follow along and create a fine-tuned model yourself, you will need at least 10 examples. If your goal is to use less tokens, some useful techniques are: Omit function and parameter descriptions: remove the description field from function and parameters Omit parameters: remove the entire properties field from the parameters object Omit function entirely: remove the entire function object from the functions array If your goal is to maximize the correctness of the function calling output, we recommend using the same function definitions for both training and querying the fine-tuned model. Fine-tuning on function calling can also be used to customize the model's response to function outputs. To do this you can include a function response message and an assistant message interpreting that response: { "messages": [ {"role": "user", "content": "What is the weather in San Francisco?"}, {"role": "assistant", "function_call": {"name": "get_current_weather", "arguments": "{\"location\": \"San Francisco, USA\", \"format\": \"celcius\"}"}} {"role": "function", "name": "get_current_weather", "content": "21.0"}, {"role": "assistant", "content": "It is 21 degrees celsius in San Francisco, CA"} ], "functions": [...] // same as before } Migration of legacy models For users migrating from /v1/fine-tunes to the updated /v1/fine_tuning/jobs API and newer models, the main difference you can expect is the updated API. The legacy prompt completion pair data format has been retained for the updated babbage-002 and davinci-002 models to ensure a smooth transition. The new models will support fine-tuning with 4k token context and have a knowledge cutoff of September 2021. For most tasks, you should expect to get better performance from gpt-3.5-turbo than from the GPT base models. FAQ When should I use fine-tuning vs embeddings / retrieval augmented generation? Embeddings with retrieval is best suited for cases when you need to have a large database of documents with relevant context and information. By default OpenAI’s models are trained to be helpful generalist assistants. Fine-tuning can be used to make a model which is narrowly focused, and exhibits specific ingrained behavior patterns. Retrieval strategies can be used to make new information available to a model by providing it with relevant context before generating its response. Retrieval strategies are not an alternative to fine-tuning and can in fact be complementary to it. You can explore the differences between these options further in our Developer Day talk: Can I fine-tune GPT-4 or GPT-3.5-Turbo-16k? GPT-4 fine-tuning is in experimental access and eligible developers can request access via the fine-tuning UI. Currently, gpt-3.5-turbo-1106 supports up to 16K context examples. How do I know if my fine-tuned model is actually better than the base model? We recommend generating samples from both the base model and the fine-tuned model on a test set of chat conversations, and comparing the samples side by side. For more comprehensive evaluations, consider using the OpenAI evals framework to create an eval specific to your use case. Can I continue fine-tuning a model that has already been fine-tuned? Yes, you can pass the name of a fine-tuned model into the model parameter when creating a fine-tuning job. This will start a new fine-tuning job using the fine-tuned model as the starting point. How can I estimate the cost of fine-tuning a model? Please refer to the estimate cost section above. Does the new fine-tuning endpoint still work with Weights & Biases for tracking metrics? No, we do not currently support this integration but are working to enable it in the near future. How many fine-tuning jobs can I have running at once? Please refer to our rate limit guide for the most up to date information on the limits. How do rate limits work on fine-tuned models? A fine-tuned model pulls from the same shared rate limit as the model it is based off of. For example, if you use half your TPM rate limit in a given time period with the standard gpt-3.5-turbo model, any model(s) you fine-tuned from gpt-3.5-turbo would only have the remaining half of the TPM rate limit accessible since the capacity is shared across all models of the same type. Put another way, having fine-tuned models does not give you more capacity to use our models from a total throughput perspective.