Prompting In the Wild #2: Date Extractor
Welcome back to our “Prompting in the Wild” series where we detail the prompts we use at Glowstick for mutual learning.
Today we’re gonna target our Renewal Date Extractor, when a speaker mentions a renewal date coming up, we would like to extract it. Most likely, there is no date and we need to take that into account.
We’ll cover some examples and then present our current prompt crafted by our fearless CEO: Masha Krol!
As usual, please let us know any tips and tricks you might have for this particular task!
Examples
| Snippet | Meeting Date | Answer |
|---|---|---|
| So our renewal is coming up on December 1st. | 2023-02-01 | 2023-12-01 |
| As we’ve discussed, our last day with JetBrains is next month. | 2022-09-01 | 2022-10-01 |
| Our contract is coming up on the 16th of January I think. | 2023-11-04 | 2024-01-16 |
| I don’t think I know the renewal date for Furia. | 2023-04-29 | NA |
Please note, that the Agent must output “NA” when there is no answer. This has been proven difficult to handle for common techniques like regexes, or more advanced NLP techniques.
Moreover, the way customer express their renewal comes in such a variety of manners that using LLMs is the perfect choice for a quick implementation.
After many iterations, we settled on this Agent crafted by Masha Krol
Renewal Date Extractor
Prompt:
Your task is to extract the renewal date for a product from a small portion of a transcribed meeting recording.
You should output the date of the renewal, taking into account the date of the meeting.
If you cannot find the exact day of renewal, output the first of that month. You must output "NA" when you do not know the answer.
Example:
Input: "Our Microsoft contract is up for renewal in June", the meeting date is February 7, 2024.
Answer: June 1, 2024
Input: "Our Apple contract is up for renewal in June", the meeting date is April 2, 2023.
Answer: June 1, 2023
Model: GPT-4
Run Instruction: the meeting date is {meetingDate}
Note that we do a bit of supervision by providing examples. After our evaluation, we found this approach results in better performance in almost all cases. This also allows us to provide instructions on the expected format which we can then parse using datetime.strptime.
If you’ve made it this far, let me know in the comment and tell me about common failure cases you see in this task!
See you next time,
Fred