Power Automate: Integrate ‘gpt-3.5-turbo’ Model And ‘GPT-4’ Model With Microsoft Teams

Update: Included the GPT-4 Model header as I got the beta access a week after publishing the article, and it’s basically the same implementation.

GPT-3.5-Turbo was released last week and added another arsenal for developers to work with OpenAI’s public API services. This week we’re going to have a look at how we can upgrade our existing Microsoft Teams integration and upgrade it with a chat functionality with the possibility to remember the conversation history and context.

We’ve previously touched on the same, but this one will be a bit different, because of involving the array of chat context and dealing with the token limits:

Power Automate: Integrate Dall•E/OpenAI (TTI) in Microsoft Teams – recursion.no

Power Automate: Integrate GPT-3/OpenAI in Microsoft Teams – recursion.no

This guide will demonstrate how to leverage Microsoft Power Automate to seamlessly integrate Microsoft Teams with OpenAI’s ‘gpt-3.5-turbo’ model. Additionally, it will provide a simplified technique for memorizing part of the conversation when the prompt exceeds the token limits.


Our goals:

  • How to work with the OpenAI ‘gpt-3.5-turbo’ Chat completion API and create an integration to Microsoft Teams, by leveraging Power Automate features.
  • Create a basic technique to keep the chat history retention for ChatGPT to answer in context of the thread, without exceeding the token threshold limit.

Prerequisites:

  • An OpenAI API account registered from – OpenAI API
  • Power Automate Premium license to use HTTP actions.
  • An active Office 365 account with access to Azure App Registration and user rights to create new / edit existing Teams channels

Attention:

Hey you! Before reading further, I would like to give you a heads up that while going through the guide, I’ve noticed that I keep using an unsafe keyword in the flow (‘chatgpt’). It’s crucial to note that this can cause a post message loop by the API, which can lead to unexpected results. I highly recommend you to change it to something else like ‘gptgpt’ or a more unusual trigger word. (the replace methods are not working as I expected they would). I’ve replaced the code snippets to match the ‘gptgpt’ keyword in the guide, but the screenshots themselves may have traces where it says ‘chatgpt’. Cheers!



Create a Teams API application for our HTTP request in Power Automate

This API application we’re setting up is designed to allow extraction of a Teams Message Reply thread through the Power Automate HTTP action, as the Microsoft Teams connector in Power Automate does not have an action for getting thread messages for each thread.

Start by navigating to your Azure tenant.

First navigate to the “App registration” service in Azure Portal and click on “New registration

When registering a new application, make sure to give it a recognizable name to identify what it’s used for. In my case I called it “Microsoft Teams API”. Click on “Register” to finish.

From the overview save the “Object ID” value

Now that’ the App has been created, let’s go ahead and configure the permissions to our Key Vaults service. Navigate to the “API permissions” and click on “Add a permission“.

Add the highlighted by using the Request API permissions “Blade” after selecting “Add a permission” as described on the below steps

The last few configurations that we need before we’re done with the whole Key Vault thing… Save the Client secret value

Request access to Teams protected APIs

Because of security concerns and restrictions you will also need to get access approval through a Microsoft forms to the Microsoft Graph team. The applications are usually approved within 3-4 days, based on my own experience. Send the application from here: https://aka.ms/teamsgraph/requestaccess


Create a new dedicated Teams channel (as administrator/owner)

Click on the three dots next to the Teams you want your new channel to be.

Give it a suitable name, and delegate it either to certain teams or publicly… In my case I’ll leave it for the public.


Create your OpenAI API account and token secret

Navigate to https://openai.com/api and create a new account by signing up, if you haven’t done so.

After registration, you need to verify your mail-account in which they will send you a verification in your mailbox. On your first login, you’ll be prompted to provide your personal details and phone no.

Login and navigate to your personal profile menu at the top right corner and click on “View API Keys

Create a new secret key, and make sure to copy and store this key in a safe place. Take note that each secret keys will last around 3 months according to the documentation (as of date 15.01.2023)


Create the Power Automate flow for integration to a Teams channel

Add the trigger for the channel

Message:

first(triggerOutputs()?['body/value'])?['replyToMessageId']

Parent message ID:

first(triggerOutputs()?['body/value'])?['replyToMessageId']

Compose the array of tables – which initializes the behavior of the AI-assistant – change and describe to your liking if you want to:

[
  {
    "role": "system",
    "content": "Assistant is a large language model trained by OpenAI."
  },
  {
    "role": "@{if(equals(outputs('Get_message_details')?['body/from/user'],'null'),'assistant','user')}",
    "content": "@{trim(replace(outputs('Get_message_details')?['body/body/plainTextContent'],'gptgpt',''))}"
  }
]

Add the composed object to an initialized array, we are going to work with it based on chat thread over time:

URI:

https://graph.microsoft.com/v1.0/teams/@{first(triggerOutputs()?['body/value'])?['teamId']}/channels/@{first(triggerOutputs()?['body/value'])?['channelId']}/messages/@{first(triggerOutputs()?['body/value'])?['replyToMessageId']}/replies

Great! Next, create a thread in the Teams channel with three messages. Once you’ve done that, trigger the flow on the last reply. This will allow us to create the JSON schema based on the payload from the last reply’s body.

Add a parse action. Generate the body schema by “Generate from sample”: Paste in the body values in clear text from last run and click “Done”.

Next up is appending the array of thread messages that was found in our Array Variable that we created earlier. And for that I want to start with initializing a new variable that only contains a blank newline – calling it “nl“:

Now apply an “Apply to each” loop:

And we’re going to loop through the list in reverse (bottom up):

reverse(body('Parse_JSON_-_Get_Teams_Chat_Thread')?['value'])

Inside the loop, add the following actions:

items('Apply_to_each_-_Thread_Reply')?['from']?['application']?['displayName']
{
  "role": "@{if(equals(outputs('Compose_-_from_-_displayName'), 'Flow'), 'assistant', 'user')}",
  "content": "@{trim(replace(replace(replace(replace(replace(replace(replace(replace(replace(replace(replace(items('Apply_to_each_-_Thread_Reply')?['body']?['content'],'  ',' '),'<div>',''),'</div>',''),'<p>',''),'</p>',''),'<li>',''),'</li>',''),'<ol>',''),'</ol>',''),'&nbsp;',' '),variables('nl'),''))}"
}
outputs('Compose_Array_Object_for_Teams_Chat_Thread')

Great, now we can continue with the OpenAI API call with the HTTP-action. In my scenario, I use a “Get secret” action to fetch my stored API token from my Azure Key Vault:

URI:

https://api.openai.com/v1/chat/completions

Header Authorization:

Bearer *Your_API_Key*

Body:

{
  "model": "gpt-3.5-turbo",
  "messages": @{json(trim(replace(string(variables('chatThread')),'gptgpt','')))},
  "max_tokens": 3500,
  "temperature": 0.7
}

Now save and trigger the flow once from your Teams channel (ask ChatGPT about anything) and wait for a response as we want to parse the JSON values from the response.

Copy the above output body as sample.

Add a Parse JSON action, click on “Generate from sample” and paste the value and click “Done“:

Add a Compose-action and give the input the following:

first(body('Parse_JSON_-_OpenAI_Response')?['choices'])?['message/content']

Now the final step is to add a “Reply with a message in a channel” action

Message ID:

first(triggerOutputs()?['body/value'])?['replyToMessageId']

Message:

outputs('Compose_-_OpenAI_Response')

Now when you test Teams thread to ChatGPT again you should get the following result:


Solve the Token limit temporarily (No embeddings… yet)

Although it’s a quick and dirty workaround, I’ve been using a temporary technique until I can find more time to learn about embeddings combined with semantic search. Specifically, I use another HTTP-action as a catcher/error handler and retry with a different prompt. If you have expertise in this area and don’t mind pair-programming with me, I would be happy to join you on a session via Discord! 🙌🙌🙌

Below the HTTP-action to call the OpenAI API, add a Compose action to limit the array.

Input:

union(outputs('Compose_-_Initial_AI_behaviour_value'),reverse(take(reverse(variables('chatThread')),2)))

Now configure it’s “run after” with “has failed” checkbox ticked:

Add another HTTP-action below it:

The only thing that’s changed above is the messages key-value and max token set to 2000 as compared to the previous OpenAI API call.

Body messages key-value:

json(trim(replace(string(outputs('Compose_-_Last_two_conversation_in_memory')),'gptgpt','')))

Body “max_tokens” ~ key-value:

2000

Now on the “Parse JSON” input. Change the Content input to the following:

Content:

if(equals(outputs('HTTP_-_OpenAI')['statusCode'],200),body('HTTP_-_OpenAI'),body('HTTP_-_OpenAI_Catch'))

Now configure Parse JSON action run after to only run when the previous action “is successful” or “is skipped” and press “Done“:

NOW YOU CAN SAVE THE FLOW! 🙌

Below you can see an example when I continue the thread it is less likely to hit the token limits, however it only checks the context with the original message and the latest two chat messages when it reaches the catch HTTP-action.

Congratulations! Now you have a fully functional ChatGPT in your Microsoft Teams!

Summary

We’ve developed an integration with Microsoft Teams using the OpenAI ‘gpt-3.5-turbo’ Chat completion API and Power Automate tools. To ensure that ChatGPT responds in context of the thread without exceeding the token threshold limit, we created a basic catch technique to retain chat history. Along the way, we gained experience working with array objects, learned how to implement an API application through Azure AD App registration, and discovered that the Teams API policy is stricter than we anticipated. Additionally, we had the opportunity to apply a Microsoft Forms. And lastly you’ve gained the experience of applying a Microsoft Forms. 😂😂😂

Below is the picture of the entire flow:

Comments

6 responses to “Power Automate: Integrate ‘gpt-3.5-turbo’ Model And ‘GPT-4’ Model With Microsoft Teams”

  1. Lampros Karalis Avatar
    Lampros Karalis

    Dear Sirs, Thank you for sharing such a great work.
    I tried to utilize your guide, but i am stuck at the step where it supposed to retrieve the client-id and the client secret. The error i am getting is “Operation against key vault ‘https://Azure Key Vault.vault.azure.net/’ failed as the vault name and/or input parameters are not valid.”
    However, i did add Get Secret, I used as authentication type ‘Service Principal authetincation’ and fill the proper fields. Do you have an idea, of what i might be missing?

    1. gchi Avatar

      Hi, Lampros! 🙌

      The Keyvault part is actually something I didn’t describe more details on because I’ve done that a few other blogs…
      There is a section from another blog in which I describe more about how I usually implement my Azure Key Vaults Secrets:

      Getting Started: An Introduction to create Key Vaults for Beginners

      It is not necessary of course (and just for the case of hiding sensitive information), in the case of you not being able to make it work with Azure Key Vaults secrets, then you can instead just input the ClientID and ClientSecret values from OpenAI in plain text in your flows (instead of fetching the values by using Azure Key Vaults service)!

      Hope the above link and my comment explains abit.

      1. Lampros Karalis Avatar
        Lampros Karalis

        Thanks a lot for your help! I did move forward 🙂
        I am close to complete the quide!

  2. Lampros Karalis Avatar
    Lampros Karalis

    Thanks a lot once again for your geat guide, I finally made it and completed all the steps, and i thought i should share a couple of things that caused me a headache, someone might find them usefull. (Most probably it is not best practice, but it worked for me)

    1. MS Graph token Expiration. Default time is 60 min, so i added an extra http step to retrieve a new token everytime the flow runs
    2. On step ‘Parse JSON – Get Teams Chat Thread’, the generated schema shows a required object (properties/value/items/properties/from/properties/user) which is null for the bot responses. So it retrieved the first response, but it failed for the rest of them. I deleted it manually, and everything worked as expected.

    All i would wish for the next step would be to change the Flow Bot icon and name. I haven find anything so far, but it is not too important.

    1. Lampros Karalis Avatar
      Lampros Karalis

      … And #3 Microsoft’s approval took 8-9 days for my case

    2. gchi Avatar

      Thank you for the tips, Lampros!

      I’m sure it would be of good use for people that haven’t gotten the context.

      1. In my case I also have my own flows to auto update all my tokens stored in Key Vaults. You can find a guide of it here as well: https://recursion.no/blogs/power-automate-store-your-api-access-tokens-in-azure-key-vaults-and-automatically-renew-them/
      2. That’s a good tips.
      3. About the bot-icons, I usually find my icons (in this case OpenAI icon) from: https://simpleicons.org/
      Note the website might be loading slowly, because it’s designed to load all icons it has in the front-page. Let it take a minute’ish and you’ll be able to search the Open AI icon there. I usually just download it and then edit it or convert it with SnagIt which I use frequently for my guides (try with other softwares that can convert svg’s to e.g. png and upload to web to use it.
      4. And yes I have a feeling that the approval is a manual process, I’ve heard rumors that they usually do the approvals on Wednesdays or Fridays…

      Hope this helps!

      GK

Leave a Reply

Your email address will not be published. Required fields are marked *