Power Automate: Integrate ‘gpt-3.5-turbo’ Model And ‘GPT-4’ Model With Microsoft Teams

Update: Included the GPT-4 Model header as I got the beta access a week after publishing the article, and it’s basically the same implementation.

GPT-3.5-Turbo was released last week and added another arsenal for developers to work with OpenAI’s public API services. This week we’re going to have a look at how we can upgrade our existing Microsoft Teams integration and upgrade it with a chat functionality with the possibility to remember the conversation history and context.

We’ve previously touched on the same, but this one will be a bit different, because of involving the array of chat context and dealing with the token limits:

Power Automate: Integrate Dall•E/OpenAI (TTI) in Microsoft Teams – recursion.no

Power Automate: Integrate GPT-3/OpenAI in Microsoft Teams – recursion.no

This guide will demonstrate how to leverage Microsoft Power Automate to seamlessly integrate Microsoft Teams with OpenAI’s ‘gpt-3.5-turbo’ model. Additionally, it will provide a simplified technique for memorizing part of the conversation when the prompt exceeds the token limits.

---

Our goals:

• How to work with the OpenAI ‘gpt-3.5-turbo’ Chat completion API and create an integration to Microsoft Teams, by leveraging Power Automate features.
• Create a basic technique to keep the chat history retention for ChatGPT to answer in context of the thread, without exceeding the token threshold limit.

Prerequisites:

• An OpenAI API account registered from – OpenAI API
• Power Automate Premium license to use HTTP actions.
• An active Office 365 account with access to Azure App Registration and user rights to create new / edit existing Teams channels

❗Attention:

Hey you! Before reading further, I would like to give you a heads up that while going through the guide, I’ve noticed that I keep using an unsafe keyword in the flow (‘chatgpt’). It’s crucial to note that this can cause a post message loop by the API, which can lead to unexpected results. I highly recommend you to change it to something else like ‘gptgpt’ or a more unusual trigger word. (the replace methods are not working as I expected they would). I’ve replaced the code snippets to match the ‘gptgpt’ keyword in the guide, but the screenshots themselves may have traces where it says ‘chatgpt’. Cheers!

---

---

Create a Teams API application for our HTTP request in Power Automate

This API application we’re setting up is designed to allow extraction of a Teams Message Reply thread through the Power Automate HTTP action, as the Microsoft Teams connector in Power Automate does not have an action for getting thread messages for each thread.

Start by navigating to your Azure tenant.

First navigate to the “App registration” service in Azure Portal and click on “New registration“

When registering a new application, make sure to give it a recognizable name to identify what it’s used for. In my case I called it “Microsoft Teams API”. Click on “Register” to finish.

From the overview save the “Object ID” value

Now that’ the App has been created, let’s go ahead and configure the permissions to our Key Vaults service. Navigate to the “API permissions” and click on “Add a permission“.

The last few configurations that we need before we’re done with the whole Key Vault thing… Save the Client secret value

Request access to Teams protected APIs

Because of security concerns and restrictions you will also need to get access approval through a Microsoft forms to the Microsoft Graph team. The applications are usually approved within 3-4 days, based on my own experience. Send the application from here: https://aka.ms/teamsgraph/requestaccess

---

Create a new dedicated Teams channel (as administrator/owner)

Click on the three dots next to the Teams you want your new channel to be.

Give it a suitable name, and delegate it either to certain teams or publicly… In my case I’ll leave it for the public.

---

Create your OpenAI API account and token secret

Navigate to https://openai.com/api and create a new account by signing up, if you haven’t done so.

After registration, you need to verify your mail-account in which they will send you a verification in your mailbox. On your first login, you’ll be prompted to provide your personal details and phone no.

Login and navigate to your personal profile menu at the top right corner and click on “View API Keys“

Create a new secret key, and make sure to copy and store this key in a safe place. Take note that each secret keys will last around 3 months according to the documentation (as of date 15.01.2023)

---

Create the Power Automate flow for integration to a Teams channel

Add the trigger for the channel

Message:

first(triggerOutputs()?['body/value'])?['replyToMessageId']

Parent message ID:

first(triggerOutputs()?['body/value'])?['replyToMessageId']

Compose the array of tables – which initializes the behavior of the AI-assistant – change and describe to your liking if you want to:

[
  {
    "role": "system",
    "content": "Assistant is a large language model trained by OpenAI."
  },
  {
    "role": "@{if(equals(outputs('Get_message_details')?['body/from/user'],'null'),'assistant','user')}",
    "content": "@{trim(replace(outputs('Get_message_details')?['body/body/plainTextContent'],'gptgpt',''))}"
  }
]

Add the composed object to an initialized array, we are going to work with it based on chat thread over time:

URI:

https://graph.microsoft.com/v1.0/teams/@{first(triggerOutputs()?['body/value'])?['teamId']}/channels/@{first(triggerOutputs()?['body/value'])?['channelId']}/messages/@{first(triggerOutputs()?['body/value'])?['replyToMessageId']}/replies

Great! Next, create a thread in the Teams channel with three messages. Once you’ve done that, trigger the flow on the last reply. This will allow us to create the JSON schema based on the payload from the last reply’s body.

Add a parse action. Generate the body schema by “Generate from sample”: Paste in the body values in clear text from last run and click “Done”.

Next up is appending the array of thread messages that was found in our Array Variable that we created earlier. And for that I want to start with initializing a new variable that only contains a blank newline – calling it “nl“:

Now apply an “Apply to each” loop:

And we’re going to loop through the list in reverse (bottom up):

reverse(body('Parse_JSON_-_Get_Teams_Chat_Thread')?['value'])

Inside the loop, add the following actions:

items('Apply_to_each_-_Thread_Reply')?['from']?['application']?['displayName']

{
  "role": "@{if(equals(outputs('Compose_-_from_-_displayName'), 'Flow'), 'assistant', 'user')}",
  "content": "@{trim(replace(replace(replace(replace(replace(replace(replace(replace(replace(replace(replace(items('Apply_to_each_-_Thread_Reply')?['body']?['content'],'  ',' '),'<div>',''),'</div>',''),'<p>',''),'</p>',''),'<li>',''),'</li>',''),'<ol>',''),'</ol>',''),'&nbsp;',' '),variables('nl'),''))}"
}

outputs('Compose_Array_Object_for_Teams_Chat_Thread')

Great, now we can continue with the OpenAI API call with the HTTP-action. In my scenario, I use a “Get secret” action to fetch my stored API token from my Azure Key Vault:

URI:

https://api.openai.com/v1/chat/completions

Header Authorization:

Bearer *Your_API_Key*

Body:

{
  "model": "gpt-3.5-turbo",
  "messages": @{json(trim(replace(string(variables('chatThread')),'gptgpt','')))},
  "max_tokens": 3500,
  "temperature": 0.7
}

Now save and trigger the flow once from your Teams channel (ask ChatGPT about anything) and wait for a response as we want to parse the JSON values from the response.

Copy the above output body as sample.

Add a Parse JSON action, click on “Generate from sample” and paste the value and click “Done“:

Add a Compose-action and give the input the following:

first(body('Parse_JSON_-_OpenAI_Response')?['choices'])?['message/content']

Now the final step is to add a “Reply with a message in a channel” action

Message ID:

first(triggerOutputs()?['body/value'])?['replyToMessageId']

Message:

outputs('Compose_-_OpenAI_Response')

Now when you test Teams thread to ChatGPT again you should get the following result:

---

Solve the Token limit temporarily (No embeddings… yet)

Although it’s a quick and dirty workaround, I’ve been using a temporary technique until I can find more time to learn about embeddings combined with semantic search. Specifically, I use another HTTP-action as a catcher/error handler and retry with a different prompt. If you have expertise in this area and don’t mind pair-programming with me, I would be happy to join you on a session via Discord! 🙌🙌🙌

Below the HTTP-action to call the OpenAI API, add a Compose action to limit the array.

Input:

union(outputs('Compose_-_Initial_AI_behaviour_value'),reverse(take(reverse(variables('chatThread')),2)))

Now configure it’s “run after” with “has failed” checkbox ticked:

Add another HTTP-action below it:

The only thing that’s changed above is the messages key-value and max token set to 2000 as compared to the previous OpenAI API call.

Body messages key-value:

json(trim(replace(string(outputs('Compose_-_Last_two_conversation_in_memory')),'gptgpt','')))

Body “max_tokens” ~ key-value:

2000

Now on the “Parse JSON” input. Change the Content input to the following:

Content:

if(equals(outputs('HTTP_-_OpenAI')['statusCode'],200),body('HTTP_-_OpenAI'),body('HTTP_-_OpenAI_Catch'))

Now configure Parse JSON action run after to only run when the previous action “is successful” or “is skipped” and press “Done“:

NOW YOU CAN SAVE THE FLOW! 🙌

Below you can see an example when I continue the thread it is less likely to hit the token limits, however it only checks the context with the original message and the latest two chat messages when it reaches the catch HTTP-action.

Congratulations! Now you have a fully functional ChatGPT in your Microsoft Teams!

Summary

We’ve developed an integration with Microsoft Teams using the OpenAI ‘gpt-3.5-turbo’ Chat completion API and Power Automate tools. To ensure that ChatGPT responds in context of the thread without exceeding the token threshold limit, we created a basic catch technique to retain chat history. Along the way, we gained experience working with array objects, learned how to implement an API application through Azure AD App registration, and discovered that the Teams API policy is stricter than we anticipated. Additionally, we had the opportunity to apply a Microsoft Forms. And lastly you’ve gained the experience of applying a Microsoft Forms. 😂😂😂

Below is the picture of the entire flow: