Power Automate: Integrate Dall•E/OpenAI (TTI) in Microsoft Teams

One of my favorite use-cases with Power Automate is leveraging its HTTP-action capabilities to integrate with external services mixed with Power Platform and Microsoft services. In one of my previous blogs – Power Automate: Integrate ChatGPT/OpenAI in Microsoft Teams – recursion.no, I shared a guide on how to integrate OpenAI/GPT3 into Microsoft Teams by using a Power Automate flow. In this blog, I aim to expand on that integration by incorporating OpenAI’s cutting-edge Dall•E technology. This allows users to generate custom images based on their input text, making it a powerful and effective tool for creative expression and content producers. Whether you’re a tech enthusiast or just curious about the possibilities of this technology, this blog is the perfect starting point. So buckle up, let’s go geek-mode and learn how to build a Power Automate flow integrating OpenAI Dall•E 2 in Microsoft Teams! 😎

Our goals:

How to work with the OpenAI/Dall•E Text-To-Image API and create an integration to Microsoft Teams, by leveraging the Power Automate services
How to convert base64 string from a JSON payload to binary data
How to use SharePoint site to store images from external services
How to display Dall•E images in Microsoft Teams by using Adaptive Cards for Microsoft Teams.

Prerequisites:

An OpenAI API account registered from – OpenAI API
Power Automate Premium license to use HTTP actions.
An active Office 365 account with access to create new/existing Teams channels

Table of Contents

Create a new dedicated Teams channel (as administrator/owner)

Click on the three dots next to the Teams you want your new channel to be.

Give it a suitable name, and delegate it either to certain teams or publicly… In my case I’ll leave it for the public.

Create your OpenAI API account and token secret

Navigate to https://openai.com/api and create a new account by signing up, if you haven’t done so.

After registration, you need to verify your mail-account in which they will send you a verification in your mailbox. On your first login, you’ll be prompted to provide your personal details and phone no.

Login and navigate to your personal profile menu at the top right corner and click on “**View API Keys**“

Create a new secret key, and make sure to **copy** **and store this key** in a safe place. Take note that each secret keys will last around 3 months according to the documentation (as of date 15.01.2023)

Create the Power Automate flow for integration to a Teams channel

For our simple scenario, create a new Power Automate flow that starts with a trigger – called “**When keywords are mentioned**” and set your preferred Teams + Channel.

Based on the values we can extract from our Teams message, we need to add a “**Get message details**” to get the text information itself.

Continue with an HTTP action and fill out as seen above. A copy of the details is written below. (Instead of pasting in my OpenAI token directly, I use my Key Vault Secret instead).

URI:

https://api.openai.com/v1/images/generations

Method:

POST

Headers:

"Authorization": "Bearer {token secret}"

Body:

{
  "prompt": "@{trim(replace(outputs('Get_message_details')?['body/body/plainTextContent'],'dalle',''))}",
  "response_format": "b64_json"
}

A few explanations and context for our body:

prompt: Can maximum contain 400 words. (When you define your picture in text, keep it short and simple).
response format: Optional input, if this one is not defined then the format is set to “url” which will give you a response with a temporary Azure Blob Storage file link. If the response is “b64_json” then a Base 64 string is returned back. For this guide we are going to set it to the latter as we want to store the files permanently in a SharePoint list.
❗Note: If you plan to change it to a URL in the future, be cautious of using the trigger word “dalle”, as this could cause an infinite trigger loop due to OpenAI’s use of Azure’s Blob Storage service, which features a link containing the word “dalle.” To avoid this issue, I highly recommend taking precautionary measures.❗

Now save and trigger the flow once from your Teams channel (ask about anything) and wait for a response, as we want to parse the JSON values from the returning body.

Post in the “**AI Tools**” channel to trigger the flow and go back to the run log to fetch the HTTP output information received back.

Click on download to expand the full content.

Mark the whole content and copy the Body value outputs and go back to editing your flow.

Paste in the content and click “**Done**“.

Your JSON schema is now defined to access the key-values for the rest of the flow.

Now we want to transform the base64 string value to a real decoded base64 object, and use this decoded base64 value and later on convert it to binary before feeding it to our SharePoint list.

The Compose expression is written below:

decodeBase64(base64(first(body('Parse_JSON_-_OpenAI_Response')?['data'])?['b64_json']))

Use the “Create file” action related to our SharePoint site connected to the teams we created. I’ve also attached another image below to locate the “Folder Path” location, in case you’re unsure about it.

File Name:

string(body('Parse_JSON_-_OpenAI_Response')?['created'])

File Content:

base64ToBinary(outputs('Compose_-_OpenAI_Base64_Image'))

If you’re unsure about your **Folder Path** in the “**Create file**” action above. Try navigate in Microsoft Teams to check where your Teams channel SharePoint location is, you need to navigate to the following address to figure out the folder path in that specific SharePoint location.

Add a “Reply with adaptive card in a channel” action, and replicate the information above.

Message ID:

@{outputs('Compose_-_Message_value_array')?['replyToMessageId']}

Adaptive Card:

{
    "type": "AdaptiveCard",
    "body": [
        {
            "type": "ColumnSet",
            "columns": [
                {
                    "type": "Column",
                    "items": [
                        {
                            "type": "Image",
                            "style": "Person",
                            "url": "{your weburl for icon image}",
                            "size": "Small"
                        }
                    ],
                    "width": "auto"
                },
                {
                    "type": "Column",
                    "items": [
                        {
                            "type": "TextBlock",
                            "weight": "Bolder",
                            "text": "Open AI",
                            "wrap": true,
                            "fontType": "Default"
                        },
                        {
                            "type": "TextBlock",
                            "spacing": "None",
                            "text": "Created @{formatDateTime(utcNow(), 'ddd, MMM dd, yyyy')}",
                            "isSubtle": true,
                            "wrap": true
                        }
                    ],
                    "width": "stretch"
                }
            ]
        },
        {
            "type": "Image",
            "isVisible": true,
            "url": "https://{tenantname}.sharepoint.com/sites/Automation@{outputs('Create_file_-_AI_Tools_folder')?['body/Path']}"
        }
    ],
    "actions": [
        {
            "type": "Action.OpenUrl",
            "title": "Full Size",
            "url": "https://{tenantname}.sharepoint.com/sites/Automation@{outputs('Create_file_-_AI_Tools_folder')?['body/Path']}"
        }
    ],
    "$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
    "version": "1.4"
}

And this was the last action of the flow! Now hurry up and save it(!) to test the solution out! Remember the trigger keyword “dalle“!

TAAADAAA! 🧆My meatballs on a plate, served IKEA style!!😎

All your historical files are now stored in the same file folder of your teams channel, using the equivalent SharePoint location in the “backend” – making it super easy to check for your image collection created previously directly in the Teams channel! Pretty neat! 😉

Our SharePoint library for the specific Teams channel which can be found under the “**Files**” tab.

Summary

We have explored how to use OpenAI/Dall•E API, Power Automate services, SharePoint sites and Adaptive Cards for Microsoft Teams to create an integration between OpenAI/Dall•E and Microsoft Teams. We have gone through the steps of converting base64 strings from a JSON payload to binary data, and how to store images from external services in a SharePoint site. Finally, we have seen how to display Dall•E images in Microsoft Teams using Adaptive Cards. With this knowledge, you can now easily create an integration between OpenAI/Dall•E and Microsoft Teams! 🤖

Our final flow overview looks like this:

Posted

January 30, 2023

Blogs, Power Automate

Dennis Chi

Tags:

api, base64, chatgpt, dall-e, guide, http, openai, power automate, power platform, Technology, tutorial

Comments

6 responses to “Power Automate: Integrate Dall•E/OpenAI (TTI) in Microsoft Teams”

Mathias S. J.

February 6, 2023

Great Article! Could you please expand the first compose action?

Reply
1. Dennis Chi
  
  February 12, 2023
  
  Hey, Mathias! I will try and explain this from most inner expression and work our way outwards. 🙂
  
  1. body(‘Parse_JSON_-_OpenAI_Response’)?[‘data’])?[‘b64_json’]
  The returned values by OpenAI API returns an array of image objects. (The reason it returns an array regardless of 1x picturereturned is because we also have an option to define number of pictures we want, which isn’t used in this article.)
  
  2. first()
  This one is not required, BUT! Because we only get 1 object inside that array, I only need to specify the first object it finds, this is a technique I use, only to avoid the “For each” wrapper that is done by Power Automate for array items, I do this to keep the flow clean.
  
  3. base64()
  Because the OpenAPI sends us a string version of a base64 object, we first need to convert it back to a real base64 object for our Power Automate flow.
  
  4. decodeBase64()
  The base64 object must be decoded before we can further convert it to binary further down the flow to store it in SharePoint.
  
  Hope this clears up abit 🙂
  
  GK
  
  Reply
Marco

April 8, 2023

Hello,
“Adaptive cards ” parameter in “Reply with adaptive card in a channel” could be also an URL?
if not
Which are in “Adaptive Card” json my specific configuration parameter I’ve to add (e.g {tenantname})?
Where the {tenantname} parameter I could find?

Thanks

Reply
1. gchi
  
  April 15, 2023
  
  Hi, Marco! Are you referring to this URL part that I’m highlighting?
  
  If so, this is actually an URL pointing to an image that I use for my Adaptive Card replies as seen below:
  
  Reply
Mostafa

August 9, 2023

I got an error while doing the trim function in the HTTP Post body;

Unable to process template language expressions in action ‘HTTP’ inputs at line ‘0’ and column ‘0’: ‘The template language expression ‘trim(replace(outputs(‘Get_message_details’)?[‘body/body/plainTextContent’], ‘Dalle’,”))’ cannot be evaluated because property ‘body/body/plainTextContent’ cannot be selected. Array elements can only be selected using an integer index. Please see https://aka.ms/logicexpressions for usage details.’.

—–

Here’s the body I used,

{
“prompt”: “@{trim(replace(outputs(‘Get_message_details’)?[‘body/body/plainTextContent’],’Dalle’,”))}”,
“response_format”: “b64_json”
}

Can you help me to fix that issue

Reply
1. Dennis Chi
  
  September 1, 2023
  
  Hey, Mostafa!
  
  Is it maybe possible that this can be the culprit when copying my code?… I know for a fact that sometimes these curly-brackets can mess up on the code when copying and pasting from a project to another. Can you try commenting out/remove the following part on the code as highlighted:
  
  Reply