Director of Innovation in AI and Analytics
Data & AI Evangelist
Subscribe to the newsletter
As data volumes continue to grow, businesses are increasingly seeking innovative methods to convert raw data into actionable insights. Data scientists and data analysts are now turning to Generative AI to concentrate on the “why” and “how” of data analysis, while the AI takes care of the “what.” Generative AI technology is empowering businesses to gain deeper insights, automate complex tasks, enhance decision-making, and expand the possibilities within the field.
Microsoft Fabric has made significant strides in Generative AI by introducing AI Skill, a new capability that enables users to craft custom Q&A experiences using their own data.
Microsoft Fabric AI Skill: A new update for creating custom Q&A experiences with your data
AI Skill lets you effortlessly select a data source in Microsoft Fabric and start querying your data right away, without any setup required. When you ask a question, the generative AI will display the query it generated to find the answer. You can further refine the Q&A experience by adding more tables, providing additional context, and adjusting settings. AI Skills empower users to explore data, create and customize AI-driven experiences, and gain the necessary insights.
Microsoft Fabric AI Skill empowers your colleagues to explore datasets more easily and extract valuable insights. Imagine the possibilities when your data becomes more accessible and understandable to everyone!
Learn more: Explore Microsoft Fabric AI Skill, a new update in MS Fabric.
But how can you implement AI Skills in Microsoft Fabric and start using this feature? This blog offers a detailed, step-by-step process for getting started with AI Skills in Microsoft Fabric. We will explore the key steps involved, from creating an AI Skill to training and deploying it.
The challenge of traditional data analysis
In many organizations, centralized or embedded analyst teams are tasked with answering a wide range of data-related questions. Among the top data analysis challenges is extracting the true value of data. While these inquiries may not always be complex, they often require knowledge of query languages like SQL and a deep understanding of the data’s context. This dependency on a select group of individuals leads to bottlenecks as these teams struggle to keep up with the constant influx of requests. Because of this challenge in data analysis, valuable data-driven insights remain trapped within the organization, accessible only to a few.
This constant demand for data answers also detracts from more strategic tasks. Analysts find themselves spending excessive time on routine questions, leaving little room for advanced analysis or proactive planning.
How can Microsoft Fabric AI Skill resolve data bottlenecks and overcome challenges in data analysis?
Generative AI has already demonstrated its ability to write queries effectively. However, many Generative AI applications lack the nuanced understanding and context required for accurate real-world data analysis. AI Skills bridge this gap.
Microsoft Fabric AI Skill enables you to train your AI systems with the critical context and nuance that analysts typically acquire over time. Without proper background information, you can’t expect an AI to deliver accurate insights. Equipping your AI with the necessary context empowers it to provide reliable and accurate answers, freeing up your analysts to focus on more sophisticated and strategic tasks.
How to create an AI Skill in Microsoft Fabric? Step-by-step process explained
Microsoft Fabric AI Skill offers a powerful way to create conversational Q&A systems that make use of cutting-edge Generative AI. However, AI technology in Microsoft Fabric is transforming businesses to manage and gain insights from their diverse datasets.
Prerequisites to get started
Prior to getting started with AI Skill in Microsoft Fabric, you should ensure that you meet the following necessary prerequisites:
- An F64 Fabric capacity or higher.
- The copilot tenant switch is enabled.
- Cross-geo sharing for AI is enabled, if relevant.
- A warehouse or Lakehouse with data.
Setting up an AI Skill in Microsoft Fabric
Creating and configuring an AI Skill in Microsoft Fabric is quick and easy. You can start testing AI skills in no time. Here are the steps to get started:
- Set up a new AI Skill
- Choose your data
- Start asking questions
- Switch the data source
- Guide AI Skill with examples
- Guide AI Skill with instructions
Now let’s look into these steps in detail and start making conversations with your data.
Step 1: Set up a new AI Skill
Creating a new AI Skill in Microsoft Fabric is as simple as creating other standard Fabric items. You can do it directly from the Fabric Data Science homepage, choose the “New” option in your workspace, or go through the Create Hub. Also, make sure to name your AI Skill.
Step 2: Choose your data
Once you’ve created your AI skill, the next step is to choose a data source. This could be a data warehouse or a Lakehouse. On the following screen, pick your desired warehouse or Lakehouse and then click “Connect.”
You’ll see the available tables from the data source listed in the left pane. You can use the checkboxes to decide which tables the AI will have access to. Remember that you’ll need to select at least one table before the AI skill can answer your questions.
Step 3: Start asking questions
Once you’ve selected your data, you can start asking questions right away. The AI system is designed to handle questions that can be answered with a single query.
When you ask a question, the system uses your credentials to access the schema. It then combines this information with the examples and instructions you’ve provided to create a prompt. This prompt is sent to the AI, which generates multiple SQL queries based on your question.
After generating SQL queries, look closely at them to ensure they only query the data and don’t make any changes- like creating, updating, or deleting anything. Once you’ve identified the best query from the list, you can make any necessary adjustments. Finally, with your credentials, the system will regenerate the query and deliver the results.
Step 4: Switch the data source
If you want to switch to a different Lakehouse or warehouse, just click on the arrows located at the top of the Explorer pane.
Step 5: Guide AI Skill with examples
In Fabric, you can fine-tune how the AI responds to your questions by providing examples. This approach, known as “few-shot learning” in Generative AI, involves giving the AI pairs of queries and questions. When you ask a new question, the AI looks at the examples you’ve provided and uses the most relevant ones as a reference, along with the corresponding SQL query, to generate the right SQL for your needs.
If the AI isn’t quite getting it right, you can always add more examples to improve its accuracy.
To add examples, simply click the edit button under “Example SQL queries” on the right side of the screen. Refer to the screenshot below:
Step 6: Guide AI Skill with instructions
You can guide the AI by providing instructions in the “Notes for model” text box. Just type your instructions in English, and the AI will use them to help generate the SQL queries.
If the AI seems to misunderstand specific words or acronyms frequently, you can define these terms in this section to improve its accuracy.
Microsoft Fabric AI Skill is available in public preview
Microsoft Fabric AI skill is currently available in public preview. This means you can start using it to create AI-driven experiences and answer data-related questions within your organization.
As outlined in the prerequisites, you must use Fabric F64 or a larger capacity to access AI Skill. Before you can try this new feature, your tenant admin must enable both the preview experience and the Fabric AI setting.
Start benefiting from AI Skills
Microsoft Fabric AI Skill is a remarkable leap when it comes to applying queries to your data. With this capability, you cannot only generate queries but also understand the deeper context of your data.
Want to explore how Microsoft Fabric AI Skill can transform your data interactions? Our experts at Confiz can offer Microsoft Fabric AI Skill implementation service and customize it to solve a problem or extract particular data. Reach out to us at marketing@confiz.com to learn how AI Skills can meet your specific business needs through Q&A experiences.
