AI for M365 Admins — Chapter 1: How Microsoft 365 Copilot Actually Works (Semantic Index, Graph Grounding & Tenant Boundary)

📚 AI for Microsoft 365 Administrators Chapter 1 of 8
View Full Book TOC →
🤖 Chapter 1 · Learning Objectives AI for M365 Admins · Book Series
What You Will Master in Chapter 1
  • The 3-Engine Architecture: How Microsoft 365 apps, Microsoft Graph + Semantic Index, and the LLM orchestrator work together during every prompt.
  • The Security Truth: Why Copilot respects every existing SharePoint, OneDrive, Teams, and Exchange ACL—and why oversharing is the real threat.
  • Tenant Data Boundary: What leaves your tenant (nothing for model training), what stays inside the Microsoft 365 service boundary, and how web grounding works.
  • Hands-On Lab: Run a read-only Microsoft Graph PowerShell readiness audit against your tenant to inventory Copilot-eligible SKUs and baseline settings.
Get-MgSubscribedSku | Select SkuPartNumber, ConsumedUnits, @{N='Enabled';E={$_.PrepaidUnits.Enabled}}

Before you configure a single Data Loss Prevention policy or assign a single Microsoft 365 Copilot license, you need an accurate mental model of what happens in the 3 seconds between a user hitting Enter and Copilot displaying an answer.

Most executive misconceptions—both "Copilot will train ChatGPT on our board decks" (false) and "Copilot won't show confidential files because they're in an HR folder" (false if permissions are broken)—come from misunderstanding the underlying architecture.

In Chapter 1, we strip away the marketing diagrams and trace the exact technical request lifecycle inside Microsoft 365.


End-to-End Microsoft 365 Copilot Request Lifecycle (Inside the Tenant Boundary) 1. User Prompt Teams, Outlook, Word, Excel, or Copilot Chat Carries user's live Entra ID OAuth token Conditional Access applied 2. Grounding (RAG) Orchestrator queries Microsoft Graph & Semantic Index vectors Filters by user ACLs + Purview DLP / RSS rules User-level index + Tenant index 3. LLM Inference Modified prompt + retrieved snippets sent to Azure OpenAI host Stateless inference: zero model training Inside M365 Service Boundary 4. Post-Processing Responsible AI safety & Sensitivity label check Citations returned to user + UAL logged CopilotInteraction event Core Principle: Copilot uses Retrieval-Augmented Generation (RAG) scoped strictly to the authenticated user's permissions.

1. The Three Building Blocks Every M365 Admin Must Understand

A. Retrieval-Augmented Generation (RAG), Not Fine-Tuning

Microsoft does not train or fine-tune a custom AI model on your company's emails and SharePoint files. Instead, Copilot uses Retrieval-Augmented Generation (RAG):
  1. The user asks: "Summarize the Q4 network migration risks from last week's emails and SharePoint docs."
  2. The Copilot orchestrator translates that prompt into search queries against Microsoft Graph and the Semantic Index.
  3. Only the relevant text chunks the user has Read permission to see are retrieved and injected into a temporary prompt context ("grounding").
  4. The LLM generates a response from that temporary context, attaches clickable source citations, and immediately discards the working memory.

B. The Semantic Index (User-Level vs. Tenant-Level)

Traditional keyword search fails when a user asks about "layoff risks" while the document says "headcount reduction plan". The Semantic Index solves this by creating conceptual vector embeddings of your organization's content:
  • User-Level Index: Covers the user's own Exchange Online mailbox, Teams chats, and personal OneDrive files. Stored inside the user's mailbox substrate boundary.
  • Tenant-Level Index: Covers SharePoint Online sites and shared libraries. When a user prompts Copilot, the index only returns vectors for documents whose Access Control Lists (ACLs) include that user.

C. Web Grounding (Bing Search Toggle)

When Allow web search in Copilot is enabled, the orchestrator can generate a short, de-identified web search query (stripped of tenant IDs and user names) and send it to the Bing search service to fetch public internet facts (e.g., current CVE numbers or public vendor pricing). Admins can control this via Cloud Policy (Allow web search in Copilot) or the Microsoft 365 Admin Center.

2. The Myth vs. Reality Table (Share With Your Leadership)

Common Executive ConcernTechnical Reality in Microsoft 365
"Will our prompts train public OpenAI models?"No. Prompts, grounded data, and responses remain inside the Microsoft 365 service boundary and are never used to train foundation models.
"Can Copilot read an HR folder if the user isn't in HR?"Only if permissions are broken. If the HR SharePoint site was shared with Everyone except external users, Copilot can read it—because the user already could.
"Does Copilot bypass sensitivity label encryption?"No. If a document is encrypted with a Purview sensitivity label that does not grant the user View and Extract rights, Copilot cannot ground on it.
"Can we see what files Copilot referenced?"Yes. Every interaction generates a CopilotInteraction record in the Microsoft Purview Unified Audit Log listing the user, app host, and accessed resources.

3. Hands-On Lab 1: Run Your Tenant Copilot Readiness Baseline

Before moving to Chapter 2 (fixing SharePoint oversharing), run this read-only Microsoft Graph PowerShell script to inventory your tenant's organization metadata, verified domains, and current Microsoft 365 / Copilot license SKUs.

POWERSHELL
<#
.SYNOPSIS
    Chapter1-CopilotReadinessAudit.ps1
    Read-only Microsoft Graph baseline check for AI for M365 Admins (Chapter 1).
#>

# Connect with read-only Organization permissions (Interactive or CBA)
Connect-MgGraph -Scopes "Organization.Read.All", "Directory.Read.All" -NoWelcome

Write-Host "=== 1. Tenant Identity & Boundary ===" -ForegroundColor Cyan
Get-MgOrganization | Select-Object DisplayName, Id, TenantType, CreatedDateTime | Format-Table -AutoSize

Write-Host "=== 2. License Inventory (Checking Base E3/E5 & Copilot SKUs) ===" -ForegroundColor Cyan
$skus = Get-MgSubscribedSku
$skus | Select-Object SkuPartNumber, CapabilityStatus, ConsumedUnits,
    @{Name = "Purchased"; Expression = { $_.PrepaidUnits.Enabled } },
    @{Name = "Available"; Expression = { $_.PrepaidUnits.Enabled  $_.ConsumedUnits } } |
    Sort-Object SkuPartNumber | Format-Table -AutoSize

# Highlight any Copilot SKUs found
$copilotSkus = $skus | Where-Object { $_.SkuPartNumber -match "COPILOT|M365_AI" }
if ($copilotSkus) {
    Write-Host "[+] Copilot SKUs detected in tenant: $($copilotSkus.SkuPartNumber -join ', ')" -ForegroundColor Green
} else {
    Write-Host "[i] No paid Microsoft 365 Copilot add-on SKUs assigned yet (ideal time to complete pre-rollout governance!)." -ForegroundColor Yellow
}

Live Lab Output (LAB 365 Dev Tenant):

TEXT
DisplayName Id                                   TenantType CreatedDateTime
----------- --                                   ---------- ---------------
LAB 365     ab85c7a1-366d-44ba-acc6-924fa0950045 AAD        2023-03-30T13:47:39Z

SkuPartNumber    CapabilityStatus ConsumedUnits Purchased Available
-------------    ---------------- ------------- --------- ---------
DEVELOPERPACK_E5 Enabled                     25        25         0
FLOW_FREE        Enabled                      5     10000      9995

4. Chapter 1 Lab Exercise & Next Step

Your Action Item Before Chapter 2:

  1. Run Chapter1-CopilotReadinessAudit.ps1 in your own dev or production tenant.
  2. Verify whether your users hold qualifying base licenses (Microsoft 365 E3/E5, Business Standard/Premium) and whether Purview E5 Compliance is available for sensitivity labels and Copilot DLP.
  3. Check whether Web Grounding is enabled or disabled in Microsoft 365 Admin Center → Copilot → Settings.

Up Next in the Book: