How to Inspect Microsoft Copilot Agent Telemetry in Application Insights

If you manage Copilot agents in Microsoft 365, you need a way to see how users interact with them and whether the agents return correct answers. Standard Copilot reports show high-level usage but do not expose individual queries, token counts, or failure reasons. Application Insights, part of Azure Monitor, captures this telemetry when you connect your Copilot agent to it. This article explains how to configure the connection, run queries in the Azure portal, and interpret the data that matters for agent performance.

What Telemetry Copilot Agents Send to Application Insights

When you configure a Copilot agent to send telemetry to Application Insights, the agent logs every user interaction as a custom event. Each event includes the prompt text, the response generated, the number of tokens consumed, and the HTTP status code of the call. This data lives in the customEvents table of your Application Insights resource. The agent also logs performance metrics such as response latency in milliseconds. No personally identifiable information from the user is included by default. You can extend the schema by adding custom dimensions in your agent code. Prerequisites: an active Azure subscription, an Application Insights resource in the same region as your Copilot agent, and Contributor or Owner role on that resource.

Tables Used for Agent Telemetry

The two main tables are customEvents and traces. The customEvents table stores each user query as a row with fields for name, timestamp, customDimensions, and customMeasurements. The traces table stores debug and error messages from the agent runtime. You can join these tables on the operation_Id column to correlate a user request with its backend logs.

Data Fields in Each Event

Each event in customEvents has a name field set to CopilotAgent. The customDimensions property contains a JSON object with keys such as prompt, response, model, and errorMessage. The customMeasurements property contains numeric values like totalTokens, promptTokens, and responseLatencyMs.

Steps to Connect Your Copilot Agent to Application Insights

Follow these steps to enable telemetry streaming from your Copilot agent to Application Insights. You need the Application Insights connection string before you begin.

1. Create or open an Application Insights resource in Azure
   Sign in to the Azure portal at portal.azure.com. Search for Application Insights and select your existing resource or create a new one. Note the Instrumentation Key on the Overview page.
2. Copy the connection string
   In the Application Insights resource, go to Settings > Access control and copy the Connection String value. This string starts with InstrumentationKey= and includes the key and the IngestionEndpoint.
3. Open your Copilot agent configuration
   Go to the Microsoft 365 admin center at admin.microsoft.com. Navigate to Copilot > Agents. Select the agent you want to monitor. Click Edit agent.
4. Paste the connection string into the agent settings
   In the agent settings, find the Telemetry section. Paste the Application Insights connection string into the field labeled Application Insights connection string. Click Save.
5. Generate test traffic
   Open your Copilot agent in a browser or Teams and send at least three prompts. Wait five minutes for telemetry to appear in Application Insights.

Common Issues When Telemetry Does Not Appear

No events appear in customEvents table

If you run a query and see zero results, the connection string may be incorrect. Double-check the string in the agent settings. Also confirm that the Application Insights resource is in the same Azure region as your Copilot agent. Cross-region telemetry is not supported. If the problem persists, regenerate the connection string in Azure and paste it again.

Events appear but contain empty prompt or response fields

This happens when the agent code does not pass the prompt or response to the telemetry API. Check your agent code for the line that calls trackEvent or trackTrace. Ensure the customDimensions dictionary includes keys for prompt and response. If you use a prebuilt agent from the Copilot Studio template, update the template to include these fields.

Error events show HTTP 429 rate limiting

A 429 status code means your Application Insights resource is receiving too many requests per second. The default ingestion limit is 10,000 events per second per resource. If your agent handles high traffic, increase the limit by requesting a quota increase from Azure Support. Alternatively, sample the telemetry by setting a sampling percentage in Application Insights under Settings > Sampling.

Copilot Agent Telemetry vs Standard Copilot Reports

Item	Copilot Agent Telemetry in Application Insights	Standard Copilot Reports in Admin Center
Data granularity	Per-user prompt and response	Aggregated usage counts only
Token consumption	Available per event in customMeasurements	Not available
Error details	HTTP status code and error message	Generic error count
Latency tracking	Response time in milliseconds	Not available
Custom dimensions	Add any data via agent code	Fixed schema
Retention period	Configurable up to 730 days	28 days

After you connect your Copilot agent to Application Insights, you can run Kusto queries to inspect every user interaction. Start with a simple query that returns the last 100 events: customEvents | where name == "CopilotAgent" | take 100. From there, filter by timestamp, user ID, or token count to find performance issues. The telemetry data also helps you debug agent responses when users report incorrect answers. For a deeper analysis, create a dashboard in Azure Workbooks that charts average latency and error rate over time.