A KQL Query Is Not a Detection: What My Microsoft Sentinel Lab Actually Validated

Summary

A KQL Query Is Not a Detection: What My Microsoft Sentinel Lab Actually Validated A KQL query compiling is not proof that a detection works. The underlying event, relevant field values, query result, and Sentinel behavior must all be verified. A reliable detection starts with reliable telemetry. This lab builds an Azure logging foundation for Microsoft Sentinel, validates the native tables, and uses the observed data to develop three Microsoft Entra ID detections. Azure Resource Hierarchy Azure resources are organized into management scopes, and Azure RBAC or Azure Policy assignments can inherit from higher scopes to the resources below them. Microsoft Entra ID tenant (identity and trust boundary) └── Tenant root management group └── Management groups └── Subscriptions └── Resource groups └── Resources The Microsoft Entra tenant stores identities and authentication configuration. Azure governance begins at the tenant root management group, then flows through management groups, subscriptions, resource groups, and resources. For this lab, the important path is: Lab subscription └── Resource group: sentinel └── Log Analytics workspace: sentinelsiem └── Microsoft Sentinel and ingested log tables A role or policy assigned at a higher Azure scope can apply to descendants. Resource locks protect against accidental modification or deletion, while tags add metadata such as environment or owner; neither creates a new hierarchy level. Azure Resource Manager and the Control Plane Azure Resource Manager (ARM) is Azure’s management layer. Portal, CLI, PowerShell, SDK, template, and REST management requests ultimately use the Azure control plane. This distinction matters during validation: control-plane changes should be investigated through the telemetry route and native table that record management activity, while identity events appear through Microsoft Entra logging. Building the Telemetry Foundation Create Log Analytics and Enable Microsoft Sentinel Create the sentinelsiem Log Analytics workspace in the sentinel resource group, then enable Microsoft Sentinel on that workspace. For this personal lab, set the workspace daily cap to 0.1 GB/day . The cap controls cost but can pause ingestion until it resets. A gap caused by the cap is not evidence that no activity occurred. This setting is not appropriate for production. Route Azure Activity Logs Creating a workspace does not export the subscription Activity Log automatically.

• Open Monitor > Activity log > Export Activity Logs.
• Add the subscription diagnostic setting activity-to-sentinelsiem . - Select the categories needed by this lab, including Administrative, Security, and Policy.
• Send them to sentinelsiem and save. This subscription-level route supplies the AzureActivity table. Route Microsoft Entra ID Logs Open Microsoft Entra ID > Monitoring & health > Diagnostic settings. Send these categories to sentinelsiem : AuditLogs NonInteractiveUserSignInLogs NonInteractiveUserSignInLogs writes to AADNonInteractiveUserSignInLogs . Validating Native Azure and Entra Telemetry Validate AzureActivity with a Harmless Tag Change Record the UTC start time, then add a reversible tag to the sentinel resource group: Resource groups → sentinel → Tags → Add Name: “LabPhase” Value: “ActivityTest” Start broad instead of guessing an operation name: AzureActivity | where ResourceGroup =~ “sentinel” | project TimeGenerated, OperationNameValue, ActivityStatusValue, ActivitySubstatusValue, Caller, CallerIpAddress, ResourceId, CorrelationId, Properties | order by TimeGenerated desc Inspect the returned operation, caller, resource, result, timestamp, and correlation ID. The preserved evidence validates AzureActivity ingestion, although the exact tag-change row was not retained. Validate AuditLogs with Temporary Entra Objects Use an authorized Azure Cloud Shell PowerShell session. Create temporary, timestamped objects that are easy to identify: Get-AzContext $group=New-AzADGroup‘-DisplayName"DetectionLab-$(Get-Date -Format HHmmss)” -MailNickname "DetectionLabTest"$app = New-AzADApplication -DisplayName “DetectionLab-OAuth-App-$(Get-Date-FormatHHmmss)"$sp = New-AzADServicePrincipal -ApplicationId $app.AppId$cred = New-AzADAppCredential -ApplicationId $app.AppId -StartDate (Get-Date) -EndDate (Get-Date).AddDays(7) Inspect raw audit rows before adding an activity-name filter: AuditLogs | project TimeGenerated, ActivityDisplayName, Result, InitiatedBy, TargetResources, CorrelationId | order by TimeGenerated desc The lab preserved these observed values: Add application Add service principal Add service principal credentials Update application - Certificates and secrets management A credential command can appear as an application update rather than the phrase expected in advance. Inspecting raw telemetry first makes later KQL more defensible. Validate AADNonInteractiveUserSignInLogs These read-only commands reuse the Cloud Shell context and can generate background token activity: Get-AzSubscription Get-AzResourceGroup Get-AzResource Get-AzADUser -First 5 Get-AzAccessToken -ResourceTypeName MSGraph Microsoft Graph can appear as the target resource in sign-in telemetry. That does not mean Microsoft Graph Activity Logs are enabled. AADNonInteractiveUserSignInLogs | project TimeGenerated, UserPrincipalName, UserId, AppId, AppDisplayName, ResourceIdentity, ResourceDisplayName, IPAddress, SessionId, AuthenticationProtocol, IncomingTokenType, DeviceDetail, ConditionalAccessStatus, UserAgent | order by TimeGenerated desc This confirms that the table and required fields are available for later detection engineering work. Building Microsoft Sentinel Detection Rules Create scheduled rules from Microsoft Sentinel > Configuration > Analytics > Create > Scheduled query rule. Use these settings unless a rule specifies otherwise:
• Alert when query results are greater than 0 . - Group all events into one alert for this small lab.
• Create incidents from alerts.
• Keep TimeGenerated in the query output. - Test the KQL in Logs before enabling the rule.
• Keep unvalidated rules Disabled. The three detections do not have the same validation state: Rule Native table(s) Validation state
• Rule 1 — Application / service principal credential change AuditLogs
• Rule 2 — Same session across multiple IPs AADNonInteractiveUserSignInLogs
• Rule 3 — Authentication followed by directory change AADNonInteractiveUserSignInLogs +AuditLogs ⚠️ This distinction is intentional: a detection is not presented as end-to-end validated until the source event, query result, alert, and incident behavior are all evidenced. Rule 1 — Entra: Application or Service Principal Credential Change Detection Objective Identify application, service principal, and credential-management activity using the exact AuditLogs activity names observed in the lab. Trigger Condition Alert when AuditLogs contains any of these activities: Add application Add service principal Add service principal credentials Update application - Certificates and secrets management Lab Trigger Run trigger commands only in the lab tenant and remove temporary objects after testing. $stamp=Get-Date-Format"yyyyMMdd-HHmmss"$rule1App = New-AzADApplication -DisplayName "DetectionRule-Trigger-$stamp" $rule1Sp = New-AzADServicePrincipal -ApplicationId $rule1App.AppId$rule1AppCred = New-AzADAppCredential -ApplicationId $rule1App.AppId -StartDate (Get-Date) -EndDate (Get-Date).AddDays(1) $rule1SpCred = New-AzADSpCredential -ObjectId $rule1Sp.Id -StartDate (Get-Date) -EndDate (Get-Date).AddDays(1) These commands create temporary lab objects. Depending on the backend activity names in the tenant, the application and service-principal credential commands can appear as the credential-management activities used by the rule. Detection KQL AuditLogs | where TimeGenerated > ago(15m) | where ActivityDisplayName in~ ( “Add application”, “Add service principal”, “Add service principal credentials”, “Update application - Certificates and secrets management” ) | extend InitiatingUser = tostring(InitiatedBy.user.userPrincipalName), InitiatingUserId = tostring(InitiatedBy.user.id), InitiatingApp = tostring(InitiatedBy.app.displayName), TargetName = tostring(TargetResources[0].displayName), TargetType = tostring(TargetResources[0].type) | project TimeGenerated, ActivityDisplayName, Result, InitiatingUser, InitiatingUserId, InitiatingApp, TargetName, TargetType, TargetResources, CorrelationId | order by TimeGenerated desc Validation Query AuditLogs | where TimeGenerated > ago(30m) | where ActivityDisplayName in~ ( “Add application”, “Add service principal”, “Add service principal credentials”, “Update application - Certificates and secrets management” ) | project TimeGenerated, ActivityDisplayName, Result, InitiatedBy, TargetResources, CorrelationId | order by TimeGenerated desc Confirm the Sentinel Alert and Incident The raw event match is only part of the validation. After enabling Rule 1 and running the lab trigger, wait for the scheduled rule to execute, then confirm that Sentinel produced both an alert and an incident. SecurityIncident | where TimeGenerated > ago(24h) | where Title contains “Application or Service Principal Credential Change” | summarize arg_max(TimeGenerated, *) by IncidentNumber | project TimeGenerated, IncidentNumber, Title, Severity, Status, ProviderName, AlertIds, IncidentUrl | order by TimeGenerated desc Observed Result and Validation Status The lab observed the four activity values targeted by the rule and confirmed that the query returns the corresponding raw AuditLogs events. The event-matching stage is therefore validated. Get Zyad Waleed Elzyat’s stories in your inbox Join Medium for free to get updates from this writer. Full end-to-end validation is complete only after the SecurityAlert and SecurityIncident checks above return the expected records and the corresponding screenshots are added. Until then, describe Rule 1 as event-match validated, not fully end-to-end validated. Rule 2 — Same Entra Session Across Multiple IP Addresses Detection Objective Identify a populated UserId and SessionId pair observed from at least two distinct source IP addresses within 30 minutes. Rule Settings
• Severity: Medium
• Run every: 5 minutes
• Look back: 30 minutes
• Initial state: Disabled until validated - MITRE ATT&CK: T1550.001 — Application Access Token Trigger Condition Alert when the same populated UserId and SessionId pair is observed from at least two distinct source IP addresses within 30 minutes. Lab Trigger There is no reliable single command that can force Microsoft Entra to reuse the same native SessionId from two IP addresses. Test the rule by keeping the same authenticated PowerShell session while changing the lab machine’s public egress path. Before changing the public IP: Connect-AzAccount Get-AzAccessToken -ResourceTypeName MSGraph | Out-Null Invoke-RestMethod “https://api.ipify.org” Change the network, for example by using a VPN, then run: Get-AzAccessToken -ResourceTypeName MSGraph | Out-Null Microsoft Entra must log the same SessionId from both IP addresses for this to be a valid trigger. If it does not, the rule has not been validated. Detection KQL AADNonInteractiveUserSignInLogs | where TimeGenerated > ago(30m) | where isnotempty(UserId) and isnotempty(SessionId) and isnotempty(IPAddress) | summarize TimeGenerated = min(TimeGenerated), LastSeen = max(TimeGenerated), Events = count(), DistinctIPs = dcount(IPAddress), IPAddresses = make_set(IPAddress, 10), AppIds = make_set(AppId, 10), AppNames = make_set(AppDisplayName, 10), ResourceIds = make_set(ResourceIdentity, 10), ResourceNames = make_set(ResourceDisplayName, 10), Protocols = make_set(AuthenticationProtocol, 10), TokenTypes = make_set(IncomingTokenType, 10), DeviceDetails = make_set(DeviceDetail, 10), CAResults = make_set(ConditionalAccessStatus, 10), UserAgents = make_set(UserAgent, 10) by UserId, UserPrincipalName, SessionId | where DistinctIPs >= 2 | order by TimeGenerated desc Validation Query AADNonInteractiveUserSignInLogs | where TimeGenerated > ago(2h) | where isnotempty(SessionId) | where isnotempty(IPAddress) | project TimeGenerated, UserPrincipalName, UserId, SessionId, IPAddress, AppDisplayName, ResourceDisplayName, AuthenticationProtocol, IncomingTokenType | order by TimeGenerated desc Observed Result and Validation Status The source does not preserve evidence that one native SessionId appeared from two distinct IP addresses or that Sentinel generated the expected alert and incident. Rule 2 remains unvalidated and must stay disabled. False Positive and Benign Considerations Expected benign causes include VPN changes, proxy changes, mobile-network changes, and legitimate session mobility. Rule 3 — Authentication Followed by Sensitive Directory Change Detection Objective Correlate a successful non-interactive authentication with a sensitive directory change performed by the same populated UserPrincipalName within 30 minutes. Rule Settings
• Severity: Medium
• Run every: 10 minutes
• Look back: 40 minutes
• Initial state: Disabled until validated - MITRE ATT&CK: T1098.001 — Additional Cloud Credentials Trigger Condition Alert when all three conditions are true:
• A successful non-interactive authentication contains a populated UserPrincipalName . - The same user performs one of Rule 1’s sensitive directory changes within 30 minutes.
• The directory change occurred during the latest 10 minutes. Lab Trigger First generate a candidate authentication event: Connect-AzAccount -UseDeviceAuthentication Get-AzContext | Select-Object Account, Tenant, Subscription Get-AzAccessToken -ResourceTypeName MSGraph | Out-Null Confirm the signed-in user: $upn=(Get-AzContext).Account.Id$upn Then create a temporary lab application from the same signed-in user: $stamp=Get-Date-Format"yyyyMMdd-HHmmss"$rule3App = New-AzADApplication ` -DisplayName “AuthThenChange-$stamp"$rule3App | Select-Object DisplayName, Id, AppId This is only a candidate trigger. The correlation is valid only if the same non-empty UserPrincipalName appears in both native tables. Detection KQL let AuthEvents = materialize( AADNonInteractiveUserSignInLogs | where TimeGenerated > ago(40m) | where tostring(ResultType) == “0” | where isnotempty(UserPrincipalName) | project AuthTime = TimeGenerated, UserPrincipalName, UserId, AppDisplayName, AppId, ResourceDisplayName, ResourceIdentity, IPAddress, AuthenticationProtocol, IncomingTokenType, SessionId ); let DirectoryChanges = materialize( AuditLogs | where TimeGenerated > ago(10m) | where ActivityDisplayName in~ ( “Add application”, “Add service principal”, “Add service principal credentials”, “Update application - Certificates and secrets management” ) | extend UserPrincipalName = tostring(InitiatedBy.user.userPrincipalName) | where isnotempty(UserPrincipalName) | project TimeGenerated, UserPrincipalName, ActivityDisplayName, Result, TargetResources, CorrelationId ); AuthEvents | join kind=inner (DirectoryChanges) on UserPrincipalName | where TimeGenerated between (AuthTime .. AuthTime + 30m) | extend MinutesToChange = datetime_diff(“minute”, TimeGenerated, AuthTime) | project TimeGenerated, AuthTime, MinutesToChange, UserPrincipalName, UserId, IPAddress, AppDisplayName, AppId, ResourceDisplayName, ResourceIdentity, AuthenticationProtocol, IncomingTokenType, SessionId, ActivityDisplayName, Result, TargetResources, CorrelationId | order by TimeGenerated desc Validation Check For validation, rerun the detection over a wider test window and inspect the two native tables separately before trusting the join. The same non-empty UserPrincipalName must appear in the authentication event first and in the sensitive directory change afterward, within 30 minutes. AADNonInteractiveUserSignInLogs | where TimeGenerated > ago(2h) | where tostring(ResultType) == “0” | where isnotempty(UserPrincipalName) | project TimeGenerated, UserPrincipalName, UserId, IPAddress, SessionId, AppDisplayName | order by TimeGenerated desc AuditLogs | where TimeGenerated > ago(2h) | where ActivityDisplayName in~ ( “Add application”, “Add service principal”, “Add service principal credentials”, “Update application - Certificates and secrets management” ) | extend UserPrincipalName = tostring(InitiatedBy.user.userPrincipalName) | where isnotempty(UserPrincipalName) | project TimeGenerated, UserPrincipalName, ActivityDisplayName, Result, TargetResources, CorrelationId | order by TimeGenerated desc Observed Result and Validation Status The source does not preserve evidence that the same populated UserPrincipalName correlated across both native tables within the required time window, or that Sentinel generated the expected alert and incident. Rule 3 remains unvalidated and must stay disabled. Entity Mapping
• Map UserId to the Account entity. - Map IPAddress to the IP entity. What This Lab Proved
• Reliable detections start with validated telemetry, not with a KQL query written from assumptions.
• Native activity names and field values should be learned from raw events generated in the target environment.
• A query returning rows proves event matching; it does not by itself prove that Sentinel generated the expected alert and incident.
• Candidate rules should remain disabled until their exact trigger and correlation assumptions are reproduced.
• Validation evidence should preserve the chain from test action → raw telemetry → KQL result → alert → incident. Cleanup Run cleanup in the same PowerShell session used to create the temporary objects. if ($rule1SpCred-and$rule1Sp) { Remove-AzADSpCredential -ObjectId $rule1Sp.Id -KeyId $rule1SpCred.KeyId‘-Confirm:$false } if ($rule1AppCred-and$rule1App) { Remove-AzADAppCredential -ApplicationId $rule1App.AppId -KeyId $rule1AppCred.KeyId‘-Confirm:$false } if (rule1Sp) { Remove-AzADServicePrincipal ` -ObjectId rule1Sp.Id -Confirm:$false } if ($rule1App) { Remove-AzADApplication -ObjectId $rule1App.Id‘-Confirm:$false } if (rule3App) { Remove-AzADApplication ` -ObjectId rule3App.Id -Confirm:$false } if ($sp) { Remove-AzADServicePrincipal -ObjectId $sp.Id‘-Confirm:$false } if (app) { Remove-AzADApplication ` -ObjectId app.Id -Confirm:$false } if ($group) { Remove-AzADGroup -ObjectId $group.Id‘-Confirm:$false } Remove the temporary LabPhase=ActivityTest tag from the sentinel resource group after validation.

By Zyad Waleed Elzyat

Original Article