Hello readers,
Today I am back with another post to discuss one of the common needs that customers have: retrieving data in a given business interval. The interval could one or more days, one or more hours as well as the combination of both. Let’s see how to do it.
As first step let’s identify the scenario in which this filter can be of help. Say that the IT department must only take care of data (and notification) which are inside the following interval:
- Monday to Friday.
- 9 a.m. to 6 p.m.
According to above scenario, the IT department needs to retrieve the events occurred during the last 60 days according to the business interval defined above.
Now that we know the scenario and the business interval, what else do we need? Of course, an existing workspace containing data, then the knowledge of the data we will going to query, a lookup table and, finally, the query we use to retrieve data.
As far as the lookup table goes, we will create it inside the query. It will be a simple step. In the Azure portal, got to the relevant workspace, open the query page and paste the code below:
// Defining business day display names and business hours lookup table
let businessInterval = datatable(dayOfWeekTimespan:int, dayOfWeekDayDisplayName:string, firstH:int, lastH:int)
[
0, "Sunday", 9, 18,
1, "Monday", 9, 18,
2, "Tuesday", 9, 18,
3, "Wednesday", 9, 18,
4, "Thursday", 9, 18,
5, "Friday", 9, 18,
6, "Saturday", 9, 18
];
The code above defines a multidimensional array, containing 7 rows, with the following schema that will be used as dimension table:
- dayOfWeekTimespan:int à the timespan value used to convert the result of dayofweek function. This function will return the integer number of days since the preceding Sunday, as a timespan.
- dayOfWeekDayDisplayName:string à the display name to convert the timespan value to.
- firstH:int à the lower boundary of the business hours.
- lastH:int à the upper boundary of the business hours.
Given the schema, you can also define different business hour interval on different days.
NOTE: The lookup table does not honor the various locales. For non-English representation, feel free to change the value in the dayOfWeekDayDisplayName column accordingly. Same concept applies to the TimeGenerated which needs to be adapted using the relevant time zone. Records in Log Analytics are stored only in UTC. You can convert the time during the presentation in the portal or from within the query.
Moving on with the query, let me first explain why I called the multidimensional array “Lookpup table”. The reason is inside the query code. It will use, in fact, the lookup operator. In short, the lookup operator performs an operation similar to the join operator with the following differences:
- The result does not repeat columns from the $right table that are the basis for the join operation.
- Only two kinds of lookup are supported, leftouter and inner, with leftouter being the default.
- In terms of performance, the system by default assumes that the $left table is the larger (facts) table, and the $right table is the smaller (dimensions) table. This is exactly opposite to the assumption used by the join operator.
- The lookup operator automatically broadcasts the $right table to the $left table (essentially, behaves as if hint.broadcast was specified). Note that this limits the size of the $right table.
With the lookup operator clear in mind, let’s assemble the query considering the following:
- Customer needs to query the last 30 days of data.
- Customer is interested on data occurred inside the business interval.
Right after the lookup table code, we will define 2 variables to set the Time Range interval according to the 60 days requirement expressed by the customer. Paste the following after the last line of the table code:
// Defining timerange interval
let startDate = ago(60d);
let endDate = now();
Now, let’s go ahead querying and presenting the data using the lookup table for the defined time range interval. The remaining part of your query, which you can write right after the time range interval lines above, could be similar to the sample one reported below:
// Querying data
Event
| where TimeGenerated between (startDate .. endDate)
| extend dayOfWeekTimespan = toint(substring(tostring(dayofweek(TimeGenerated)), 0, 1))
| where dayOfWeekTimespan in (1, 2, 3, 4, 5)
| lookup kind=leftouter businessInterval on dayOfWeekTimespan
| where datetime_part("Hour",TimeGenerated) between (firstH .. lastH)
// Presenting the data
| project TimeGenerated, Computer, Source, EventLog, dayOfWeekDayDisplayName
| sort by TimeGenerated asc
The complete code will look as follow:
The results will be shown according to the project operator:
As you can understand from the query code, the relevant part for the filtering concept is represented by the lookup table and by the lines from 18 to 21 in the screenshot:
You can use this approach in your workbook, in your alerts and in all your queries. Of course, the charts will show some blank or missing values, but this is in line with the filter.
Hope that this is going to help taking care of or analyzing the data only during the desired business interval.
Thanks for reading 
Disclaimer
The sample scripts are not supported under any Microsoft standard support program or service. The sample scripts are provided AS IS without warranty of any kind. Microsoft further disclaims all implied warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The entire risk arising out of the use or performance of the sample scripts and documentation remains with you. In no event shall Microsoft, its authors, or anyone else involved in the creation, production, or delivery of the scripts be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the sample scripts or documentation, even if Microsoft has been advised of the possibility of such damages.
Posted at https://sl.advdat.com/3Ay519v