Friday, 15 May 2020

A Date with Microsoft Power BI - Part 2

This blog post is about how to create a date table in Microsoft Power BI. In the last blog post, we looked at what happens when you import data with a date column into Power BI. We identified a few limitations when you don't change the defaults. If you’ve not read it, take a look.

So why do you need a separate date table? Here’s two reasons:
  • To report on time periods that are different from the standard ones
  • To report on time periods that span more than one year.

There are several ways of creating a date table:
  • Import it – from a database or data warehouse or another service
  • Create it using M
  • Create it using DAX
To create a new table with a single column of type date/time using DAX:
  • In Power BI Desktop, select Modeling.
  • From the Calculations Group, click New Table. The DAX panel appears.
  • Rename the table name to MyCalendar and to the right of the equals sign type:
    • MyCalendar = CALENDAR (DATE(2020,01,01), DATE(2020,12,31))
  • Click the tick symbol to create the table.
This creates a single column date table from the first date to the second date. The syntax is:

CALENDAR(<start_date>, <end_date>)

The start_date and end_date must be valid date values, so use the DATE function with the syntax:

DATE(<year>, <month>, <day>)

Alternatively, you can use a value from your data, such as:

MyCalendar = CALENDAR(MIN(Sale[OrderDate]), MAX(Sale[OrderDate]))

The important thing is that the date table contains the full range of dates in your data, with no gaps.

There is an alternative DAX function - CALENDARAUTO() which scans your data, and finds the first and last dates and creates an appropriate date table. This has one optional parameter – fiscal_year_end_month.


If your tax year ends in March, for example, the DAX is:

MyAutoCalendar = CALENDARAUTO(3)

This creates a table with a single column called date, with the first date being the 1st April of the earliest year in your data, and the last date being the 31st March of the last year in your data.
Just two things to remember:
  • In Options, remove the Auto date/time selection
  • In Table Tools, select Mark as data table. Then select the date column. A warning is given to tell you that the hidden date table will be removed. Click OK.

Now that you have a table with dates in it, you can add columns to suit your reporting purposes.

To create a new column for month name:

MonthName = FORMAT(Sale[OrderDate], “MMMM”)

To create a new column for the month number:

MonthNo = MONTH(Sale[OrderDate])

To create a new column for year number:

OrderYear = YEAR(Sale[OrderDate])

To create a new column that combines the year and month values:

YearMonth = Sale[OrderYear] & "-" & Sale[OrderMonth]

Your calendar table is now beginning to take shape, and you can now create slicers containing months from specific years.  This is progress - but there's a lot more to working with dates in Power BI.

In the meantime, leave a comment to let me know how it's going. And if you want some help with your Power BI reporting or data modeling, then get in touch.

Monday, 11 May 2020

A Date with Power BI - Part 1

Are you creating your first report in Power BI? Or are you a bit more experienced, but finding that things are not working as you expected? Most business data contain dates. That means that most data analysis involves handling dates in both visualizations and calculations.

Getting to know how Power BI handles dates is well worthwhile.

For example, sales data might include the date that something was sold, plus when it was despatched, and maybe when payment was received. Training data will include the date someone books a training course, as well as the date they attend.

The passing of time and the events that happen in time are a fundamental building block to the way we analyse things.

Below is a simplified sales data table imported into Power BI. Count of OrderID provides the volume of sales, and it is analysed by date. To create a line chart:
  • From the Fields panel, drag OrderID to the Values area on the Visualizations panel. Check the aggregation is Count.
  • In the Fields panel, expand the date hierarchy
  • From the Fields panel, drag Month to the Axis area on the Visualizations panel. 

The line chart shows the volume of sales by month. 

Alternatively, delete Month from the Axis area and drag Date hierarchy in its place, you can see that now a drill down symbol has been added to your chart. Use it to drill up and down the date hierarchy, such as from Year to Quarter, from Quarter to Month, and from Month to day level.

But what if you do not want a date hierarchy? What if you want the date that was in your original data? Right click on the date filed to toggle between date hierarchy and date.

But let us just stop a moment. When you imported your data, did it include a date hierarchy? Mine just had a simple date, with no hierarchy. So where did it come from? At this stage you may say “I’m happy with my report, I don’t care where it came from!” Which is fair enough, but let me ask you some questions:
  • Does your company’s financial months and quarters align with those provided by Power BI?
  • Do you ever need to analyse data by week?
  • Do you use a completely different time period to analyse data, such as a sales period?
  • If you drag “Month” to a date slicer, which month is it referring to?
  • What if your data had included two dates, and not just one? How should that be handled?
These are not trick questions, honestly! But they are issues that emerge once you start building real reports for real decision-makers, and once your data starts to become more complicated.

We did not do anything to the date field to create a date hierarchy. So how did Power BI separate the month, quarter, and year from the date? The answer is a clue to the data model we need, rather than the data model we have.

Spoiler alert! There is a difference between what you see, and what is actually happening.

What you see is Power BI adding a date hierarchy, presumably because it understands dates. What is actually happening is that Power BI has created a completely new and hidden table.

To check that there really is a table, use Dax Studio to have a look. Below is a screen picture of the Power BI generated date table with the new fields: Day, Month, MonthNo, Quarter, QuarterNo, and Year.

There are a few important issues arising from this:
  • To create a chart with increments other than days, Power BI created a new, hidden table.
  • The new table had separate fields for day, month, quarter and year. Plus, month number and year number.
  • The table is hidden, and you cannot change it.
The functionality is provided by a setting in Options, under Time intelligence called “Auto date/time for new files”. When Power BI is installed it is Selected by default. So if you don’t want this functionality go to:
  • File
  • Options & Settings
  • Options
  • Time Intelligence
  • Deselect Auto date/time for new files

Next time I will look at how to create your own date table using DAX. This is a more flexible approach, and answers at least some of the questions raised earlier.

In the meantime, leave a comment or ask a question in the comments section. Or if you want some immediate help with your Power BI development, get in touch.

Wednesday, 29 April 2020

Accurate Record Keeping

Never has data been so much in the news. Whilst the numbers tell us nothing of the heartache behind them, they do tell scientists how best to handle Coronavirus and how to minimize its impact. We have all become familiar with R0 values, curves, and moving averages. Software like Microsoft Power BI is being used to analyse and visualize the data, and to make it more understandable.

At the same time, the importance of accurate record keeping has become more apparent. As the Head of Mortality Analysis at the Office of National Statistics (ONS) points out, counting the number of Covid-19 deaths may be important, but it is not simple. Issues such as the speed with which data can be recorded and disseminated, accuracy and consistency of classification, and missing data all contribute to how trustworthy the data is.

These issues are important when lives depend on the decisions that are made on our behalf.

Thankfully in business we are not often dealing with life and death. But we still need accurate data. Getting a good understanding of how many people have been trained in specific courses, by role and by date, for example. Or understanding how many meetings with clients by industry type, and outcome. These are not complex ideas, but getting the data is not always straightforward.

The issues with health data and business data are the same in how the data is stored. Both count something and then analyse it by a number of attributes. In the business intelligence terms, these are known as facts and dimensions. Facts are the things you count, and dimensions are how you analyse the numbers. The number of cases is a fact, and age group is a dimension. There are normally many dimensions for each fact.

As more businesses are working to become more data led, and less opinion led, keeping accurate data becomes vital. If you are making decisions based on inaccurate data, your decisions could be less than optimal, or just plain wrong.

The good news is that data has never been easier to manage, analyse and visualize. Microsoft Power BI is being used extensively to help manage the Coronavirus pandemic, and to help businesses make better decisions. It is an inexpensive piece of software, very powerful, and encourages sharing of data.

If you want to move from an opinion led business to a data led business, get in touch.  If you want help getting your data into shape so you can analyse it, we offer consultancy and development on the Microsoft Data Platform, including Power BI, SQL Server and Excel.

Friday, 24 April 2020

Business as normal, but not as we know it

There will be many legacies from the Coronavirus pandemic, not least the adoption of technologies we have had for some time. Many companies have been using video conferencing successfully for years, but many more were flying their people around the globe for reasons that were plausible at the time. Now grounded, they are using video conferencing and will find that travel budgets are reduced once the health dangers are over.

Cometh the pandemic, cometh the ability to get to grips with Teams.

It is not just video conferencing; it is also cloud adoption. Cloud providers such as Microsoft Azure have been struggling to keep up with the sudden surge in demand. The myriad of reasons why the cloud was bad for business have melted as home workers need the same information as they had in the office.

Do any of us think we will go back to life as it was two month’s ago? No. It is impossible to unlearn a new skill, and it is impossible to become less efficient once we know how to be more efficient. So wider spread use of technology to do business will become the new normal. Which is good news for some, but for others their hearts will sink.

The harsh reality is that we cannot cherry pick. The rapid demise of the high street has demonstrated that all too graphically. If we want to flourish, we must understand how to offer value wherever and however people want to buy. Whether that is on their mobile, at their desk, or in store.

Value is where the internet scores. On the internet we can flick from product to product, compare prices and companies quickly and easily. We may want the cheapest price, or we may want specialist advice; value is not the same for everyone. Nor is it the same at every step in the customer journey.

The difference between online and physical stores is that we can measure perceived value, as well as sales. If I read an article, and return to it, there is an indication I am getting value from it. If the article gets very few readers, it indicates a lack of value for readers. Diligent measurement, analysis, and tests enables us to understand where the value lies for our customers.

The shift to online will bring many benefits, not least a cleaner planet. But after social distancing ends, we may well find it more convenient to get advice online rather than face to face. We may prefer to video call rather than pick up a new infection in the doctor’s surgery (and the doctor may feel the same). We may find it easier to chat to our hairdresser about the best shampoo to use, rather than go into the salon. And there will be many other areas online were value can be found for businesses willing to provide it.

Behind all these opportunities, and very many more, will be measurement. Knowing for sure where people find value, and where they do not. Which is where Microsoft Power BI scores so highly. You do not have to be Amazon or eBay to use analysis and AI technologies, they are available to businesses of all sizes. Measurement, analysis, and better understanding are available to everyone.

Pandemics are not generally the best way to adopt a new technology, but the lesson is that one way or another there is no way to hold back progress.

As always, if there is anything in this blog you found interesting, valuable, or you would like to argue with, then get in touch. We are Microsoft Business Intelligence specialists, and Gold Partners for the Microsoft Data Platform and if you are grappling with a data problem, we might be able to help.

Tuesday, 10 March 2020

Achieving Goals with Microsoft Power BI

Goals are an important part of business; they provide focus, direction, and motivation. Good goals can inspire us to perform better than our best – what Jim Collins calls Big Hairy Audacious Goals (BHAGs). But there is a difference between setting a goal and achieving it. Otherwise we would all be slim and complaining about the worldwide shortage of yachts.

Microsoft Power BI can help you achieve the goals you set. How? Research by Deloitte  suggests there are three principles you should follow with goals:
  1. Goal clarity
  2. Ongoing review
  3. Making goals matter

Goal clarity

Goal clarity comes from defining how the goal will be measured.

By setting a goal and defining the measurement criteria at the same time, including which system the data will come from, you remove any discussion about what the goal means.

What often happens is that we set a seemingly simple goal, let’s say sales per month, without defining the data that will be used because it seems obvious. A sale is a sale, isn’t it? Three months into the year, and discussions start. Is it:
  1. When the Sales department get agreement from the customer to proceed?
  2. When the contract has been signed?
  3. When the deposit has been paid?
  4. When some of the work has been invoiced?
  5. When all of the work has been invoiced?
  6. When all of the work has been paid for?
  7. Including commission/charges/carriage? 

Unless the measurement of the goal is clearly defined, debates and discussion will follow. Or worse, some people will work to one set of rules, and others will work to another set of rules.

Ongoing review

The Deloitte research also found:

“Organizations that have employees revise or review their goals quarterly or more frequently were three-and-a-half times more likely to score in the top 25 percent of business outcomes.”

With clear measurement criteria in place, Power BI makes it possible to review as frequently as required. Power BI has a range of options to review progress to targets, from tables to cards, gauges to KPI indicators.

What’s important is that once the report is linked to the data it can be regularly refreshed, so everyone is working on current information.

Make goals matter

Power BI helps you clarify your goals, and review progress. It also helps with setting goals that matter to everyone, by allowing employees to set and monitor they own goals.

Goal setting power is in everyone’s capability with Power BI due to the range of data sources it works with, including Excel and Access databases. In addition, it gives everyone the ability to create powerful visuals that compare actuals to targets.

As most of us have repeatedly found, it’s not setting the goal that’s difficult, it’s tracking progress and keeping it visible that’s hard.

Power BI is a game changer when it comes to goals, targets, key performance indicators (KPIS). Far from “just” being able to visualize progress, it helps us clarify how we will measure our goals and track our progress towards them. And it gives us the ability to share that progress with others.
And for the first time, this isn’t something that happens within IT – everyone has the ability to create their own hard-hitting reports that show progress towards important goals.

If you’d like to find out more about using Power BI to track your goals, get in touch to find out more.

Bersin by Deloitte - Effective Employee Goal Management is Linked to Strong Business Outcomes