5 Ways Sustainability Data Reporting Can Be Misleading

Presenting data in an eye-catching way to engage and inform all audiences is an essential part of an organisation’s sustainability journey, allowing effective monitoring and reporting of important progress.

However, choosing the most appropriate style of visualisation can be challenging. An in-depth understanding of the patterns and contextual information behind the data needs to be grasped if you are to uncover narratives that can stand up to scrutiny. If a chart or graph is misused, audiences may draw conclusions that are not based on the evidence, or the true picture that lies within the data can be obscured.

Here are five ways unsuitable visualisations can lead to misunderstandings:

1. Using the Wrong Scale

This is usually done to “zoom in” on what might seem to be the most interesting place to examine variances in data or to provide a cleaner, more aesthetically pleasing look, but it can remove the opportunity for the audience to gain a full overview and remove important points of reference. This could include:

    • Starting the y-axis above zero
    • Removing the baseline altogether
    • Using non-linear intervals

This could also occur when you have a large data set collected over many years. You may be tempted to select only the more recent years for presentation, but again, the true picture may be lost as a result of significant omissions.

Charts 1 and 2 below both show temperature anomalies recorded per year as averages. Chart 1 only shows data since 2000, while Chart 2 shows data going back to 1850. If you only saw Chart 1 you could be persuaded that the difference year to year may have originated solely in natural fluctuation. However, Chart 2 shows a much starker picture, displaying a significant rise over the longer period and so is a more appropriate time frame to use.

2. Using comparisons incorrectly

When data from different sources are compared on a like-for-like basis, it can lead to incorrect assumptions without any clarifying information.

Charts 3 and 4 show the natural gas use from several factories. Chart 3 shows this as a percentage of the whole for each month, while Chart 4 shows the actual volumes consumed. Because percentages are used in Chart 3, this tells you little about the true scale of the emissions, unlike Chart 4 which provides more clarity. However, neither chart can provide contextual information such as the size and activities of the factories, and so they may be utilised to conceal a factory with a high emissions-to-square-footage ratio.

3. Selecting the wrong visualisations

Sometimes, the wrong visualisation method for the type of data can be chosen. A common mistake is to use pie charts for data that does not add up to a whole.

Chart 5 shows fatality ratios per country for Covid-19 data reported on 21st December 2020. In addition to being an inappropriate choice of chart type, some of the nuance is lost due to most ratios being much smaller than the largest. Because this is shown as a percentage, the scale of variation can be misleading. When taking a global view, sensible sampling could be employed using a bar chart or similar, to present a clearer picture.

4. Using emotive or distracting themes

An audience will have pre-conceived ideas around colour schemes and visuals that can be manipulated, unconsciously or deliberately, in order to present ideas that may be unsubstantiated or irrelevant.

Colours will have connotations to everyone such as green being “good” and red being “bad”. For this reason, data can be coloured in a way that can adversely affect the way it is processed by an audience. Increments of colour can also be used in a leading way, such as making the difference between two data points seem greater by using highly contrasting colours.

Creative forms of visualisation using cartoons or icons can be an effective and arresting way of communicating, as long as it does not over-dramatise or distract. The data should be able to inform on its own, with visualisations only used to serve in its accurate communication.

Chart 6 shows the kg CO2e per kg of several different products. The colour transitions from green for the product with the lowest value, product A, to red for the highest, product E. This may serve to validate the carbon emissions of product A when it may be higher than average for a product of its type, while product E may have mitigating factors in its production, meaning it is a greener product than appears at face value. Using a colour-scale in this manner may dissuade from any deeper analysis. Also, in this example, although the icon is eye-catching, it may be too distracting or imprecise for some uses and may require additional data labels.

5. Drawing conclusions

Accompanying information or text that frames a visualisation, such as its title or any analysis should bear in mind that correlation does not equate to causation. This means that any patterns in data may not have an underlying cause that is directly connected, and one should not be assumed purely from the data itself.

Any limitations in data collection or analysis should also be stated such as the use of a small sample size.

Chart 7 shows the number of Covid-19 deaths per country as of 21st December 2020. The title in itself may be misleading as each country tabulates their Covid-19 deaths differently and it is problematic to compare them directly in order to draw conclusions. In addition, this Chart does not account for the significant factor of population size. When deaths per 100,000 people is plotted on Chart 8, the ranking of each country is greatly changed. The title has also been changed to provide greater context. However, this is still only a small indication of the wider picture and should not be relied upon to draw conclusions.

 

Allowing an audience to jump to conclusions due to misleading presentations of data can have dramatic and far-reaching consequences upon decision making and significant problems may go undetected. For this reason, the choice of data visualisation should always be made in a responsible, measured way, with awareness of any internal bias. Lastly, the limitations of any data should always be appreciated, and concrete conclusions should be avoided where there is not enough evidence available.

Back to Our Stories