Notes on Data Variance, Mismatches, and Curiosities
This page contains guidance on how to interpret changes over time in the data featured in our Global Digital Reports series. Please also refer to our separate notes on data sources and methodologies for further information.
It includes general notes, which are applicable to most of the charts and data points in our reports, and specific notes, which capture important changes and anomalies in individual datasets or data points.
You can find the specific notes further down the page. Please note that specific notes are listed in reverse chronological order, with the most recent updates towards the top of the page.
General notes
The reports and analysis articles that we feature on DataReportal use data from a wide variety of sources, including data published by market research companies, internet and social media platforms, governments and public bodies, and many more. We also include extrapolations of this data, and our own proprietary analysis.
Wherever possible, we prioritise data sources that provide broader geographical coverage, in order to minimise potential variations between data points, and to offer more reliable comparison across countries. However, where we believe that individual, standalone metrics provide a more reliable reference, we use those numbers to provide more accurate reporting and more representative insights. Please note that some data points may only be available for a limited selection of countries or on an ad hoc basis, so we may not be able to report the same data for all geographies, or in each update to our reports.
From time to time, we may also change the source(s) that we use to calculate or inform specific data points. As a result, some figures may appear to change in unexpected ways between different reports in the Global Digital Reports series. Wherever we’re aware of these changes, we include details in the footnotes of each relevant chart. However, please use caution when comparing any data from different reports, because changes to research samples, base data, research methodologies, and how the source organisation reports its data may mean that values for what may appear to be the same data point are not directly comparable.
Furthermore, due to differing data collection and treatment methodologies, and the different periods during which data have been collected, there may be significant differences in the reported metrics for what appear to be similar data points throughout our reports. For example, data from surveys often varies over time, even if that data has been collected by the same organisation using the same approach in each wave of their research.
In particular, reports of internet user numbers may vary considerably between different sources and over time. In part, this is because there are significant challenges associated with collecting, analysing, and publishing internet user data on a regular basis, not least because research into internet adoption necessitates the use of face-to-face surveys. Different organisations may also adopt different approaches to sampling the population for research into internet use, and variations in factors such as the age range of the survey population, or the balance between urban and rural respondents, may play an important role in determining eventual findings.
Prior to our Digital 2021 reports, we included data sourced from social media platforms’ self-service advertising tools in our calculations of internet user numbers, but we no longer include this data in our internet user figures. This is because the user numbers reported by social media platforms are typically based on active user accounts, and may not represent unique individuals. For example, one person may maintain more than one active presence on the same social media platform (i.e. “duplicate” accounts). Similarly, some accounts may represent “non-human” entities, including: pets and animals; historical figures; businesses, interests and causes, groups and organisations; places of interest; etc.
As a result, the figures that we report for social media users may now exceed the figures that we report for internet users, and in some instances may even exceed figures for total population. However, such instances do not necessarily represent ‘errors’ in the data, or in our reporting. For example, these differences may indicate delays in the reporting of internet user numbers, or they may indicate higher instances of duplicate or “non-human” social media accounts. In situations where social media user numbers exceed figures for population, it may be worth considering that official population figures might not include migrant workers or temporary residents, but these individuals may still be counted in social media platforms’ reports of users for that location.
Similarly, the figures that we report for the advertising audience reach of certain social media platforms may be quite different to the total monthly active user figures we report for those same platforms. This difference may be caused by various factors, including:
The impact of economic sanctions, which may prevent companies from selling advertising in certain territories.
Individual user behaviours, which may result in some active users not being shown any advertising in the relevant reporting period. For example, if a Facebook user only checks a friend’s profile, they would still qualify as an active user, but they may not see any advertising, because adverts do not currently appear on people’s Facebook profile pages.
Company policies which may restrict the serving or targeting of advertising to specific users or user groups.
A lack of infrastructure enabling the sale of advertising in certain locations.
Specific notes
2024
January 2024 – new TikTok dataset: we’ve recently been able to access an enlarged dataset for TikTok, which has enabled us to report the platform’s ad reach figures across a much wider range of countries. We still source this dataset directly from Bytedance, and the included values are directly comparable with those in our previous dataset. However, because the new dataset includes so many new countries, our latest figures for global TikTok use are not comparable with figures published in previous reports.
January 2024 – impact of previous changes: please read the extensive notes below for changes we introduces during 2023, which may impact values for change over time published in our 2024 reports.
2023
April 2023 – changes to core metrics: starting with our Digital 2023 April Global Statshot Report, we’ve changed the labels we use to describe some of our “headline” metrics, and we’ve also made some changes to how we calculate these figures:
We now refer to “unique mobile phone users“ as “unique mobile phone subscribers”, but our methodology for calculating this figure has not changed.
We now refer to “internet users” as “individuals using the internet”, but our methodology for calculating this figure has not changed.
We now refer to “social media users” as “social media user identities”. The sources we use to calculate this metric have not changed, but we have changed how we use that source data in our calculations, in order to reduce the potential for duplication (e.g. where the same person operates multiple social media accounts). However, despite our best efforts and those of our source data partners, please note that this metric may still not represent unique individuals, due to the potential influence of both duplicate and “false” accounts. Please note that these changes may result in meaningful differences in the figures we report for specific countries, as compared with the figures published in our reports prior to March 2023.
April 2023 – changes to GWI’s methodology: As part of its regular program of updates, GWI recently made some important revisions to its underlying methodology, and these updates have resulted in meaningful changes to the values for various data points in our April 2023 report, as compared with similar data points included in previous reports in this series. The GWI team has offered the following guidance to help you make sense of how these changes may impact findings in this report. However, note that these changes only affect data sourced from GWI, so please refer to the sources detailed in the footnotes of each page to identify which specific charts and data points may have been impacted by these revisions.
“Survey response styles differ around the world. In some countries, people tend to select a greater number of answer options, while in others they’re more reserved. This isn’t inherently bad, and these cultural differences are insights in and of themselves. However, starting with our Q4 2022 wave of research, we’ve taken a number of steps to reduce the potential impact that these differences can sometimes have on research findings.
GWI has always had checks in place to capture and remove over-selectors, as well as other checks to identify respondents who provide contradictory answers, or complete the survey too quickly. However, the introduction of new questions in recent waves of research has allowed us to implement some additional checks. In particular, these checks involve taking a stricter approach to distinguishing between respondents who’re responding in a genuinely enthusiastic way, and those who’re over-selecting in a way that’s inaccurate or insincere.
That isn’t to say that results from prior waves are invalid; simply that the “noise” associated with different cultural response styles has been reduced, allowing for more effective comparisons between markets. In many cases, the results are now more in line with other sources, and differences between markets are less extreme.
What this means for GWI’s data: values have declined across multiple categories and across a number of markets that are prone to over selection, particularly India and China. Because these markets are so big and account for such a large share of the global online population, these declines are also apparent in global figures, albeit to a lesser extent.
What this means for you: GWI data is now even more comparable across markets than it was before. However, you should be aware of and consider these changes when looking for trends – particularly at a global level and in APAC.
Further information: If you want to know more about GWI respondents and response styles, please refer to this article. If you want to know more about GWI’s data checks more broadly, please refer to the GWI Core Research and Methodology deck on this webpage.”
April 2023 – social media ad reach data: our ongoing analysis has identified some unusual trends in the “official” audience reach data published in the ad planning tools of various social media platforms over recent weeks, which may impact the reliability of metrics such as change over time. Specific to our Digital 2023 April Global Statshot Report, these trends are particularly apparent in the data published in Twitter’s Ads Manager tool, and in Meta’s Ads Manager tool. While the ad reach numbers that we publish in our Global Digital Reports are a faithful and accurate representation of the audience estimate figures that the platforms themselves publish, significant anomalies in source data over time mean that readers should proceed with caution when exploring trends over time in these datasets.
April 2023 – Indonesia: Partly due to the anomalies in social media ad reach data highlighted above, but also due to increasing incongruities in other source data, we’ve changed the sources and approaches that we use to inform and calculate key digital adoption and use metrics for Indonesia. As a result, the figures that appear in our latest update may appear to be lower than those that appear in our previously published reports. However, these changes reflect revisions in data, and do not represent a drop in actual use of connected tech in the country. We advise readers not to compare figures for internet use and social media use included in our reports published after March 2023 with reports published before March 2023.
January 2023 – social media users: significant changes in source data mean that our latest values for social media users may appear to be meaningfully lower than figures we published in our previous reports. However, in most cases, we do not believe that such changes correspond with an actual drop in social media use. Rather, these changes are the result of revisions and corrections in source data. As a result, we strongly advise readers not to compare figures for social media user numbers published in our Digital 2023 reports with similar figures published in previous reports in this series. For context, we use advertising reach data published by various internet companies – including but not limited to Meta, Google, ByteDance, Snapchat, and Twitter – to inform our overall figures for social media users. The data “signals” that we collect from these companies are subject to methodology changes and data revisions at source, which may result in meaningful changes to our calculated totals, and in some instances, these changes may result in a lower figure for overall social media use than the figure that we published for the same country or region in previous reports in the Global Digital Reports series. During the course of 2022, we detected multiple revisions in source data from various sources – including advertising reach figures published by Meta and Google – and these revisions have resulted in significant changes in our calculated values for social media user numbers. Some of these changes may relate to US sanctions on the Russian Federation as a result of the conflict in Ukraine. However, most changes appear to be updates and corrections in audience reach figures. Notes in the companies’ respective advertising tools offer no detailed explanations for these revisions, but we believe that such revisions may relate to how these companies define and identify duplicate and inauthentic accounts.
January 2023 – internet user numbers: we use data from a number of different sources to inform the values that we publish for internet user numbers, but data from the ITU and GSMA Intelligence influences our figures in the greatest number of geographies. Both of these organisations made significant revisions to their base data in the final quarter of 2022, including significant updates to historical data. Based on these revisions, we have also updated our historical figures, which may mean that figures for change over time in internet user numbers published in our Digital 2023 report do not correlate with figures for internet use published in previous reports in the Global Digital reports series. In addition, due to the widespread reporting of internet adoption as a percentage of population, the changes in underlying population data detailed in our entry for July 2022 below have also played a role in reshaping our latest internet user totals. As a result, we strongly advise readers not to compare figures for internet user numbers published in our Digital 2023 reports with similar figures published in previous reports in this series. It’s also important to highlight that collecting, analysing, and reporting internet user research is a highly complex process. For example, research into adoption of the internet should be conducted face-to-face, because even phone interviews would skew results in favour of people who already have access to a device that enables them to access the internet. Because of these complexities, it often takes many months – and sometime years – before the “latest” internet user research is ready for publication. As a consequence, our latest figures for internet user growth between 2022 and 2023 may not fully represent actual growth, because source data for those years will not yet be available for all locations.
January 2023 – impact of US sanctions on Russia: US sanctions on the Russian Federation due to the continuing conflict in Ukraine mean that many Western companies are currently unable to sell their advertising services in the country. These restrictions have resulted in meaningful drops in the global advertising audience reach figures for various social media platforms versus the figures published by those platforms in early 2022. In particular, this issue may affect YouTube, Instagram, Twitter, and Facebook, but it may also impact other platforms and non-social media figures, including ecommerce spend.
January 2023 – audience reach figures for Meta’s platforms (Facebook, Instagram, Facebook Messenger): Our analysis of Meta’s published figures for potential advertising reach indicate that the company continues to make significant revisions to the advertising audience numbers that it reports for Facebook, Messenger, and Instagram, as well as for its platforms’ combined audience reach. These revisions may be the result of changes in the company’s reporting methodologies and definitions, and / or due to the removal of duplicate and inauthentic accounts. Indeed, because these changes affect almost every country, we believe that these ongoing revisions reflect “corrections” in Meta’s underlying data, and we’re confident that these changes do not reflect changes in actual platform use. However, it’s worth highlighting that we’ve seen a series of downward corrections in Meta’s audience reach numbers over recent months, and given the size and increased frequency of these corrections, we have decided to publish change figures “as is”, to enable readers to make better informed decisions about the potential opportunities offered by advertising activities on the company’s platforms.
January 2023 – Facebook Messenger: certain Messenger ad formats are still unavailable in Australia, Canada, France, and the United States. Likely as a consequence of these limitations, the company’s ad planning tools show unusually low figures for potential Facebook Messenger ad reach in these countries. Because of this, we have chosen not to publish figures for these four countries in our Digital 2023 local country reports, because we do no believe that the data currently showing in the company’s ad planning tools is representative of overall use of the Messenger platform.
January 2023 – TikTok: we have identified significant fluctuations in the potential reach figures published in TikTok’s advertising resources over recent months, so advise reader caution when interpreting values for change over time.
January 2023 – Twitter: readers may wish to interpret the figures for potential advertising reach in certain countries published in Twitter’s advertising resources with caution. Specifically, our analysis has revealed large and seemingly implausible changes in the published figures for advertising reach in Singapore and the Seychelles, such that Twitter’s published figures for advertising reach in these countries now exceeds each country’s total population. Recent trends in audience growth in Saudi Arabia may also point to the need for extra caution.
2022
October 2022 – Meta (Facebook, Instagram, and Messenger): Meta appears to have made yet more revisions to the advertising audience reach numbers that it reports for Facebook, Messenger, and Instagram, as well as for its platforms’ combined reach. Our analysis suggests that Meta makes such revisions on a regular basis, and they may be the result of changes in the company’s reporting methodologies and definitions, and / or due to the removal of duplicate and inauthentic accounts. Indeed, because these changes affect almost every country, it seems likely that the most recent round of revisions reflects corrections in Meta’s underlying data, and these changes do not necessarily reflect similar changes in actual platform use. However, it’s worth highlighting that we’ve seen a series of downward corrections in Meta’s audience reach numbers over recent months, and this is the first time we’ve seen the company make such extensive revisions for all platforms at the same time.
October 2022 – TikTok: changes in the way that Bytedance reports advertising audience reach figures for its various platforms, combined with changes in our methodology for assessing TikTok’s advertising reach, mean that audience figures for TikTok published in our reports after September 2022 are not comparable with similar data points included in reports published before September 2022.
October 2022 – YouTube: Google added data for a number of new countries to its advertising resources since July 2022, so the advertising audience reach figures for YouTube included in our reports published after September 2022 may appear meaningfully higher than figures included in reports published prior to September 2022.
July 2022 – population: the United Nations published a comprehensive update to its World Population Prospects data this month, which included some important revisions to population numbers around the world. In addition to impacting the figures we report for population, these revisions may also affect all data points where we report digital adoption or activity as a percentage of total population, or as a percentage of a specific age group within the broader population (e.g. people aged 13 and above). As a consequence, please note that any comparisons to equivalent data points in the Global Digital Reports series published prior to July 2022 may result in unexpected trends, including potential drops in user figures.
July 2022 – social media users: source corrections and revisions to the published ad audience reach figures for certain social media platforms (as detailed below) have resulted in some changes to our published figures for social media users by country. As a result, social media user and adoption figures in our reports published after 01 July 2022 may not correlate with similar data points included in our reports published before that date.
July 2022 – YouTube audiences: Google appears to have revised its ad audience reach data for YouTube sometime during Q2 2022, resulting in some important corrections to the published audience figures in certain countries. These revisions may be due to changes in Google’s methodologies and / or definitions, and / or due to the removal of duplicate or inauthentic accounts. YouTube audience data for Russia is also currently unavailable due to the conflict in Ukraine, resulting in a significant drop in global YouTube reach. As a result, user figures and audience reach figures for YouTube included in our reports published after 01 July 2022 are not directly comparable with similar data points included in our reports published before that date.
July 2022 – Instagram audiences: Meta appears to have revised its ad audience reach data for Instagram sometime during Q2 2022, resulting in some important corrections to the published audience figures in certain countries. These revisions may be due to changes in Meta’s methodologies and / or definitions, and / or due to the removal of duplicate or inauthentic accounts. As a result, user figures and audience reach figures for Instagram included in our reports published after 01 July 2022 are not directly comparable with similar data points included in our reports published before that date.
April 2022 – Meta’s audiences in Russia: due to the ongoing conflict in Ukraine, Meta’s advertising tools have not published any updates to audience data for the Russian Federation, which may impact various figures across the company’s platforms. For reference, in January 2022, Meta’s tools reported that Facebook ads reached 8.6 million users in Russia, while Instagram ads reached 63.0 million users in the country. However, due to Meta’s revised reporting, we are currently unable to include any figures for Russia in the ad audience data that we report for Facebook, Instagram, or Messenger. This has also had a negative impact on the global figures that we report for each of these platforms.
January 2022 – internet connection speeds: Starting with our Digital 2022 local country reports, the figures that we publish for internet connection speeds reflect median connection speeds, as opposed to the mean values that we featured in previous reports. Median values offer a more representative indication of the speeds that “typical” users can expect, but this change means that the internet connection speeds published in our Digital 2022 reports will not be comparable with the values published in our previous reports. Learn more about the difference between median and mean here.
January 2022 – internet users: Research into internet adoption relies heavily on face-to-face interviews, because even phone-based interviews can inflate adoption rates by focusing on people who already have access to technologies that enable internet connectivity. COVID-19 has made it more difficult to conduct these face-to-face interviews, resulting in a meaningful reduction in internet user research over the past two years. Research has started to return to normal, but because of these delays, new data may reflect changes over two or more years. As a result, figures for year-on-year growth in internet users in our Digital 2022 reports may represent growth over periods of more than one year, especially where updated figures were not available in time for the production of our Digital 2021 reports.
January 2022 – social media users: The figures we report for social media users rely heavily on advertising reach data published by social media platforms, and any changes in how those platforms report reach can have a significant impact on the numbers we report too. Various changes in platform reporting over recent months (see below) mean that some numbers in this year’s reports are not directly comparable with figures for the same data points published in previous reports, even though all figures were correct at the time of production. Where we’ve been able to calculate representative growth figures, we’ve included those figures in this year’s reports, but where we haven’t included figures for change over time, it’s likely that any comparisons to historical data will result in inaccuracies. As a result, please avoid comparing social media user figures in this year’s reports with figures included in our previous reports.
January 2022 – Meta: In Q4 2021, Meta made some important changes to how its self-service tools report the potential advertising reach of its platforms, including a move to publishing audience figures as a range, instead of as an absolute number. Our analysis suggests that Meta also revised its base data around this time, resulting in some important corrections to published audience figures for Facebook and Messenger. As a result, audience figures for Facebook and Messenger included in our reports published after 01 January 2022 are not comparable with similar data points included in our reports published before that date.
2021
October 2021 – Twitter: In Q3 2021, Twitter made important changes to how its self-service tools report potential advertising reach. As a result, audience figures for Twitter included in our reports published after 01 October 2021 are not comparable with similar data points included in our reports published before that date.
January 2021 – internet users: starting 01 January 2021, we no longer include social media user numbers in the figures we publish for internet users, except where no other data are available. As a result, internet user numbers included in our reports published after 01 January 2021 may not be directly comparable with similar data points included in our reports published before that date.
Further information
If you have any questions about specific data points used in our reports, please see our notes on data sources and methodologies.
And if you’d like to offer your organisation’s data for potential inclusion in our future reports, please click here to email us.