whatsapptwitterteamslinkedinfacebookworkplace

Are you part of your own data? 

The inspection paradox

Imagine you’re driving on a highway; it’s busy but traffic is still moving smoothly. Here in Belgium the maximum speed allowed on the highway is 120km/h. I want to be at my destination as quick as possible but as a law abiding citizen I would not go over the speed limit, so let’s say I average around 110km/h. My initial assumption is that other people would behave more or less as I do, so we expect the speed of all cars on that highway to be normally distributed with an average of 110km/h. A policeman with a speed radar is standing next to the highway, and the graph on the left of the figure shows the distribution of the speed of the cars they observe. In my car however I experience something completely different! It seems like I’m the only responsible driver here, everybody else is driving either rather slowly or much faster than me. The distribution of the speed that I observe from my car looks like the one on the right:

Figure 1: Speed of cars (km/h) according to somebody witching from the side (left) or as perceived by somebody going 110 km/h (right)

What is happening here? Basically I’ve became part of my own data. Most cars you see are either passing you or being passed by you; in other words they are faster or slower. You will notice cars going the same speed as you but they will stay right in front or right behind you for a longer period of time. Other cars driving the same speed as you but further in front or behind you are just way less likely to pass you or be passed by you. Formally we call this a case of the inspection paradox which happens when the probability of observing something (e.g. a car going at a certain speed) is not independent of its magnitude (the speed of that car). Using yourself as a point of reference co-occurs with cognitive anchoring and should therefore always be avoided.

The friendship paradox

A related topic is known as the friendship paradox which states that your friends – on average – have more friends than you do. This sounds contradictory but does make sense if we represent friendships (or whatever relationship for that matter) as a graph. Every node (the dots) in the following drawing represents a person, and the number in the node corresponds with the number of connections to friends. Most nodes in this example connect to two or three others, but the ones they connect to are more likely to be the node with higher numbers. The “popular” node in this case, i.e. those with six or seven connections, are rather rare but account for more connections.

Figure 2: Graph representation of connections between people.

There are more people (potential friends of yours) with only a few rather than a lot of friends, so if you would pick a person at random it’s more likely that this person doesn’t have a lot of friends, rather than have many friends. The wrong assumption that results in the paradox is that the choice is not random. The reason it is less likely to have a friend with fewer friends is due to the fact that a person with few friends is simply less likely to be a friend of yours than someone with lots of friends.

Don’t think that it’s only your friendships that are victim of this paradox: your sexual partners have had – on average – more sexual partners than you, your co-authors have published more than you, and your street probably ends in a bigger street. Basically this counts for all types of networks, but especially when we’re discussing social networks the paradox is apparent as we tend to switch between different points of view on the data: that of the observer looking at the graph versus the one where you’re part of the graph and look towards your friends.

If you want to read more about the inspection paradox and the more specific friendship paradox, I recommend the article The Inspection Paradox is Everywhere by Prof. Allen Downey (2019).

IF YOU WORK WITH DATA

You as a researcher should never be part of your own data and if you adhere to this you’ll not come to wrong conclusions in your research. Ask yourself if the observations could have had a different probability distribution if they were taken by someone else. If so, your own position might be of influence which should be removed from the equation.

Your choice of measurement instrument can influence your conclusions 

Measuring length is not that straightforward

Today I wondered what the total perimeter of Belgium is. As the weather was nice I decided not to Google it but instead go outside with a ruler and put it on the country borders. In total I had to lay down my ruler exactly twenty times (it was a rather big one); this is represented on the left of the following picture. During measuring I noticed my ruler didn’t allow me to measure some parts as accurately as I was hoping to as I had to cut some corners and approach the non-straight parts. I went out again but this time with a ruler exactly half the size of the previous one. The result of the second attempt is visualized on the right. With the smaller ruler I counted forty-two units. Given the fact that the small ruler was half the size of the big one that is an increase in perimeter of 5%.

Figure 3: The total perimeter of Belgium measured twice with rulers of different lengths. Obviously both my rulers are way too large and inaccurate to measure the perimeter of a country. As you further decrease the size of your ruler (i.e. increase the size of your accuracy in measurement) the resulting perimeter will increase. You might think that at some point this would converge to the “actual” perimeter, but this is where we arrive at the paradox: the total perimeter measured will keep on increasing as you keep refining your measurement even though you can’t imagine the perimeter of a country to be infinity long.

What happens – especially at natural borders like coastlines, so that’s where the name coastline paradox comes from – is that the closer you look (which you do if you’re using a smaller ruler) the more detail starts to arise As you walk at the coast you measure around a boulder something you might not have done with a less accurate ruler. On closer inspection you see that the boulder does not have a smooth surface so you take a smaller ruler to take the dents into account thus increasing the total length of your coastline. Theoretically you can go on to a sub-microscopic level, to eventually come up with the answer that the coast has an infinite length.

If you work with geographical data this is a real issue, but there are more common types of data where this problem arises like when a continuous variable is represented in a discrete way. Common examples of this are signal data (waves) and graphical data (pictures). A digital picture most commonly represents objects using pixels which we call raster-based images. Vector-based images exist too but are generally limited to specific applications. Problems arise once we start measuring on those raster-based images. If you draw a circle using a raster-based imaging program like Paint or GIMP you actually get an approximation which is worse as the resolution is lower. Take for example the 12 by 12 pixel circle in the following picture:

Figure 4: A 12 by 12 pixel circle. The red line is the radius; in blue a quarter of the circumference is marked.

The red line is the radius (r) of the circle which is 6 pixels long. Let’s look what the circumference (C) would be. A quarter of the circumference is highlighted in blue. If we sum the length of the segments we get (from left to right): 4 + 1 + 2 + 1 + 1 + 2 + 1 = 12. As this is exactly one quarter of the circle the whole circumference is 12 · 4=48. We all know the relationship between the radius of a circle and its circumference:

C = 2 · r · π

If we put our radius of 6 and circumference of 48 in the equation we find that π = 4 which is obviously wrong. The reason is of course that what we were measuring was not the actual circumference but the way around the pixels. This applies not only to circles but any non-straight path. There are formulas to accurately measure non-straight lines on a discrete grid to correct for this problem.

Measurement in social sciences

Some years ago a student working on her Master’s thesis asked me to take a critical look at a questionnaire she had developed for use in her research. She wanted to measure the consumption of porn in today’s adolescents. One of the questions was “How often do you watch porn?” with different possible answers ranging from “Never” to “Multiple times per week”. I was impressed with the construction of the questionnaire and only had minor suggestions, one of which was to add an extra possible answer to said question being “Multiple times a day”. The reason was not that I expected people would actually use this answer but because when it comes to socially sensitive subjects people have the tendency not to choose the most “extreme” answer. Only the adjusted version of the questionnaire was used so we were not able to prove that adding the “Multiple times per day” option had a significant effect on how people would answer this question. However it’s worth mentioning that despite our expectations a rather significant number of the participants chose this option.

Most mistakes regarding the use of a too limited range in possible answers in questionnaires are not intentional but the result of poor questionnaire design. These problems are then visible during the data-analysis when lots of the items show an extremely skewed distribution which often makes it very hard for the statistician working with the data. There are however cases where this is intentional:

Figure 5: Something missing? (Watson, 2017) There is a whole field of research when it comes to designing questionnaires so that they measure as objectively as possible. Avoiding bias due to effects of socially desirable answering as well as making sure you capture the right range are just two of many known points of attention. If you even plan to work with this kind of data, especially if you need to work with non-standardised tests and want to develop questionnaires yourself you should study this thoroughly as bad design will lead to wrong conclusions. Even if you don’t do this kind of research yourself, when you read conclusions by others check out how they measured it.

IF YOU WORK WITH DATA

Accurate measuring is key regardless of what kind of data you work with as the quality of your data always determines the upper level of quality of a model you build on top of. We illustrated this with two totally different types of data: images and questionnaires. At first sight they seem completely unrelated but they do share a common pitfall.

Conclusion for this part

 

I hope that by now you’re already looking a lot more critically towards your own data-related work and see ways to improve it in the future by being aware of all those pitfalls. There’s still some more to come though which will be covered in the third and final part.

References 

Downey, A. (2019). The Inspection Paradox Is Everywhere. https://towardsdatascience.com/the-inspection-paradox-is-everywhere-2ef1c2e9d709

Watson, K. (2017). Trump ’Approval Poll’ Offers No Negative Options. https://www.cbsnews.com/news/trump-approval-poll-offers-no-negative-options/

Any questions? Don’t hesitate to contact me: joris.pieters@keyrus.com

newsletter.svg

Never miss an insight

Stay updated on the latest articles, events, and more

Your email address is only used to send you the Keyrus newsletter. You can use the unsubscribe link in each newsletter sent at any time. Learn more about the management of your data and your rights.

Continue reading

Blog post

DEEP LEARNING FOR UNSTRUCTURED DATA? YES, YOU CAN !

August 9, 2021

Today, you take a picture of a paper bill and it gets suddenly processed by your banking app without you doing anything but confirming through Face Id recognition. Today, you speak to your microphone’s car while driving and it starts calling someone from your contact list. Today, you are probably old-fashioned if you never used google translate to process some sentence in another language, right?

Expert's opinion

UPGRADE OF A SEMARCHY XDM SOLUTION

August 9, 2021

In 2014, one of our clients (leading provider of packaging worldwide) sought a solution to bring structure to their customer base. They reached out to Keyrus who designed and developed the Customer Data Integration (CDI) tool.

Blog post

BE MORE EFFECTIVE THAN DOLLY PARTON ON OPEN BANKING

August 9, 2021

Appropriate action is a combination of marketing automation and of the personal touch by your frontline staff. Make it data driven.

Blog post

WHY DATA SCIENCE NEEDS BOTH MACHINE LEARNING AND CAUSAL DATA ANALYSIS

August 9, 2021

Data Science is running complex machine learning algorithms on ever growing datasets. The promise towards business stakeholders is to replace gut decisions and experience with objective and improving algorithms. But is machine learning the only game in town data scientists need to help business decision making?

Blog post

RISE OF THE CITIZEN DATA SCIENTIST

August 9, 2021

And why you still can’t replace your employees with software completely...

Blog post

DATA SCIENCE EXPLAINED BY BAKING CAKE

August 6, 2021

A few times I have been asked what it is I do exactly as a data-scientist, and managers and potentials data-scientists especially are interested in the common struggles we as data-scientists have to deal with. Just listing all issues we comes across would not result in an interesting read, so I will present it to you in the form of an analogy you’re all familiar with: baking cake.

Blog post

Data Visualization and Decision Making

August 6, 2021

“In 2019, one of the leading actors in the Oil Industry, was assessing different possibilities for the implementation of a mobile payment solution in their B2B segment. In order to be able to take data driven decisions, they reached out to Keyrus to set-up a data visualization solution.”

Blog post

DATA AND ANALYTICS: THE FOUR TYPES OF DATA ANALYTICS

August 6, 2021

You might have heard the saying Data is the new oil. This mainly refers to their potential value and in both cases this value is not merely in the raw product but rather results from the way it is processed. In this article we present a commonly used classification of data and analytics into descriptive, diagnostic, predictive, and prescriptive analytics. We’ll discuss each of these separately including some of the commonly used methods. Thereafter follows how these four types of data analytics relate to each other. First however we’ll explain what we exactly mean with data and analytics.

Blog post

THE HUMAN BEHIND THE DATA (PART 3 OF 3)

August 3, 2021

You’ve made it to the third and final part in our series ‘The human behind the data’. This will all be about (illusionary) patterns and the importance of some good old probability theory.

Blog post

THE HUMAN BEHIND THE DATA (PART 1 OF 3)

August 3, 2021

Humans are not very rational beings, even though we think we are. This impacts our personal as well as our professional lives, and the latter is of particular importance if you often work with data. If you work in business intelligence, data science, or any related field, people expect you to deliver them an objective truth. In this article we’ll discuss a lot of pitfalls that undermine this goal. Knowledge of these will help you avoid these mistakes and also to spot them in other people’s work. Many of the topics covered in this article involve pitfalls that can be classified as biases or paradoxes.