Play big on small data

Chris HOHMANN

Chris HOHMANN – Author

This weird title “Play big on small data” suggests the utilization of big data principles on small data sets. “Small” is to be considered relatively to huge amount of data big data can manage, which is not necessarily only a handful.


I came across big data with former colleagues who were IT experts and got a kind of epiphany about big data with the eponym book.

Since that reading I do not collect, structure and analyze data the same way anymore. I tend to be more tolerant about inaccuracies, mess and lack of data because what I am looking for is insight and big picture rather than certitude and accuracy.

As poorly tended datasets are the norm rather than exception, starting an analysis with this mindset saves some stress. The challenge is not to filter out valid data for a statistical significant analysis, but a way to depict a truthful “good enough” picture, suitable for decision-making.

Playing big on small data does not mean apply the technical solutions for handling huge amount of data or fast calculation on them, simply get inspired by an approach favoring the understanding of the “what” rather than the “why”, in other words, favor correlation instead of causation.

In many cases, a good enough understanding of the situation is just… good enough. Going down to the very details or make sure of the accuracy would not change much but would take time and divert resources for the sake of unnecessary precision.

When planning a 500km journey, you don’t need to know each meter’s details, some milestones are just good enough to depict the way.

Accepting to trade, when it’s meaningful, correlation for causation helps to get around the few and messy data usually available. Even so data may be plenty, for a given analysis they are too often few fitting the purpose and in the right format. It is then smart to look at other data sets, even if they are in the same state, and search for patterns and correlations that can validate or invalidate the initial assumption.

The conclusion is most of the time trustworthy enough to make a decision.

Bandeau_CH36
View Christian HOHMANN's profile on LinkedIn

If you liked this post, share it!

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s