Blog

Why do i suddenly care about statistics and data?

Posted admin
www.geoinfo-solutions.com

In his book How to Lie with Statistics Darrel Huff had themes that included “Correlation does not imply causation” and “Using random sampling“. It also shows how statistical graphs can be used to distort reality, for example by truncating the bottom of a line or bar chart, so that differences seem larger than they are, or by representing one-dimensional quantities on a pictogram by two- or three-dimensional objects to compare their sizes, so that the reader forgets that the images do not scale the same way the quantities do.

Of late I have taken a great interest in learning data science, this bug bit me some time back, but I hadn’t really answered the whole important question required in all worthwhile endeavors…WHY?? As the saying goes, “When the WHY is clear, the HOW will take care of itself”

I have become obsessed with the whole idea of data analytics, predictive models, regression etc. and using data to understand how businesses are using data to make better decisions, I got to understand how some of the biggest and most valuable companies today e.g. Google, Facebook, Apple, LinkedIn are built around data, they have mastered the art of turning data given to them by customers into products or data driven services.

Of course this is exciting to me, because in the many years working with GIS Ltd, data analytic is a great part of what we do, we approach all this from an information management perspective, our focus is on helping NGO’s use data to answer questions like; Who are our beneficiaries and where are they? “beneficiary targeting” —- which project should we implement first to achieve the greatest impact and where? “project location – prioritization”, “project – beneficiary matching” —–To what extent will this project meet the needs of our targeted beneficiary?

The ability to properly answer these questions is something NGO’s & CBO’s continue to struggle with, it’s a common sight in many former “NGO hot zones” to go and find many communities still struggling with poverty, poor sanitation etc., one may ask, is this a question of causation or correlation?

We’ve all heard the joke that eating pickles causes death, because everyone who dies has eaten pickles. That joke doesn’t work if you understand what correlation means.

NGO’s & Governments sit on terabytes of data that is only minimally put to use in decision making, of course this trend is changing as more pressure is put on these institutions to have a scientific based approach to responding to humanitarian challenges and extending much needed service to the population.

In one of our many capacity building events we organize for NGO’s as we advocate for a scientific approach to humanitarian response, I receive questions of various challenges that these technical people go through as they try to provide much needed support to various communities, one that stood out for me was a participant from one of the NGO’s focusing on drilling water points, who asked how they could improve their chances of striking water, as you may have guessed “effective” use of data was a major part of my recommendation.

As humanitarian challenges around the world get more complicated, NGO’s will not just be required to respond faster, but act in a more informed manner that allows beneficiaries to quickly feel the impact of their interventions. Effective data use will play a major factor in this.

The question facing every company today, every startup, every non-profit, is how to use data effectively — not just their own data, but all the data that’s available and relevant. Using data effectively requires something different from traditional statistics. Let’s not confuse statistics, M&E, with data scientist.

So who is a data scientist and who is not?

Data Science is an interdisciplinary field about processes and systems to extract knowledge or insights from data in various forms, either structured or unstructured, which is a continuation of some of the data analysis fields such as statistics, data mining, and predictive analytics, similar to Knowledge Discovery in Databases (KDD).

But merely using data as many professional disciplines do isn’t really what we mean by “data science.”

What differentiates data science from statistics and M&E is that, data science is a holistic approach. We’re increasingly finding data in the wild, and data scientists are involved with gathering data, massaging it into a tractable form, making it tell its story, and presenting that story to others. This is the most boring but also the most crucial.

In the last 5-10 years, there has been an explosion in the amount of data that’s available. Whether we’re talking about web server logs, tweet streams, online transaction records, “citizen science,” data from sensors, government data, or some other source, the problem isn’t finding data, it’s figuring out what to do with it. And it’s not just companies using their own data, or the data contributed by their users. For even more mind numbing statistics, check out the infographics below produced by Domo

Data Science in action

During the recent earth quake catastrophe in Nepal, millions of OpenStreetMapvolunteers took online to provide much needed support in updating the map of Nepal, these maps which are data products was instrumental in making life saving decisions especially when it came to delivering much needed humanitarian relief to the affected people.

I have always wondered how LinkedIn is able to suggest people to connect with?LinkedIn uses patterns of friendship relationships to suggest other people you may know, or should know, with sometimes frightening accuracy.

Techniques like sentiment analysis also known as opinion mining can be used to determine the attitude of the speaker, voters, community opinion on new projects etc. I witnessed this first hand during an event at technology hub Outbox, where R was used to analyze twitter text messages streamed during the first live televised presidential debate in Uganda, through the hashtag #UGDebate16

The ability to take data — to be able to understand it, to process it, to extract value from it, to visualize it, to communicate it — that’s going to be a hugely important skill in the next decades.

34 thoughts on “Why do i suddenly care about statistics and data?”

  3. I seriously love your site.. Excellent colors &
    theme. Did you build this site yourself? Please
    reply back as I’m looking to create my own personal site
    and want to learn where you got this from or just what the theme is called.
    Appreciate it!

    1. Thanks for the compliments, sure we built these ourselves. The theme we used was called Sydney

  4. I’m not that much of a online reader to be honest but your
    blogs really nice, keep it up! I’ll go ahead and bookmark your website to come
    back later. Cheers

  5. I wish to express some appreciation to this writer just for bailing me out of such a circumstance. After exploring throughout the world-wide-web and seeing concepts that were not pleasant, I thought my life was done. Existing without the presence of solutions to the issues you’ve solved by means of the short post is a crucial case, and ones which could have badly affected my entire career if I had not noticed your web page. Your main understanding and kindness in controlling every item was very helpful. I’m not sure what I would have done if I hadn’t come upon such a step like this. I can at this moment relish my future. Thanks so much for your impressive and effective help. I won’t be reluctant to suggest the website to any person who ought to have assistance about this area.

  7. Wow that was unusual. I just wrote an incredibly long comment but after I clicked submit my comment didn’t show up. Grrrr… well I’m not writing all that over again. Regardless, just wanted to say fantastic blog!

  8. I am really loving the theme/design of your blog. Do you ever run into any web browser compatibility issues? A small number of my blog visitors have complained about my website not operating correctly in Explorer but looks great in Firefox. Do you have any ideas to help fix this issue?

  9. Terrific post but I was wanting to know if you couyld write a
    litte more on this topic? I’d be very thankful if you
    could elaborate a little bit more. Many thanks!

  15. Great web site you have here.. It’s difficult to find high quality writing
    like yours nowadays. I really appreciate people like you!
    Take care!!

  17. Thanks for ones marvelous posting! I really enjoyed reading it,
    you will be a great author.I will make sure to
    bookmark your blog annd will eventually come back down the road.
    I wznt to encourage continue your great work, have a nice day!

  23. Simply wish to say your article is as surprising. The clearness in your post is simply cool and
    i can assume you’re an expert on this subject. Fine with your permission let me to grab your RSS fered to keep updated with forthcoming post.
    Thanks a million and please carry on the gratifying work.

  24. I just want to mention I am newbie to blogs and seriously savored you’re page. Very likely I’m going to bookmark your blog post . You definitely have excellent posts. Cheers for sharing your website.

  25. I have been exploring for a bit for almost any good quality articles
    or blog posts within this form of space . Exploring
    in Yahoo I finally found this website. Studying this info So i’m satisfied to exhibit that I’ve a remarkably perfect uncanny feeling
    I discovered just what I needed. I so much surely is likely to make sure to don?t overlook this website and give it a peek on a continuing basis.

  26. Does your site have a contact page? I’m having a tough time locating it
    but, I’d like to shoot you an e-mail. I’ve got some creative ideas for your personal blog you may well be
    enthusiastic about hearing. In any event, great blog and
    I look ahead to seeing it improve as time passes.

  27. Hello there! I recently would want to give you a huge thumbs up for the great info you’ve got on this site with this
    post. I will be coming back to your site for further soon.

  28. Today, while I was at work, my sister stole my apple
    ipad and tested to see if it can survive a 25 foot drop,
    just so she can be a youtube sensation. My iPad is now destroyed and
    she has 83 views. I know this is totally off topic but I had to share it with someone!

  29. Hey there I am so glad I found your blog page, I really found you by
    error, while I was looking on Yahoo for something else, Anyways I am here
    now and would just like to say cheers for a fantastic post and
    a all round enjoyable blog (I also love the theme/design),
    I don’t have time to look over it all at the moment but I
    have bookmarked it and also added in your RSS feeds, so when I have time I will be
    back to read much more, Please do keep up the superb job.

Comments are closed.