For a quick overview to big data, read Bernard Marr’s The Complete Beginner’s Guide to Big Data Everyone Can Understand (Forbes, March 2017). He sums up the basics you need to know in just a few paragraphs. Marr is a consultant, lecturer, and author of several books, including Data Strategy.
We are in the midst of a data explosion. Machine-related data is collected via sensors, transactions, and home monitoring and security devices. People-related data is collected via our smartphones, social media, shopping, and communications. Additionally email, text, social messaging, photographs, videos, webcam and satellite images, and other kinds of unstructured data are captured and stored in vast data centers around the world.
Data scientists mine data by building data models and running simulations against structured and unstructured data to look for patterns and make predictions. Sophisticated yet easy-to-use data tools make it easier to access the data. “As a service” subscriptions make it possible to mine data without expensive overhead, thereby democratizing data access for even small-scale researchers.
The upside of collecting and analyzing all this data is that we have the potential of solving some of our biggest problems – curing diseases, feeding the hungry, exploring space, preventing crime, predicting and responding to disasters, and making life generally better. The downside can be invasion of data privacy, threats to data security, and potential data discrimination.