Mar 13, 2012

7 ways to leverage Big data and data visualization

A Yotta is no longer something out of Star Wars or a Jedi master, but when you run a formal vision workshop you may find that you need one. Exabyte’s are a reality in many organizations, like Google, Facebook, Twitter, Yahoo etc. A Yottabyte is one quadrillion gigabytes. So far while the world has very few Yottabytes internet information driven companies are already planning on ways to harness this kind of data at the speed of thought. Industries like homeland security, climate research, environmental studies, genomics, Google, Facebook are already planning at these levels of data scanning while they currently work in the Petabyte and Exabyte ranges on a daily basis. While all this seems like a distant happening more and more proactive companies are realizing that they need to proactively plan for handling exceptionally large volumes of data faster than their competition – that’s the competitive advantage.
Today almost every company is experiencing unprecedented data growth. They have one of three options. The first is to live with longer and longer running queries. 100 seconds today, 500 seconds in six month and 900 seconds in a year. The second option is to start trimming data and look at smaller and smaller data sets. The third is to look at big data options. Companies harnessing this big data challenge in meaningful ways are changing the very competitive landscape they operate in. Here are some ways that companies are leveraging big data and changing their playing fields. Just look at the digital world of entertainment. It contains images, audio, movies, TV Shows and all this constitutes a lot of data. It does not belong to the enterprise but has a lot of data. It is possibly the largest data consumer on the planet today. Some examples are:-

1.      Music Industries need to churn a lot of data in order to be able to predict the next Madonna or Lady Gaga, with their experience they have all the signatures that can predict a great musician way ahead of competition. Proactive companies are looking at their license and contracts in real-time due to the emergence of private labels and their competitive push in the music industry. The other area of impact is the digital music distribution

2.      Utilities depend on energy production and consumption. Unfortunately there is no efficient method to store excess energy. So utilities loose if they produce more and lose more if they produce less. Energy consumption, Smart meters and making our planet a little green is a quest that is realizable but the amount of data is humongous. While each utility does their job the central governments or environment agencies can collate all the data from various utilities and conduct consolidated national environmental analysis for a greener nation.

3.      Tourist economies like Hawaii and Singapore cannot afford the spread of any disease and thus are looking at ways to monitor targeted drug sales in real time across their tourism economy. Haiti actually managed to track cholera from their twitter streams. In order to accomplish this they need to sort over 250 million records on a daily basis, but the effort is worth the prevention. Harvard Medical School already has a study that analyzes tweets for infectious disease signals.

4.      The world of self-publishing is getting more and more digital and the diamonds are in blogs and self-publishing contents. Getting information from the blogosphere does not require tons of equipment of thousands of servers. All it needs is a few very smart people. While publishers are very content sensitive there are ways to search by tagging, and big data scientists accomplish all this with smart algorithms and search programs. Almost on a weekly basis companies are reporting success of searching the blogosphere, while global security folks are catching terrorists and criminals across the planet.

5.      Healthcare and hospitals are under tremendous pressure to cut costs and prevent hospital errors, while optimizing their information systems and patient care. Up until recently most hospitals were driven by EMR, or electronic medical records- systems that converted paper into digital images. However, these customers realized that paper was scalable but not information relevant. A larger problem is that ERM captures only part of the patient care information. The modern goal is to create one big database that encompasses the hospitals, doctors, equipment, medication, patients, medication, etc. Firstly hospitals have to deal with images like CAT, PET, X-Rays, etc. Secondly they have to encompass structured data currently residing in relational databases like accounts payable, charges, patient medication etc. The third category is the unstructured data, which can be as large as 80% of the data includes post-operative data, contra indicative data, radiology reports, external reports, audio reports, etc. Proactive companies, like UPMC, are now providing services to hospitals and Healthcare Corporations. Hospitals themselves have started taking small steps into harmonized data and mobile analytics whereby a nurse can have full access to a patients medication as she walks into a room along with where the nearest equipment is in case of an emergency. “When we look at healthcare we cannot place our data into silos” said one hospital dean, “the advantage of having it all accessible, from X-rays to medication for each patient in real-time is going to be priceless. It will revolutionize the medical industry”.

6.      For Telecom companies their goldmine is in the digital records they store. The cellphone bill tells person related contextual information. Each text sent by a cell phone may contain a wealth of information for disease control, homeland security, public healthcare, crime prevention, welfare potentials and a host of other information that is invaluable to government and corporations worldwide. Mining cellphone data can solve a lot of the world’s social problems and can also empower governments on development tasks. It can lead to social dynamics and even assist in catching terrorists as it was done in the 9/11 bombing. It is an excellent source for collating how people move around and how people think as particles and within a collective ecosystem. When particles oscillate within a group known to have specific properties then their participation or result may become predictable. For example if a tourist oscillates within a infectious group or locality their possibility of contacting the disease becomes predictable.

7.      All this comes to Mobile Analytics and Data Visualization. The iPad has changed the face of personal computing and along with it brought finger-tip analytics for people on the go. iPad is a clear example of a disruptive technology, with its multi-touch capabilities, always connected capabilities and a host of mobile analytical tools that now make getting the right information at your fingertips a reality. Companies like Roambi and SAP’s Mobi allow enterprises to easily create visual reports and analytics and take them to new boundaries of informatics. A visual analytic can now be swiped, flipped, pulled down and across, and with a tap of the finger take the user to the underlying data that rolls up into the visualization. User expectations are getting very high and the iPad has capabilities to meet them all, and in most cases exceed them. It is not uncommon to see executives carrying iPads. They do not use them for movies or games but carry their whole enterprise in real-time mode with them all the time. Executive decisions taken on the golf-course or while on a holiday are becoming more and more routine. Hospital staff having a transparent view of all their patients and all their support equipment routine. Executives and managers are viewing their subordinates in realtime, while musicians can see their license and revenues in real-time too. Behind all this real-time analytics is a host of technologies that can parse big data in exceptional speeds.

Come and enjoy the world where the size of data is no longer a limitation. Where your business needs to find their vision where big data can be harnessed in true real-real time, and where data visualization tools enable them to access this information anywhere, anytime and when they need it. Ten years ago this represented the Promised Land. Today we have the stairway to reach it.

No comments:

Post a Comment