5 critical factors to turn data into insight

When I first started my career in IT in the mid 90’s I was a Data Analyst. I absolutely loved it. I felt like Sherlock Holmes trying to find the one key ‘clue’ that would explain everything. It was a natural fit for me as I am very inquisitive and love to seek answers. I believe that no matter how great business is, it can always be better if we can just find the answer in the data.

Specifically, as a data analyst I worked with Point of Sales (POS) data from retail giants Wal-Mart and Target.

Wal-Mart was at the forefront of big data long before it even had a name. In the mid 1980’s they decided to build their own Consumer Behavior software that reported on this new technology called Barcodes. This was no easy feat; according to Frontline they started on development in 1985 and invested 4 billion dollars building it over the next seven years. They officially launched “Retail Link” in 1992. It was truly revolutionary. All of a sudden, suppliers had access to all of their inventory and sales data by SKU, by hour, by store.  It was a massive amount of data and it was powerful.

As an analyst, I combined the sales and inventory data with weather patterns, major events, holidays and trends in the news. I also performed market basket analysis (aka Affinity Analysis) to see what products sold well together and when. I analyzed pricing, packaging and shelving strategies to see what trends and what worked and what didn’t.

It’s crazy when I think back to all I accomplished with so much data and ‘primitive’ analytical tools like Excel and SQL. I wonder what my job would look like with today’s tools and techniques?


Back then I didn’t have the tools to sift through so much data and information so I would start with a question and a hypothesis, such as “Will sales increase in years where Easter is earlier in the year?”, and then examine the data to see if it was true or not. It was cumbersome and time-consuming.

Compare that to today’s Data Mining that looks at the data and then presents patterns and outliers.

What used to be impossible for a human is now accomplished by technology. And not just any technology but fast, powerful machines that can learn and adapt.

My POS data fit nicely in CSV files with clear column labels and consistent data, whereas today’s most insightful data is often unstructured and come from many sources such as online blogs and social posts and photos.

While I had to wait until I had actual sales data to make changes to inventory, pricing, and shelving, today’s analysts can make changes proactively based on external leading indicators. For example if a recipe for Raspberry Chipotle Chicken suddenly starts trending on Pinterest the smart grocer will not only place the three key items near each other but will also add a QR code near the shelf with a link to the recipe online.

With online and social data, the amount of data only continues to grow. So does the need for analysts. McKinsey Global Institute predicts that “by 2018, the U.S. alone may face a 50 percent to 60 percent gap between supply and requisite demand of deep analytic talent.”


Data Source:

The data doesn’t lie.


Posted in: Big data, Business Intelligence, Collaboration, Digital, High Performance Analytics      
Comments: 0
Tags: , , , , , , , , , , , ,


We’ve been hearing about machine learning and IOT for quite some time but with all the complexities and high-costs it has simply been out-of-reach for many companies. Salesforce is hoping to change that. Their goal is to provide easy and powerful tools to make IoT possible for every organization.

Earlier this month, I was able to watch the keynote for the Salesforce Developer Conference held in San Francisco, Salesforce excited and motivated developers to start building around their IoT cloud powered by the Thunder event processing engine.

The highlight of the keynote was a demo of what the future will look like for home buying. In the mock presentation the buyer visited to look for a home–pretty typical stuff. Where it got exciting, was when the system looked at the buyer’s Pinterest page to see what styles and types of homes she preferred. In other words, a machine was able to quickly determine her needs based on a third party images and non-structured data. In the past this would require a 20-30 minute conversation to get to the heart of her likes and dislikes.

Speaking of conversations, in the demo, the system actually carried on a text conversation with the home-buyer that looked and felt like talking with a live person.

There were able to do this thanks to the recent acquisition of AI company MetaMind. Metamind technology can not only use natural language to answer text questions and determine the sentiment of the statement but it can also utilize image-recognition and analysis.

They also demo’d how the Thunder powered system could listen to multiple separate events to determine if a property had a lot of interest. If it did, it would automatically schedule an open house and notify the interested parties.

Imagine how this can transform just about everything we do from buying cars to picking travel destinations.

The hard job of machine learning will be that humans are inconsistent and unpredictable. When my brother and his wife were home shopping they said they hated split-level homes and would never, ever, live in one.  You guessed it–they ended up buying a split-level. And they love it.

IoT is changing home showings already. Now a real estate agent can use their Smartphone to unlock doors and lock boxes instead of fumbling around to enter in a code. Both the Zillow system and the home owner could know instantly when someone accesses the house. What an amazing benefit to the seller. I’ve personally spent countless hours killing time because an agent wanted to bring a client by sometime that day. I had no idea when it was safe to return home.

Gartner predicts that more than half of major new business processes and systems will include some element of the IoT by 2020. That’s just 4 years from now! It will be fascinating to see what the future holds.


Posted in: Cloud, Developers, Digital strategy, Internet of Things, IT strategy, project management, Quality Assurance, Research, Social Aspects, Software Development, Testing and innovation      
Comments: 0
Tags: , , , , ,