Highlights: DataSift at GigaOm’s Structure: Europe 2012

Zuzanna Pasierbinska-Wilson

We had the pleasure of being a sponsor at GigaOM’s Structure: Europe 2012, a 2-day event that housed the leaders in cloud computing. Throughout the event, we spoke with numerous attendees about their perspective on Big data trends and the cloud SaaS model in Europe.

Here are the highlights we wanted to share with you:

The Demise of the Original Equipment Manufacturer (OEM)

Companies are moving away from the OEM model for real-time demands. Jay Parikh, Facebook’s VP of Infrastructure Engineering told attendees that the company will be using 100 percent non-OEM servers and moving away from traditional hardware OEM vendors, the reasoning behind this was:

“We have to fine-tune performance given the scale and real-time nature of our application,” Parikh said. “We want to push forward aggressively not only on the user experience but all the underlying pieces of infrastructure to support it. We can iterate with the hardware we’re designing and deploying. Most of the time it’s cheaper and consumers less energy. It’s better for the environment and better for us, gives us more flexibility.”

Big data and Cloud Equals Big Opportunity

A key area discussed at the conference was around the use of Big Data and what customers are actually doing with the data itself. The challenge is bringing together siloed data sets in order to analyze and visualize the context of the information. This will ultimately answer the issues raised in many forums – is this a technology challenge or a internal policy issue?

Cloud storage has been around for a long time, but getting large volumes of data in and out of the cloud is tricky and expensive. It is still seen as a challenge to transport data around cloud-based services, due to the transport protocols that are in place. With many new companies like Aspera popping up, this issue will start to resolve – allowing the deployment of new and faster data applications.

Infrastructure was also an arising problem. Reliable and fast data delivery is obviously crucial for business. DataSift has resolved this issue for our clients by building a technology that is able to move terabytes of data across the internet within a given timeframe to a chosen destination. Switch technology has also moved forward over the last three years, DataSift has switched to Arista to accommodate the high volume of packets and network speeds.

Graphs of Data are Not Enough

The promise of valuable data analysis can only be guaranteed when you have the ability to bring different data sets together. General themes become biased when using only 1 or 2 sources of social data. The goal is to include multiple sources to establish a baseline for normalization.

Understanding the consumer on a deeper level with demographics is highly important. Up until a few years ago, whether a customer is male or female, what city they live in, their profession, etc. was done via clipboards, but now we are able to run analytics on multiple data points from multiple public feeds. And we have only skimmed the surface of this area. The ability of understanding your audience on a deeper level and optimizing your message accordingly is one of the areas we will see over the next few years with the biggest growth. The depth of data available now is allowing precision targeting, enabling messages that are highly focused to your audience niches.

Dark Social

Although the amount of conversation and sharing continues to grow via social media, source and referral data can be very difficult or impossible to track. There is still a lot of organic conversation in private mediums that we cannot search or measure – a link to your website can be shared in an email from a colleague, an IM from a friend, or even talked about in a conversation over lunch. This proves difficult for data analyses and makes room for error and bias in your data. The Atlantic published their findings on Dark Social, as well as some pretty interesting statistics from the social analytics company, Chartbeat. An excerpt from their study shows just how important Dark Social is, “Dark social is even more important across this broader set of sites. Almost 69 percent of social referrals were dark! Facebook came in second at 20 percent. Twitter was down at 6 percent.”

Check out the video below of our Founder, Nick Halstead who shares his views on big data trends, drinking from the twitter firehose, and “Dark Social”:

Zuzanna Pasierbinska-Wilson

Written by Zuzanna Pasierbinska-Wilson

Zuzanna is VP, Marketing & Communications at DataSift. You can follow her @fattypontoonski.