by Rob Bailey, CEO
The last few months have been an amazing journey for us. It was only back in November last year that we launched DataSift. That seems like a lifetime ago when we look back at what we’ve achieved since then…
- with the help and support of Twitter we’ve launched our Historics platform to unlock insights from 2+ years of Tweets
- we’ve on-boarded 20+ new data sources (including YouTube, Blogs, Forums and NewsCred powered News )
- we’ve added around 200 new customers, both entrepreneurs building their socially-intelligent applications to large enterprises that recognize that social is becoming table-stakes for business.
On top of that, as passionate Big Data junkies we held 50 events across the world last week as part of www.bigdataweek.com, including a live Twitter Q&A with Todd Park the CTO of the United States. Huge kudos goes to our whole team in getting us where we are today, but especially our founder, Nick Halstead, who has the vision and passion that keep us all moving.
Maybe our biggest surprise to me is the breadth of use-cases that we’re seeing companies use our Social Data platform for. Social Media Monitoring and “Breaking News” are obvious applications that companies build. But we’re seeing everything from Business Intelligence, Stock-Trading models, Public-health applications and social-TV guides being built with DataSift. A big part of Nick’s vision has always been to democratize the social data market – opening it to both small and large companies. We are starting to see this play-out.
But we’ve still got a lot of work ahead of us. Our combo of Social Data + Big Data + Real-Time has the opportunity to transform the industry. With Lean Start-up as our guide, we’ve tested and seen the demand. Now we need to grow faster to help us move faster in response to this.
That’s why we’re delighted to announce that our existing investors have invested additional $7.2M investment in DataSift to accelerate this growth. And I think that’s the real-story behind our story – having awesome investors that share your vision and are eager to double-down with us on this market. We’re lucky to have Roger Ehrenberg (IA Ventures) and Mark Suster (GRP Partners) as part of our team. They’ve been instrumental in helping us get to where we are today.
So what’s next? We don’t want to spoil the news, but over the next few months you’ll see new services and applications from DataSift as we continue to build out our social-data platform. One of the biggest reasons that people choose DataSift is not just that we can filter and deliver raw social data at massive-scale, but we can structure it into a “ready-to-analyze” format with sentiment, Klout Topics Natural Language Topic analysis, etc. for companies to consume. We’re unique in providing this in the market. What we’re working on next will take these capabilities further to give our customers not just a way to monitor social interactions, but to measure and metric them.
And if you want to help shape the future of social and big-data, we’re hiring!
by Tim Barker, Chief Marketing Officer @ DataSift (@timbarker)
Today we’re proud to announce a partnership with NewsCred – enabling companies for the first time to be able to understand, track and analyze news-articles as they flow through the Social Web.
You can read the coverage, press release and visit our sign-up page and pinterest board for some examples, but I wanted to put this in perspective as a marketeer who’s struggled with challenges of analyzing and understanding this for the last few years, and why this is a big deal for us at DataSift.
In my previous life as the head of marketing for Salesforce.com Europe, I could clearly see the impact that social was having on how customers expect to engage with companies, and how employees want to collaborate using social platforms. As well as creating a vision and products for the market (called the Social Enterprise) the imperative was also that every function, every team in Salesforce would need to align and rally around social.
This is where I first saw the chasm that every company needs to cross to understand and evolve into a social organization; and why platforms that can integrate traditional-media with social-media have become critical to help companies transform into socially-savvy businesses.
Nowhere is this social-transformation more apparent and challenging than in the world of Public Relations and Corporate Communication. If you’re in a company large enough to have its own PR team, you’ll know they are on-point to engage with market-influencers to carry the corporate message out to the market. Of course, in a market where everyone has influence (and a Klout score), identifying who in the market is a social-influencer for your business has become critical. But increasingly the gap has always been – how do you evolve how you measure your impact socially. This is where we need to fuse-together traditional-metrics with social-metrics to get a clearer picture. For me, I think of this as a 3rd generation of how companies can measure their social impact and influence for communications:
1) First there were News Clippings Services: Many companies today will subscribe to a news-clipping service. These are born from the era of “paper and glue”, giving you a summary of the news-output around your company or brand. Useful to know, but not actionable.
2) Then Social Media Monitoring: Next up, PR/Marketing teams were some of the first to embrace Social Media Monitoring solutions to help them understand the social-conversations around their company, products, and competitors. “Share of Conversation” is becoming a benchmark metric for an increasing number of companies, enabling them to see how their brand is perceived.
The problem is, there’s never been a way to correlate the “news clippings output” world with the “social media monitoring” world. Put another way – you can know that a killer article was published about your company, but trying to get an insight into how that news was shared, what the conversation around it was socially, how it impacted your “share of conversation”, or who made this news go-viral – have been near-impossible to measure.
What the combination of NewsCred and DataSift provides is a bridge between the news-production and the social-consumption of news. So now, you can monitor for articles that mention your company, products, or any topic you want to track, then measure the social-distribution and engagement around the news.
In the above example you can see the:
- Social Reach: How many people does the news reach (x-axis)
- Social Resonance: How many people are engaging by retweeting and commenting on the news (y-axis)
- Sentiment of Conversation around the News: How does the sentiment of public conversation change as the news goes-viral. (color)
- Influencer (Klout) Trends: What’s the weight of Klout-influence sharing each article (size)
And best of all, what we provide is a platform to integrate and embed this into your existing social and news applications.
We’re pretty excited about the potential for this. Let us know what you think. And if you’re interested in adding NewsCred content into your DataSift sources, get in touch!
We’re pleased to announce our new partnership with enterprise social intelligence platform company NetBase.
NetBase provides its customers with a comprehensive platform for real-time and long-term historic analysis of social media with the context of its clients’ overall business intelligence, discovering actionable insights. That’s exactly the type of business intelligence that DataSift was created to support.
Through the partnership, NetBase customers and reseller partners—including the SAP Social Media Analytics by NetBase solution and the J.D. Power and Associates Web Intelligence offering —will be able to access and filter from DataSift’s real-time feed of Tweets. NetBase customers will benefit from some of the extremely robust filters that DataSift enables, including location, language, and even influence based on Klout score. DataSift’s technology can also apply filtering to any content that is represented as a link within the post itself, providing companies with an accurate, holistic picture of their business.
“NetBase delivers the exact solutions that make the DataSift platform so powerful,” said DataSift CEO Rob Bailey. “We’re thrilled to work with them to give companies an even more complete picture of their representation online and better understanding of consumer opinions and passions. The most robust information drives the best business decisions, and that’s what NetBase provides with the help of DataSift.”
To learn more about DataSift partner, NetBase visit www.netbase.com and http://www.facebook.com/NetBaseInc. You can follow them on Twitter @NetBase.
Today we are pleased to announce the launch of Twitter Historic Data. This has been a very long term project for us with an immense amount of specialized engineering required to deliver a solution that is both scalable and easy to use. Below is a short video to explain how it works.
Introduction to DataSift Historics
Coverage:

At DataSift we are obsessed with making it easier for companies to access Social data, analyze it, and get value from it. This week we are proud to announce the launch of a new way to access/use the Social Data produced by the DataSift platform. Starting today, DataSift customers will have access to new Record and Export features, that allow even non-technical users to extract their social media data direct through our website into easy to use files like Microsoft Excel.
Creating customized Social Data streams with DataSift is already easy.
Thousands of DataSift customers are already using the DataSift platform to create personalized streams. DataSift created a special filter creation language called CSDL which enables users without a technical background to set up complex filters in a matter of minutes. You can immediately build a stream using CSDL, define the time and date you want to start and stop the recording, then extract it, the whole data set or just the specific time segment or data types that interest you.
Rapid access to Twitter data sets for your research and social media monitoring.
It takes seconds to set up an account with DataSift and there’s no need to invest in a specific application or dashboard for consuming the data. This feature lets anyone start using social media intelligence without any prior investment, training or time delay.
How to turn your data stream into an excel file in 3 easy steps:
1) Set the time that you want the recording to start and end.

2) Once the recording has started you can see the volume of data that you’ve gathered through your stream.

3) When you’ve gathered all the data that you want for this particular recording, then extract it by selecting the format that you would like to receive it in and wait for it to be delivered.

It’s Free
Try out these new features for free with $10 free credit that DataSift gives to every new user and start using social data to inform your strategic decision making.
Get Started Now with a Free Trial.
Find more detailed information about recording and exporting a stream.
Haters Are Going To Hate…But They Aren’t Always Right
Here at DataSift we are big fans of YouTube and really love the new UI they launched a few weeks ago. At the same time that our team was admiring the sleek new site design that You Tube had just launched, we noticed YouTube was taking a hammering from online commentators and press. It seemed like EVERYONE hated the new UI! Were the “haters” right that the UI was terrible, or was a minority of “haters” not representative of the YouTube user base? We turned to Twitter to find out. The results will surprise you.
 |
 |
Our Inital Read: Lots of Complaints!
Our first step was to take a look at the The YouTube blog. We were really surprised at how the boards seemed to be full of complaints and calls for a return to the original layout that users were familiar with. All the while we were left thinking ‘well, actually we kinda like it’.
We thought the complainers might be wrong, and set out to find out what the truth was.
The main rollout of the new YouTube UI went live on the 2nd of December. We used the DataSift platform to quickly set up some streams to capture the real-time public reaction of You Tube users to the new design. (Our platform is so easy to use that someone from our Marketing team actually did most of the work.) The rule we set up for this stream captured any tweet that contained the word “YouTube” AND a word relating to the redesign, such as layout, style, interface, etc. We also added sentiment tagging to see how people generally felt about the redesign. We ran a filter for about 40 hours, and collected over 35,000 tweets from all over the world commenting about the YouTube UI update.
We found the “Haters” were wrong–a lot of people actually liked the design.
While one might have gotten the strong impression from online blogs of widespread public disappointment with the changes, a quick analysis of the tweets that the stream gathered over a 40 hour period told a different story. We found that there was actually a greater expression of positive rather than negative sentiment expressed by Twitter users in reference to the changes. 32.25% of the Tweets captured by our stream expressed positive sentiment, 57.26% of the tweets were neutral and only 29.30% expressed any negative sentiment. 29.30% is still high—but we wondered if there will always be a percentage of users who hate any new change to a UI.
The big surprise – most of the tweeting was being done by influential people!
Although the average Klout score of Twitter users is close to 10, we were fascinated to find out that 53.5% of the individuals tweeting within this stream had a Klout score of 26 or higher. This meant that not only was there a strong expression of positive sentiment but most of the tweets featured in the stream were also being written by highly influential Twitter users. It seems to us that YouTube must definitely be doing something really well if so many highly influential Twitter users were tweeting about the new UI.
What did we learn?
1. Twitter is a great place to get a complete picture of what users think about a new product.
If as YouTube say on their blog, they ‘rely on your feedback to figure out when we’ve gotten it right and when it needs further tweaks’, it’s important that it’s not just the views of a passionate minority of extreme YouTube lovers posting on the company blog, rather than the opinions of the greater YouTube user base, that drive forward future site developments and strategy.
2. The US is important, but so is the UK and Brazil!
The 37,385 tweets that DataSift collected deliver an objective image of the spread of public reaction immediately after launch. A broad data set such as this gives a reliable aggregate view of how people are responding online. This data can be be consumed though our API in real time and funneled through any application (CRM solution, dashboard etc.) or simply recorded and extracted as an excel file. It can also be analyzed by geographical region, gender, degrees of positive and negative sentiment and social influence level, url content as well as the trends within those streams. For instance one insight that we gleamed was that the tweets within this stream came mostly from the US but also the UK, and perhaps surprisingly Brazil.
3. DataSift is a great platform for mining Twitter content.
The DataSift platform is so easy to use were able to set up trackers in just a few minutes that were able to provide very insightful information. With all the noise that comes with the real-time web, extracting the information that will drive productive changes in your company’s communication with it’s customer base requires heavy duty technology. DataSift was architected for enterprises to manage the unstructured data of real-time social media with finesse so they get just the information they need to confidently derive effective strategy from the data.
4. For the daily price of a Starbucks Latte, YouTube could get really interesting insights into their users.
See the stream running live here. This stream costs $4.80 per day to run (not including licensing costs at 0.10 cents for 1000 tweets). Have a product launch coming soon? Set up your own stream on DataSift in less than a minute.

“Those that fail to learn from history, are doomed to repeat it”
- Winston Churchill
Today DataSift is excited to announce the launch of signs-ups for alpha access to Twitter Historical data. You can sign-up and reserve a space on our alpha list and be one of the first to test drive this new functionality when it is available in January. The Alpha Program will give you full access to 60 days of the entire Twitter feed. When the Twitter Historical program goes into a full commercial launch next year, even more historical data will be available.
“We have been flooded with customer interest from lots of different industries in Twitter data. Twitter is not only an incredible source of real time information, it also provides a very rich history of trends, customer interactions, consumer opinions, and other information. We want to make that information available to customers,” said Rob Bailey, CEO of DataSift.
DataSift will not only provide access to historical data, but also offer customers the most powerful platform for deriving value from that data. They can can now apply all the DataSift filtering tools that they currently use to manage real-time content, to extract and analyse streams from our archive of the entire Twitter output. In short, DataSift now makes accessing historic Twitter data rapid and easy.
The applications of historical Twitter data are immense. For example, if a brand manager missed a customer review or thought of a campaign that needs evidence to support it, they can now gain access to the data that they need retrospectively in minutes not weeks and use DataSift’s powerful tools for getting the information they need quickly and cost effectively. This includes filtering the historical data on a given topic by sentiment levels, links, demographics and social influence, as measured by Twitter user Klout scores.
DataSift’s Historic data access provides a huge leap forward for customers. Until now customers wanting historical acccess were limited to very simple keyword searches or other limited filtering methods. The overwhelming feedback from customers is that these search methods were not sufficient.
“We kept hearing from customers how valuable historical Twitter data was, so we wanted to offer that to them,” said Nick Halstead, founder and CTO.
With 250+ million tweets being published every day, extracting meaningful data sets from this immense archive is even more challenging technologically than doing so for real-time data. DataSift’s platform architecture simplifies the historical data processing using our CSDL language to make Big Data simple. Based upon the Cloudera Hadoop stack our solution gives immense scalability to our customers to be able to process data on demand.
UPDATE: coverage from Techcrunch

Start tracking and analyzing Twitter data for your organization in less than 10 minutes!
DataSift, the most sophisticated platform for managing and filtering social data, is live today at 2pm GMT. We invite you to explore this revolutionary new platform that has been designed to make tracking and analyzing Social Media data easy. DataSift offers access to the complete range of Twitter content, but you only pay for the tweets that are actually useful to you that are found using filters that are fully customizable. Getting started is easy-you can start setting up filters very quickly without having any technical knowledge.
We’d like to thank you for supporting DataSift during testing and through the development of our platform during the past few months. Our Beta was a lot more successful than we expected and we are excited for our full commercial release.
During the coming years the engine of growth for DataSift will be the US market. With this in mind, today we are proud to announce the elevation of Rob Bailey as the new CEO of DataSift in preparation for our full US launch later this week. Rob has already been working with DataSift for the past few months as the GM of the US, and in the first few months has already closed revenue-generating customer deals and expanded our relationship with partners like Twitter, Klout and Endeca. Rob has spent more than a decade building experience building organizations in tech, including time at Yahoo! in the Mobile and Broadband group, and at eScene (Acquired by Inktomi) and SimpleGeo (acquired by Urban Airship). Rob has an extensive network of relationships throughout the tech world which will help accelerate DataSift’s growth in the US. DataSift plans to begin hiring aggressively in the US and will be launching an office in San Francisco. Rob also has proven experience working with developers, while at SimpleGeo and as an advisor to numerous start-ups, something that will be critical for DataSift’s growth.
Founder Nick Halstead will assume the Chief Technology Officer which you can read more about on his personal blog. Nick will continue to drive the technology and product vision for the company.

DataSift will be integrating with TrendSpottr to provide customers with the ability to deride early and predictive insights from their real-time curated data streams
TrendSpottr is a real-time data analytics and business intelligence service that identifies and predicts trends from Twitter, Facebook and other real-time data streams. It works by improving the signal-to-noise ratio by identifying the most timely, relevant and trending information from a firehose of streaming content.
Through DataSift, TrendSpottr users will receive trending issues as real-time enriched data with added augmentations (including social influence scoring and geo-location amongst others) that can be analyzed for deep demographic and commercial insight.
“DataSift offers one of the most robust and advanced data curation platforms for social media data sources. By integrating with the DataSift platform, TrendSpottr will be offering DataSift customers access to relevant real-time trends from their curated data streams. This real-time intelligence will provide DataSift customers with early and predictive insights about major trending events, social behaviors, customer preferences and impending crises.” – Mark Zohar, Founder and CEO of TrendSpottr
“DataSift is pleased to integrate TrendSpottr into the DataSift platform to offer our customers real-time predictive coverage of trending issues on 100 percent Twitter feed and other sources…..DataSift and TrendSpottr’s partnership will give trend hunters deep insight into the issues driving specific trends.”
- Nick Halstead, Founder and CEO of DataSift

The full Press Release can be found here.
Page 1 of 712345»...Last »