Showing results for 
Search instead for 
Did you mean: 

SparkStreaming and Twitter

Hi everyone

We would like to create a Spark Streaming Application using the Twitter Search API with .NET for Spark. Because we don’t know much about Spark and its Streaming we were following examples and tutorials, but unfortunately we couldn’t find any examples showing how to use a data source other than localhost or a local file for Streaming. We only found tutorials in Scala/Java showing that there is a class TwitterUtils, that can be used to create Streams with the Twitter API. We were not able to find something similar in .NET for Spark.

We also tried getting a Stream with help of some library (Tweetinvi) but can’t use that as source for the Spark.Sql.DataFrame

Is there something similar to the TwitterUtils class in .NET for Spark? Or if not, how could we achieve Spark Streaming with the API?