spark

.NET for Apache Spark - Write and Read data from Azure Data Lake Storage Gen1

August 22, 2020

azure data lake storage, csharp, dotnet, fsharp, microsoft azure, spark

For the second article in the .NET for Apache Spark series, we’ll be looking into how to interact with Azure Data Lake Storage Gen1. We’ll be creating a DataFrame, writing the DataFrame into a Data Lake and retrieving the written data back into our application. Prerequisites # To get started: You’ll need an Azure Data Lake Storage Gen 1 account. Download and install .NET Core 3. ...

.NET for Apache Spark - Write and Read data from Azure Blob Storage

August 6, 2020

.NET, Development, Microsoft Azure

csharp, dotnet, fsharp, microsoft azure, spark, azure blob storage

In this article, we’ll look into how to create a DataFrame, write the DataFrame into Azure Blob Storage, and read the written data back into our application. Prerequisites # To get started: You’ll need Azure Storage account. Complete .NET for Apache Spark Tutorial - Get started in 10 minutes. This guide will show you how to download and install .NET Core 3.1 SDK, JDK 8 and Apache Spark 2. ...