Implementing functions and more fun in Spark Connect using gRPC and .NET

All Spark Connect Posts
Code
Goal of this post The goal of this post is to look at creating a SparkSession and a DataFrame that will wrap the Range relation and...

Moving towards the DataFrame API using the Spark Connect gRPC API in .NET

All Spark Connect Posts
Code
Goal of this post So there are two goals of this post, the first is to take a look at Apache Arrow and how we can...

Exploring the Spark Connect gRPC API more

All Spark Connect Posts
Code
Goal of this post In this post we will continue looking at the gRPC API and the AnalyzePlan method which takes a plan and analyzes it....

Exploring the Spark Connect gRPC API

All Spark Connect Posts
Code
Goal of this post In the first two posts, we looked at how to run some Spark code, firstly against a local Spark Connect server and...

7 sept, schedlued article

Shared Work Spaces: Hell, Heaven, Both?

I've just finished working in my fourth shared work space. I am not a fan. The endless hallways with all these little glassed in rooms where I can see everyone, and everyone can see me, are not my favorite places. I can see white boards with content that maybe I shouldn't be seeing. There are […]

Understanding Shared Access Signature and Access Policy in Azure Storage

This article covers how you can give access with a shared key to Azure Storage Accounts and apply a policy for how users can access the files.

Dealing with Transaction Log Growth

Introduction Relational databases are designed to track changes introduced to a database by data modification language (DML) commands. The fundamental reason for this construct is to ensure that changes are durable and that they can be rolled back reliably. The typical DML command used in SQL are INSERT, UPDATE and DELETE. When INSERT introduces new […]

Will Hosted Services Take Your Job?

This week I was honored to be able to attend, and present a session at, SQLDay in Wroclaw Poland. I fell in love with Poland the very first time I attended this event, so I look forward to any time I can go again. This year, Pavel Potasinski presented the keynote: The Evolution of the […]

Standards Matter

A short story about why standards matter and where they can help you with limiting the off hour calls from others in your organization.