How to Retrieve a File from Azure Blob Storage with an Azure Function (2023)

By: Koen Verbeeck | Updated: 2023-01-30 | Comments (1) | Related: > Azure Functions

Problem

I need to read a file stored in an Azure Blob Storage container. However, I cannotuse Azure Data Factory (ADF) because I need to do some transformations on the data.Can I achieve my goal with Azure Functions instead?

Solution

There are several different methods for reading files in the Azure cloud. Theeasiest options don't require code: a Copy Data activity in ADF or an AzureLogic App. However, both copy the data to a destination. What if we want to do morecomplex transformations? Another option is dataflows in ADF (both mapping dataflowsor Power Query), but these might be a bit too expensive to process small files.

A solution that involves a bit more code is Azure Functions. Using C# libraries,we can read a file from Azure Blob Storage into memory and then apply the transformationswe want. In this tip, we'll cover a solution that retrieves a file from AzureBlob storage into the memory of the Azure Function.

How to Retrieve a File with an Azure Function

Creating a New Project

Let's start by creating a new project in Visual Studio. This tiputilizes screenshots taken from VS 2022.

Choose the Azure Function project template. In Visual Studio, you can only usethe C# programming language. You can use Visual Studio Code or program directlyin the Azure Portal if you want another language, such as PowerShell.

Give the project a name and choose a location to store it. If you want to addmore Azure Functions later, you might want to give the solution a different name.

In the next screen, you can leave the defaults. Make sure the trigger is HTTP.

Note -You could also use a blob trigger. In this case, theAzure Function will be executed every time a new file is created or changed. Theblob itself will be passed along to the Function, so there's not much effortrequired to read it. However, in this solution, we want more control over when andhow the Function is triggered, hence the choice for the HTTP trigger. If you wantmore info about blob triggers, check out the tipProcess Blob Files Automatically using an Azure Function with Blob Trigger.

A new Azure Function is created. It contains a bunch of template code:

In short, it does the following:

1. The function is triggered by an HTTP trigger. The body and other informationare stored in the HttpRequest object named req.
2. The function logs that it is started the processing.
3. It assumes a property called name is passed along in thebody of the HttpRequest. It tries to read this directly (line 22), then parsesthe body (which is in JSON format) and tries to retrieve name again (lines 24-26).
4. Both results from the previous steps are coalesced together in line 26.
5. A response is created containing the name that was parsed in the previoussteps. The response is returned to the client in line 32. If no name was found,a message is displayed saying a name should be passed.

First, we will add a connection string to the Azure Blob Storage account. Inthe project, there's a file called local.settings.json. Inthere, we can find a key with the name AzureWebJobsStorage. Fill in thename of the Azure Blob Storage account and the account key (which can be found inthe Azure Portal).

If the storage account you want to use differs from the storage account usedby the Azure Function, you can add a new key-value pair to the JSON file.

By adding a few lines of code, we can read this config file into our Azure Function:

This ConfigurationBuilder object actually does a lot more. It will firstcheck if configuration values can be found in the Azure Function App (which willhost the Azure Function when deployed in the cloud). This means we can specify configurationparameters for when the Azure Function is running in its serverless environmentin Azure. In the next step (line 32), the contents of the local.settings.json fileare checked. This is needed to debug our Azure Function locally in Visual Studio.This settings file is not deployed to the Azure Function App! In fact, itshould not even be in source control since it contains secrets. If bothare not found, the configuration builder checks environment variables or build parameters(lines 33-34).

Finally, in line 36, we read the connection string for the Azure Storage accountinto a variable from the config.

Connecting to an Azure Blob Container

Before we can connect to a container, we need to install some additional packagesinto our project. This is done through the Nuget Package Manager,which comes with Visual Studio. Right-click Packages under Dependencies and choose Manage Nuget Packages...

In the package manager, go to Browse and search for azure.storage.blobs.Install the package. It's possible you have to accept one or more licenseagreements.

In the code of the Azure Function, add a reference to our new package:

Let's add some code that will create a connection to a specific blob container,loop over all the items in that container, and write each filename to the log. Thisis done by creating an instance of the classBlobContainerClient, which accepts two arguments for its constructor: a connectionto a storage account – which we retrieved earlier from the config –and the name of the container ("mssqltips" in this use case).

Then we can use the methodGetBlobsByHierarchy, which will return a list ofBlobHierarchyItems. For each item, we get the blob and log its name.

string blobconnection = config["AzureWebJobsStorage"];string container = "mssqltips"; BlobContainerClient containerClient = new BlobContainerClient(blobconnection, container);foreach(BlobHierarchyItem item in containerClient.GetBlobsByHierarchy()){ log.LogInformation(item.Blob.Name);}

When we run the Azure Function in Visual Studio (hit F5), a command line promptwill open where you can retrieve the local URL of the Azure Function:

The Azure Function will be triggered if you open a browser and go to this URL.When it's triggered, you can see all of the names of the different blobs inthe blob container being logged to the console:

Now that we've verified that we can connect successfully to an Azure Storageaccount and blob container, we can try to retrieve a single blob file.

Retrieving a Blob from a Blob Container

The rest of the code looks like this:

First, we define some variables holding the filename of the blob we want to downloadand the name of the folder where this blob is located. We concatenate this togetherinto a full file path.

Then we create an instance of aBlobDownloadResult. Inside a try-catch block, we try to download the file. Nowwe're using the methodsGetBlobClient and the asynchronousDownloadContentAsync to retrieve the blob from the Azure service. The resultof the method DowloadContentAsync is stored in the BlobDownloadResultinstance.

When we debug our Azure Function again (I've put a breakpoint on the lastline), we can see that the function has retrieved the file successfully. However,we cannot inspect its contents directly since it's a binary file.

In the next tip, we'll look at how we can read the contents of an Excelfile inside an Azure Function.

Next Steps

• As an exercise for the reader, try to pass the file name and the folderas parameters through the HTTP request body instead of hardcoding them.
• You can find more Azure tips inthis overview. You can filter on "function" to find all thetips related to Azure Functions.
• You can download the sample codehere.

About the author

Article Last Updated: 2023-01-30