Today I would like to intoduce my side project, AzureFS. It is an open source command-line tool to mount Windows Azure Blob Storage (an AWS S3 alternative) on your local filesystem and play with it just like you are doing with your local files. It is implemented in Python and it is a FUSE wrapper that works on UNIX environments.
Sometimes we need to list files under a container, transfer a bunch of local files to the cloud, remove files matching a specific name pattern, rename files on the cloud, move files accross containers etc. There are not any file explorer tools for Azure Storage on Linux that I know of, so I developed this.
AzureFS allows you to do such everyday task practically with commands like ls, mkdir, rm, cp, mv etc.
Tutorial: Store database backups on the cloud
Installation of the environment is described on GitHub project page. It is up to you. Now let’s create a storage account from azure.com management portal. (You don’t need to sign up, of course, I’ll show you the demo here). Then obtain your primary access key: Create a directory with mkdir then we’ll mount our cloud storage onto it. (I’ll use my test account named “azurefs” here):
Don’t shut down this process, jump to some other tab, navigate into cloud directory and let’s create a directory named databases:
Then I’ll simply copy my database backups with cp command:
That simple! I have just backed up my data to the cloud. Let’s say I’d like to remove some of them as the newer backups come in:
Voila! We successfully removed these files. Don’t forget that it is a wrapper for your local filesystem so you can execute commands like:
rm *.sql (delete all .sql files) cat 12.sql >> 13.sql (append one file to another) grep (search in files, may actually take some time)
In fact, last summer I have started this project with C++ however it turned out that it would be troublesome and overkill for such a project. Basically what is does is to implement methods of FUSE (filesystem of userspace) interface and making these methods communicate Windows Azure through its REST API. For that reason, I choose fuse.py project which provides a neat Python bindings for FUSE with ctypes. They have done a brilliant job with that project and it is enjoyable to write a FUSE wrapper on fuse.py with the joy of Python language. For Azure communication, it uses official Python SDK for Windows Azure published by Microsoft.
My objective was not to provide an API-like something that your software uses to communicate Windows Azure Storage. This should be done directly though Azure REST API. However, imagine there’s a case where you would like to upload your daily log files to the cloud with a cronjob, then you can mount AzureFS upon startup and let your cronjob just copy the files to the cloud very easily.
This project has became a handy tool to explore and intervene files at Azure Blob Storage. It has some certain limitations listed at README file on GitHub. At this point, I published code under Apache 2 license and all your contributions are welcomed.
It works quite good on my GUI-less Ubuntu Server installation, however it may cause freezes on GUI environments. Also I didn’t test it on OS X, probably it won’t work. So give it a try if you’re on Linux (and have an Azure account!) and let me know about your impressions.