Comparison of distributed file systems explained

In computing, a distributed file system (DFS) or network file system is any file system that allows access from multiple hosts to files shared via a computer network. This makes it possible for multiple users on multiple machines to share files and storage resources.

Distributed file systems differ in their performance, mutability of content, handling of concurrent writes, handling of permanent or temporary loss of nodes or storage, and their policy of storing content.

Locally managed

Client	Written in	License	Access API	High availability	Shards	Efficient Redundancy	Redundancy Granularity	Initial release year	Memory requirements (GB)
	Alluxio (Virtual Distributed File System)	Java		HDFS, FUSE, HTTP/REST, S3			[1]	[2]	2013
	Ceph	C++		librados (C, C++, Python, Ruby), S3, Swift, FUSE			[3]	[4]	2010	1 per TB of storage
	Coda	C		C				[5]	1987
	GlusterFS	C		libglusterfs, FUSE, NFS, SMB, Swift, libgfapi			[6]	[7]	2005
	HDFS	Java		Java and C client, HTTP, FUSE[8]			[9]	[10]	2005
	IPFS	Go		HTTP gateway, FUSE, Go client, Javascript client, command line tool			[11]	[12]	2015[13]
	JuiceFS	Go		POSIX, FUSE, HDFS, S3					2021
	Kertish-DFS	Go		HTTP(REST), CLI, C# Client, Go Client					2020
	LizardFS	C++		POSIX, FUSE, NFS-Ganesha, Ceph FSAL (via libcephfs)			[14]	[15]	2013
	Lustre	C		POSIX, NFS-Ganesha, NFS, SMB			[16] [17]	[18] [19]	2003
	MinIO	Go		AWS S3 API, FTP, SFTP			[20]	[21]	2014
	MooseFS	C		POSIX, FUSE			[22]	[23]	2008
	OpenAFS	C		Virtual file system, Installable File System				[24]	2000[25]
	OpenIO[26]	C		Native (Python, C, Java), HTTP/REST, S3, Swift, FUSE (POSIX, NFS, SMB, FTP)			[27]	[28]	2015	0.5
	Ori[29]	C, C++		libori, FUSE				[30]	2012
	Quantcast File System	C		C++ client, FUSE (C++ server: MetaServer and ChunkServer are both in C++)			[31]	[32]	2012
	RozoFS	C, Python		FUSE, SMB, NFS, key/value			[33]	[34]	2011[35]
	SeaweedFS	Go, Java		HTTP (REST), POSIX, FUSE, S3, HDFS			[36]	[37]	2015
	Storj	Go		HTTP (REST), S3, Native (Go, C, Python, Java)					2018
	Tahoe-LAFS	Python		HTTP (browser or CLI), SFTP, FTP, FUSE via SSHFS, pyfilesystem			[38]	[39]	2007
	XtreemFS	Java, C++		libxtreemfs (Java, C++), FUSE			[40]	[41]	2009

Client	Written in	License	Access API
	BeeGFS	C / C++	FRAUNHOFER FS (FhGFS) EULA,[42] GPLv2 client	POSIX
	ObjectiveFS[43]	C		POSIX, FUSE
		C, C++		POSIX, NFS, SMB, Swift, S3, HDFS
	MapR-FS	C, C++		POSIX, NFS, FUSE, S3, HDFS, CLI
	PanFS	C, C++		DirectFlow, POSIX, NFS, SMB/CIFS, HTTP, CLI
	Infinit[44]	C++	(to be open sourced)[45]	FUSE, Installable File System, NFS/SMB, POSIX, CLI, SDK (libinfinit)
	Isilon OneFS	C/C++		POSIX, NFS, SMB/CIFS, HDFS, HTTP, FTP, SWIFT Object, CLI, Rest API
	Qumulo	C/C++		POSIX, NFS, SMB/CIFS, CLI, S3, Rest API
	Scality	C		FUSE, NFS, REST, AWS S3
	Quobyte	Java, C++		POSIX, FUSE, NFS, SMB/CIFS, HDFS, AWS S3, TensorFlow Plugin, CLI, Rest API

Remote access

Name	Run by	Access API
	Amazon S3	Amazon.com	HTTP (REST/SOAP)
	Google Cloud Storage	Google	HTTP (REST)
	SWIFT (part of OpenStack)	Rackspace, Hewlett-Packard, others	HTTP (REST)
	Microsoft Azure	Microsoft	HTTP (REST)
	IBM Cloud Object Storage	IBM (formerly Cleversafe)[46]	HTTP (REST)

Comparison

Some researchers have made a functional and experimental analysis of several distributed file systems including HDFS, Ceph, Gluster, Lustre and old (1.6.x) version of MooseFS, although this document is from 2013 and a lot of information are outdated (e.g. MooseFS had no HA for Metadata Server at that time).^[47]

The cloud based remote distributed storage from major vendors have different APIs and different consistency models.^[48]

See also

• Distributed file system
• List of file systems, the Distributed file systems section

References

1. Web site: Caching: Managing Data Replication in Alluxio.
2. Web site: Caching: Managing Data Replication in Alluxio.
3. Web site: Erasure Code Profiles .
4. Web site: Pools.
5. Mahadev . Satyanarayanan . James J. . Kistler . Puneet . Kumar . Maria E. . Okasaki . Ellen H. . Siegel . David C. . Steere . Coda: A Highly Available File System for a Distributed Workstation Environment .
6. Web site: Erasure coding implementation . . 2 November 2021 .
7. Web site: Setting up GlusterFS Volumes.
8. Web site: MountableHDFS.
9. Web site: HDFS-7285 Erasure Coding Support inside HDFS.
10. Web site: Apache Hadoop: setrep.
11. Erasure coding plan: Web site: Reed-Solomon layer over IPFS #196. GitHub., Web site: Erasure Coding Layer #6. GitHub.
12. Web site: CLI Commands: ipfs bitswap wantlist.
13. Web site: Why The Internet Needs IPFS Before It's Too Late. 4 October 2015 .
14. Web site: Configuring Replication Modes.
15. Web site: Configuring Replication Modes: Set and show the goal of a file/directory.
16. Web site: Lustre Operations Manual: What a Lustre File System Is (and What It Isn't).
17. Reed-Solomon in progress: Web site: LU-10911 FLR2: Erasure coding.
18. Web site: Lustre Operations Manual: Lustre Features.
19. File-level redundancy plan: Web site: File Level Redundancy Solution Architecture.
20. Web site: MinIO Erasure Code Quickstart Guide.
21. Web site: MinIO Storage Class Quickstart Guide. GitHub.
22. Only available in the proprietary version 4.x Web site: [feature] erasure-coding #8]. GitHub.
23. Web site: mfsgoal(1).
24. Web site: Replicating Volumes (Creating Read-only Volumes).
25. Web site: OpenAFS.
26. Web site: OpenIO SDS Documentation. docs.openio.io.
27. Web site: Erasure Coding.
28. Web site: Declare Storage Policies.
29. Web site: Ori: A Secure Distributed File System.
30. Ali Jose . Mashtizadeh . Andrea . Bittau . Yifeng Frank . Huang . David . Mazières . Replication, History, and Grafting in the Ori File System .
31. Web site: The Quantcast File System.
32. Web site: qfs/src/cc/tools/cptoqfs_main.cc. GitHub. 8 December 2021.
33. Web site: About RozoFS: Mojette Transform.
34. Web site: Setting up RozoFS: Exportd Configuration File.
35. Web site: Initial commit.. GitHub.
36. Web site: Erasure Coding for warm storage. GitHub.
37. Web site: Replication. GitHub.
38. Web site: zfec -- a fast C implementation of Reed-Solomon erasure coding. GitHub. 24 February 2022.
39. Web site: Tahoe-LAFS Architecture: File Encoding.
40. Web site: Under the Hood: File Replication.
41. Web site: Quickstart: Replicate A File.
42. Web site: FRAUNHOFER FS (FhGFS) END USER LICENSE AGREEMENT. 2012-02-22. Fraunhofer Society.
43. Web site: ObjectiveFS official website.
44. Web site: The Infinit Storage Platform.
45. Web site: Infinit's Open Source Projects. 13 August 2019.
46. Web site: IBM Plans to Acquire Cleversafe for Object Storage in Cloud. https://web.archive.org/web/20151008001155/http://www-03.ibm.com/press/us/en/pressrelease/47776.wss. dead. October 8, 2015. 2015-10-05. www-03.ibm.com. en-US. 2019-05-06.
47. Web site: Séguin. Cyril. Depardon. Benjamin. Le Mahec. Gaël. Analysis of Six Distributed File Systems. HAL.
48. Web site: Data Consistency Models of Public Cloud Storage Services: Amazon S3, Google Cloud Storage and Windows Azure Storage. SysTutorials. 4 February 2014. 19 June 2017.