Since computers exist, the performance of processors increases faster than the throughput of networks and permanent storage. As a result, the performance bottleneck is the throughput between the processor the other system components. To improve overall cluster performance, the processor can be used to compress the data before sending it to a slower component. In order to apply the compression efficiently to all kind of data intensive applications, compression can be performed in an underlying file system which is widely used in clusters. One of these file systems is Lustre. LZ4 is focused on speed and is currently one of the fastest lossless compression algorithms. The algorithm can compress up to 4.51 GB/s while decompression reaches about 9.14 GB/s. All available LZ4 implementations use continuous buffers. Since Lustre is a Linux kernel module, the memory is accessed via page arrays. To be able to use LZ4 the data has to be copied into a continuous buffer and back later. The aim of this thesis to improve the throughput of the compression by reducing the memory utilization and the number of copying processes. Therefore a modified version of LZ4 is introduced, which works directly on page-based buffers. After the implementation of the LZ4 algorithm with page-based buffers, it became clear that the performance was not sufficient. Therefore, a new algorithm called BeWalgo is designed that doubles the compression throughput when page arrays are used as buffers. The drawback of the BeWalgo algorithm is that the compression ratio may be lower in comparison to LZ4.

@misc{PCITLKW17,
	author	 = {Benjamin Warnke},
	title	 = {{Page-Based Compression in the Linux Kernel}},
	advisors	 = {Michael Kuhn and Anna Fuchs},
	year	 = {2017},
	month	 = {08},
	school	 = {Universität Hamburg},
	howpublished	 = {{Online \url{https://wr.informatik.uni-hamburg.de/_media/research:theses:benjamin_warnke_page_based_compression_in_the_linux_kernel.pdf}}},
	type	 = {Bachelor's Thesis},
	abstract	 = {Since computers exist, the performance of processors increases faster than the throughput of networks and permanent storage. As a result, the performance bottleneck is the throughput between the processor the other system components. To improve overall cluster performance, the processor can be used to compress the data before sending it to a slower component. In order to apply the compression efficiently to all kind of data intensive applications, compression can be performed in an underlying file system which is widely used in clusters. One of these file systems is Lustre. LZ4 is focused on speed and is currently one of the fastest lossless compression algorithms. The algorithm can compress up to 4.51 GB/s while decompression reaches about 9.14 GB/s. All available LZ4 implementations use continuous buffers. Since Lustre is a Linux kernel module, the memory is accessed via page arrays. To be able to use LZ4 the data has to be copied into a continuous buffer and back later. The aim of this thesis to improve the throughput of the compression by reducing the memory utilization and the number of copying processes. Therefore a modified version of LZ4 is introduced, which works directly on page-based buffers. After the implementation of the LZ4 algorithm with page-based buffers, it became clear that the performance was not sufficient. Therefore, a new algorithm called BeWalgo is designed that doubles the compression throughput when page arrays are used as buffers. The drawback of the BeWalgo algorithm is that the compression ratio may be lower in comparison to LZ4.},
}