Developed Technology to Accelerate String Tokenization at Speeds Up to 1000% Faster Than CPU Use

MIRACLE LINUX CORPORATION conducts a wide range of technical research and development related to open-source software. As a part of that, the company made an announcement today to herald research and development results that indicate the acceleration of string tokenization using field programmable gate arrays (FPGAs).

Within the research and development effort, MIRACLE LINUX has developed parallel algorithms that work within an FPGA and a framework for efficiently transferring data between the host computer and the FPGA via PCI Express interface, to increase the speed of string tokenization, one of the most basic features of text processing. Increases in speed were successfully achieved at up to ten times the speed recorded for CPU use alone, as a result of benchmarking with these two new features that assumed the batch processing of large amounts of text logs.

Background for the Current Research and Development
Processing a flood of data from both devices and gateways for the Internet of Things (IoT) has become an issue as data volumes increase and the practical uses for the IoT rapidly expand. MIRACLE LINUX has been focusing on the acceleration of string tokenization, using FPGAs in IoT gateways and data center servers. This effort was based in MIRACLE LINUX’s technical prowess in developing and supporting operating systems, given our knowledge of the hardware environment.

It is anticipated that FPGAs will have a wide variety of applications within enterprise systems. Altera, a leading company in the FPGA field, has become an Intel subsidiary, and Intel has announced plans to offer Altera’s FPGA products with Intel Xeon processors as highly customized, integrated products. There remain significant barriers for ordinary users to adopt FPGA use: FPGA circuit designs not only require knowledge of digital circuits and a broad, deep knowledge of things like system buses and operating systems, but also considerable modification to the architecture would be required so that existing applications can utilize FPGA.

MIRACLE LINUX conceived a solution for these issues through offering an operating system to transparently use FPGAs from basic libraries, like the GNU C Library.

Benchmarking Overview
Off-loading1 the Linux OS core libraries onto FPGAs enables applications to transparently use FPGAs. MIRACLE LINUX performed benchmark testing, assuming the batch processing of large amounts of text logs and the running of functionality equivalent to that of the strtok function 2 in the C programming language, using an FPGA.

Processing speeds for the CPU (Intel Core i7-4790 3.6GHz) increased up to approximately 1000% as a result of off-loading string tokenization to FPGAs. Processing times remained nearly fixed for data volumes up to several kilobytes due to a transfer overhead, and were longer than those for the CPU. When the data volume increased beyond one megabyte, however, processing speeds were approximately ten times faster when compared to CPU processing.

These results indicate that the processing of large amounts of data, including string parsing with Hadoop for example, will be effective when using this technology developed by MIRACLE LINUX. This technology speeds up the processing itself as well as the overall system, owing to the decrease of CPU loads.

This technology, using FPGAs transparently inside the operating system and keeping library APIs compatible, allows applications to be used as is.

Future Plans
MIRACLE LINUX is going to study the use of FPGAs for string processing functionality that many programming languages and libraries support, including tasks like converting characters into numeric values, converting between cases, and partial searching, for further efficient processing in combined with the string tokenization feature in an FPGA. Future plans include using FPGAs for other functions such as encryption, and compression and expansion.

MIRACLE LINUX will advance the use of FPGAs in the enterprise field by accelerating Linux applications without modification, to support the needs of processing capacity as IoT devices proliferate and the integration of FPGAs into Intel processors.

More information on this research and development effort can be found here.

Source: MIRACLE LINUX CORPORATION
with Business Wire