Native Libs

The native Libs are a mechanims to effectively replace the NIO layer in the rubris library. Most of the idioms are retained, however, the epoll set and selectors are delgated to the Native C code. In practice this is not a huge difference but gives us ~5-10% improvement in memory management for the socket level objects and latency in internal processing.

In addition it allows the library to register multiple acceptors on the same listener to enable a more efficient accept call for new socket connections.

The library is only available for Linux and requires at least Kernel 3.9.0 to run.

Maven dependencies

The maven dependency required for the native library is:

<groupId>rubris</groupId>
<artifactId>rubris-io</artifactId>
<classifier>native</classifier>
<type>jar</type>

Currently only Linux 64-bit under x86 is supported. The jar contains:

  /META-INF/
  /linux-x86-64/librubris-io-native.so

The NativeLoader can load the library from either the LD_LIBRARY_PATH or the classpath. The simplest mechanisnm is to unpack the shared library into the classes directory using Maven. To achieve this in your POM you need to add this to the plugins section.

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-dependency-plugin</artifactId>
  <executions>
    <execution>
      <id>unpack</id>
      <phase>generate-resources</phase>
      <goals>
          <goal>unpack</goal>
      </goals>
      <configuration>
        <artifactItems>
          <artifactItem>
            <groupId>rubris</groupId>
            <artifactId>rubris-io</artifactId>
            <classifier>native</classifier>
            <type>jar</type>
            <overWrite>true</overWrite>
            <outputDirectory>${project.build.directory}/classes</outputDirectory>
             <includes>**/*.so</includes>
             <exclude>META-INF/**</exclude>
          </artifactItem>
        </artifactItems>
      </configuration>
    </execution>
  </executions>
</plugin>

For use with m2eclipse you will need to add the following to the m2eclipse plugin in order to delegate the correct lifecycle for the unpack phase:

<pluginManagement>
    <plugins>
      <plugin>
        <groupId>org.eclipse.m2e</groupId>
        <artifactId>lifecycle-mapping</artifactId>
        <version>1.0.0</version>
        <configuration>
          <lifecycleMappingMetadata>
            ...
               <pluginExecution>
                    <pluginExecutionFilter>
                      <groupId>org.apache.maven.plugins</groupId>
                      <artifactId>maven-dependency-plugin</artifactId>
                      <goals>
                        <goal>copy-dependencies</goal>
                        <goal>unpack</goal>
                      </goals>
                      <versionRange>[0.0,)</versionRange>
                    </pluginExecutionFilter>
                    <action>
                      <execute />
                    </action>
                  </pluginExecution>

            </pluginExecutions>
          </lifecycleMappingMetadata>
        </configuration>
      </plugin>
        ....

Using the above pom entries will enable the shared library to be available as part of the normal build cycle.

Similarly for the deployed application the so library is required to be bundled in the classpath or the LD_LIBRARY_PATH outside the jar.

Native Logging

The logging for the native library is output to stdout for TRACE, DEBUG, INFO and stderr for WARN and ERROR.

The log level at the JVM startup is passed into the C library using the com.rubris.io.linux.NativeLogLevelSetter. The level for the native can be controlled specifically by setting the level for this class. However, in general it is expected that this class will pick up the general level.

For applications running in the IDE all the outputs will end up on the console. For command line applications it will be necessary to redirect stdout and stderr to a file if you do not want to lose them (as log4j2 does not capture these outputs OOTB).

The format for the logging is :

12:39:21.377[thread-2] WARN rubris_jni.c:509:jniSetLogLevel():Setting level to 7

The logger outputs: TIME [thread-id] LEVEL FILE:LINE_NO:METHOD: MESSAGE

If STDOUT or STDERR is redirected to another stream from the defaults then this can be reset by calling:

resetLogLevel(int level, int outFD, int errFD)

The level is an int defined on RubrisCallbackLogger which acts as a mask to define multiple levels. As with Log4j2, setting TRACE enables all other levels, DEBUG all levels except TRACE etc.

  public static int TRACE=31;
  public static int DEBUG=15;
  public static int INFO=7;
  public static int WARN =3;
  public static int ERROR=1;

Note: as the above code takes the File Descriptor of out and err it is possible to use this method to set the logging to be Files by passing in different FD integers. Normally these are the 1 & 2 handles associated with STDOUT and STDERR.

Native behaviours

The native library specifically replaces Socket Accept, Socket Read and Socket Write. All these functions are delegated to the C library using Linux’s epoll support.

While the native functions provide some performance advantage at lower connection levels, in general its advantages are focused around a lower heap memory, lower cpu usage and better scale up behaviour. The library tries to allocate very little memory in its normal operation and pretty much all memory is incurred on the connection itself.

Native multi-accept

One of the main changes that the native library adds is the ability to run multiple accept threads that are coordinated using the OS. This is achieved using SO_RESUSEPORT socket option. The new socket option allows multiple sockets on the same host to bind to the same port, and is intended to improve the performance of multithreaded network server applications running on top of multicore systems.

As a result the native libs (if multi-accept is enabled) will start multiple listener threads (1 for for each module) and the OS will try and ensure a fair-ish split of connections across the listeners without the application needing to provide any locking or coordination of the threads.

As the Listeners only interact with their own module’s reader/writer pair this should provide better isolation of the connections. Using a single listener results in a round-robin allocation of batches of accepts to modules which depending on the accept batch behaviour can cause very asymmetric allocation of connections to one module or another.

Native read-write

In the NIO libraries the file descriptors are continually boxed in and out of the underlying accept HashSet. This means that for high frequency epoll activity the JVM is constantly allocating boxed versions of the file handles and HashMap Node allocations as they are passed to and from the underlying OS.

The native library instead uses an idiom of a DirectByteBuffer (whose size is set by the maximum connections allowed per module) whose memory location is shared by the C and Java code. On an epoll notification the long values of the Socket Id (or the int value of the FD depending on the type of activity)) is written to the DirectByteBuffer (as if it was a C array). On the Java side this is read as an int or long straight from the Buffer using UNSAFE.readLong/readInt and used for all other activity. On a subsequent cycle the same memory is overwritten with the new set of active sockets.

The processing of the read/write is the same for both the Java only and the Native mechanism as far as the application is concerned. The only differences is how the list of socket Ids is obtained and how the memory is read from or written to the underlying socket handle in the C library.