5. SuperSockets

Please consult the software release note for details.

Heterogeneous Cluster Operation (Endianness)

SuperSockets supports clusters where Cluster Nodes use different endian representation (mixing little endian or big endian). Managing the data in a bi-endian configuration is the responsibility of the application.

Sending and Receiving Vectors

The vector length for the writev() and sendmsg() functions is limited to 16. For readv() and recvmsg(), the vector length is not limited.

Socket Options

The following socket options are supported by SuperSockets for communication over PCI Express:

  • SO_DONTROUTE (implicit, as SuperSockets don't use IP packets for data transport and thus are never routable)

  • TCP_NODELAY

  • SO_REUSEADDR

  • SO_TYPE

The following socket options are passed to the native (fall-back) socket:

  • SO_SENDBUF and SO_RECVBUF (the buffer size for the SuperSockets is fixed).

All other socket options are not supported (ignored).

Fall-forward for Stream Sockets

While SuperSockets offer fully transparent fall-back and fall-forward between PCI Express-based communication and native (Ethernet) communication for any socket (TCP or UDP) while it is open and used, there is currently a limitation on sockets when they connect: a socket that has been created via SuperSockets and connected to a remote socket while the PCI Express interconnect was not operational will not fall forward to PCI Express communication when the interconnect comes up again. Instead, it will continue to communicate via the native network (Ethernet).

This is a rare condition that typically will not affect operation. If you suspect that one Cluster Node performs not up to the expectations, you can either contact support (Section 12, “Support”) to help you diagnose the problem, or restart the application making sure that the PCI Express interconnect is up.

Removal of this limitation as well as a simple way to diagnose the precise state of a SuperSockets-driven socket is scheduled for updated versions of SuperSockets.

Resource Limitations

SuperSockets allocate resources for the communication via PCI Express by means of the IRM. Therefore, the resource limitations listed for the IRM indirectly apply to SuperSockets as well. To resolve such limitations if they occur (i.e. when using very large number of sockets per Cluster Node), please refer to the relevant IRM section above. SuperSockets logs messages to the kernel message log for two typical out-of-resources situations:

  • No more VCs available. The maximum number of virtual channels needs to be increased (see Section 3, “IRM”).

  • No more segment memory available. The amount of pre-allocated memory needs to be increased (see Section 3, “IRM”).

SELinux

SuperSockets don't support systems with SELinux enabled.

RHEL/Centos 7 enables the SELinux security extensions by default. They should be disabled since they cause problems with SuperSockets. To disable SELinux at runtime run the following command:

[root@XD1-CentOS7 ~]# setenforce 0

In order to permanently disable SELinux edit the file /etc/sysconfig/selinux and change the line:

SELINUX=enforcing

to

SELINUX=disabled

Reboot for the changes to take effect.