Session

To advance kTLS over DCTCP in datacenter networking [1], we propose Homa-Level Security (HomaLS), a transport-level encryption integrated with the Homa transport protocol. Homa is available as an out-of-tree Linux kernel module [2], which outperforms DCTCP by a large margin; Homa provides 1) receiver-driven congestion control, 2) packet scheduling that prioritizes small requests using multiple in-network queues, 3) one-to-many socket abstraction that preserves message boundaries, and 4.) reliable data transfer. Strawman design for secure communication over Homa would use TLS in the application, but that approach introduces the same challenge as TLS over TCP, that is, to prevent the application from using transparent, opportunistic NIC offloading, which is done by kTLS today. We thus propose HomaLS, transport-level encryption integrated with Homa, where applications read or write plain-text data. HomaLS performs segment-level encryption, because Homa utilizes TSO by overlaying the TCP header including the TCP options space.

In this talk we first present our initial protocol design. Since utilizing hardware offloading is crucial, we test whether hardware TLS offloading, which is far more complicated than TSO, works for a Homa segment, which has a different protocol number than TCP in the IPv4 header. We examined the Nvidia ConnectX-6 DX NIC, and found that it works with small driver modification, indicating the viability of the HomaLS approach. We then present experimental results. If the encryption overhead diminishes the advantage of Homa, HomaLS would be not attractive (over kTLS over TCP). Our prototype implementation that encrypts data in software confirms that HomaLS exhibits shorter RTT than kTLS over TCP by 26–30%, achieving 18–23µs of message RTT.

[1] T. Herbert, “Data center networking stack”, https://legacy.netdevconf.info/1.2/session.html?tom-herbert [2] J. Ousterhout “A Linux Kernel Implementation of the Homa Transport Protocol”, USENIX ATC 2021