rdma apps segfaulting

[fwmiller]

I've got rdma-core building successfully into the yocto build now and it configures correctly. However, when I run any of several test apps that ping between the Arria10 board and a PC, I'm getting nasty segfault errors. The issue appears to be with registration of memory regions. Here's an example when I'm running the server side of a udaddy ping pong test:

root@arria10:~# udaddy -s 10.0.1.16
udaddy: starting client
[ 1883.526301] rdma_rxe: null vaddr
udaddy: connecting
failed to reg MR
udaddy: failed to create messages: -1
test complete
Segmentation fault

I'm running kernel 4.17 and the most recent (v18) version of rdma-core

Thanks,
FM

[fwmiller]

An update:

I have a build of rdma-core in kernel 4.17 using yocto for an Altera Arria10 with a dual-core A53 ARM processor. The system is build and rxe configures correctly, i.e. I can rxe_cfg start, rxe_cfg add eth0 and ibv_devices looks good:

root@arria10:~# rxe_cfg status

Name Link Driver Speed NMTU IPv4_addr RDEV RMTU

eth0 yes st_gmac 1500 10.0.1.28 rxe0 1024 (3)

root@arria10:~# ibv_devices

device                 node GUID

------              ----------------

rxe0                085697fffec1059b

root@arria10:~# ibv_devinfo rxe0

hca_id: rxe0

    transport:                      InfiniBand (0)

    fw_ver:                         0.0.0

    node_guid:                      0856:97ff:fec1:059b

    sys_image_guid:                 0000:0000:0000:0000

    vendor_id:                      0x0000

    vendor_part_id:                 0

    hw_ver:                         0x0

    phys_port_cnt:                  1

            port:   1

                    state:                  PORT_ACTIVE (4)

                    max_mtu:                4096 (5)

                    active_mtu:             1024 (3)

                    sm_lid:                 0

                    port_lid:               0

                    port_lmc:               0x00

                    link_layer:             Ethernet

This all looks good. However, when I try to ping this machine against a PC running rdma-core, I'm getting some strange errors including a segfault when the Arria10 acts as server for udaddy.

root@arria10:~# udaddy -s 10.0.1.16

udaddy: starting client

[ 1883.526301] rdma_rxe: null vaddr

udaddy: connecting

failed to reg MR

udaddy: failed to create messages: -1

test complete

Segmentation faultrxe_mem_init_user

I traced the first error, rdma_rxe: null vaddr to rxe_mem_init_user() in /drivers/infiniband/sw/rxe/rxe_mr.c It appears that a page address, perhaps from a virtual to physical translation is failing. Any thoughts on how to solve this?

[kraj]

@fwmiller have you gotten past this error ? if not can you reproduce it on master and see if we can make some progress

[fwmiller]

Yessir. At this time I was trying to put together pieces from disparate places. Once the code bases were unified this problem went away. Thanks!