IB Network Configuration — xCAT 2.16.4 documentation

IB Network Configuration¶

xCAT provides a script configib to help configure the Infiniband adapters on the compute nodes.

The Infiniband adapter is considered an additional interface for xCAT. The process for configuring Infiniband adapters complies with the process of Configure Additional Network Interfaces.

Below are an simple example to configure Mellanox IB in Ubuntu 14.04.1 on Power8 LE

If your target Mellanox IB adapter has 2 ports, and you plan to give port ib0 4 different IPs, 2 are IPV4 (20.0.0.3 and 30.0.0.3) and another 2 are IPV6 (1:2::3 and 2:2::3).

  1. Define your networks in networks table

    chdef

    -

    t

    network

    -

    o

    ib0ipv41

    net

    =

    20.0.0.0

    mask

    =

    255.255.255.0

    mgtifname

    =

    ib0

    chdef

    -

    t

    network

    -

    o

    ib0ipv42

    net

    =

    30.0.0.0

    mask

    =

    255.255.255.0

    mgtifname

    =

    ib0

    chdef

    -

    t

    network

    -

    o

    ib0ipv61

    net

    =

    1

    :

    2

    ::

    /

    64

    mask

    =/

    64

    mgtifname

    =

    ib0

    gateway

    =

    1

    :

    2

    ::

    2

    chdef

    -

    t

    network

    -

    o

    ib0ipv62

    net

    =

    2

    :

    2

    ::

    /

    64

    mask

    =/

    64

    mgtifname

    =

    ib0

    gateway

    =

  2. Define IPs for ib0

    chdef

    <

    node

    >

    nicips

    .

    ib0

    =

    "20.0.0.3|30.0.0.3|1:2::3|2:2::3"

    \

    nicnetworks

    .

    ib0

    =

    "ib0ipv41|ib0ipv42|ib0ipv61|ib0ipv62"

    nictypes

    .

    ib0

    =

    "Infiniband"

  3. Configure ib0

  • To configure during node installation

    chdef

    <

    node

    >

    -

    p

    postscripts

    =

    "confignetwork --ibaports=2"

    nodeset

    <

    node

    >

    osimage

    =<

    osimagename

    >

    rsetboot

    <

    node

    >

    net

    rpower

    <

    node

    >

    reset

  • To configure on a node which has had operating system

    updatenode

    <

    node

    >

    -

    P

    "confignetwork --ibaports=2"