Configure channels

This guide shows how to give a device its channels: with --channel-config, with a JSON file, or as properties from code. The result is a topology in which each device binds or connects its sub-channels on the addresses that you select.

Before you start

• You have a device made with runDevice.h, for example from Your first device.
• You know the channel names that your code uses, for example "data" in Send(msg, "data") or in OnData("data", ...). The configuration must give channels with the same names.

Steps

1. Select the socket type of each channel, for example push for the sender and pull for the receiver. Refer to Socket types and patterns and Select the socket pattern.

2. Select the device that binds. Usually this is the device with a fixed place in the topology, for example the sink. Give it the method bind and an address such as tcp://*:5555.

3. Give the peers the method connect and the address of the binding device, for example tcp://host1:5555.

4. Give the configuration to the devices in one of the three ways.

5. Start the devices, and check the result.

Three ways to give the configuration

--channel-config

Give one value for each channel. A value is a list of key=value pairs, separated by ,, without spaces. Put the value in quotes if it contains *, so that the shell does not change it.

fairmq-ex-1-n-1-processor --id processor1 \
  --channel-config 'name=data1,type=pull,method=connect,address=tcp://localhost:5555' \
                   'name=data2,type=push,method=connect,address=tcp://localhost:5556'

• Give all channels as values after one --channel-config, as the examples do.
• name=data1,... and data1:... give the same name (SuboptParser.cxx).
• Each address= key makes one sub-channel. All other keys apply to all sub-channels of the channel, at each position in the list (SuboptParser.cxx).
• The parser logs Ignoring unknown argument in --channel-config for an unknown key.
• Because , separates the keys, an address with several endpoints is not possible here.

JSON file

Write one file for all devices of the topology, with one entry for each device. Give the file with --mq-config. The device reads the entry that matches --config-key, or --id if there is no --config-key. An entry matches by its key, or by its id if it has no key.

fairmq-ex-1-n-1-processor --id processor1 --mq-config ex-1-n-1.json --config-key processor

• The values of a channel are the defaults for its sockets. Each socket object makes one sub-channel.
• Several devices can use one entry with --config-key, for example the two processors of the 1-n-1 example.
• If you give --mq-config and --channel-config, the device reads only the JSON file.

JSON configuration describes the format and the errors. The complete example below shows a file.

Properties from code

Set the properties chans.<name>.<index>.<property> before the controller requests the transition COMPLETE INIT. Use the exact types: std::string, int or bool. Each sub-channel needs a type property, otherwise the device does not see it. For a plugin, set them when the device goes into INITIALIZING DEVICE:

#include <fairmq/Plugin.h>

#include <string>
#include <utility>

struct ChannelSetup : fair::mq::Plugin
{
    template<typename... Args>
    ChannelSetup(Args&&... args)
        : Plugin(std::forward<Args>(args)...)
    {
        SubscribeToDeviceStateChange([this](DeviceState state) {
            if (state == DeviceState::InitializingDevice) {
                SetProperty<std::string>("chans.data.0.type", "push");
                SetProperty<std::string>("chans.data.0.method", "bind");
                SetProperty<std::string>("chans.data.0.address", "tcp://*:5555");
                SetProperty<int>("chans.data.0.rateLogging", 0);
            }
        });
    }

    ~ChannelSetup() override { UnsubscribeFromDeviceStateChange(); }
};

Register and load the plugin as Write a plugin describes. The device runner makes the plugins of -P before the config plugin. If --mq-config or --channel-config also gives the channel data, the config plugin replaces these values.

In the device, use the hook Init(). It runs after the request of COMPLETE INIT, before the device reads the channel properties:

void Init() override
{
    fair::mq::Channel channel("push", "bind", "tcp://*:5555");
    channel.UpdateRateLogging(0);
    AddChannel("data", std::move(channel));
}

A device without runDevice.h can add its channels before it starts, as this test helper does:

test/device/_data_callbacks.cxx

void AddInput(Device& device, string const& name, test::IpcAddress const& address, string const& transport = "")
{
    Channel channel("pull", "bind", address);
    channel.UpdateRateLogging(0);
    if (!transport.empty()) {
        channel.UpdateTransport(transport);
    }
    device.AddChannel(name, std::move(channel));
}

test/device/_data_callbacks.cxx:55–63

Caution

• AddChannel() adds a sub-channel at the next free index, also when the channel exists. After RESET DEVICE, Init() runs again and adds one more sub-channel. If your device can go through INITIALIZING DEVICE again, set the properties with fConfig->SetProperty() instead.
• AddChannel() does not copy sndTimeoutMs and rcvTimeoutMs. Set them as properties.
• Do not set channels in the constructor of a device that uses Device(). With runDevice.h, the device runner replaces the store of the device after the constructor.

Several sub-channels

Method	Several sub-channels
--channel-config	Give one address= key for each sub-channel: name=data,type=push,method=bind,address=tcp://*:5555,address=tcp://*:5556. Or give numSockets=3 for sub-channels without an address.
JSON file	Give one object for each sub-channel in sockets, or give numSockets in the channel.
Properties	Set the properties of the indexes 0, 1, 2 and so on, without gaps.

The copypush example has a channel with two sub-channels. Channel properties describes numSockets.

Addresses on one host

• tcp://localhost:5555 or tcp://127.0.0.1:5555 on the connecting side, and tcp://*:5555 or tcp://127.0.0.1:5555 on the binding side.
• ipc:///tmp/fmq_data gives a Unix domain socket file. The test scripts of the examples use ipc:// addresses.
• inproc://data connects two sub-channels of the same device with the same transport.

The shmem transport uses the same addresses. It sends the metadata of the messages through its ZeroMQ socket on this address, and the data through the shared memory.

Addresses on several hosts

• On the binding host, bind to tcp://*:5555 (all interfaces) or to the IPv4 address of one interface.
• On the connecting hosts, give the host name or the IPv4 address of the binding host. The device uses the first IPv4 address of a host name (Network.cxx).

A binding sub-channel without an address binds to the IPv4 address of the interface that --network-interface names, for example eth0 (Device.cxx). With the default default, the device uses the interface of the default route. On Linux, it reads /proc/net/route and then the output of ip route. On macOS, it uses route -n get default (Network.cxx). If it finds no interface, it binds to tcp://*:1. Without root rights, the bind to port 1 fails, and autobind selects a port. For the functions that find the interface and the addresses, see Network.h.

The peers must then get the address. After the bind, the device writes the address to the property chans.<name>.<index>.address. A plugin can read it and give it to the peers. A connecting device reads its address property again every 50 ms in CONNECTING, until the time of --init-timeout is over.

Automatic ports

By default, a binding sub-channel takes a random port from 22000 to 23000 when its TCP bind fails. For fixed ports, switch this off:

fairmq-ex-1-n-1-sink --id sink1 \
  --channel-config 'name=data2,type=pull,method=bind,address=tcp://*:5556,autoBind=false'

In a JSON file, write "autoBind": false. The keys portRangeMin and portRangeMax change the range. Refer to Channel properties.

A complete example

The 1-n-1 example has one sampler, two processors and one sink:

Device	Channel	Type	Method	Address
sampler1	data1	push	bind	tcp://*:5555
processor1, processor2	data1	pull	connect	tcp://localhost:5555
processor1, processor2	data2	push	connect	tcp://localhost:5556
sink1	data2	pull	bind	tcp://*:5556

The JSON file has one entry for each device. The two processors use the entry with the key processor:

examples/1-n-1/ex-1-n-1.json

{
    "fairMQOptions": {
        "devices": [
            {
                "id": "sampler1",
                "channels": [
                    {
                        "name": "data1",
                        "sockets": [
                            {
                                "type": "push",
                                "method": "bind",
                                "address": "tcp://*:5555",
                                "sndBufSize": 1000,
                                "rcvBufSize": 1000,
                                "rateLogging": 0
                            }
                        ]
                    }
                ]
            },
            {
                "key": "processor",
                "channels": [
                    {
                        "name": "data1",
                        "sockets": [
                            {
                                "type": "pull",
                                "method": "connect",
                                "address": "tcp://localhost:5555",
                                "sndBufSize": 1000,
                                "rcvBufSize": 1000,
                                "rateLogging": 0
                            }
                        ]
                    },
                    {
                        "name": "data2",
                        "sockets": [
                            {
                                "type": "push",
                                "method": "connect",
                                "address": "tcp://localhost:5556",
                                "sndBufSize": 1000,
                                "rcvBufSize": 1000,
                                "rateLogging": 0
                            }
                        ]
                    }
                ]
            },
            {
                "id": "sink1",
                "channels": [
                    {
                        "name": "data2",
                        "sockets": [
                            {
                                "type": "pull",
                                "method": "bind",
                                "address": "tcp://*:5556",
                                "sndBufSize": 1000,
                                "rcvBufSize": 1000,
                                "rateLogging": 0
                            }
                        ]
                    }
                ]
            }
        ]
    }
}

examples/1-n-1/ex-1-n-1.json:1–73

The start script gives the file to all devices, and --config-key processor to the processors. The build writes it to examples/1-n-1/fairmq-start-ex-1-n-1.sh in the build directory, with the paths filled in:

examples/1-n-1/fairmq-start-ex-1-n-1.sh.in

#!/bin/bash

export FAIRMQ_PATH=@FAIRMQ_BIN_DIR@

ex2config="@EX_CONF_DIR@/ex-1-n-1.json"

SAMPLER="fairmq-ex-1-n-1-sampler"
SAMPLER+=" --id sampler1"
SAMPLER+=" --mq-config $ex2config"
xterm -geometry 80x23+0+0 -hold -e @EX_BIN_DIR@/$SAMPLER &

PROCESSOR1="fairmq-ex-1-n-1-processor"
PROCESSOR1+=" --id processor1"
PROCESSOR1+=" --mq-config $ex2config"
PROCESSOR1+=" --config-key processor"
xterm -geometry 80x23+500+0 -hold -e @EX_BIN_DIR@/$PROCESSOR1 &

PROCESSOR2="fairmq-ex-1-n-1-processor"
PROCESSOR2+=" --id processor2"
PROCESSOR2+=" --mq-config $ex2config"
PROCESSOR2+=" --config-key processor"
xterm -geometry 80x23+500+330 -hold -e @EX_BIN_DIR@/$PROCESSOR2 &

SINK="fairmq-ex-1-n-1-sink"
SINK+=" --id sink1"
SINK+=" --mq-config $ex2config"
xterm -geometry 80x23+1000+0 -hold -e @EX_BIN_DIR@/$SINK &

examples/1-n-1/fairmq-start-ex-1-n-1.sh.in:1–27

The same topology with --channel-config, in four terminals in the directory examples/1-n-1 of the build directory:

./fairmq-ex-1-n-1-sampler --id sampler1 \
  --channel-config 'name=data1,type=push,method=bind,address=tcp://*:5555,rateLogging=0'
./fairmq-ex-1-n-1-processor --id processor1 \
  --channel-config 'name=data1,type=pull,method=connect,address=tcp://localhost:5555,rateLogging=0' \
                   'name=data2,type=push,method=connect,address=tcp://localhost:5556,rateLogging=0'
./fairmq-ex-1-n-1-processor --id processor2 \
  --channel-config 'name=data1,type=pull,method=connect,address=tcp://localhost:5555,rateLogging=0' \
                   'name=data2,type=push,method=connect,address=tcp://localhost:5556,rateLogging=0'
./fairmq-ex-1-n-1-sink --id sink1 \
  --channel-config 'name=data2,type=pull,method=bind,address=tcp://*:5556,rateLogging=0'

Check the result

Start the devices with --severity debug. For each endpoint, the device logs the address and the method:

Attached channel data1[0] to tcp://*:5555 (bind) (push)

Attached channel data1[0] to tcp://127.0.0.1:5555 (connect) (pull)

The connecting device shows the resolved IPv4 address of localhost. In the interactive controller (the default --control dynamic), press p. The device logs the number of peers of each sub-channel, for example in sampler1 with two processors:

data1[0]: 2 peers connected

Problems

Message or symptom	Cause	Fix
GetChannel(): 'data[0]' does not exist.	The configuration has no channel with the name that the code uses, or the device did not read the configuration.	Compare the names. Look for parser errors at the start of the log.
fair::mq::plugins::Config: no channels configuration provided via --mq-config or --channel-config	The device has no channel configuration. A misspelled option, for example --chanel-config, has the same effect, because the device ignores unknown options.	Give --channel-config or --mq-config. For a misspelled option, the debug log shows detected unregistered option.
No device ID or config-key provided for the configuration parser.	--id and --config-key are empty.	Give --id.
missing channel name in argument of option --channel-config	A value of --channel-config has no name.	Add name=.
Invalid channel type: 'unspecified'	A sub-channel has no type.	Give type.
Cannot update configuration. Socket method (bind/connect) for channel 'data[0]' not specified.	A sub-channel has no method and no modifier.	Give method=bind or method=connect.
1 of the binding channels could not initialize	A bind failed, for example because the port is in use and autoBind is false.	Free the port, or select another port.
could not connect all channels after 120 attempts	A connecting sub-channel has no valid address, or its host name does not resolve.	Correct the address.
The peers do not find the binding device.	The port was in use, and autobind selected a random port.	Set autoBind=false.

Related pages

• Channels and sockets: sub-channels, socket types, methods and addresses.
• Channel properties: all keys, with their defaults.
• JSON configuration: the format of the file for --mq-config.
• Configuration and properties: how the device reads its configuration.
• 1-n-1 example: the example of this guide.
• Design a topology: the patterns, the binding devices and the transports of a topology.