🚧
This feature is only available in BindPlane OP Enterprise. Learn more here.

Enable Kafka as the Event Bus

Kafka can be used as the event bus for BindPlane OP Enterprise, and is a good option for distributed on prem deployments.

Prerequisites

In order to use Kafka as the event bus we need to ensure a topic exists. The below command can be used to generate the topic just replace <YOUR_KAFKA_SERVER> with a broker address. Auto topic creation can also be use if configured in your Kafka environment.

kafka-topics.sh --create --topic bindplane-op-message-bus --partitions 1 --replication-factor 1 --bootstrap-server <YOUR_KAFKA_SERVER>

NOTE: Ordering is required for BindPlane OP events so the topic must only have 1 partition.

Configuration

In order to use Kafka as the event bus the eventBus.type field must be set to kafka and the eventBus.kafka config must be filled out.

Here is an example of a full configuration using Kafka as the event bus.

name: default
apiVersion: bindplane.observiq.com/v1
auth:
    # A random uuid which is used as a shared secret between bindplane and
    # deployed agents.
    secretKey: your-secret-key

    # Basic auth should use a username other than
    # admin along with a secure password.
    username: admin
    password: password

    # A random uuid which is used for generating web ui session cookies.
    sessionSecret: your-session-secret
network:
    # Listen on port 3001, all interfaces.
    host: 0.0.0.0
    port: "3001"

    # Endpoint for which clients and collectors will interfact
    # with the server's http interface.
    remoteURL: http://bindplane.c.bindplane.internal:3001
store:
    bbolt:
    	path: /var/lib/bindplane/storage/bindplane.db
eventBus:
    type: kafka
    kafka:
      # A list of all brokers that are in the Kafka cluster
    	brokers:
      	- broker1:9092
        - broker2:9092
      authType: none
      # protocolVersion should be specified as MAJOR.MINOR.PATCH and should match your version of Kafka
      protocolVersion: '3.5.0'
logging:
    filePath: /var/log/bindplane/bindplane.log

Configuration Parameters

Parameter Name	Description	Default Value
`brokers`	List of brokers to connect to.
`protocolVersion`	The protocol version the Kafka brokers are using. Should be specified as `MAJOR.MINOR.PATCH`. This usually matches the version of the brokers.
`topic`	The name of the topic to use. See Prerequisites.	`bindplane-op-message-bus`
`authType`	The type of authentication to use. Valid values are `none`, `plainText`, and `sasl`.	`none`
`plainText`	Authentication configuration when `authType` is `plainText`. See PlainText Auth.
`sasl`	Authentication configuration when `authType` is `sasl`. See SASL Auth.
`enableTLS`	Signals whether or not to use TLS.	`false`
`tls`	TLS Configuration when `enableTLS` is `true`. See TLS Configuration.

PlainText Auth

Below are the configuration parameters when using PlainText authentication.

Parameter Name	Description	Required	Default Value
`plainText.username`	The username for authentication.
`plainText.password`	The password for the user.

Here is an example configuration using PlainText authentication:

name: default
apiVersion: bindplane.observiq.com/v1
auth:
    # A random uuid which is used as a shared secret between bindplane and
    # deployed agents.
    secretKey: your-secret-key

    # Basic auth should use a username other than
    # admin along with a secure password.
    username: admin
    password: password

    # A random uuid which is used for generating web ui session cookies.
    sessionSecret: your-session-secret
network:
    # Listen on port 3001, all interfaces.
    host: 0.0.0.0
    port: "3001"

    # Endpoint for which clients and collectors will interfact
    # with the server's http interface.
    remoteURL: http://bindplane.c.bindplane.internal:3001
store:
    bbolt:
    	path: /var/lib/bindplane/storage/bindplane.db
eventBus:
    type: kafka
    kafka:
      # A list of all brokers that are in the Kafka cluster
    	brokers:
      	- broker1:9092
        - broker2:9092
      # protocolVersion should be specified as MAJOR.MINOR.PATCH and should match your version of Kafka
      protocolVersion: '3.5.0'
      # PlainText Authentication Configuration
      authType: plainText
      plainText:
      	username: 'user1'
        password: 'password1234'
logging:
    filePath: /var/log/bindplane/bindplane.log

SASL Authentication

Below are the configuration parameters when using SASL authentication.

Parameter Name	Description	Default Value
`sasl.username`	The username for authentication.
`sasl.password`	The password for the user.
`sasl.mechanism`	The mechanism by which to use SASL authentication. Valid values are `plain`, `scramSHA256`, and `scramSHA512`.	`plain`
`sasl.version`	The version of SASL authentication to use. Valid values are `0` or `1`.	`0`

Here is an example configuration using SASL authentication:

name: default
apiVersion: bindplane.observiq.com/v1
auth:
    # A random uuid which is used as a shared secret between bindplane and
    # deployed agents.
    secretKey: your-secret-key

    # Basic auth should use a username other than
    # admin along with a secure password.
    username: admin
    password: password

    # A random uuid which is used for generating web ui session cookies.
    sessionSecret: your-session-secret
network:
    # Listen on port 3001, all interfaces.
    host: 0.0.0.0
    port: "3001"

    # Endpoint for which clients and collectors will interfact
    # with the server's http interface.
    remoteURL: http://bindplane.c.bindplane.internal:3001
store:
    bbolt:
    	path: /var/lib/bindplane/storage/bindplane.db
eventBus:
    type: kafka
    kafka:
      # A list of all brokers that are in the Kafka cluster
    	brokers:
      	- broker1:9092
        - broker2:9092
      # protocolVersion should be specified as MAJOR.MINOR.PATCH and should match your version of Kafka
      protocolVersion: '3.5.0'
      # SASL Authentication Configuration
      authType: sasl
      sasl:
      	username: 'user1'
        password: 'password1234'
        mechanism: 'scramSHA256'
logging:
    filePath: /var/log/bindplane/bindplane.log

TLS Configuration

Below are the configuration parameters when using TLS.

Parameter Name	Description	Default
`tls.tlsCert`	Path to the TLS certificate file.
`tls.tlsKey`	Path to the TLS key file.
`tls.tlsCa`	Path to the TLS certificate authority file.
`tls.tlsSkipVerify`	If `true` use TLS without verification.	`false`

Here is an example configuration using TLS:

name: default
apiVersion: bindplane.observiq.com/v1
auth:
    # A random uuid which is used as a shared secret between bindplane and
    # deployed agents.
    secretKey: your-secret-key

    # Basic auth should use a username other than
    # admin along with a secure password.
    username: admin
    password: password

    # A random uuid which is used for generating web ui session cookies.
    sessionSecret: your-session-secret
network:
    # Listen on port 3001, all interfaces.
    host: 0.0.0.0
    port: "3001"

    # Endpoint for which clients and collectors will interfact
    # with the server's http interface.
    remoteURL: http://bindplane.c.bindplane.internal:3001
store:
    bbolt:
    	path: /var/lib/bindplane/storage/bindplane.db
eventBus:
    type: kafka
    kafka:
      # A list of all brokers that are in the Kafka cluster
    	brokers:
      	- broker1:9092
        - broker2:9092
      # protocolVersion should be specified as MAJOR.MINOR.PATCH and should match your version of Kafka
      protocolVersion: '3.5.0'
      authType: none
      enableTLS: true
      tls:
      	tlsCert: /opt/kafka.crt
        tlsKey: /opt/kafka.key
logging:
    filePath: /var/log/bindplane/bindplane.log