Getting Started With Board Cloud Connector (Hybrid Data Pipeline)

Davide Guardamagna

Abstract

Every Board solution relies on connections to external data sources. In this article, we describe step by step how to link Board to an on-premises data source.

Context

Board Cloud Data Pipeline is a service that allows Board applications to access data from sources that are hosted on a public cloud or on-premises. The architecture is made of two components, the Data Pipeline service and the On-Premises Connector. The Data Pipeline service is hosted on Board Azure infrastructure and is the main component. It exposes a web portal where the user can define connections and rules. The On-Premises Connector - from now on OPC - is installed remotely and allows to securely access data sources that reside behind a firewall. This module connects to the Data Pipeline server via https without requiring firewall configuration changes on the inbound traffic.

Content

3.1 Getting started - Change Data Pipeline user password

From the link on the Cloud Administration portal shown below

open the Data Pipeline portal and entry the user and password displayed under the button.

Upon first login, you will be prompted to change the password.

For security reasons, passwords must meet the following requirements.

At least eight character long

Include a combination of upper and lower-case letters

Include at least a number

Include punctuation marks and/or other special characters

3.2 Provision the On-premise Connector Host

The OPC can be installed either on a physical server or on a virtual machine. Minimum hardware requirements are

• CPU 4-cores
• RAM 8 GB
• Disk 40GB

The following Windows 64-bit operating systems are supported.

• Window 10
• Windows Server 2012 R2
• Windows Server 2016
• Windows Server 2019
• Windows Server 2022

Network Requirement. The hosting machine must have access to the internet and TCP port 443 must be opened to allow outbound traffic.

3.3 Install the On-Premise Connector

1. Download the OPC installer from the link on the Cloud Administration portal onto the file system of the Windows host where you want to install it.

2. Unzip the downloaded file.3. Run as Administrator or double-click on file PROGRESS_DATADIRECT_HDP_OPCONNECTOR_4.6.1.HOTFIX_WIN_64.exe4. The following window appears. Click Next to continue

5. Specify the installation directory in one of the following ways:

Accept the default installation directory and click Next.

Enter the full path of the installation directory and click Next.

Click on Choose... to browse and select an installation directory and click Next

6. Select the type of installation:

For default installation, select Standard Installation and Next

To customize the installation, select Advanced Installation and select one or more of the following options.

The OPC communicates with the Hybrid Data Pipeline service over the internet. If your network requires a proxy to access the public internet, select Proxy Connection and Next (for additional details, read Proxy Connection Configuration section below).

If you need to connect to a MySQL data store, select MySQL Community Edition and provide the name and location of the MySQL Community Edition driver (the .jar file). Click Next and proceed.

When a proxy is used in the customer’s private network, you need to specify the connection details and the type of proxy authentication.

Hostname: the host name and, optionally, the domain of the proxy server. The value can be a host name, a fully qualified domain name, or an IPv4 or IPv6 address.

Port Number: the port number where the proxy server is listening.

User Name: the user name required to connect to the proxy server if you are using HTTP Basic or NTLM authentication. If NTLM Proxy Authentication is selected, the username must be in the form Domain\User

Password: the password of the user that can authenticate to the proxy server if you are using HTTP Basic or NTLM authentication.

From the Proxy Authentication drop-down list, select the type of proxy authentication needed in your environment:

Select No Proxy Authentication if the proxy server does not require authentication.

Select HTTP Proxy Authentication if the proxy server requires HTTP Basic authentication.

Select NTML Proxy Authentication if the proxy server requires NTLM authentication.

These settings can be changed any time after the installation by using the Hybrid Data Pipeline On-Premises Connector Configuration Tool.

Important Note. The Cloud connector does not support Secure Sockets Layer (SSL) inspection using an HTTPS certificate. If you must use your SSL proxy, we strongly recommend that you update the server policy to pass through the Board certificate so that no certificate is altered within the OPC-Datapipeline communication.7. Provide the User ID and Password for your Hybrid Data Pipeline account. If you want, you can change the default Connector Label. Click Next. The installer will validate these credentials.

8. Review the Pre-Installation Summary window. If the summary is correct, click on Install, otherwise click on Previous and review the previous steps.

9. Click on Done to exit the installer when the installation is complete.

10. After the installation is finished, the OPC Configuration tool opens and verifies the access to the Hybrid Data Pipeline service.

To open the configuration tool, search for this executable on the Windows menu, as shown below.

Important Note. For application maintenance reasons, we recommend not to install multiple instances of OPC on the same host. In fact, the same OPC instance can be used by different connections from the Data Pipeline service.

If you are running an outdated version of the OPC that requires upgrading, here are the steps to follow. Make sure you arrange a maintenance window to prevent that any import process is triggered from Board while the upgrade takes place.

1. Download the new OPC installer from the same link mentioned in the installation instructions above.

2. Unzip the installer.
3. “Run as Administrator” or double-click the installation file PROGRESS_DATADIRECT_HDP_OPCONNECTOR_4.6.1.HOTFIX_WIN_64.exe
4. The installation Introduction window appears. Click on Next, agree to the license agreement terms and the following window appears. Select Upgrade Existing Instance and click on Next.

5. Click on Next to upgrade the existing instance.

6. Select Java home directory and click on Next.

7. Select No in the following window and click on Next.

8. On the Pre-Installation summary, click on Install and wait for the installation to complete.

9. In the next window, select No, I will restart my system myself if you prefer to restart the machine in a different moment and click on Done.

10. When installation is complete, click on Done.

3.5 Create the connection on the Data Pipeline portal

When the OPC is installed and configured, you can setup your connections on the Data Pipeline portal.1. On the left-hand navigation panel, click on Data Sources.

2. Click on + New Data Source on the right-hand.

3. From the list of available Data Stores, choose the one you need to connect to.

4. The Create Data Source page opens up. This page will require different connection parameters based on the source chosen. Here is an example of a SQL Server source.

5. On the General tab, enter all the required fields.

6. For any on-premise data source, choose the Connector ID of the OPC previously installed.7. Press Test to test the connection and Save to save it.

Connecting Board Cloud to cloud or public sources

When connecting to any source exposed to the internet, like a cloud SaaS or public REST APIs, the data source administrators or network engineers may need to implement IP restriction rules for inbound traffic. In such case, they will require the IP addresses of Board Data Pipeline clusters. These network rules are not implemented when the OPC is installed.

Here are the Board Cloud Connector IP ranges listed by region.

Australia (datapipeline-au.board.com)	20.40.191.92/30
North America (datapipeline-us.board.com)	52.252.168.80/30
Europe (datapipeline-eu.board.com)	51.145.189.128/30
Middle East (datapipeline-ae.board.com)	20.203.65.168/30
Far East (datapipeline-jp.board.com)	20.27.188.88/30

3.6 Linking a Board environment to a Data Pipeline connection

After setting up a connection on the Data Pipeline portal, you can map it to your Board Cloud instances so it can be used by read and write processes such as data readers, drill-through, ROLAP cubes and bulk insert.

To do so, open the Cloud Administration portal and go to the Data Pipeline section. Here, you can define a connection alias.

Click on ADD NEW and type a connection name. This name will appear among the available ODBCs when creating an SQL data source from Board. The Data Pipeline name must match with the connection name previously created on the Data Pipeline portal.

Important Note. A connection alias must be defined for each Board instance (Development, Test, Production) that needs to connect to that data source.

In the Data Pipeline field type the data source name you have previously created under Data Pipeline web site.
NOTE: an alias must be defined for each Cloud instance.