Introduction

A connector is a simplified layer above one or a series of channels. It is like a (or multiple) predefined channel(s) with only a few holes to fill. The goal of a connector is to simplify and speed up the integration process by avoiding the end-user to build the integration from scratch. Once the connector has been set up and installed, it will automatically create all the related channels needed for the integration with the parameters exposed and filled by the user.

The connector will be as generic as possible to allow onboarding multiple customers.

One of the most used examples of such a connector is PDF2XML where the customer has only to upload and extract some fields from its PDF invoice to a set of pre-defined fields to integrate its invoice.

The connector definition can be built by any team and changes on the connector definition itself (e.g. change in the transformation) can be released any time (no dependency on Babelway team).

Glossary

This document will use extensively these two concepts:

Connector definition: The definition of the connector which is in the form of a ZIP file containing all the channels + the steps and parameters that the user will go through when he uses the connector.

Typically, the user responsible to build the connector definition is a user with a very good knowledge of Babelway and the overall business which allows him to find a common integration flow (i.e. similar channels in Babelway) and define/build a connector that will be used by many buyers or sellers.

Connector: Instance of one connector definition.

Connector will be used by buyers/sellers willing to integrate to Tradeshift, with a very poor knowledge of Babelway but thanks to the concept of connector, they will be able to fill the few parameters needed for the integration, without seeing/understanding specific concept of Babelway (channels, transformation, …)

Walkthrough - Example

Connector definition can be quite complex, but let’s start with a simple example to get an introduction of all the steps needed to build a connector definition:

1) Create the channel(s) inside Babelway

2) Export the channels - this produced in ZIP file

3) Customise the connector definition

1. Choose which parameters you want to expose to the end-user
2. Customize all the steps that the user will go through

4) Upload the ZIP file in Babelway

5) Create & configure the connector

Create channel(s) & export channel(s)

The first step to build a connector is actually to build the integration flow in one or multiple channels in Babelway. Once the integration is complete, channel(s) will be exported in a ZIP file.

The connector built in this document is based on the Babelway demo channel, which is getting an XML by email, transforming it into a CSV and sending it back by email. The customer is sending one XML file by email, that will be transformed into a CSV and sent back to another email address.

Once the channel(s) is (have) been configured in Babelway, they have to be exported in a ZIP file. There are two options to export the ZIP file:

Single channel: Export only one channel and its all related items (other connected channels, lookup tables, ..)
Related channels: Export the current channel, all the related linked channels (connected via internal gateways), lookup tables, notifications, partners, documents and routing.

The channel(s) is (are) then exported in a ZIP file containing the following structure:

Definition.xml

The definition.xml file defines which parameters are exposed to the user, and defines all the steps that the user will go through to configure and deploy the connector.

The first thing to configure is to give a name to the connector:

<name>My First Connector</name>
<description>This connector allows you to transform an XML to a CSV.</description>

In this connector, we would like to configure two steps for the user:

One step called “Inbound” to configure the email address that will receive the XML
One step called “Outbound” to configure the email address where the resulting CSV will be sent to.

The definition.xml file included in the ZIP file is just an example (a skeleton) to avoid starting from scratch.

In our example we will configure 2 parameters (EMAIL IN, EMAIL OUT).

<parameters>
        <parameter>
            <name>InEmailAddress</name>
            <type><class>SmtpUserName</class></type>
            <label>IN Email Address</label>
            <tooltip>Email that will receive the XML file to be transformed.</tooltip>
        </parameter>
        <parameter>
            <name>OutEmailAddress</name>
            <type><class>EmailAddress</class></type>
            <label>OUT Email Address</label>
            <tooltip>Email address on which the CSV will be sent to</tooltip>
        </parameter>
</parameters>

NB: All the XML tags will be fully explained in details in the section about the structure of definitions.xml

And we will define two steps that the user will go through to configure these two parameters. Of course, multiple parameters can be defined at each step, or the two parameters could be configured in one unique step in this example as well.

<steps>
        <step>
            <name>Inbound</name>
            <parameter>InEmailAddress</parameter>
            <label>Inbound parameter configuration</label>
            <introduction>This step allows you to define on which email address the XML is 
                          received
            </introduction>
        </step>
        <step>
            <name>Outbound</name>
            <parameter>OutEmailAddress</parameter>
            <label>Outbound parameter configuration</label>
            <introduction>This step allows you to define on which email address the CSV will
                          be sent to
            </introduction>
        </step>
</steps>

Once the parameters have been defined, they need to be referenced in the channels.xml file.

Channels.xml

The channels.xml file contains the XML representation of the exported channel(s) with their parameters.

By default, this file is also static but this will not be very useful to create static channels where all parameters are hardcoded. We want to use the parameters defined at the previous step in the definition.xml file.

To be able to use this parameter, the file has to become dynamic and Babelway uses JSP for the templating language (more information can be found in the section JSP templating language). There are two steps to JSPify the file:

Add the JSP header
Rename channels.xml to channels.xml.jsp

Let’s reference the parameters that have been defined in the channels.xml.jsp file (note the renaming here so Babelway can detect which file is using JSP).

 <GatewayInfo>
      <Gateway>43134</Gateway>
      <Type>GATEWAY_IN_SMTP_IN</Type>
      <CreationMoment>2021-06-09T16:04:33.632+02:00</CreationMoment>
      <LastUpdateMoment>2021-06-09T16:04:33.648+02:00</LastUpdateMoment>
      <Name>Smtp in tutorial560595</Name>
      <Hub>101</Hub>
      <Parameters type="UnmodifiableMap">
        [...]
        <username type="SmtpUserName">${InEmailAddress}</username>
      </Parameters>
       [...]
 </GatewayInfo>
 <GatewayInfo>
      <Gateway>43135</Gateway>
      <Type>GATEWAY_OUT_SMTP_OUT</Type>
      <CreationMoment>2021-06-09T16:04:33.655+02:00</CreationMoment>
      <LastUpdateMoment>2021-06-09T16:04:33.698+02:00</LastUpdateMoment>
      <Name>Smtp out</Name>
      <Hub>101</Hub>
      <Parameters type="UnmodifiableMap">
        [...]
        <toRecipients type="String">${OutEmailAddress}</toRecipients>
        [...]
      </Parameters>
 </GatewayInfo>

Import the connector definition

Connector is now complete and files can be zipped and uploaded to Babelway (Manage & Build → Connector → Manage Connector Definition → Create Connector Definition).

Once the connector has been uploaded, you can now instantiate & configure a new connector.

NB: The visibility parameter will be discussed later in this document.

Configure and deploy connector

Go to the page “Manage & Build → Connector → View Catalog” and the list of available connectors will be displayed.

The last step is to go through the two steps and deploy the connector. The deployment of the connectors will trigger the creation & deployment of the underlying channels defined by the connector definition.

All underlying channels, message definitions, transformations and gateways created by a connector are listed in the Related Items tab, but are in a read-only mode. If an update is needed on one of these elements, the update must be done through the connector definition by re-uploading the ZIP with the changes.

There is a way to “unlink” the channels created by the connector to have the possibility to modify them. This is discussed in the section “Delete and keep channels”.

Connector definition

Structure of definition.xml file

The definition.xml file defines which parameters are exposed to the user, and defines all the steps that the user will go through to configure and deploy the connector. The file is static in the sense that we cannot make it dynamic with JSP and introduce conditional parameters or steps.

Structure

The structure of the definition.xml file consists of the following sections:

Name / Description of the connector.
Definition of parameters (that will be later used in the channels.xml.jsp file or in any other files)
Steps that the user will go through to configure the connector.

Example

Support of languages

As shown in the example above, the definition.xml file supports multiple languages to allow the user to go through the configuration steps & parameters in the same language as the one configured in the Babelway platform.

XML tags that support the “lang” attribute (by default, if omitted, english will be used):

<name lang=”[fr|de|en|nl]”>
<description lang=”[fr|de|en|nl]”>
<label lang=”[fr|de|en|nl]”>
<tooltip lang=”[fr|de|en|nl]”>
<introduction lang=”[fr|de|en|nl]”>

Parameters

Parameters have the following mandatory structure:

<parameters>
     <parameter>
         <name>InEmailAddress</name>
         <type><class>SmtpUserName</class></type>
         <label>IN Email Address</label>
         <tooltip>Email address on which CSV file are received</tooltip>
     </parameter>
     [...]
</parameters>

The name must be unique and must be referenced in one of the steps of the connector. If not, an error message will be displayed when the zip file is uploaded. It’s a technical name and is never displayed to the end-user when he will configure his connector.

The type of the parameter must be the exact same type as the parameter defined in the channels.xml file. In the example above, the parameter type is SmtpUserName, which is the exact same type as the parameter of the gateway that we would like to replace.

<GatewayInfo>
   <Gateway>43134</Gateway>
   <Type>GATEWAY_IN_SMTP_IN</Type>
   [...]
   <Parameters type="UnmodifiableMap">
      [...]
      <username type="SmtpUserName">${InEmailAddress}</username>
      [...]
   </Parameters>
</GatewayInfo>

The label and tooltip will be displayed to the end-user when he configures his connector. The label must tell what the parameter is, and the tooltip can contain some additional information that the end-user will see when he mouse-over the parameter. They both support the “lang” attribute to display them in the same language as the end-user.

Example:

Mandatory/Optional

By default, all parameters defined in the definition.xml file are mandatory, meaning that if the end-user does not enter any value for them, the connector will not be deployable.

The XML tag “mandatory” allows to specify this:

<parameter>
    <name>InEmailAddress</name>
    <type><class>SmtpUserName</class></type>
    <label>IN Email Address</label>
    <tooltip>Email address on which CSV file are received</tooltip>
    <mandatory>true|false</mandatory>
</parameter>

AllowedValues/StringAllowedValues

In case you want to suggest a list of possible values to the user, the StringAllowedValues/AllowedValues tag can be used to pre-defined the list which will be displayed in a dropdown to the end-user using the connector.

Some examples:

<parameter>
  <name>MyEmailAddress</name>
  <type>
      <class>EmailAddress</class>
      <AllowedValues type="EmailAddress[]">
      <AllowedValue>test@babelway.com</AllowedValue>
      <AllowedValue>info@babelway.com</AllowedValue>
      </AllowedValues>
  </type>
  <label>Email address</label>
  <tooltip>List of possible email addresses where to send the message</tooltip>
</parameter>
<parameter>
   <name>GatewayOutType</name>
   <type>
       <class>String</class>
       <StringAllowedValues type="String[]">
       <StringAllowedValue>CloudScan</StringAllowedValue>
       <StringAllowedValue>ScanIO</StringAllowedValue>
       </StringAllowedValues>
   </type>
   <label>Tradeshift Connection Type</label>
   <tooltip>CloudScan or ScanIO</tooltip>
</parameter>

Default value

You can also define a default-value that will be displayed when the user instantiates the connector. This is done with the default-value XML tag as follow:

<parameter>
    <name>username</name>
    <type>
          <class>String</class>
    </type>
    <label>Username</label>
    <tooltip>The username to connect to SFTP</tooltip>
    <default-value>christian</default-value>
</parameter>

For parameters of type Array ([]), the structure is a bit different. Below you can find a complex example on how to define a list of supported PEPPOL documents with some default value.
You can also notice some other parameters that can be set on Array like MinSize, MaxSize, AllowNullElements

<parameter>
  <name>P6</name>
  <type>
   <class>PeppolDocument[]</class>
   <MinSize>1</MinSize>
   <MaxSize>2</MaxSize>
   <AllowNullElements>false</AllowNullElements>
   <ElementsValueDescription>
      <ValueClass>PeppolDocument</ValueClass>
      <AllowedValues type="PeppolDocument[]">
          <AllowedValue>PEPPOL_INVOICE_P5_2_C10_2_V2_1</AllowedValue>
          <AllowedValue>PEPPOL_INVOICE_04_T10</AllowedValue>
      </AllowedValues>
   </ElementsValueDescription>
  </type>
  <label>P6</label>
  <tooltip>P6T</tooltip>
  <default-value>
   <Array type="PeppolDocument[]">
    <Element>PEPPOL_INVOICE_P5_2_C10_2_V2_1</Element>
   </Array>
  </default-value>
</parameter>

Structure of channel.xml file

JSP templating language

To be able to use the parameter defined in the definition.xml file in the channel.xml file, we need to use a templating language. For the moment, Babelway only supports the JSP templating language. There are two steps to migrate the channels.xml file into a JSP file:

Change the extension of the file to JSP - rename channels.xml to channels.xml.jsp
Add the following JSP headers at the beginning of the file

<%@ page contentType="text/xml; charset=UTF-8" %>
<%@ taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core" %>
<%@ taglib prefix="fn" uri="http://java.sun.com/jsp/jstl/functions" %>
<%@ taglib prefix="display" uri="/babelway/display" %>

That’s it, you can now reference the parameter defined in the definition.xml in your channels.xml.jsp file with the placeholder ${parameterName}. Note that you can use the whole power of JSP (<c:if>, <c:choose>, ...) to do additional complex logic in your channels.xml.jsp.

PDF template parameter

Steps:

As soon as you export a channel with a PDF template parameter, the export will generate a parameter of type PdfTemplate[] in the definition.xml file. The structure will be as follows.

Note that the validations and the description in the node of the Message Definition tree are exported automatically as well, so you can configure everything in the Message Definition tree before the export.

<parameter>
       <name>PdfTemplate</name>
       <type>
          <class>PdfTemplate[]</class>
          <ForceFieldsToBeExtracted>
              <Element type="PdfMessageDefinitionSimpleFieldDescription"> [...]</Element>
              <Element type="PdfMessageDefinitionSimpleFieldDescription"> [...]</Element>
              <Element type="PdfMessageDefinitionSimpleFieldDescription"> [...]</Element>
              [...]
              <Element type="PdfMessageDefinitionTableFieldDescription">
                   <Name>LineItems</Name>
                   <TableFields>
                     <TableField>
                       <Name>InvoiceLineQuantity</Name>
                     </TableField>
                     [...]
                   </TableFields>
             </Element>
         </ForceFieldsToBeExtracted>
      </type>
      <label>PDF Template</label>
      <tooltip>Define extraction rules based on the PDF template you will be sending</tooltip>
</parameter>

In the channel.xml file, delete the whole XML tag (and children) <Templates> and replace it with the following instruction.

<MessageDefinitionInfo>
      <MessageDefinition>787116</MessageDefinition>
      <Direction>IN</Direction>
      <Type>PDF</Type>
      [...]
      <Parameters type="UnmodifiableMap">
         [...]
        <forcedFieldsToExtract type="PdfMessageDefinitionFieldDescription[]">
          <Element type="PdfMessageDefinitionSimpleFieldDescription">
            <Name>InvoiceNumber</Name>
          </Element>
          <Element type="PdfMessageDefinitionSimpleFieldDescription">
            <Name>InvoiceDate</Name>
          </Element>
          <Element type="PdfMessageDefinitionSimpleFieldDescription">
            <Name>InvoiceTypeCode</Name>
          </Element>
          <Element type="PdfMessageDefinitionSimpleFieldDescription">
            <Name>InvoiceNote</Name>
          </Element>
           [...]
        </forcedFieldsToExtract>
        <SplitTextWhenDoubleSpace type="Boolean">true</SplitTextWhenDoubleSpace>
         <!-- Instead of the Templates XML tag, replace it with the following line - 
             it will serialize the parameter ‘PdfTemplate’ defined in the definition.xml 
             to produce a new XML tag ‘Templates’ -->
         ${display:xmlSerialize('Templates', PdfTemplate)}
        <ValidationMode type="ValidationMode">loose</ValidationMode>
        [...]
     </Parameters>
</MessageDefinitionInfo>

Description and tooltip

To ease the integration with the end-user, a description can be added to each PDF field that will be displayed in the PDF Editor as a tooltip when the user mouse-over the field.

<Element type="PdfMessageDefinitionSimpleFieldDescription">
          <Name>InvoiceTypeCode</Name>
          <Description>Please indicate the document type by inputting a 
                       document type code: either 380 for invoice, or 381 for credit note
          </Description>
</Element>

Validation of PDF fields

It is possible to add some validations on each PDF extracted field, so that when the end-user will extract fields from its PDF, he will get an immediate feedback whether the extraction is valid or not.

The recommended way to add validation is to add them before the export in the MD tree, so that the validation is not only on the PDF UI screen, but also when processing messages.

<Element type="PdfMessageDefinitionSimpleFieldDescription">
         <Name>InvoiceTypeCode</Name>
         <Description>Please indicate the document type by inputting a document type code: 
                      either 380 for invoice, or 381 for credit note
         </Description>
         <Validations>
               <Validation>
                     <Type>mandatory</Type>
                     <Assertion></Assertion>
               </Validation>
               <Validation>
                     <Type>maxLength</Type>
                     <Assertion>3</Assertion>
               </Validation>
               <Validation>
                     <Type>minLength</Type>
                     <Assertion>3</Assertion>
               </Validation>
               <Validation>
                     <Type>regexp</Type>
                     <Assertion>380|381</Assertion>
               </Validation>
         </Validations>
</Element>

Validation will be displayed in the same tooltip when the user mouse-over the field:

Generic gateway

In some use cases, we would like that the gateway IN or OUT of the connector is generic and we want to llet the end-user decide which protocol suits him best. For this purpose, connector allows to define a generic gateway IN / OUT as parameter in the definition.xml file:

Below is an example of such a parameter with a list of allowed gateway types. In this example, the end-user can choose between 4 types of gateways: HTTP, EMAIL, SFTP Server or AS2.

<parameter>
   <name>GatewayIN</name>
   <type>
      <class>GatewayInfo</class>
      <direction>IN</direction>
      <GatewayTypeAllowedValues type="GatewayType[]">
           <GatewayTypeAllowedValue>GATEWAY_IN_SFTP_IN</GatewayTypeAllowedValue>
           <GatewayTypeAllowedValue>GATEWAY_IN_SMTP_IN</GatewayTypeAllowedValue>
           <GatewayTypeAllowedValue>GATEWAY_IN_HTTP_IN</GatewayTypeAllowedValue>
           <GatewayTypeAllowedValue>GATEWAY_IN_AS2_IN</GatewayTypeAllowedValue>
      </GatewayTypeAllowedValues>
  </type>
  <label>Gateway IN</label>
  <tooltip>Specify how you want to upload the messages!</tooltip>
</parameter>

Afterwards, this parameter must be referenced in the channels.xml.jsp file as follows. Note that the function xmlSerialize is a specific Babelway defined function.

<GatewayInfo>
      <Gateway>1</Gateway>
      <Type>${GatewayIN.type}</Type>
      <CreationMoment>2019-10-25T01:12:03.258+02:00</CreationMoment>
      <LastUpdateMoment>2019-10-25T01:12:03.308+02:00</LastUpdateMoment>
      <Name>Gateway IN</Name>
      <Hub>101</Hub>
      ${display:xmlSerialize('Parameters', GatewayIN.parameters)}
      <ExtraProcessings>
      </ExtraProcessings>
</GatewayInfo>

Lookup table

Lookup tables used in channels will also be exported in the channels.xml file, but only their structure. Lookup table rows are not exported.

Certificate

Trusted certificates and private keys are not part of the export, for security reasons. However, it is still possible to build a connector that relies on a certificate (public or private).

Prerequisites:

The certificate must always be present in the environment where the connector definition has been built and uploaded.

Steps:

Find the parameter in the channels.xml where the certificate alias is defined.
Prepend the certificate alias with the keyword importFromHub:
This will indicate to Babelway that when deploying the connector, this certificate must be imported from the environment (hub) where the connector definition has been uploaded.

Here is an example with an AS2 gateway that has a trusted certificate to verify the AS2 signature generated by Tradeshift. The certificate has the alias “tradeshift” in this example and has been uploaded to the environment where the connector definition is installed.

<GatewayInfo>
      <Gateway>53550</Gateway>
      <Type>GATEWAY_IN_AS2_IN</Type>
      [...]      
      <Name>AS2 IN (TRADESHIFT)</Name>
      <Hub>101</Hub>
      <Parameters type="UnmodifiableMap">
        [...]
        <messageSignatureEnforced type="Boolean">true</messageSignatureEnforced>
        <certificateForVerification type="KeystoreCertAlias">importFromHub:tradeshift
        </certificateForVerification>
        [...]
      </Parameters>
</GatewayInfo>

Visibility

hub: the connector will be available only in the same environment as the connector definition
account: the connector will be available only in the same environment as the connector definition
med: the connector will be available in the specified Messaging Engine definition
world: the connector will be available for anyone using Babelway (note that this option is only available for Babelway Admin)

Internal properties

It is possible to define properties that are not displayed to the user, but that can be used in all JSP files. An example of this could be a parameter for the versioning of the connector, that can be then used in all channel, lookup table names,...

How to?

Create a new file called internal.properties at the root of the connector (same level as channels.xml & definition.xml)
Add your properties in the file using the name=value (more on the format on https://en.wikipedia.org/wiki/.properties)
Use the property as any other parameter in JSP files: ${name}

Manage update

Updating a connector definition and all the related connector instances is easy and does not require any dependency on Babelway teams & releases.

Here are the usual steps to update a connector definition and all related connector instances.

Apply the changes on the original channel(s) that was used to build the connector. If you do not have the channel(s) anymore, you can still instantiate & deploy one connector, and then click on “Delete but keep channels”.
Once the changes have been made, export the channel(s).
Unzip the zip file on your file system
Re-apply the changes in the channels.xml & definition.xml files
Zip all files & folders
Upload the new ZIP file in Babelway in the connector definition details page and click on Save.

Once the new connector definition has been uploaded, there are two ways of pushing the changes on the already installed connectors. Note that new instances of connectors will already use the newly uploaded connector definition.

1) Push Changes
This allows pushing the changes to the related connectors without deploying the changes. Connector can then be manually deployed by each end-user and they will see a small “Deploy updates” in their connector page:

2) Push Changes and Deploy
This allows pushing the changes to the related connectors and directly deploying the changes. No action is needed by the end-user who will benefit from the updates instantaneously. This will deploy connectors that are already deployed only. Connectors that are not yet deployed will get the changes as soon as they are deployed.

After pushing the change (and deploy), the state of each connector is displayed in the “Related items” tab of the connector definition:

Paid connector

In the context of the TradeshiftApps, connectors can be associated with a specific price plan. This is only taken into account if the price plan of the overall environment is “Tradeshift App”. Otherwise the price plan defined in the connector is not taken into consideration.

To define a price plan inside a connector, the following XML tags must be added to the definition.xml file:

<pricePlan>[default2021|free|app99]</pricePlan>
<billedChannels>45,85</billedChannels>

The pricePlan XML tag must be either free, default2021 or app99. There are only 2 paid price plans and the details of the price plan are displayed when the end user instantiates the connector in the tooltip (mouse-over on the price plan).

The billedChannels must be a comma-separated list of channel ID(s) defined in the channels.xml. All messages going through the defined channels will be billed to the user according to the price plan.

However, this XML tag might not be relevant to some price plan as there is only fixed fee per month (for example app99 will bill 99USD every month, no matter the number of messages processed, while default2021 will bill 75USD per month + extra fee after x messages per month).

When a Tradeshift user will search for a connector, the paid connector will have a small banner Paid on their box.

After choosing his connector, the Tradeshift end-user will see the price plan in the General tab of the connector:

Delete connector & keep channels

Connector allows to have a simplified layer above channels, and ease the integration of suppliers/buyers. However, it can happen that some suppliers/buyers have really specific needs and would like to customise some specific part of the connector (for example the transformation from PDF to UBL if we take the example of PDF2XML).
Once the connector has been deployed and that the underlying channels have been created and deployed as well, there is a button “Delete connector and keep channels”. This will remove the link to the connector but keep all channels that were created by the connector

Be careful when doing so because you will not benefit from the two main advantages of connectors:

You do not benefit anymore from automatic update of the connector definition
No way to revert and to link the channels back to the connector