By Boomi
At its heart, event-driven integration means enhanced real-time customer engagement and better-informed, predictive decision-making. The Solace PubSub+ Connector for Boomi enables organizations to reap those benefits faster by creating a combined Boomi and Solace experience that’s greater than the sum of its parts.
In a previous post, I covered the first three steps in a 6-step methodology. I recommend you read that post first, to understand how we got to event-driven design.
In this post, I will go over the final three steps needed to help you get the most out of your event-driven integration investment.
Step 4. Event-Driven Design
Now that you’ve landed on a single project for a pilot, it’s time to roll up your sleeves and start designing. As mentioned at the beginning, a key part of the event-driven methodology is an API-first philosophy. Prior to PubSub+ Event Portal, there was no API gateway equivalent for events. With the introduction of our event portal, the designs you create here will serve to document your architecture, but also serve as the raw material for the PubSub+ connector as you implement your design in the next section using the Boomi AtomSphere Platform and Solace PubSub+ Platform.
Solace PubSub+ Event Portal contains three main entities: events, schemas, and applications. When developing event-driven integrations with Boomi, you can think of the components like this:
An application represents a piece of software that produces and consumes events. For us, that will be a Boomi Integration Process or an external entity like Salesforce or Bamboo. Applications connect to the event broker in an event-driven architecture and communicate with other applications via events.
An event represents a business moment that can be communicated to interested applications. An application (here a Boomi process or external application) produces an event, containing a payload that matches a schema on a specific topic.
A schema describes the payload of an event. Producers and consumers of an event can trust that the payload of the event matches the schema definition assigned to that event. Schemas define a type of payload through JSON, XML, Binary, or Text. JSON and XML schemas have content that describes each property of the schema. The content is either in JSON Schema format or XSD/DTD format.
Going from pilot project choice to a full-fledged design means knowing the details of your applications, events, and schemas. That means asking a lot of questions. In some ways it’s like a high school journalism class: figuring out the 5Ws+H (who, what, when, where, why, and how) of the events flowing through the enterprise. Equally important is documenting all the answers in a repeatable way that developers can use our design-time processes later.
Each of the key questions below flesh out some aspect of the applications, events, and schemas. We can then enter them into PubSub+ Event Portal.
What information is available from the source application and what does it look like?
The first step is figuring out what information from the source application is available that could be used by the rest of your enterprise. A schema represents what that data looks like. If you’re using a SaaS or other commercial product, there is a well-defined schema available that you can find in its documentation. If it’s an internal application, you might need to dig a little more to find or create a schema representation. Event Portal catalogues schemas for easy access, and allows Boomi to import them when designing processes:
How can the source application send data?
This is also known as the protocol. Luckily for us, Solace and Boomi support many protocols—from event-driven protocols like JMS and MQTT, to REST over HTTP, to application specific ones like database connections with JDBC or just reading from a flat file. The protocol is key for determining the structure of your Boomi processes and picking the right entry from the process library, which will be addressed shortly.
What destination applications are interested in the data?
Flexible, one-to-many event distribution is a key benefit of event-driven integration. That means Solace can distribute an event arriving from a single information source to many interested consuming applications. Each of those applications can be defined within Event Portal.
What do destination applications need the payload to look like?
Just because an event starts with a certain data format, doesn’t mean that consuming systems can accept it in that format. That’s especially true for commercial SaaS applications. The first step is adding the destination schema to the Event Portal. This catalogues it and makes it available to the PubSub+ connector at design time. At which point, Boomi’s graphical data transformation tools can convert the data as required.
When are destination applications interested in data?
By using wildcards wisely, the flow of events to a consumer isn’t all or nothing. Some consumers may only be interested in a geographic region. Others may only care about events involving a large amount of money. Fortunately, with event-driven architecture, the event broker can handle the details of event distribution. This feeds into topic string design. As you start your EDA journey, it’s important to pay attention to topic taxonomy – setting up a topic naming convention early on and governing it. A solid taxonomy is probably the most important design investment you will make, so best not to take short cuts here. Having a good taxonomy will help with event routing, and the conventions will become obvious to application developers. For expert advice on this, refer to my colleague Ken Barr’s post on topic best practices. Event Portal calls the combination of a schema and a topic an event. As you input them into Event Portal, you start to build a catalog of events that you can use in this pilot and in future projects.
Boomi’s graphical tooling often comes into play here, as it lets you extract information from the payload and create a topic string with it.
How can destination applications receive data?
Again, this is the protocol, but this time it is the destination protocol that the consuming application will use. The combination of Boomi and Solace gives you a wide array of supported protocols.
How should we structure the Boomi processes?
One guiding principle: decouple information sources from information consumers.
What does decoupling look like for a Boomi implementation?
A traditional Boomi process receives information, then uses the branch or process route shape to interact with multiple information consumers. Moving to decoupling unbinds information receivers from information senders, with separate Boomi processes for both. PubSub+ Event Bbroker distributes information between them.
Why is decoupling so important?
Decoupling allows you to reap the benefits of event-driven architecture you evangelized in Step 1, including simplified error handling, preserving data in the case of disaster, allowing free movement of data from on-premises to the cloud, and rapid deployment of innovative solutions. While decoupling is the key, it can take several different forms, depending on what protocol your source and destination applications can use. If it’s an event-driven protocol like JMS or MQTT, then Solace might be the right choice to interact directly within the information source.
In this architecture, an information provider – which could be an internet of things device like a sensor – interacts directly with the event broker. The event broker routes the incoming event and places copies of it on queues for individual Boomi sender processes. The queuing insulates the Boomi processes from sudden rushes of traffic; if a downstream system can’t handle an event at the time, the event broker will hold it. And because each Boomi sender process gets its own copy of the message, it can 1) complete independently, without a dependency on other processes, and 2) implement business logic that is solely related to the end consumer. So, the Boomi sender for Salesforce logic is completely isolated from the logic for ServiceNow. If it’s not an event-driven protocol, then using Boomi to interact directly with the information source is probably the right call.
If the information provider doesn’t use an event-driven protocol like JMS, then connectors within Boomi can liberate information in legacy protocols into events by grabbing the incoming payload, determining the correct topic string, and publishing the event to Solace. In this case the information provider interacts with Boomi first, then Solace.
What does the overall flow of events look like in Event Portal?
Finally, you’ve completed the design work. The reward is a top down view through Event Portal of how your event-driven integration will work. That view is invaluable for communicating your vision for the pilot out to the team and re-evangelizing and re-educating your enterprise for your next project.
Instead of the haphazard Visio diagram that lives only on the architect’s laptop and is likely outdated, we now have a living, visual document that developers, architects and support personnel can access.
An outdated Visio diagram crammed with information
A standardized Event Portal diagram
What’s more, we can now dive into each of the entities to get more information about the schemas, applications, and events that make up our architecture. For example, we can see what schema, topic string, and applications interact with the ResourceHired event that was discussed earlier on.
And as we turn to implementation in the next section, the PubSub+ Connector for Boomi seamlessly brings this information into the Boomi UI.
Step 5. Implement a Quick Win
Now that the Async-API first design completed, it’s time to implement the design. Event Portal work continues to be important; you’ll be using it to import event definitions that ensure that the topic strings and data formats that are used match the design. But now the toolset expands to the Cloud Management capabilities of Solace, the Integration component of the Boomi AtomSphere Platform, and the crucial the Solace PubSub+ Connector, which allows them to integrate.
Note, the implementation and configuration instructions here are intentionally high-level. You can get much more information after importing an appropriate process from the process library, which will guide you step-by-step through the process or from more technical blog posts from Mike Hilmen.
Match your decomposed processes with process libraries
To help you get started, Solace and Boomi created a process library leveraging the PubSub+ connector and Boomi Integration. These processes bring together best practices developed through years of shared experience between Solace and Boomi — all you do is fill in the blanks and enhance with your business logic.
Many of these processes feature key technical aspects specifically aimed to meet the business objectives that made event-driven integration appealing in the first place. The key is to match the use case of your application to a use case in the process library.
Step through the code lab configuration
For each process in the Process Library, in addition to the skeleton code, there is extensive documentation in the form of a Google Code Lab. The documentation shows how to configure the Connector, Connector Operation, and the process itself.
Configure the PubSub+ Connector
This is where the design-first work you did in the earlier step comes into play. The PubSub+ Connector for Boomi integrates with Event Portal, which gives visibility into your event-based architecture. After picking an event, the connector imports the schema and the format of the topic string, then generates a recommended input queue name based on the name of the connector operation.
Round out business logic using Boomi’s integration capabilities
Even though the process library covers many use cases, you’ll still need to enhance the processes to make them your own. That could include custom data mapping, enriching the event with information from a flat file or database, or running business rules against them.
Create the underlying Solace infrastructure
Now that you’ve created the Boomi process, it’s time to create the underlying Solace infrastructure event services, the queues that your applications will read off, and the subscriptions that will attract messages to the queues. You can create all of these using the Cloud Management console. Again, the code lab guides you through each click you need to create the objects
Step 6. Repeat
With the pilot, as you define events and refine the flow of events through your processes, your Event Portal event catalog also takes shape. The initial event catalog serves as a starting point of reusable events which future applications can consume off the event broker queues. Here, the cycle begins again: searching for additional business uses cases that could benefit from event-driven integration. Only this time, you are ready with some lessons learned from the pilot and a handful of events already available on your mesh.
Conclusion – Why Use PubSub+ Connector for Boomi?
To briefly recap: why would you use the PubSub+ Connector for Boomi? The main reason is that it guides you towards repeatable best-practices to help you get the most out of your event-driven integration investment. It serves as the glue in a combined Boomi/Solace environment, making your design-first artifacts living, breathing entities that your developers can utilize as they make life better for your customers.
Boomi has partnered with Solace to extend the value of the industry leading Boomi platform with enterprise grade event streaming and event management capabilities. Learn more here.