Journey Platform: A low-code device for creating interactive person workflows | by Arjun Raman | The Airbnb Tech Weblog | Might, 2023
Journey Platform: Low-code notification workflow platform that permits technical and non-technical customers to create advanced workflows by means of a easy drag and drop person interface.
By: Arjun Raman, Ken Snyder, Mengting Li
Efficient communication hinges on delivering the proper message, to the proper viewers, on the proper time. At Airbnb, our aim is to interact our customers — each friends and hosts — by delivering inspirational and informational notifications by means of numerous channels, resembling e mail or in-app messages.
Traditionally at Airbnb, advanced notification workflows have been solely managed by engineering groups, with every workflow requiring the deployment of code. As our platform developed, we acknowledged the necessity for a low-code or no-code answer to streamline the creation of those intricate notification workflows. In response, the Advertising and marketing Expertise group developed the Journey Platform, a robust device that permits non-technical customers to construct and ship personalised notifications primarily based on our customers’ engagement with Airbnb.
The objectives of the Journey Platform are:
- Empower customers to simply create event-driven notification workflows utilizing an intuitive drag and drop interface.
- Allow real-time execution of those notification workflows for well timed and related communication.
- Supply a unified interface for managing transaction notifications, resembling upcoming journey reminders and promotional notifications.
- Assure Service Stage Agreements (SLAs) for processing numerous forms of notification workflows, together with transactional and promotional communications.
- Scale back the time required to develop advanced notification workflows.
Journey Platform permits customers to iterate sooner by permitting self-serve workflow creation. It has diminished the time taken to help a brand new use-case from 1–2 months to simply 1–2 weeks.
The important thing parts of the Journey Platform are:
- Journey Platform UI: WYSIWYG device permits customers to pull and drop parts and create a workflow. The workflow definition is then transformed to a customized DSL (Domain-specific language) which will be interpreted and executed by the workflow orchestrator.
- Workflow Orchestrator: Mind of the system, the workflow orchestrator takes within the workflow definition DSL from the UI. As soon as a workflow is launched, it listens for occasions from the occasion retailer that may begin the execution of a workflow, interprets then parses the DSL to execute workflows on the workflow engine, and depends on the Motion retailer to carry out particular duties.
- Platform Retailer:
- Occasion Retailer: Pre-configured catalog of Kafka occasions which Journey Platform can hearken to and set off new executions of a workflow or go occasions to present workflow execution.
- Motion retailer: Repository of predefined, specific-purpose features permits customers to carry out numerous duties, resembling sending emails, push notifications, or emitting Kafka occasions. Customized actions will be outlined and built-in into the device, making them accessible to all Journey Platform customers.
- Attribute retailer: Central repository for important knowledge, resembling person metadata (e.g. person’s geolocation, Airbnb search historical past, and many others.) and contextual data. It helps decision-making in workflow branching processes by exposing these knowledge as a parameter to set situations upon by means of the parameter supervisor.
- Customized shops: Means to create customized motion or attribute shops which aren’t already outlined within the platform.
- Workflow Orchestrator: Mind of the system, the workflow orchestrator takes within the workflow definition DSL from the UI. As soon as a workflow is launched, it listens for occasions from the occasion retailer that may begin the execution of a workflow, interprets then parses the DSL to execute workflows on the workflow engine, and depends on the Motion retailer to carry out particular duties.
When crafting the UI for the workflow automation system, we aimed to create a well-known and intuitive expertise. Drawing inspiration from circulate charts, productiveness instruments with “inspector panels,” and incorporating drag and drop performance, we needed a platform the place customers might begin instantly with out consulting the guide.
We additionally had a aim of utilizing progressive disclosure to incrementally allow the complete depth of the platform capabilities, whereas maintaining it easy for customers who solely want a small subset of the options. Through the use of smart defaults, and transferring extra advanced options into tabs and sub-screens, our superior customers might create distinctive options, going past the pre-planned use circumstances.
To edit the graph, we leveraged React Flow, an open-source library. This enabled us to show the graph, in addition to present primary operations like zooming, panning, transferring, and connecting nodes. On prime of this basis, we added our customized node and edge parts, together with drag and drop performance for including new nodes and an inspector panel for enhancing present ones.
To create the kinds displayed within the inspector panel, we applied a schema-based type system. This method supplies a excessive stage of flexibility, permitting us to declaratively specify the UI for particular node enter/output fields as a part of their sort definitions. The system is inbuilt a type-safe method, making use of Thrift annotations and Java reflection. Based mostly on the schema data and UI-specific annotations, the interface shows the suitable type fields, assist textual content, and validation, guaranteeing our UI is routinely up-to-date with the platform’s capabilities.
Area-specific language
DSL supplies a excessive diploma of flexibility and customization, permitting us to outline the construction and habits of the workflow. As a substitute of getting to hardcode a workflow within the workflow engine, we as an alternative have a generic workflow outlined that may execute any DSL-based workflow. Nodes and edges make up a workflow, with nodes representing particular person actions or duties and edges defining the dependencies and relationships between them.
The nodes and edges embrace all the required data to outline a workflow resembling inputs, outputs, and parameters handed between nodes. The DSL generated by the UI is handed to the workflow orchestrator, the place the DSL parser executes it.
Journey Shops
The occasions, attributes, and actions shops are an integral a part of the backend, as they permit listening to occasions to start out workflow executions, filter customers, and execute duties within the journey. All these parts work collectively seamlessly to create a versatile and customizable backend that may be tailor-made to the particular wants of the platform.
Occasion Retailer
Journey Platform helps listening to totally different Kafka occasions and utilizing them to set off new executions of a workflow, or use the occasion to go alerts to a operating execution. For instance, begin a brand new execution when a visitor books a keep, go a sign to a operating execution when a person receives a push notification, and many others. Just like the motion retailer, as soon as an occasion is on-boarded, all of the groups at Airbnb can use it.
Attribute Retailer
The attribute retailer features as a central repository for fetching all crucial knowledge, resembling contextual knowledge, person preferences, and machine data, which can be utilized to counterpoint the workflow branching course of and enhance decision-making capabilities. These shops are supported by a knowledge storage system that manages numerous attributes or traits of entities.
Think about you might have a brand new person who simply signed up for Airbnb, and also you’re all for figuring out whether or not they’ve carried out any itemizing searches on the platform. If the reply is sure, you’ll ship a customized message primarily based on their search historical past, and if it’s no, you’ll ship a static message.
It is a concrete instance of how the Airbnb Journey Platform leverages attributes, resembling “itemizing search historical past,” to reinforce the person expertise. These attributes are extracted and outlined as parameters, which can be utilized for numerous functions. Every workflow execution has its personal parameter knowledge assortment, which will be accessed within the parameter supervisor. Extra details about parameters shall be mentioned within the parameter supervisor.
Motion Retailer
The motion retailer is used to execute numerous duties, resembling sending an e mail or updating a database file, when a person reaches a selected level within the journey. It’s a frequent library the place every operate will be shared and reused by totally different customers of their workflow.
Every motion implements a typical interface, together with its metadata required for the UI schema-based kinds talked about above, and its habits through the precise workflow execution.
Parameter Supervisor
Managing a posh workflow that entails a number of steps with various inputs and outputs is usually a difficult process, particularly if the enter and output parameters change regularly or are totally different for every person. As an example, you would possibly want conditional branching in your workflow or personalised communication content material primarily based on person search. That is the place parameterized workflows and parameter managers can show to be invaluable parts.
By specifying inputs and outputs (of attribute node / occasion node / customized node) as parameters, you possibly can reuse them all through your complete workflow execution. A parameter supervisor is a vital element that may retailer and handle your workflow parameters, streamlining the method of making, storing, retrieving, and modifying them.
Along with offering an environment friendly parameter administration system, a parameter supervisor additionally supplies a spread of options resembling parameter creation, storage, retrieval, modification, versioning, entry management, and auditing. These options be certain that your workflow is executed reliably and constantly whereas additionally correctly managing and storing your parameters all through your complete workflow.
The Workflow Orchestrator executes workflows by deciphering the that means of every DSL node and performing the corresponding actions. It manages low-level features resembling storing state, interacting with the motion retailer to carry out an motion, listening for callbacks by means of the occasion retailer, and permitting builders to focus on workflow logic somewhat than technical particulars. Journey Platform makes use of Temporal because the underlying workflow engine for state upkeep and orchestration. Temporal helps orchestrate workflows by means of Temporal Workers.
Builders can incorporate customized performance resembling new nodes or edges to broaden platform capabilities, making it easier to create workflows that fulfill the platform’s and customers’ distinctive necessities. Moreover, it helps superior options like parallel execution and computerized retries and enhancing platform reliability and efficiency.
Guaranteeing SLA for processing several types of workflows (i.e. transactional and promotional) is vital at scale. Transactional notifications initiated by person motion (e.g. reserving affirmation, visitor/Host messaging, and many others.) have a strict SLA and require greater precedence when in comparison with promotional notifications. To realize this, now we have applied the next at totally different elements of the system:
Occasion pre-processing:
- Pre-filter: As a substitute of passing all of the occasions on to the Workflow Handler, the occasion processor filters out occasions that don’t match the factors. e.g. solely go if the occasion sort is reservation_complete and filter out for all different reservation occasions. This drastically reduces the QPS feeding into the system.
- Combination excessive QPS occasions: Occasions like searches have a excessive QPS. As a substitute of instantly processing, we batch and mixture them over a time window. This reduces the QPS by at the least just a few orders of magnitude.
Devoted lanes:
- We have now devoted lanes for various classes of workflows by means of the system. The occasion listener has totally different shopper teams with built-in throttling. The workflow handler has devoted Temporal namespaces for every class and strict limits on the processing QPS, max QPS to the database, and many others.
The Journey Platform empowers non-technical and technical customers to create advanced stateful workflows by means of a easy drag and drop interface. By leveraging a generic workflow definition DSL, together with motion retailer, occasion retailer, and attribute retailer, the platform facilitates the creation of workflows that reply to real-time occasions, streamlining communication, and enhancing person experiences.
Thinking about working at Airbnb? Try these open roles.
Due to Balaji Kalaimani, Davis Wamola, Iris Feng, Jesse Garrison, John Bernardo, Kumar Arjunan, Michael Endelman, Priyank Singhal, Steve Krulewitz, Tej Sudha, Victoria Gryn, Xin Tu, and Zhentao Solar for his or her contributions in constructing Journey Platform.
Due to Sagar Naik and Michael Kinoti for his or her management and supporting us on this Journey.
All product names, logos, and types are property of their respective house owners. All firm, product and repair names used on this web site are for identification functions solely. Use of those names, logos, and types doesn’t suggest endorsement.
