As a software product developer, efficiently managing background processes is crucial for creating robust and reliable applications. Temporal, an open-source platform, offers a powerful solution for orchestrating these processes. Temporal is the successor to Cadence (developed by Uber), created by the same team to address similar needs for workflow orchestration. In this article, we will explore how Temporal can be utilized to manage background processes more effectively, using a PoC KYC System project as a case study.

What is Temporal?

Temporal is a workflow orchestration engine that helps developers build scalable and reliable applications. It provides a framework for managing long-running, stateful workflows and background processes. Temporal abstracts away the complexities of handling retries, state management, and failure recovery, enabling developers to focus on writing business logic.

Key Features of Temporal

Durable Execution: Ensures that workflows and activities are durable and can survive process failures.Stateful Workflows: Temporal allows developers to write workflows as code, managing complex business logic and stateful operations without worrying about infrastructure.Scalable Architecture: Supports high scalability and concurrency.Monitoring and Visibility: It provides tools for monitoring and debugging workflows in real-time, offering visibility into the execution and performance of applications.Fault Tolerance: Automatically handles retries and failures, ensuring robustness.Flexible APIs: Provides language-specific SDKs, making it easy to integrate into various environments. (Still waiting on the Rust SDK)

KYC PoC

The motivation behind creating a KYC system stems from personal experience with a complex KYC system, addressing nuanced issues. The system requires the following components:

Notification on signup to prompt users to submit KYC details.Internal systems pinged upon submission of KYC details for preprocessing.Preprocessing of submitted KYC data via internal systems.Waiting for KYC agents (human input) to update details.Processing and notifying the user about actions taken.Restarting the KYC process if the user’s KYC is rejected.
These requirements were implemented using temporal workflows, activities, and signals.

Workflows

Temporal workflows are a way to define and manage long-running, stateful processes within the Temporal platform. These workflows allow developers to write business logic that can span days, months, or even years, while Temporal handles the complexities of state persistence, retries, and failure recovery.

// Workflow to manage KYC process
func KYCWorkflow(ctx workflow.Context, user model.User) (string, error) {
// … workflow code
}

Child Workflows

Child workflows are initiated and managed by another parent workflow. They encapsulate specific tasks or subprocesses within the broader context of a parent workflow, enhancing maintainability and scalability:

// Workflow to manage KYC process
func KYCWorkflow(ctx workflow.Context, user model.User) (string, error) {
// …
workflow.ExecuteChildWorkflow(ctx, RequestKYCWorkflow, user).Get(ctx, &response)
// …
}

Activities

Temporal activities represent individual units of work or tasks within a workflow. They are executed asynchronously and reliably by Temporal:

func SendKYCNotification(ctx context.Context, data SendNotificationData) (string, error) {
// … send notification to User
}
// Workflow to manage KYC process
func KYCWorkflow(ctx workflow.Context, user model.User) (string, error) {
// …
// Trigger notification for KYC results
var result string
err := workflow.ExecuteActivity(ctx, kyc_activity.SendKYCNotification, request_kyc_notification).Get(ctx, &result)
return result, err
// …
}

Signals

Temporal signals enable communication with running workflows, allowing external processes or users to send asynchronous messages:

// Keep sending notifications to user until we get a KYC submission
func RequestKYCWorkflow(ctx workflow.Context, user model.User) (*model.KYCRequest, error) {
// …
// set up signal
var kyc_request model.KYCRequest
kyc_signal_channel := workflow.GetSignalChannel(ctx, app.NEW_KYC_SIGNAL)
// following blocks until signal is received
kyc_signal_channel.Receive(ctx, &kyc_request)
//…
}
// KYC API Submitted by User and Signal workflow with new kyc data
func KycSubmit(w http.ResponseWriter, r *http.Request) {
// … process user input and connection to temporal server
err = client.SignalWorkflow(context.Background(), kyc_request.UserId, “”, app.NEW_KYC_SIGNAL, kyc_request)
// … manage errors and process response
}

Timers

Temporal timers schedule actions or events within workflows, essential for implementing timeouts and scheduling retries:

workflow.NewTimer(ctx, NOTIFICATION_INTERVAL)

Selectors

Selectors in Temporal allow workflows to wait for multiple events concurrently and react to the first event that occurs:

// Keep sending notifications to user until we get a KYC submission
func RequestKYCWorkflow(ctx workflow.Context, user model.User) (*model.KYCRequest, error) {
// …
kyc_signal_channel := workflow.GetSignalChannel(ctx, app.NEW_KYC_SIGNAL)
// Create selector to wait for signal as well as timer
// Which ever one triggers first will decide what happens in the if check later
selector := workflow.NewSelector(ctx)
selector.AddReceive(kyc_signal_channel, func(c workflow.ReceiveChannel, more bool) {
c.Receive(ctx, &kyc_request)
log.Info(“RECEIVED IN CHILD”)
})
// Wait for Interval
// If timer triggers first then signal will be received in the next run
selector.AddFuture(workflow.NewTimer(ctx, NOTIFICATION_INTERVAL), func(f workflow.Future) {
err = f.Get(ctx, nil)
})
selector.Select(ctx)
// …
}

Implementation

The full working implementation of the Proof of Concept is available on GitHub.

Results

In the PoC, everything was orchestrated with retries and a built-in UI to monitor the background system. The developer experience was satisfactory, with efficient implementation and minimal time investment.

Temporal: A better way to manage background processes? was originally published in Level Up Coding on Medium, where people are continuing the conversation by highlighting and responding to this story.

​ Level Up Coding – Medium

about Infinite Loop Digital

We support businesses by identifying requirements and helping clients integrate AI seamlessly into their operations.

Gartner
Gartner Digital Workplace Summit Generative Al

GenAI sessions:

  • 4 Use Cases for Generative AI and ChatGPT in the Digital Workplace
  • How the Power of Generative AI Will Transform Knowledge Management
  • The Perils and Promises of Microsoft 365 Copilot
  • How to Be the Generative AI Champion Your CIO and Organization Need
  • How to Shift Organizational Culture Today to Embrace Generative AI Tomorrow
  • Mitigate the Risks of Generative AI by Enhancing Your Information Governance
  • Cultivate Essential Skills for Collaborating With Artificial Intelligence
  • Ask the Expert: Microsoft 365 Copilot
  • Generative AI Across Digital Workplace Markets
10 – 11 June 2024

London, U.K.