Skip to main content

Failure detection - .NET SDK

Topics covered in this section

Workflow timeouts
→
Workflow retries
→
Activity timeouts
→
Activity Retry Policy
→
Heartbeat an Activity
→
Heartbeat Timeout
→

Workflow timeouts​

How to set Workflow timeouts using the Temporal .NET SDK

Each Workflow timeout controls the maximum duration of a different aspect of a Workflow Execution.

Workflow timeouts are set when starting the Workflow Execution.

These values can be set in the WorkflowOptions when calling StartWorkflowAsync or ExecuteWorkflowAsync.

Available timeouts are:

  • ExecutionTimeout
  • RunTimeout
  • TaskTimeout
var result = await client.ExecuteWorkflowAsync(
    (MyWorkflow wf) => wf.RunAsync(),
    new(id: "my-workflow-id", taskQueue: "my-task-queue")
    {
        WorkflowExecutionTimeout = TimeSpan.FromMinutes(5),
    });

Set Workflow retries​

How to set Workflow retries using the Temporal .NET SDK

A Retry Policy can work in cooperation with the timeouts to provide fine controls to optimize the execution experience.

Use a Retry Policy to retry a Workflow Execution in the event of a failure.

Workflow Executions do not retry by default, and Retry Policies should be used with Workflow Executions only in certain situations.

The RetryPolicy can be set in the WorkflowOptions when calling StartWorkflowAsync or ExecuteWorkflowAsync.

var result = await client.ExecuteWorkflowAsync(
    (MyWorkflow wf) => wf.RunAsync(),
    new(id: "my-workflow-id", taskQueue: "my-task-queue")
    {
        RetryPolicy = new() { MaximumInterval = TimeSpan.FromSeconds(10) },
    });

Activity Timeouts​

How to set Activity Timeouts using the Temporal .NET SDK

Each Activity Timeout controls the maximum duration of a different aspect of an Activity Execution.

The following Timeouts are available in the Activity Options.

An Activity Execution must have either the Start-To-Close or the Schedule-To-Close Timeout set.

These values can be set in the ActivityOptions when calling ExecuteActivityAsync.

Available timeouts are:

  • ScheduleToCloseTimeout
  • ScheduleToStartTimeout
  • StartToCloseTimeout
return await Workflow.ExecuteActivityAsync(
    (MyActivities a) => a.MyActivity(param),
    new() { StartToCloseTimeout = TimeSpan.FromMinutes(5) });

Set an Activity Retry Policy​

How to an Activity Retry Policy using the Temporal .NET SDK

A Retry Policy works in cooperation with the timeouts to provide fine controls to optimize the execution experience.

Activity Executions are automatically associated with a default Retry Policy if a custom one is not provided.

To create an Activity Retry Policy in .NET, set the RetryPolicy on the ActivityOptions when calling ExecuteActivityAsync.

return await Workflow.ExecuteActivityAsync(
    (MyActivities a) => a.MyActivity(param),
    new()
    {
        StartToCloseTimeout = TimeSpan.FromMinutes(5),
        RetryPolicy = new() { MaximumInterval = TimeSpan.FromSeconds(10) },
    });

Override the Retry interval with nextRetryDelay​

When you throw an Application Failure and assign the nextRetryDelay field, its value replaces and overrides the Retry interval defined in the active Retry Policy.

For example, you might scale the next Retry delay interval based on the current number of attempts. Here's how you'd do that in an Activity. In the following sample, the attempt count is retrieved from the Activity Execution context and used to set the number of seconds for the next Retry delay:

var attempt = ActivityExecutionContext.Current.Info.Attempt;

throw new ApplicationFailureException(
    $"Something bad happened on attempt {attempt}",
    errorType: "my_failure_type",
    nextRetryDelay: TimeSpan.FromSeconds(3 * attempt));

Heartbeat an Activity​

How to Heartbeat an Activity using the Temporal .NET SDK

An Activity Heartbeat is a ping from the Worker Process that is executing the Activity to the Temporal Service. Each Heartbeat informs the Temporal Service that the Activity Execution is making progress and the Worker has not crashed. If the Temporal Service does not receive a Heartbeat within a Heartbeat Timeout time period, the Activity will be considered failed and another Activity Task Execution may be scheduled according to the Retry Policy.

Heartbeats may not always be sent to the Temporal Service—they may be throttled by the Worker.

Activity Cancellations are delivered to Activities from the Temporal Service when they Heartbeat. Activities that don't Heartbeat can't receive a Cancellation. Heartbeat throttling may lead to Cancellation getting delivered later than expected.

Heartbeats can contain a Details field describing the Activity's current progress. If an Activity gets retried, the Activity can access the Details from the last Heartbeat that was sent to the Temporal Service.

To Heartbeat an Activity Execution in .NET, use the Heartbeat() method on the ActivityExecutionContext.

[Activity]
public async Task MyActivityAsync()
{
    while (true)
    {
        // Send heartbeat
        ActivityExecutionContext.Current.Heartbeat();

        // Do some work, passing the cancellation token
        await Task.Delay(1000, ActivityExecutionContext.Current.CancellationToken);
    }
}

In addition to obtaining cancellation information, Heartbeats also support detail data that persists on the server for retrieval during Activity retry. If an Activity calls Heartbeat(123, 456) and then fails and is retried, HeartbeatDetails on the ActivityInfo returns an collection containing 123 and 456 on the next Run.

Set a Heartbeat Timeout​

How to set a Heartbeat Timeout using the Temporal .NET SDK

A Heartbeat Timeout works in conjunction with Activity Heartbeats.

HeartbeatTimeout is a property on ActivityOptions for ExecuteActivityAsync used to set the maximum time between Activity Heartbeats.

await Workflow.ExecuteActivityAsync(
    (MyActivities a) => a.MyActivity(param),
    new()
    {
        StartToCloseTimeout = TimeSpan.FromMinutes(5),
        HeartbeatTimeout = TimeSpan.FromSeconds(30),
    });