Learn › Background Service, Azure WebJob Setup

Applies to background service version 26.08.12 and later · August 2026

The background service performs the Data Quality App’s long-running jobs – bulk duplicate detection, bulk merge, phonetic field fill and data quality score calculation. It polls Dynamics 365 every few seconds for scheduled background operations and runs them.

This guide sets it up as a continuous WebJob inside an Azure App Service, so it runs in the cloud without any customer-hosted machine. Setup takes about 15 minutes.

Before you start

The Data Quality App solution must be installed in the target Dynamics 365 environment, and its version must match this package.
You need the WebJob package: DuplicateDetection.BackgroundService.Standalone-<version>.zip (get it from Downloads). Do not extract it – it is uploaded as-is.
You need a connection string for the environment. An application user with a client secret is recommended:

AuthType=ClientSecret;Url=https://myorg.crm4.dynamics.com/;ClientId={app-id};ClientSecret={secret};

Only one background service instance may run against an environment – whether WebJob or the on-premises Windows service. Two instances would compete for the same jobs.

Step 1 – Create the Web App

Select Web » Web App.

Fill in the basics. The two settings that matter:

Runtime stack must be ASP.NET V4.7 or any later .NET Framework stack – not .NET Core.
Plan size: pick one suited to continuous operation. The size in the screenshot is only for testing; for production use at least a Basic (B1) plan. The detection engine runs several worker threads, so the smallest tiers will slow it down.

Application Insights is not required.

Tags are not required.

Review the configuration and click Create.

Wait for the deployment to finish, then click Go to resource.

Step 2 – Turn on Always On

Go to Configuration » General settingsswitch Always on to On, and click Save.

Important: Without Always On, Azure unloads the site when idle and the WebJob stops polling. Jobs will then sit in Scheduled and nothing appears to happen.

Step 3 – Configure the connection and settings

Go to Configuration » Application settings. Everything the service needs is entered here – the zip itself never has to be edited.

Add the polling interval

Under Application settingsclick New application setting and add:

Name: SleepTimeSeconds Value: 15

This is how long the service waits between polls for new jobs when idle. 15 seconds is a good default.

Adding the SleepTimeSeconds application setting

Add the CRM connection string

Under Connection stringsclick New connection string and add:

Name: CRM Type: Custom

AuthType=ClientSecret;Url=https://myorg.crm4.dynamics.com/;ClientId={app-id};ClientSecret={secret};

Note: The name must be exactly CRM and the type must be Custom. The screenshot shows the older username/password format, which still works, but an application user with a client secret is recommended – it is unaffected by MFA and password expiry.

Click Save at the top of the Configuration page when both are in.

Optional performance settings can be added the same way – see Settings reference at the end of this guide. The service is already at full speed without them; values entered in the portal override the defaults packaged in the zip.

Step 4 – Upload the WebJob

Go to WebJobs and click Add.

Configure it exactly like this:

File Upload: the DuplicateDetection.BackgroundService.Standalone-<version>.zip package, unmodified.
Type: Continuous.
Scale: Single Instance – required. Multiple instances would process the same jobs twice.

WebJob configuration, Continuous, Single Instance

Step 5 – Check that it is running

Click Refresh until the WebJob shows Running.

Then open Logs to confirm the service started cleanly.

On startup the service logs its version, its effective settings, and the CRM user it signed in as. A healthy start looks like this:

If a setting you entered in the portal is not reflected in this list, the key name does not match – check the Settings reference table.

Startup log showing version, settings and CRM sign-in

Step 6 – Verify in Dynamics 365

Schedule a job in the Data Quality App (for example a bulk detection). Within about 15 seconds the service picks it up, sets it to In Progressand writes live progress to the job’s Log field:

Processing records 201-400/4284…
Bulk merge started. 38 duplicates to merge.
Merged 20 of 38 duplicates. 18 remaining.
Bulk merge finished.

Note: The screenshot below is from an earlier release, where the log showed one record per line. The current service processes records in batches, so the log advances in ranges – that is normal and much faster.

Background operation log in Dynamics 365

Updating to a new version

Settings live in the App Service, not in the zip – so an update never touches your configuration.

In WebJobs, select the job and click Stop. Never replace a running job mid-merge; an interrupted job resumes from its saved state when the service restarts.
Delete the job, then Add the new zip with the same settings (Continuous, Single Instance).
Open Logs and confirm the new version number in the startup lines.

Settings reference

All keys go under Configuration » Application settings. Every one of them is optional: the defaults below are already the fast, recommended values. The effective values are always visible in the startup log.

Setting	Default	What it does
`SleepTimeSeconds`	15	Seconds between polls for new jobs when idle.
`BulkDetectionServerSide`	true	Run detection queries inside Dynamics 365, next to the database. The single biggest speed factor – leave on.
`BulkDetectionBatchSize`	200	Records detected per background step (roughly parallelism × 25).
`BulkDetectionDegreeOfParallelism`	8	Concurrent detection workers. 1 = sequential.
`BulkDetectionUseExecuteMultiple`	true	Batch result-row writes. Only applies when server-side execution is off.
`BulkMergeServerSide`	true	Run merges inside Dynamics 365, so related-record reparenting happens next to the database.
`BulkMergeMergesPerStep`	20	Merges performed per background step before progress is saved and logged.
`BulkMergeDegreeOfParallelism`	1	Experimental. Leave at 1 unless advised otherwise.

Throughput is ultimately limited by Dataverse service protection (roughly 6,000 API requests per 5-minute window per user), not by the App Service plan. Raising the parallelism past the point where Dataverse starts throttling does not make jobs faster.

Troubleshooting

The WebJob log (Step 5) is always the first place to look. The service names its problem there.

“No CRM connection string found”

The connection string is missing or misnamed. It must be under Connection strings (not Application settings), named exactly CRMtype Custom.

“Could not connect to CRM: …”

The connection string was found but sign-in failed, and the log line contains the actual reason from the platform – for example an expired client secret, a wrong ClientId, a disabled application user, or a firewall blocking the connection. Fix the named cause; the service retries on its own every poll.

The job stays in Scheduled and nothing happens

Is the WebJob Running (Step 5), and is Always On enabled (Step 2)?
Does the startup log end with “Logged into CRM as user …”?
Is another background service instance running elsewhere – an on-premises Windows service or a second WebJob – against the same environment?
Does the job’s status show Scheduled (not Done or Error), with no future Scheduled On date?

Jobs run slower than expected

Large jobs are throttled by Dataverse service protection, which the service handles automatically by backing off and retrying. Occasional pauses of one to two minutes during very large runs are normal and resolve on their own.