Now the Duplicate detection feature in CRM isn’t great, but it does work providing you know how to set up the correct filtering rules, in this quick guide I’ll walk you through setting up and then applying a set of Duplicate Detection rules on contacts and then how to actually run the Duplicate Detection followed by removing the duplicates.

Setting up Duplicate Detection Rules

Coming up with the right detection rules can be the most difficult part of trying to remove duplicates, you need something specific enough that you’re only picking up duplicates but general enough that you’re getting as many duplicates as possible, with that said once you’ve agreed on a set of rules I’ll show you below how to easily go about adding them into CRM.

1.  Go to the SETTINGS menu in CRM then select Data Management in the Settings menu.
2. Open Duplicate detection Rules at the top-right of the page.

Unpublish any default rules already in CRM for the entity you’re about to modify, this can be done easily by selecting a rule then clickingUnpublish in the command bar.

3. Click New and a new window will open allowing you to start setting up duplicate detection rules, to start change Base record Type to the entity you’ll apply the rule to, which in my case is Contact.
4. Where it says Select in blue near the bottom of the window add fields which should be used for the duplicate detection then one column to the right select how it should be detected.
5. You should end up with something similar to the image below.
6. Unless you’ve got a reason to search for disabled records also I would suggest making sure that ‘Exclude inactive matching records’ is checked so that no inactive records are included in duplicate detections.
7. Once done click Save & Close to close the window.
8. Select the new rule you created then click Publish in the command bar so it’s activated.

Running a Duplicate Detection Job

Once you’ve decided on and implemented a rule set the next task is to run a duplicate detection job, this can be done in two ways, it can be run from a view displaying the entity to run it on for example select several contacts then the command bar will change to show Duplicate Detection, and it can also be done from the Duplicate Detection options menu.

Running Duplicate Detection from an Entity View

1. Go to the entity you wish to run a Detection on.
2. Select any record in the view if you’ll be scanning all records or select the records you wish to run detection on if you’re only doing a select view.
3. The Command bar will now change to show a new button, click Detect Duplicates.
4. Depending on what you’re doing click either ‘For Selected Records’ or ‘For All Records on All Pages’.
5. A new window will appear similar to the image below, change any details you want to then click OK.
6. Once the detection job has completed you will receive an email, go to your CRM Deployment then go to Data Management again and select the detection job you just ran. I’ll explain the different elements in the Removing Duplicates section below.

Running Duplicate Detection from the Duplicate Detections Jobs Page

1. Once you’re in the Duplicate detection jobs page, click the New button in the command bar.
2. A window similar to below should pop-up, click next.
3. You must now either create a set of filters to return just the records you want or you can select a saved view by changing the ‘Use Saved view’ drop down. Click preview Records at the bottom-left of the window to make sure you’re viewing the right records.
4. When you’re sure you’re returning all the records to be scanned click Next again.
5. The Select Options page has a few options that are self-explanatory, change any details you wish then click Next.
6. Finally click Submit to start the duplicate detection job.
7. Once the detection job has completed you will receive an email, go to your CRM Deployment then go to Data Management again and select the detection job you just ran. I’ll explain the different elements in the Removing Duplicates section below.

Removing Duplicates

1. Once you’ve received an email confirming that a duplicate detection job has completed go to the Duplicate Detection Jobs page which should show a list of your personal jobs by default.
2. Click the name of the latest Duplicate detect you want to view which will open up the Information page for that job.
3. Click View Duplicates in the left-hand navigation to view any/all duplicates that were detected.
4. Providing any duplicates were found it will display the original record in the top-most box and any possible duplicates in the bottom box.
5. To remove a duplicate your options are to merge the records which I would recommend or you may disable the detected duplicates. To do either select the original record and then the duplicate and depending on what you want to do look in the More Actions menu to delete a record or click Merge then Select master.. to merge the records.