Managing Contacts

There are several ways contacts get added to your database.  You might add them one at a time, they might add themselves through your website, or you might import them in bulk using CiviCRM's import functionality.

Adding contacts

First decide whether your contact is an individual, a household, or an organisation, and then click on the New Individual/Organisation/Household from the CiviCRM home page.

You will be redirected to the contact add form where you can add personal and contact details. You can enter one or more postal addresses, phone numbers and email addresses.  Contact details are identified by location. For example, you may want to record a constituents home phone, mobile phone and work phone.  Click "another phone" to see additional phone fields within each Location.

TIP: Preferred phone numbers and email addresses should appear first and are marked "Preferred".

TIP: Only the location marked primary will be displayed in search results.

Other ways to add contacts

• Via the Quick Add Block - For Joomla users, this always appears in the left sidebar when within CiviCRM. If you are using Drupal and don't see the Quick Add block, you may need to enable it in your Drupal blocks menu. To configure your Drupal blocks go to Administer » Site Building » Blocks. You must fill in at least a First Name and Last Name, or an Email address. If the system detects a potential match based on the Matching Rules it will alert you to this fact at this point. This is covered in more depth later in this chapter. 
  

• Via a profile - you can set up a profile as a data entry portal (read on to learn how to use CiviCRM profiles).  This might be useful if you want to display a simplified data entry screen to some staff, or allow members of the public to add data.
• Via event registration, membership or contributions - when someone interacts with your database through your website. When someone registers for an event, for example, CiviCRM checks to see if they are already on your database using a duplicate matching rule.  If no duplicate is found CiviCRM will create a contact record associated with that interaction.
• By importing them - bringing contact information from an external spreadsheet or database into CiviCRM.
  

Preparing to import data

Importing data is a reasonably involved process and it pays to be familiar with the concepts before you start your first import. You can import both core and custom data for contacts as well as data for event attendances, activities, memberships and contributions.  This chapter will focus solely on the import process for contacts.

There are two ways in which data can be imported:

• From CSV (comma separated value) files.  Each column in your CSV file will map to a field in CiviCRM so make sure you use a different column for every bit of information.  Most database and spreadsheet applications can create and manipulate CSV files.
  
• From another database held on the same server by use of an SQL statement.

Warning: If you do not have a clear understanding of your existing data and how it will map to CiviCRM fields, you will experience frustrations and problems when you try to import the data. Please read other sections of the CiviCRM Manual and visit the CiviCRM online documentation for more information: http://wiki.civicrm.org/confluence/display/CRMDOC/Importing+Data

You may well encounter problems when running the import because every imported data has its own quirks. But you can help cut down on problems by following these recommendations: 

Import tips

• You can add all imported contacts to a new or existing group or tag.  Adding contacts to a new tag enables you to easily identify when a contact was imported and undo an import if necessary. Tagging and grouping contacts is quite limited during import.  You can only assign a contact to one particular group or tag.  One way to get around this is to create custom data fields representing the data you want to create and then converting that custom data to groups after the import using Advanced Search and selecting Add to Group from the more actions menu.
• If your imports are timing out or taking too long, try splitting up imports into smaller parts.
• CiviCRM stores first names and last names in separate fields so these should appear as separate columns in your CSV file.  The same goes for city and postal code / zip code. There are tools for common spreadsheet programmes that automate the process of splitting names across fields.  Google for "Split text among columns by using functions." 
• Do as much data cleansing as possible prior to the import. That is, make sure data has the right type, is represented consistently (is a state shown as California or CA?), and contains no unnecessary fields
• Always test your data import with a small subset of your records. After importing the test set, visit the records within CiviCRM and ensure that the data was imported and functions as you expected
• Consider creating a dummy contact that has every attribute you've defined in your existing data set. Then import the contact and check results to ensure that CiviCRM correctly represents all the data labels and options in your old data
• If the import you are doing is one that you are likely to want to use again for a different import file, CiviCRM allows you to save the the import maps (i.e which field in the csv file should go into which CiviCRM field). You will be required to give it a name. Just use something that makes it obvious (to you) what the particular mapping is for.

• If you are importing multi-choice options (e.g. checkbox or radio etc.) with custom data, you can use either the label or the value of the field the import will accept both the Label and the Value. With core data you can only use label, and not value.
• If you are updating multiple choice options, new values will replace the entire field - not just add values that aren't there, for example, if you update colors to be 'orange', for a contact that currently has colors set to blue, the result will be that colors are set to orange, not orange and blue.
• If you are importing multiple locations, the first location will be set as the primary location.   You may want to move your columns around to ensure the desired Location becomes the Primary. You may also need to split your import so that some records have one type of record as their Primary, while others have a different one.
• Ensure that your country names are expressed in the same way as they are in CiviCRM, i.e. 'United States', not 'United States of America', and 'United Kingdom', not 'England'.
• Make sure you are using an accepted date format (e.g. DD/MM/YY is not accepted but DD/MM/YYYY is)
• Make sure any name prefixes and suffixes you use have been set up in the administration interface (Administer >> Option Lists).

Running an import

The import process has five steps.

STEP 1: SET UP

Set up lets you specify some basic details of your import including the source of the data.  Data can either come from a CSV file, or from an SQL query of a database on your server.

Import uses the default strict rule to decide whether a contact record is a duplicate.  You can specify what action to take when import encounters a duplicate.

• Skip the duplicate contact, i.e. leave the original record as it is.
• Update the original record with the database fields from the import data.  Fields that are not included in the import data will be left as they are.
• Fill in the additional contact data only and leave fields which currently have values as they are.
• No Duplicate Checking. i.e. inserts all valid records without comparing them to existing contact records for possible duplicates.

Import mappings are the way in which you tell CiviCRM how the fields of data in your import file correspond to the fields in CiviCRM.  If you have a pre-existing field mapping, you can load it here.

TIP: if you have a saved mapping, and want to use it, but have a table to import that has some new columns added, then by moving those columns to the end of your spreadsheet, you can reuse the Saved Mapping and simply map the new fields in at the bottom. 

STEP 2: MATCH THE FIELDS

At this stage of the Import you have the option to add the imported contacts to a new or existing Group or Tag. Even if you do not need to do this as part of your outcome, it is worth using this in order to be able to quickly find the imports and if necessary delete and reimport if required.

STEP 4: SUMMARY

The final screen reports on the successful imports, and provides you with reports for Duplicate Contacts and Errors.  If you have set the import to add all contacts to a Group or Tag, you can click through to see your imported contact records.

Deduping and Merging

Duplicates can appear in your data for many reasons including mistakes by members of your staff.  They can also be created when people fill in forms about themselves on your site (e.g. event registrations).  CiviCRM has 'duplicate matching rules' to minimise the chance of this happening but there are still cases when some human intelligence needs to be applied to decide if two entries are the same person or not. F When duplicates do occur, CiviCRM provides a set of tools for finding and merging duplicate contacts.  These are accessed through the admin interface (Administer CiviCRM >> Find and Merge Duplicate Contacts).

You can set up different rules for identifying suspected duplicates.  When you use these rules (by clicking on 'use rule'), suspected duplicates are shown side by side in a table.

Editing Rules

You can configure up to five fields to evaluate when searching for 'suspected' duplicate contact records of a particular contact type. For each field, set a numeric weight which determines the relative importance of a match on that field.

You can also set a length value which determines how many characters in the field should be compared.

EXAMPLE: If you set a length of 2 on 'First name', then 'Mike' would match 'Michael', because the first 2 characters are the same. If length is left blank, then the comparison is done on the entire field value.

Finally, you set a numerical threshold for the rule. When the total of the weights of each match is equal to or greater than this threshold - the contacts will be listed as suspected duplicates by the Find and Merge Duplicate Contacts task.

EXAMPLE: A match based on First = 2, Last = 8, Cell=10 and Threshold

In the example below, the match has been set to first three characters for First and Last name. This will therefore spot potential duplicates such as Pete Davies and Peter Davis. 

 

 

The rule will then return any contacts whose results meet the threshold. 

 

Clicking on merge for any pair of contacts brings up a table showing both sets of details.  For each detail you can choose whether to keep the original data (shown on the right) or copy the duplicate data.

 

If you realise you want to move the information in the opposite direction, there is an option to "Flip between original and duplicate contacts" which will swop the two sets of results over, so can move data from A to B, rather than B to A.

Note associated tags, groups and activity data (including event attendance, contributions, etc.) will appear in addition to data already recorded in the original record, not in place of it.

Please note, you can restrict the search to particular Groups, this is a good option with large datasets to reduce the time each search will take. To do this, make some Smart Groups based on a section of the alphabet eg a group of all surnames beginning with A-D.

Strict rules vs Fuzzy rules

There are two categories of dedupe rules - 'Strict' and 'Fuzzy'. 'Strict' rules should be configured with a relatively tight definition of what constitutes a match (for example... email address + first and last name for Individuals). It is more important with Strict rules that you avoid false matches than that you catch every possible duplicate as it is easier to sort out duplicates later on than it is to disentangle two incorrectly merged contacts. The default 'Strict' rule is automatically used when new contacts are created via the frontend of your website (including contacts created by constituents during online contributions, etc.). They are also used when you import contacts. You can only have one default 'Strict' rule for each contact type.

'Fuzzy' rules should be configured with a looser definition of what constitutes a 'match'. They pick up a wider range of possible matches because they are used in instances where human intelligence can be applied to decide if they are a match. The default Fuzzy rule is automatically used to check for possible duplicates when contacts are added or edited via the user interface. You'll also want to use a Fuzzy rule when scanning your database periodically for possible duplicates. Click Use Rule to scan for duplicate contacts (of that contact type) using the selected rule.