Importing Data

Records from the NEC Complete Members database and the NEC Income Files database were imported to CiviCRM during June, 2010 by Andy Alm (me).

I hit a limit the number of records that can be imported at one time, about 500 or fewer, due to CiviCRM or Dreamhost (or both, in combination).  So all the data that was exported from filemaker to comma-separated values (CSV) had to be broken up into chunks.

Importing contacts

Importing contacts was difficult.  Individuals and organizations had to be imported separately, so first I had to identify which was which.  When both an individual and an organization were in the same record, I needed to import once for the person and once for the organization. These still need work to join the individuals with their organizations.

City and state were all stored in one field in Filemaker, so I had to figure out how to separate them. Some of this could be automated, some was grisly handwork, but it got done.

International and first-class mailings still need atttention.
I didn't even deal with bulk bundles; those will need their own database.

Errors on importing contacts were saved to an Excel file, attached:

Contact_Import_Errors.xls


Importing contributions

Donations were a simple and fast import, since all that was required was the date, MemberID, amount, and donation type.

Errors on import were saved to an Excel file:

contrib_import_errors.xls


I think there is another database containing rebuild-it fund donations that I do not have, so these are not imported yet.

Importing memberships

Memberships had to be imported first as membership records, then separately as contribution records (the payment associated with the membership).  This is due to the way CiviCRM treats memberships separately from contributions (money transactions).

The CSV data was broken up into 10 files with roughly 500 records apiece for importing. each file was imported twice (once to CiviMember and once to CiviContribute).

Errors on import were saved to Excel files:

renewal_import_errors.xls
renewal-contrib_import_errors.xls

Most of the errors on importing member records fell in two categories:

Memberships/membership contributions for which there was no matching contact in the CiviCRM database (matched based on NEC MemberID field).

Memberships/membership contributions for in which the matching contact is an organization, not an individual.

The 517 LIFE members did not necessarily have contributions associated with their memberships, nor is there necessarily any record of when they became life members.  So I imported these separately.  This may result in some folks having -two- life memberships, and being counted twice, so they should be checked. Where there was a date in the First Entered field, I used it.  Otherwise, I used 1/1/2010.