Match Correction and Magic Tool

Added by Susan Bramhall, last edited by Susan Bramhall on Feb 12, 2009  (view change)

Error scenarios and correction guide

Document on share point
A document describing all the different matching and mismatching error scenarios and how to correct them is stored on the IDM sharepoint site at https://collaborate.its.yale.edu/sites/projects/idmanagement/Shared%20Documents/Architecture%20and%20Implementation%20Design/OIM%20Reconciliation%20Process%20v5.doc.

The correction process will use a custom developed web application to ensure the highest probability of correct data and lowest likelihood of additional errors and unintended consequences.

Error Correction Magic Tool

  • The Magic Tool will do all updates requested by the user under JTA transaction control courtesy of JBoss so the data will always be correct to any other process.

Uncertain Match List

Yale's IdM system uses a sophisticated matching algorithm to determine if new records from HR and SIS are

  • exact matches to existing records,
  • definite unmatched new arrivals
  • uncertain matches

The uncertain matches are ignored by the create / update process and saved for later reporting and correction. The process that does the matching is Fuzzy Matching.  As stated on that page the matching process will save the incoming record to Incoming Uncertain Match Table ( names to be determined) in the YU_IDM schema.  Additional tables will contain one or more rows with the key of the incoming record (HR_PERSON_ID or PIDM depending on the source) and all the keys of each possible match (UPIs).

Incoming Uncertain Match Table
HR_PERSON_ID
PIDM
Name fields
SSN and other match fields
HR ID to UPI Possibles
HR_PERSON_ID
UPI
PIDM to UPI Possibles
PIDM
UPI

The incoming record will contain HR_PERSON_ID if coming from HR and will contain PIDM is coming in from SIS. The possible match tables will contain only the key to the incoming and keys to potential OIM user record matches.

The tool will allow the administrator to browse and take action on the Uncertain Match Report.  The tool will allow the user to select a record from those that are currently stalled as uncertain matches.  The tool will display one or more records that are current possible matches.  The administrator will be able to select an option to cause a new record to be created by running a one record create recon.  The alternate choice will be to select a matching record.  In this case the tool will update the matching OIM record with the desired matching key and run the one record recon.  If the matching record already contains a key from the incoming system a match cannot be made.  Instead the duplicate in the source system must be resolved.

The user will select one or two records to act on either forcing the creation of a new record or matching to an existing record.

Screen Mockup:

Updates to OIM using the API

To correct OIM data the tool will allow the administrator to copy HR_PERSON_ID, PIDM between OIM records. 

Merging records to correct a duplicate (Two records become one.)

When the administrator is correcting duplicate records, the tool will show the details of two records.  The administrator must select the record to be retained based on the UPI that is in use I.E. the one on the user's ID card.  The desired values for HR_PERSON_ID, PIDM, Netid and email alias are then moved to the record to be retained from the record to be deleted.  When the data to be retained is correct, the administrator submits the changes.  The tool begins a transaction and the "good" record is updated and the "bad" record is "deleted".  When the transaction is committed provisioning events triggered by the chages take place.

Duplicates always involve changes in the HR system as well.  See Functional documentation.
Screen mockup for correcting duplicates:

Separating data to correct incorrectly linked records (One record becomes two.)

When the administrator is correcting incorrectly linked records, the first step will be to create the new OIM record using the Single Record Recon.  The administrator selects the old record which has data for two people and the newly created record.  The tool allows the administrator to move HR_PERSON_ID, PIDM, Netid and email alias between the records to satisfy the requirements.  Once the user submits the changes for both records, the tool begins a transaction and both records are updated with new data.  When the transaction is committed provisioning events triggered by the changes will take place.

Screen Mockup for unlinking incorrectly matched records:

Updating Keys

Since the HR_Person_id is the mapping attribute for student provisioning into HR adjustments need to be made to student "accounts" when the key is moved from one OIM user to another. To transfer the ownership of the student HR attributes use the tcUsrOperationsIntf to 
changeToServiceAccount
moveServiceAccount
changeFromServiceAccount

This same approach is required for the HR Keys "accounts" and any other accounts where the primary key of the external system may be changed.

Changes to external systems

  • In Phase I the Netid system will remain authoritative for Netids. This means that the correlation between UPI and Netid is maintained outside of OIM. If a Netid needs to be moved or if two must be swapped this action will require writing to the Netid system.
  • Email Alias would have to be changed externally if the primary alias is to be moved.
    Since the ID System feeds the library and the gate security system with data captured only at the time a card is created, we will notify the ID center when we make a change to the association between *UPI, HR_PERSON_ID, PIDM, ID card number. It will be up to them to correct their data if they determine they should. We meed to review this with someone...who
        • Amy & Adriene reviewed with Paul Broniek of the ID Card Center and he advised that the report would be great so that he could proactively update his information. 
  •  
  • From Tim Hinckley
     
    Once the user queries (by Yale ID/SID/SSN, Person ID, NetID, UPI, or Name) and selects a record to be processed, PreAuth will query HR (yuhr_ids_new view) by person ID... if a match is not found, it will pull the data by SSN (Note:  There has been talk about removing the SSN from the ID system, so the SSN check may be replaced with a UPI check).   So, yes, there appears to be no major impacts to PreAuth when the Person ID and/or UPI are changed.
     
    We did note that the changes to the Person ID/UPI will not automatically flow into the ID System... those columns will be refreshed when someone in the ID center issues a replacement card.  This may have an impact on the library and security systems.
     
    When you are ready, we should set up test cases for each section/scenario in your "OIM Reconciliation Process and Issue Resolution" document and test both PreAuth and Tempcard.
     
    >> ID Card System View - Tim will add UPI to the HR view in test for Brian.
     
    I added YALE_UPI to the IDS_ID_ISSUANCE view in the test environment ( IDS2/SFA2 ).
     
    >> Tim will check on what system feeds the library and if the feed contains UPI and/or ID Card Number.
     
    We have a view in the ID System called LIB_CARD_V.  The view was created back in 2005 for the SSN card replacement project.
     
    LIB_CARD_V columns...
     
     Name                 
     ----------------------
     YALE_UPI             
     PERSON_ID            
     LEGACY_YALEID        
     LEGACY_ISSUANCE_CODE 
     CARD_PRINT_DATE      
     
    We also have a view in Banner (SYVLIBE) that has UPI and Legacy YaleID/Issuance Code.
     
    The technical contact is Audrey Novak (email: audrey.novak@yale.edu  phone: 203-432-2365).
     
     
    >> Tim will check on security system to see what feeds they get and determine if
    >> HR_ID is being used grant access or UPI/Card ID Number.
     
    Technical Contact:  Bill Horowitz (email: bill.horowitz@yale.edu  phone: 203-737-6330)
     
    We are passing UPI and Person ID to the security system.  Bill said that changes to the Person ID may impact some of his housekeeping reports and his student fall download.
     
    CH_CASIRUSCO_V columns...
     Name                 
     ----------------------
     CARD_ID_NUM          
     ISO_ID_NUM           
     CARD_HOLDER_ID       
     LAST_NAME            
     FIRST_NAME           
     MIDDLE_NAME          
     SUFFIX                
     AFFILIATION_TEXT     
     LEGACY_YALEID        
     LEGACY_ISSUANCE_CODE 
     SCHOOL               
     COLLEGE              
     CLASS_YEAR           
     RECORD_CREATION_DATE 
     RECORD_CHANGE_DATE   
     UNIX_PHOTO_FILENAME  
     PHOTO_FILENAME       
     PROX_NUM             
     CARD_STATUS_ID       
     CARD_PRINT_DATE      
     DEPENDENT_PERSON_ID  
     BIRTH_DATE           
     SEX                  
     PERSON_ID            
     BANNER_ID            
     CARD_TYPE_ID         
     SSN                  
     USERID                
     LYID_VALUE           
     NET_ID               
     YALE_UPI             
     EXPIRATION_DATE      
     SOURCE               
     AFF_FLAG            
     
     
     
    From: Lohman, Amelia
    Sent: Tuesday, July 08, 2008 10:48 AM
    To: Hinckley, Timothy; Susan Bramhall; Young, Brian
    Cc: Lohman, Amelia; Westgard, Clint
    Subject: IDM: Meeting Minutes 07/08/08 - Impacts to ID Card System when UPI and HR_ID Link Changes Attendees:  Tim Hinckley, Susan Bramhall, Brian Young, Amy Lohman
     
    Meeting Objective:  To determine what impacts there would be to the ID Card system when the link between HR_person_id and UPI is broken (OIM System owns the UPI).  This type of scenario would occur if 2 OIM records were created for the same person from different source system (Duplicate) or if 2 individuals are incorrectly linked and only 1 OIM record is created.  Payment history determines which HR record should be retained and which individual has the ID Card (w/ UPI printed on it) determines which OIM record to retain.  The resolution of some of these mismatches may mean that that the original HR_ID that was attached to the UPI (printed on the ID card) will be changed.
     
     
    Outcome:
     
                At this point there appears to be no impacts to the ID Card system, however, Tim will be checking on data sent to library and security systems to ensure no downstream impacts.
                ID Card System View - Tim will add UPI to the HR view in test for Brian.
                HR Logic Change - Brian will make logic change to match on UPI rather than HR_ID. Then ID Card number will be updated on the correct HR record
                Test data my not be available immediately to test, will need to develop a use case.
     
    Next Steps:
     
                Tim will check on what system feeds the library and if the feed contains UPI and/or ID Card Number.
                Tim will check on security system to see what feeds they get and determine if HR_ID is being used grant access or UPI/Card ID Number.
                Susan & Brian need to determine if contacts need to be sent to OIM because they are assigned UPI's
                Tim will make change to ID Card view for HR to add UPI in test environment
                Brian will work on making logic change in test to accept UPI from ID Card center and match on that rather than HR_ID
     
     
     

Labels