Skip to end of metadata
Go to start of metadata

You are viewing an old version of this content. View the current version.

Compare with Current View Version History

« Previous Version 32 Next »

 Anh’s Status 7/6/2015 – 7/10/2015 (1 week status)

Completed section:

Completed adding stored procedure, latest sql files to https://services.adcom.uci.edu/jira/browse/KFSDW-590

Completed rescheduling on unix, deploy2 to run start running jobs on Sunday nights (was Monday nights) due to kfs running on Sunday.

https://services.adcom.uci.edu/jira/browse/KFSDW-800 

https://services.adcom.uci.edu/jira/browse/KFSDW-801

INC0031493 for Jeric to create oracle_prd linked server on solar.  This is completed.

Austin created new connection stage_kfs_infor on lunar, stage_kfs_infor_prod on solar.

Status on load_budget_fund_groups_fromExcel:  We told them not to convert.  It is not needed since Durendal built a front end for users.

https://services.adcom.uci.edu/jira/browse/KFSDW-804

 

Not Completed section:

Configure jobs to run by file triggers

Set up and configure connection to run informatica staging jobs on lunar to load oracle tables from production to stage_kfs_infor

set up and configure informatica dim, fact jobs to run on lunar to dss schema

set up and configure informatica dim, fact jobs to run on solar to dss_kfs schema

Continue working, debugging with Analytix team

Anh’s Status 6/822/2015 – 7/03/2015

Completed section:

Completed the preliminary test of Informatica Asset fact job.  It failed because it loads way too many more records, inserts records every time the job runs without truncating the table.  The conversion team is working on fixing it.   Will need to continue test on this job as soon as they complete their fix.

Completed https://services.adcom.uci.edu/jira/browse/KFSDW-745

Completed https://services.adcom.uci.edu/jira/browse/KFSDW-744

Completed https://services.adcom.uci.edu/jira/browse/KFSDW-748

Completed testing informatica jobs using queries from Manav.  They passed. 

Created a new tab "UCI Test By Queries" in the spreadsheet InformaticaJobsCompletionStatus_{today}.xlsx to test using queries.  I've found problems in their converted balance, gl entries fact jobs. I  sent them to the conversion team to fix. 

Completed INC0029671: Jeric to copy data from dss_kfs to dss schema.  Plan for informatica jobs to run on production to test. 

Created 13 new jiras (KFSDW-771 throught 783).  Per our conversation,  I’ve assigned most to Bryce, to myself the most difficult ones (balance, gl entries facts)to assign to Bryce to verify informatica jobs by comparing data between Talend and Informatica tables.

Worked with conversion team to debug their jobs:  They used wrong type as real instead of decimal(19,2) for amount columns.

Reopened jira for Jeric to create the sql link server on solar for production https://services.adcom.uci.edu/jira/browse/DWH-1604.  He finished.

INC0030044 Jeric to fix problem not able to see oracle tables in oracle _dev linked server on nebula. 

Added more amount columns to balance, gl entries fact table to test.  Sent problems found to Conversion team.

Cleaned up talenddev01 server to gain up to 7GB of disk space.  It was full, unable the Conversion team to perform their debugging.

wf_staging_cg has error running.  Reported to Conversion team.

Created tables KFS_D_DOC_FISCAL_PERIOD, KFS_D_PROJECT and their primary keys for talend and informatica schemas to get ready for sqldiff tool to run.

Not complete yet section: 

(A Friendly reminder:  UCI off on Friday 7/3.  My vacation day or loose it on Thurs. 7/2) 

Remaining fact jobs to verify: Need to add more tests on gl entries fact, AssetFact, AssetPaymentFact, Account dimension. 

 

Error rendering macro 'jira' : Unable to locate Jira server for this macro. It may be due to Application Link configuration.

Unable to locate Jira server for this macro. It may be due to Application Link configuration.

Unable to locate Jira server for this macro. It may be due to Application Link configuration.

Unable to locate Jira server for this macro. It may be due to Application Link configuration.

Export all talend jobs on new server

ACCOUNT_CG_PRPSL_NBR not being populated in dss_kfs.KFS_D_ACCOUNT    

Anh’s Status 6/8/2015 – 6/19/2015

Highlight:

Completed section:

Completed Talend pager duty week 6/6 - 6/12.

       Completed solving fiscal year end/month end problems in outbound jobs, CAFP and CFSBAL due to duplicated records error.  These jobs ran fine after that.  Data matched the Oracle side. 

       Worked with Analytix daily to solve issues.  We found out that some staging tables in Oracle kfscfg were empty or wiped out preventing them from doing their fact jobs.  Tom R. copied data from stage production to dev.  They had data to complete dimension and fact jobs.  

Completed Unable to locate Jira server for this macro. It may be due to Application Link configuration.

Completed Unable to locate Jira server for this macro. It may be due to Application Link configuration.

Completed Unable to locate Jira server for this macro. It may be due to Application Link configuration.

Completed Unable to locate Jira server for this macro. It may be due to Application Link configuration.

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Not complete yet section: 

Remaining fact jobs to verify: AssetFact, AssetPaymentFact, balance fact 3,4,5 (1,2 finished, matched records count between Informatica and Talend).

Need to work with DBA to restore data to dss schema.  Plan for informatica jobs to run on production to test.

Unable to locate Jira server for this macro. It may be due to Application Link configuration.

  Unable to locate Jira server for this macro. It may be due to Application Link configuration.

           Unable to locate Jira server for this macro. It may be due to Application Link configuration.

           Unable to locate Jira server for this macro. It may be due to Application Link configuration.

           Unable to locate Jira server for this macro. It may be due to Application Link configuration.

           Unable to locate Jira server for this macro. It may be due to Application Link configuration.

Unable to locate Jira server for this macro. It may be due to Application Link configuration.

Export all talend jobs on new server

ACCOUNT_CG_PRPSL_NBR not being populated in dss_kfs.KFS_D_ACCOUNT    

Anh’s Status 5/25/2015 – 6/5/2015

Highlight: 

  • Completed section:

    Push all kfs jobs (1 by 1) back 1 hour:  Unable to locate Jira server for this macro. It may be due to Application Link configuration.

    Completed modifying unix cripts to push back as well to alert when jobs are not done. 

    Created service now ticket INC0025478 to request zena installation on informatica server so that Jarrod's app can create etl file triggers.  Update:  Jason Lin said that there's no more license.  We're replanning this task.

    Provided input to Analytix regarding ods jobs, mysql linked server.

    Resolved discrepancies in ODS jobs with Analytix. 

    Completed integrating tests for all staging jobs which took double amount of time to run.  They're fixing these so that the can complete running nightly within 1-hour window. 

    Created ldap_people,ldap_people_his under schema 'people_m' in MYSQL per Analytix's request.

  • Not complete yet section: 
    Talend pager duty week 6/6 - 6/12.

    Unable to locate Jira server for this macro. It may be due to Application Link configuration.
    Unable to locate Jira server for this macro. It may be due to Application Link configuration.
    Unable to locate Jira server for this macro. It may be due to Application Link configuration.

           Unable to locate Jira server for this macro. It may be due to Application Link configuration.

           Unable to locate Jira server for this macro. It may be due to Application Link configuration.

           Unable to locate Jira server for this macro. It may be due to Application Link configuration.

           Unable to locate Jira server for this macro. It may be due to Application Link configuration.

           Unable to locate Jira server for this macro. It may be due to Application Link configuration.

           Unable to locate Jira server for this macro. It may be due to Application Link configuration.  same as Unable to locate Jira server for this macro. It may be due to Application Link configuration.     

Export all talend jobs on new server

ACCOUNT_CG_PRPSL_NBR not being populated in dss_kfs.KFS_D_ACCOUNT    

Anh’s Status 5/11/2015 – 5/22/2015

 Highlight: 

  • Completed section:

Talend pager duty week 5/9 - 5/15.

Analytix Informatica staging_uci job is now solved, completed. 

They confirmed that they are able to access our Informatica server. 

System DSN Connection was created, inform them.

  Unable to locate Jira server for this macro. It may be due to Application Link configuration.

  Unable to locate Jira server for this macro. It may be due to Application Link configuration.  

Unable to locate Jira server for this macro. It may be due to Application Link configuration.

Unable to locate Jira server for this macro. It may be due to Application Link configuration.

  • Not complete yet section: 

Unable to locate Jira server for this macro. It may be due to Application Link configuration.

Unable to locate Jira server for this macro. It may be due to Application Link configuration.

Unable to locate Jira server for this macro. It may be due to Application Link configuration.

Talked to Jarrod to have trigger files created on the Informatica server (Windows) for Informatica jobs to run for month-end.  He mentioned that the best approach is to have Zena agent install on this server so his app can tell Zena to create files there.  We need to discuss this more as soon as Analytix finishes their tests with smsfeed jobs which they already imported in our repository.

Grace mentioned that she needs to have table, columns in MySQL.

Conversion:  on-going task.

Providing MySQL connections to Analytix as soon as DBA created them.

Export all talend jobs on new server

ACCOUNT_CG_PRPSL_NBR not being populated in dss_kfs.KFS_D_ACCOUNT 

Anh’s Status 4/27/2015 – 5/08/2015

 Highlight: 

  • Completed section:

    Competed Unable to locate Jira server for this macro. It may be due to Application Link configuration.
    Completed: Unable to locate Jira server for this macro. It may be due to Application Link configuration.
    Completed Unable to locate Jira server for this macro. It may be due to Application Link configuration.
    Completed sending DDL script to create ldap_peole table to Analytix
    Completed Unable to locate Jira server for this macro. It may be due to Application Link configuration.

    Informatica Conversion:  Analytix sent the first job, staging_uci.   This job encountered errors (timestamp type columns, non-existing tables in kfscfg).  After debugging it, I sent them the errors with suggestions to resolve.

    Updated Unable to locate Jira server for this macro. It may be due to Application Link configuration.

  • Not complete yet section: 

Unable to locate Jira server for this macro. It may be due to Application Link configuration.

Informatica Conversion:  ongoing task. 

Talend pager duty next week 5/9 - 5/15

Export all talend jobs on new server

ACCOUNT_CG_PRPSL_NBR not being populated in dss_kfs.KFS_D_ACCOUNT 

Anh’s Status 4/13/2015 – 4/24/2015

 Highlight: 

  • Completed section:

1) Completed fixing staging krns job Unable to locate Jira server for this macro. It may be due to Application Link configuration.

2) Pager duty this week. 

3)  Completed Unable to locate Jira server for this macro. It may be due to Application Link configuration.

     Completed setting up to schedule this job to run nightly.

 

4)  Emailed ddls scripts to Analytix per their request to create tables below:

MYSQL Tables : title_db2

 SQLSERVER Tables:      UCI_LD_CSF_TRACKER_SNAPSHOT_T, employee_basic, agingreport_det_kfs, agingreportcon_kfs

To protect sensitive data in the file, I used webfiles with password to send agingreportdet.dat 

5)  Completed  Unable to locate Jira server for this macro. It may be due to Application Link configuration.

6) Completed new job syncInvoiceNumberSundryDebtors for   Unable to locate Jira server for this macro. It may be due to Application Link configuration.

7) Completed the change to sd-nebula to remaining talend jobs

  • Not complete yet section: 

1)  Make changes to new host sd-nova

2)  Export all talend jobs on new server

3)  ACCOUNT_CG_PRPSL_NBR not being populated in dss_kfs.KFS_D_ACCOUNT

4)  Change host name to ods talend jobs to replace Nebula

5)  Need to fix  Unable to locate Jira server for this macro. It may be due to Application Link configuration.

6)  Need to schedule syncInvoiceNumberSundryDebtors to run on production

 

Anh’s Status 3/28/2015 – 4/10/2015

 Highlight: 

  • Completed section:

1)  Fixed uciSmsGIFeed.  ods_kfs.UCI_SMS_FEED_DETAIL_T table now has all records for Feb. data with batch date ’06-MAR-15’, matching Oracle side:  31,647 records, debit amount total is $3,647,165,482.52 Unable to locate Jira server for this macro. It may be due to Application Link configuration.

2)  dwhreader: 

    • Completed ap,ar,cm,fp,fs,kr,krew,krim,ld,pdp,pur,sh,tem,uci 
    • Completed staging_uc:  Talend is buggy that it loads uci tables not just uc (without the i)  when using UC_%.  Talend sucks.  The syntax to make the staging_uc job work is:
      "1 = 2) union select table_name as tname from all_tables where (owner = '" + context.KFSINT1_Schema  + "' and TABLE_NAME like 'UC_%' and TABLE_NAME not like 'UC_HOME_DEPT_T' and TABLE_NAME not like 'UC_ACCT_NBR_GEN_T' and TABLE_NAME not like 'UCI_%'"
    • Completed  OLTP jobs: utilities_award_data_to_dwhm_fs, utilities_award_data_to_dwhm_ods, utilities_ldap_all_tables, utilities_CostSharing, utilities_webservices, copy_ods_data_dss, copy_dss_kfs_data_to_dss, copy_dss_data_to_dss_2, prod_verify_kfs_data, kill_long_suspended_queries, uciLoadTitle
    • Completed ODS jobs: ods_ca, ods_pur_po, prod_ods_etl_and_accounting, ods_kfs_pcard, ods_kfs_tem, ods_kfs_pdp, ods_kbm_employee_snapshot, ods_kfs_ldap_people_employee_basic, Outbound_External_Interface_Table_ETL, uciBud100PopulateTablesJob,uciStf100PopulateTablesJob, uciStf200PopulateTablesJob, uciUcopCafpJob, uciUcopCfsbalJob, uciUcopEfa100Job, uciSmsGlFeedJob, uciUcopEfa200Job

3)  Staging_TEM job ran fine after Warren cleaned up in Oracle side.

4)  Received ODBC installed on PC.  Informatica can connect to Oracle production

5)  Analytix:  Provided our versions of our DBs that they asked.  They confirmed access to talenddev server.  Attended weekly status meeting this Friday morning.  They're going to send me a list of sqlsver tables so I can send them DDLs to them to create these tables.

6)  uciSmsGIFeed:  The job loaded month-end data this month without problem.  FYI -  Following day, this was the 2nd time this month that Jarrod reran, regenerated the trigger.  The job ran fine 2nd time. 

7)  Completed Unable to locate Jira server for this macro. It may be due to Application Link configuration.

  • Not complete yet section: 

    1)  Staging krns job loaded all tables but Talend sent false notification.  I need to debug this job.
    2)  Export all talend jobs on new server
    3)  ACCOUNT_CG_PRPSL_NBR not being populated in dss_kfs.KFS_D_ACCOUNT

          4)  Change host name to ods talend jobs to replace Nebula

5)  SBSO:  Unable to locate Jira server for this macro. It may be due to Application Link configuration. with the newest revision to the re-mapping logic:  When I came back from my 1-day vacation (due to many accumulated vacation hours, use it or loose it), I was told that we hold off on this task until further direction. 

6) Pager duty next week. 

 

Anh’s Status 3/16/2015 – 3/27/2015

 

Highlight: 

  • Completed section:

1)  Fixed prod_all_ldap_tables by increasing length of LAST_UPDATE_TS from 23 to 30. 

2)  Fixed doc details job from failing due to error "java.lang.OutOfMemoryError: GC overhead limit exceeded".   I increased to 15G.  Job ran fine.  Data is up-to-date.

3)  dwhreader: 

a)  Completed dev_AssetDimensionMaster_creates_trigger

b)  The staging_cg job failed at the iterate in toraclelist component due to tables as synonyms in dwhreader.   Thanks Valerie.  I put in the fix.  This job now ran fine.   Completed.

c)  Completed dev_dim_account_master_creates_trigger.

d)  Completed dev_staging_ca has the new change.

 

4)  JIRA KFSDW-612:  Users confirmed the fix.  It's now closed. 

5)  Fixed prod_dim_asset_master, prod_dim_account_master by using the older version in tac.  The cause of the jobs failed is unknown, and is under investigation.  The only change made was in tOracleList component in the staging_cg job in dev for talend to see dwhreader schema.   To solve this problem, I used the previous version before the change in tOracleList.  The job has been running fine without problem. 

 6)  Created JIRA,  linked with JIRA  

7)  Debugged prod_staging_krew job that failed on production last night.  I’m in the process of debugging all 50+ krew tables, 1 by 1. So far loaling table KREW_DOC_HDR_EXT_DT_T from kfsprd caused error "java.sql.SQLException: Only dates between January 1, 1753 and December 31, 9999 are accepted.".   Some records that with value of VAL (only date column in this table) has date that is out-of-range mentioned above.  I brought this up to Warren's attention.  He deleted these records.  The nightly job ran fine after that.

8)  Staging_TEM job failed with date out-of-range error.  Warren identified the mistyped date from users, said that he will clean up data this week end.

  • Not complete yet section:
        
          1)  dwhreader: 
        
                    a)       staging_uc has Incorrect syntax near ')' after trying many different      syntaxes.
                    b)       Remaining staging jobs:      ap,ar,cm,fp,fs,kr,krew,krim,krns,ld,pdp,pur,sh,tem,uci 
        
         2)  uciSmsGIFeed: The request was not approved for adding the time      next to the date to Jarrod's JIRA kfsi-11864.    I still      need to proceed with creating an additional key using batch_date in order      to load the Feb data.  Steps required are:  Move all data      from 'ods_kfs.UCI_SMS_FEED_DETAIL_T to a temp table before dropping      it in order to add another key to this table.  Move data from the      temp table back to it, including data for Feb that was saved in      [dss_kfs].[ods_kfs].[z_temp_from_oracle_UCI_SMS_FEED_DETAIL_T]
        
         3)  Export all talend jobs on new server
        
         4)  ACCOUNT_CG_PRPSL_NBR not being populated in dss_kfs.KFS_D_ACCOUNT

          5)  SBSO:  with the newest revision to the re-mapping logic

  6)  Change host name to ods talend jobs to replace Nebula

 

 

 

 

 

 

 

 

 

  • No labels