• Section Widget

  • Recent Articles

    jsiwila

    Processing Records Rejected by a Read File Operator

    This is a companion article to Processing Rejected Records. That deals with records rejected by the Read Excel operator and applies to Read operators in other expressor Extensions and to the Read Custom operator.

    Parsing records rejected by the Read File operator is simpler than parsing records from Read Excel and related operators. The Read File operator retains the sequence of fields and attributes when it writes them to the RecordData field of the Reject Record Schema. Because of that, it is not necessary for the Read File operator to write header records to describe the field and attribute sequence. That greatly simplifies the datascript required to parse the RecordData field and reconstruct the rejected records.

    As explained in the Processing Rejected Records article, when a Read operator... read more
    jsiwila 05-14-2012, 03:33 PM
    jlifter

    expressor tutorials

    In this section you will find a collection of older tutorials that demonstrate the features of expressor Studio and the entire expressor Data Integration Platform. Though... read more
    jlifter 05-08-2012, 04:08 PM
    jsiwila

    Release Notes - expressor 3.6.0, 3.6.1, 3.6.2, 3.6.3, and 3.6.4

    expressor 3.6.4 fixes four bugs, three in Studio (STU-4728, STU-4730, and STU-4744) and one having to do with reading Informix databases from dataflows running on Linux (PRO-2634). See the resolved issues section below for a description of each of the fixed bugs.

    Also, see STU-4729in the known issues section below. STU-4729 describes an issue encountered when upgrading artifacts in a Repository Workspace.

    Note that the Informix ODBC drivers shipped with expressor software do not support Unicode on Linux.

    expressor 3.6.3 fixes three bug in Studio: one covering memory leaks, another dealing with binding to the Oracle NUMBER data type, and the third corrects unnecessary rounding when converting a double precision value to decimal. See STU-4641, STU-4625, and PRO-2622 under resolved issues below. Also see STU-4657 in the known issues section immediately following for a workaround to a problem dealing with NUMBER... read more
    jsiwila 05-08-2012, 09:02 AM
    jsiwila

    Processing Rejected Records

    When records produce errors because they violate constraints set on Composite Type
    attributes or other reasons, the operator that encounters the error can handle them
    by skipping them, aborting the dataflow, or rejecting the offending records. In some
    cases, it is sufficient to simply send rejected records to a Write File operator
    and examine the records in the output file. If the intent is, however, to correct
    or otherwise use those records, examining each error and changing the data could
    be very cumbersome. The more efficient approach would be to reprocess the records
    as they come out the reject port.

    Records rejected by input operators such as Read File, Read Table, and Read Excel are structured into the following fields:

    RejectType
    RecordNumber
    RecordData
    RejectReason
    RejectMessage

    The record data as it was constituted before being rejected is contained in the RecordData field. To process that data, it must first be reconstructed from the rejected record format. Several factors affect the reconstruction. The order of the record data fields can be different from the order represented in the original Schema, and some of the records emitted from the reject port do not contain record data. For example, RejectType 1 errors are constraint violations, but before they are emitted, a RejectType 4 record is emitted. The RejectType 4 record contains the record data field order for the subsequent RejectType 1 errors in its RecordData field. The RejectTypes are fully explained in the Using the Reject Port section of the Read Custom operator topic in the product documentation.

    Note: All non-input operators that have a reject port emit rejected records with the existing attributes of the record, that is, they do not restructure the records the way input operators do. Reprocessing records rejected by non-input operators do not have to be reconstructed
    ... read more
    jsiwila 05-07-2012, 10:00 AM

  • Advanced RSS Feed

    IP address to country conversion 

    by
    jlifter
    Published on 07-21-2011 05:18 PM
    Article Preview

    This example illustrates how an expressor data integration application can be used to associate an IP address with the name of the country to which the IP address is assigned. For example, you may be collecting the IP addresses of individuals who download documents from your Web site and you want to determine whether the information you are providing is as interesting to European developers as developers in the United States. Fortunately, a list that associates numeric representations of IP addresses to the country of origin is available, so the task resolves to simply converting the IP addresses into their numeric representations and then looking up the country in the list.

    The formula ...
    Read More Read More

    Using lookup tables 

    by
    jlifter
    Published on 09-22-2011 05:55 PM
    content/attachments/294-lookup_table1.png.html

    New in expressor 3.4 are lookup tables, which are implemented by a light-weight relational database management system integrated into the expressor Data Integration Engine. The database stores the lookup table content in a file and depending on the file system location of this file, the scope of the lookup table can be limited to a single project (or library) or to all projects (and libraries) in a workspace, or to all projects (and libraries) deployed onto the computer. In fact, if the file system location of the lookup file is mapped to a drive, the lookup table's contents can be accessed by expressor data integration applications running on networked computers. In developing the examples described in this tutorial, the lookup table files will be stored in a directory (C:\lookups) accessible to all expressor ...
    Read More Read More

    Implementing a type 2 slowly changing dimension application 

    by
    jlifter
    Published on 09-24-2011 09:30 AM
    content/attachments/315-scd_dataflow_step1.png.html

    One question that is frequently raised by attendees in the expressor training course is how to design a drawing that manages a type 2 slowly changing dimension table in a data warehouse. In this type of data integration application, changes to the data are tracked by creating multiple records with different primary key (surrogate key) values. Each record also includes a natural key, which is the column value that identifies the record within the business use case. Consequently, within the dimension table there will be multiple records for a given natural key each of which has a unique surrogate key.

    Since the surrogate key values are not necessarily sequential, the dimension table record will also contain a column, or columns, indicating the most recent entry. For example, a column named VERSION could contain an incrementing version number, or columns named EFFECTIVE_DATE and END_DATE, could be used to indicate which record is current. A historical record ...
    Read More Read More

    Using the pivot row operator 

    by
    jlifter
    Published on 09-22-2011 05:55 PM
    content/attachments/268-pivot_row1.png.html

    The Pivot Row operator converts a single record into multiple records where each of these records contains selected fields from the original record. That is, this operator performs a one-to-many transformation. Using this operator requires no coding; all configuration is performed through a graphical interface.

    Let's suppose you have a file with monthly corporate sales data. The first line in your file is a header and the following lines are sales data for selected companies, as illustrated in the following fragment.

    Code:
    Company,Year,JanRev,FebRev,MarRev,AprRev,MayRev,JunRev,JulRev,AugRev,SepRev,OctRev,NovRev,DecRev
Telco1,2011,1101,1102,1103,1104,1105,1106,1107,1108,1109,1110,1111,1112
    ...
    Read More Read More

    Using the pivot column operator 

    by
    jlifter
    Published on 09-22-2011 04:55 PM
    content/attachments/282-pivot_column1.png.html

    The Pivot Column operator converts multiple records into a single record that contains selected fields from each of the multiple records. That is, this operator performs a many-to-one transformation. Using this operator requires no coding; all configuration is performed through a graphical interface.

    Let's suppose you have a file with monthly corporate sales data. The first line is a header and the following lines are the sales data by month, by year, for selected companies.

    Code:
    Company,Year,RevenueMonth,Revenue
Telco1,2011,JanRev,1101
Telco1,2011,FebRev,1102
Telco1,2011,MarRev,1103
Telco1,2011,AprRev,1104
Telco1,2011,MayRev,1105
Telco1,2011,JunRev,1106
    ...
    Read More Read More
Follow us
 
copyright  |  privacy  |  contact  |  terms of use  |  sitemap
 
Style & Customization by Boston Web Developer
Gravatar as Default Avatar by 1e2.it

SEO by vBSEO 3.6.0