This is a companion article to Processing Rejected Records. That deals with records rejected by the Read Excel operator and applies to Read operators in other expressor Extensions and to the Read Custom operator.
Parsing records rejected by the Read File operator is simpler than parsing records from Read Excel and related operators. The Read File operator retains the sequence of fields and attributes when it writes them to the RecordData field of the Reject Record Schema. Because of that, it is not necessary for the Read File operator to write header records to describe the field and attribute sequence. That greatly simplifies the datascript required to parse the RecordData field and reconstruct the rejected records.
As explained in the Processing Rejected Records article, when a Read operator... read more
In this section you will find a collection of older tutorials that demonstrate the features of expressor Studio and the entire expressor Data Integration Platform. Though... read more
expressor 3.6.4 fixes four bugs, three in Studio (STU-4728, STU-4730, and STU-4744) and one having to do with reading Informix databases from dataflows running on Linux (PRO-2634). See the resolved issues section below for a description of each of the fixed bugs.
Also, see STU-4729in the known issues section below. STU-4729 describes an issue encountered when upgrading artifacts in a Repository Workspace.
Note that the Informix ODBC drivers shipped with expressor software do not support Unicode on Linux.
expressor 3.6.3 fixes three bug in Studio: one covering memory leaks, another dealing with binding to the Oracle NUMBER data type, and the third corrects unnecessary rounding when converting a double precision value to decimal. See STU-4641, STU-4625, and PRO-2622 under resolved issues below. Also seeSTU-4657 in the known issues section immediately following for a workaround to a problem dealing with NUMBER... read more
When records produce errors because they violate constraints set on Composite Type
attributes or other reasons, the operator that encounters the error can handle them
by skipping them, aborting the dataflow, or rejecting the offending records. In some
cases, it is sufficient to simply send rejected records to a Write File operator
and examine the records in the output file. If the intent is, however, to correct
or otherwise use those records, examining each error and changing the data could
be very cumbersome. The more efficient approach would be to reprocess the records
as they come out the reject port.
Records rejected by input operators such as Read File, Read Table, and Read Excel are structured into the following fields:
The record data as it was constituted before being rejected is contained in the RecordData field. To process that data, it must first be reconstructed from the rejected record format. Several factors affect the reconstruction. The order of the record data fields can be different from the order represented in the original Schema, and some of the records emitted from the reject port do not contain record data. For example, RejectType 1 errors are constraint violations, but before they are emitted, a RejectType 4 record is emitted. The RejectType 4 record contains the record data field order for the subsequent RejectType 1 errors in its RecordData field. The RejectTypes are fully explained in the Using the Reject Port section of the Read Custom operator topic in the product documentation.
Note: All non-input operators that have a reject port emit rejected records with the existing attributes of the record, that is, they do not restructure the records the way input operators do. Reprocessing records rejected by non-input operators do not have to be reconstructed
Constraints are business rules that are applied at the level of an attribute in a composite type. Whenever you are working with a composite type, whether it is within the Output Type Editor of the Transform Editor or in the Type Editor,
...
The SQL Query operator allows you to execute any SQL SELECT statement (not a stored procedure) against a database. The advantages of using this operator in place of the Read Table operator is that you can specify
...
I have just posted a companion piece to the article I wrote last week on how to process rejected records. Last week I dealt with the more complicated case of processing records rejected by the Read...
[LEFT][SIZE=3][FONT=arial]On the phone or on the road, I have been positioning expressor solutions that solve many data and application integration challenges. On a...
Section Widget
Recent Articles
Recent Blog Posts
Processing Records Rejected by a Read File Operator
Parsing records rejected by the Read File operator is simpler than parsing records from Read Excel and related operators. The Read File operator retains the sequence of fields and attributes when it writes them to the RecordData field of the Reject Record Schema. Because of that, it is not necessary for the Read File operator to write header records to describe the field and attribute sequence. That greatly simplifies the datascript required to parse the RecordData field and reconstruct the rejected records.
As explained in the Processing Rejected Records article, when a Read operator... read more