Informatica - Java Transformation

This article explains the use of Java Transformation with an example.

We have input data as below.

And need to transform data in such a way that it creates 5 different records with EMI Sequence Number, due date and the outstanding or balance loan amount.

This transformation can be active or passive. When we think about transforming single record to multiple records, the first transformation comes to our mind is Normalizer Transformation. But for Normalizer transformation, a single record can be converted to fixed number of output records (Please refer my earlier post about Normalizer Transformation). In the example, different customers will have different tenure for loans. Lets see how this can be achieved with Java transformation.

Java Transformation: The ouput ports for the java transformation is created manually and uncheck the Input Check Box.

Below are the settings for the transformation.

And here is the java code written to transform the data under "On Input Row" tab. Code written under this tab will take each record and transform it.

Java Code

try
{
  DateFormat formatter = new SimpleDateFormat("dd/MM/yyyy");
  Calendar cal = Calendar.getInstance();
  int Yr, Mon;
  String DtStr;
  int loan_tenure;
  BigDecimal loan_amount, emi_amount;
   
  Date loan_st_dt = (Date)formatter.parse(LOAN_START_DT); 
  cal.clear();
  cal.setTime(loan_st_dt);
  cal.add(Calendar.MONTH, 1);
  Yr = cal.get(Calendar.YEAR);
  Mon = cal.get(Calendar.MONTH)+1;
  DtStr = "01/" + Mon +"/" + Yr;
  loan_st_dt = (Date)formatter.parse(DtStr);
  loan_tenure = 1;
  loan_amount = LOAN_AMT;
  emi_amount = LOAN_AMT.divide(new BigDecimal(LOAN_TNR), 6, RoundingMode.HALF_UP);

  do
  {
     loan_amount=loan_amount.subtract(emi_amount); 
     O_CUST_ID = CUST_ID;
     O_CUST_NM = CUST_NM;
     O_EMI_AMT = emi_amount;
     O_EMI_NUM = loan_tenure;
     O_EMI_DUE_DT = formatter.format(loan_st_dt);
     O_OS_EMI_AMT = loan_amount;
     generateRow();
     loan_tenure = loan_tenure +1;
     cal.clear();
     cal.setTime(loan_st_dt);
     cal.add(Calendar.MONTH, 1);
     loan_st_dt = (Date)cal.getTime();
  }while(loan_tenure <= LOAN_TNR);
}
catch (Exception e)
{
  System.out.println(e+" This is a wind message. - "+e.getMessage());
}

Expression Transformation: It converts the string to date.

Issues You might face are below:
Issue Snap:

Resolution:
Enable High Precision in Java Transformation and in the session properties.

Dowload Workflow

Thanks for reading my article !

Read More

Informatica - Error: FnName: Execute -- [Informatica][ODBC SQL Server Wire Protocol driver]Inconsistent descriptor information.

This article tries to find solution for the below error scenario in Informatica.

Error Message:

WRT_8229    Database errors occurred:

FnName: Execute -- [Informatica][ODBC SQL Server Wire Protocol driver]Inconsistent descriptor information.

FnName: Execute -- [DataDirect][ODBC lib] Function sequence error

Error Scenario: When SMALLDATETIME column from source is moved to SMALLDATETIME column at target, all the records will be rejected at target.

Database: Both source and target are MS SQL Server 2008 database.

Solution:

1. Convert the SMALLDATETIME column to string using Expression Transformation. (SMALLDATETIME datatype length is 19 and the format is 'YYYY-MM-DD 24HH:MI:SS')

SUBSTR(DATE_COL, 1, 19) 

2.Change the Target Definition of the SMALLDATETIME column from SMALLDATETIME to VARCHAR(19) and keep the database table definition as it is.

3. Connect the converted date from expression to target and the issue is resolved.

It worked for me. You can also try the same !

Read More

Informatica - Normalizer Transformation Example

This article explains the use of Normalizer Transformation with an example.

We have source data as below.

And want output as below.

Normalizer Transformation: In this example, There are 4 different expense columns and it needs to be normalized. So define the output ports in the "Normalizer Tab" and set the "Occurs" to 4, since we have 4 columns to be normalized.

Expression Transformation: The "GK_" column (1, 2, 3, 4.....) is the sequence number for the normalized records and "GCID_" is the ordinal sequence of expense columns from the source (Here it is 1, 2, 3 and 4). Based on the "GCID_" value the Expense Type is calculated with DECODE function.

Download Workflow

Thanks for reading this article !

Read More

Informatica - Splitting a Flat File based on a column using Transaction Control

This article explains how to split records based on a particular column into multiple flat files using Transaction Control Transformation. Below given the Mapping and transformation Snapshots.

Mapping Snapshot

Sorter Transformation: Sorts the records based on column that used to split the file. In this example, it is JOB column.

Expression Transformation: After sorting based on JOB column. The first job column will be marked as '1' and rest marked as '0'. This helps the Transaction control to decide where to commit. It also calculates the file name for the group.

Transaction Control Transformation: It continue the transaction when the flag is 0 and commits the transaction before on 1.

Target Definition: Add the file name using the "Add Filename column to this table" button in the taget definition.

Download Workflow

Hope this helped you and thanks for reading this article !

Read More

MySQL - String Functions

Hi Everyone,

Here are the some of the string functions in MySQL . Before we start we should keep below points in mind.

• String functions return NULL, if the length of the result would be greater than the value of the max_allowed_packet system variable.
• For functions that operate on string positions, the first position is numbered 1.
• For functions that take length arguments, non-integer arguments are rounded to the nearest integer.

ASCII(str) 

It returns the numeric value of the leftmost character of the string str. Returns 0 if str is the empty string. Returns NULL if str is NULL.

mysql> SELECT ASCII('2');
+------------+
| ASCII('2') |
+------------+
|              50 |
+------------+

mysql> SELECT ASCII(2);
+----------+
| ASCII(2) |
+----------+
|            50 |
+----------+

mysql> SELECT ASCII('dx');
+-------------+
| ASCII('dx') |
+-------------+
|              100 |
+-------------+

BIN(N)
It returns a string representation of the binary value of N, where N is a longlong (BIGINT) number. This is equivalent to CONV(N,10,2). Returns NULL if N is NULL.

mysql> SELECT BIN(12);
+----------+
| BIN(12)  |
+----------+
| 1100        |
+----------+

BIT_LENGTH(str)
It returns the length of the string str in bits.

mysql> SELECT BIT_LENGTH('Tech Volcano');
+------------------------------------+
|   BIT_LENGTH('Tech Volcano') |
+------------------------------------+
|                                                   96 |
+------------------------------------+

CHAR(N,... [USING charset_name])
This function interprets each argument N as an integer and returns a string consisting of the characters given by the code values of those integers. NULL values will be skipped.

mysql> SELECT CHAR(84,101,99,104,32,86,111,108,99,97,110,'111');
+---------------------------------------------------------+
| CHAR(84,101,99,104,32,86,111,108,99,97,110,'111') |
+---------------------------------------------------------+
| Tech Volcano                                                               |
+---------------------------------------------------------+

mysql> SELECT CHAR(111,111.5,111.7,'111.2');
+---------------------------------+
| CHAR(111,111.5,111.7,'111.2')  |
+---------------------------------+
| oppo                                          |
+---------------------------------+

CHAR_LENGTH(str) / CHARACTER_LENGTH(str)
It returns the length of the string str, measured in characters. A multi-byte character counts as a single character. This means that for a string containing five 2-byte characters, LENGTH() returns 10, whereas CHAR_LENGTH() returns 5.

mysql> SELECT CHARACTER_LENGTH('Tech Volcano');
+----------------------------------------------+
| CHARACTER_LENGTH('Tech Volcano') |
+----------------------------------------------+
|                                                                  12 |
+----------------------------------------------+

CONCAT(str1,str2,...) It concatenates all the arguments and produce the result. If all arguments are non-binary strings, the result is a non-binary string. If the arguments include any binary strings, the result is a binary string. A numeric argument is converted to its equivalent binary string form. It returns NULL if any argument is NULL.

mysql> SELECT CONCAT('Tech',' ','Volcano');
+--------------------------------+
| CONCAT('Tech',' ','Volcano') |
+--------------------------------+
| Tech Volcano                          |
+--------------------------------+

CONCAT_WS(separator,str1,str2,...)
It stands for Concatenate With Separator and is a special form of CONCAT(). The first argument is the separator for the rest of the arguments. The separator is added between the strings to be concatenated. The separator can be a string, as can the rest of the arguments. If the separator is NULL, the result is NULL.

mysql> SELECT CONCAT_WS(' ','Tech','Volcano');
+--------------------------------------+
| CONCAT_WS(' ','Tech','Volcano') |
+--------------------------------------+
| Tech Volcano                                  |
+------------------------------- ------+

ELT(N,str1,str2,str3,...)
It returns the Nth element of the list of strings: str1 if N = 1, str2 if N = 2, and so on. Returns NULL if N is less than 1 or greater than the number of arguments.

mysql> SELECT ELT(5,'Anything','and','Everything','in','IT');
+------------------------------------------------+
| ELT(5,'Anything','and','Everything','in','IT') |
+------------------------------------------------+
| IT                                                                     |
+------------------------------------------------+

FIELD(str,str1,str2,str3,...)
Returns the index (position) of str in the str1, str2, str3, ... list. Returns 0 if str is not found.

mysql> SELECT FIELD('IT','Anything','and','Everything','in','IT');
+-----------------------------------------------------+
| FIELD('IT','Anything','and','Everything','in','IT') |
+-----------------------------------------------------+
|                                                                               5 |
+-----------------------------------------------------+

FORMAT(X,D)
It formats the number X to a format like '9,999,999.99', rounded to D decimal places, and returns the result as a string. If D is 0, the result has no decimal point or fractional part.

mysql> SELECT FORMAT(12345.123456, 3);
+--------------------------------+
| FORMAT(12345.123456, 3)   |
+--------------------------------+
| 12,345.123                               |
+--------------------------------+

mysql> SELECT FORMAT(12345.1,4);
+-------------------------+
| FORMAT(12345.1,4)   |
+-------------------------+
| 12,345.1000                 |
+-------------------------+

mysql> SELECT FORMAT(12345.2,0);
+-------------------------+
| FORMAT(12345.2,0)   |
+-------------------------+
| 12,345                           |
+-------------------------+

HEX(str)/ HEX(N) /UNHEX(HxN)
For a string argument str, HEX() returns a hexadecimal string representation of str where each character in str is converted to two hexadecimal digits. The inverse of this operation is performed by the UNHEX() function.
For a numeric argument N, HEX() returns a hexadecimal string representation of the value of N treated as a longlong (BIGINT) number. This is equivalent to CONV(N,10,16). The inverse of this operation is performed by CONV(HEX(N),16,10).

mysql> SELECT 0x5465636820566F6C63616E6F, HEX('Tech Volcano'), UNHEX(HEX('Tech Volcano'));
+-------------------------------------+----------------------------------+------------------------------------+
| 0x5465636820566F6C63616E6F  | HEX('Tech Volcano')                | UNHEX(HEX('Tech Volcano')) |
+-------------------------------------+----------------------------------+------------------------------------+
| Tech Volcano                                 | 5465636820566F6C63616E6F | Tech Volcano                                |
+-------------------------------------+----------------------------------+------------------------------------+

mysql> SELECT HEX(255), CONV(HEX(255),16,10);
+------------+----------------------------+
| HEX(255) | CONV(HEX(255),16,10) |
+------------+----------------------------+
| FF              | 255                                   |
+------------+----------------------------+

INSERT(str,pos,len,newstr)
It returns the string str, with the substring beginning at position pos and len characters long replaced by the string newstr. Returns the original string if pos is not within the length of the string. Replaces the rest of the string from position pos if len is not within the length of the rest of the string. Returns NULL if any argument is NULL.

mysql> SELECT INSERT('Tec-----cano', 4, 5, 'h Vol');
+------------------------------------------+
| INSERT('Tec-----cano', 4, 5, 'h Vol')   |
+------------------------------------------+
| Tech Volcano                                         |
+------------------------------------------+

mysql> SELECT INSERT('Tec-----cano', -1, 5, 'h Vol');
+------------------------------------------+
| INSERT('Tec-----cano', -1, 5, 'h Vol')  |
+------------------------------------------+
| Tec-----cano                                          |
+------------------------------------------+

mysql> SELECT INSERT('Tec-----cano', 4, 50, 'h Vol');
+------------------------------------------+
| INSERT('Tec-----cano', 4, 50, 'h Vol') |
+------------------------------------------+
| Tech Vol                                                 |
+------------------------------------------+

INSTR(str,substr)
It returns the position of the first occurrence of substring substr in string str.

mysql> SELECT INSTR('Tech Volcano', 'vol');
+--------------------------------+
| INSTR('Tech Volcano', 'vol') |
+--------------------------------+
|                                              6 |
+--------------------------------+

mysql> SELECT INSTR('Tech Volcano', 'Voll');
+---------------------------------+
| INSTR('Tech Volcano', 'Voll') |
+---------------------------------+
|                                               0 |
+---------------------------------+

LCASE(str) / LOWER()
It returns the string str in lowercase.

mysql> SELECT LCASE('TECH VOLCANO');
+---------------------------------+
| LCASE('TECH VOLCANO')   |
+---------------------------------+
| tech volcano                            |
+---------------------------------+
mysql> SELECT LOWER('TECH VOLCANO');
+---------------------------------+
| LOWER('TECH VOLCANO') |
+---------------------------------+
| tech volcano                            |
+----------------------------- ---+

LEFT(str,len)
It returns the leftmost len characters from the string str, or NULL if any argument is NULL.

mysql> SELECT LEFT('TECH VOLCANO', 6);
+---------------------------------+
| LEFT('TECH VOLCANO', 6)  |
+---------------------------------+
| TECH V                                    |
+---------------------------------+

LPAD(str,len,padstr)
It returns the string str, left-padded with the string padstr to a length of len characters. If str is longer than len, the return value is shortened to len characters.

mysql> SELECT LPAD('Tech Volcano',15,'?');
+-------------------------------+
| LPAD('Tech Volcano',15,'?') |
+-------------------------------+
| ???Tech Volcano                   |
+-------------------------------+

mysql> SELECT LPAD('Tech Volcano',10,'?');
+---------------------------------+
| LPAD('Tech Volcano',10,'?')    |
+---------------------------------+
| Tech Volca                                |
+---------------------------------+

LTRIM(str)
It returns the string str with leading space characters removed.

mysql> SELECT LTRIM('  Tech Volcano');
+-----------------------------+
| LTRIM('  Tech Volcano')  |
+-----------------------------+
| Tech Volcano                     |
+-----------------------------+

MAKE_SET(bits,str1,str2,...)
It returns a set value (a string containing substrings separated by “,” characters) consisting of the strings that have the corresponding bit in bits set. str1 corresponds to bit 0, str2 to bit 1, and so on. NULL values in str1, str2, ... are not appended to the result.

mysql> SELECT MAKE_SET(1,'a','b','c');
+--------------------------+
| MAKE_SET(1,'a','b','c') |
+--------------------------+
| a                                     |
+--------------------------+

mysql> SELECT MAKE_SET(1 | 4,'a','b','c');
+-------------------------------+
| MAKE_SET(1 | 4,'a','b','c')   |
+-------------------------------+
| a,c                                          |
+-------------------------------+

mysql> SELECT MAKE_SET(1 | 4,'a','b',NULL,'d');
+--------------------------------------+
| MAKE_SET(1 | 4,'a','b',NULL,'d') |
+--------------------------------------+
| a                                                       |
+--------------------------------------+

mysql> SELECT MAKE_SET(0,'a','b','c');
+--------------------------+
| MAKE_SET(0,'a','b','c') |
+--------------------------+
|                                         |
+--------------------------+

MID(str,pos,len)
MID(str,pos,len) is a synonym for SUBSTRING(str,pos,len).

mysql> SELECT MID('Tech Volcano',3,5);
+----------------------------+
| MID('Tech Volcano',3,5) |
+----------------------------+
| ch Vo                                |
+----------------------------+

OCT(N)
It returns a string representation of the octal value of N, where N is a longlong (BIGINT) number. This is equivalent to CONV(N,10,8). Returns NULL if N is NULL.

mysql> SELECT OCT(12);
+------------+
| OCT(12)    |
+------------+
| 14              |
+------------+

OCTET_LENGTH()
It is a synonym for LENGTH().

mysql> SELECT OCTET_LENGTH(14);
+--------------------------+
| OCTET_LENGTH(14) |
+--------------------------+
|                                     2 |
+--------------------------+

QUOTE(str)
Quotes a string to produce a result that can be used as a properly escaped data value in an SQL statement. The string is returned enclosed by single quotation marks and with each instance of backslash (“\”), single quote (“'”), ASCII NUL, and Control+Z preceded by a backslash. If the argument is NULL, the return value is the word “NULL” without enclosing single quotation marks.

mysql> SELECT 'Don\'t!';
+-----------+
| Don't!       |
+-----------+
| Don't!       |
+-----------+

mysql> SELECT QUOTE('Don\'t!');
+---------------------+
| QUOTE('Don\'t!')  |
+---------------------+
| 'Don\'t!'                  |
+---------------------+

REPEAT(str,count)
It returns a string consisting of the string str repeated count times. If count is less than 1, returns an empty string. Returns NULL if str or count are NULL.

mysql> SELECT REPEAT('#', 5);
+------------------+
| REPEAT('#', 5)  |
+------------------+
| #####               |
+------------------+

REPLACE(str,from_str,to_str)
It returns the string str with all occurrences of the string from_str replaced by the string to_str. REPLACE() performs a case-sensitive match when searching for from_str.

mysql> SELECT REPLACE('w.techvolcano.co.in', 'w', 'www');
+--------------------------------------------------+
| REPLACE('w.techvolcano.co.in', 'w', 'www')  |
+--------------------------------------------------+
| www.techvolcano.co.in                                     |
+--------------------------------------------------+

REVERSE(str)
It returns the string str with the order of the characters reversed.

mysql> SELECT REVERSE('onacloV hceT');
+------------------------------+
| REVERSE('onacloV hceT') |
+------------------------------+
| Tech Volcano                       |
+------------------------------+

RIGHT(str,len)
It returns the rightmost len characters from the string str, or NULL if any argument is NULL.

mysql> SELECT RIGHT('Tech Volcano', 4);
+------------------------------+
| RIGHT('Tech Volcano', 4) |
+------------------------------+
| cano                                     |
+------------------------------+

RPAD(str,len,padstr)
It returns the string str, right-padded with the string padstr to a length of len characters. If str is longer than len, the return value is shortened to len characters.

mysql> SELECT RPAD('Volcano',10,'*');
+-------------------------+
| RPAD('Volcano',10,'*') |
+-------------------------+
| Volcano***                    |
+-------------------------+

mysql> SELECT RPAD('Volcano',6,'*');
+------------------------+
| RPAD('Volcano',6,'*') |
+------------------------+
| Volcan                         |
+------------------------+

RTRIM(str)
It returns the string str with trailing space characters removed.

mysql> SELECT RTRIM('Tech Volcano     ');
+-------------------------------+
| RTRIM('Tech Volcano     ')  |
+-------------------------------+
| Tech Volcano                        |
+-------------------------------+

SOUNDEX(str)
It returns a soundex string from str. Two strings that sound almost the same should have identical soundex strings. A standard soundex string is four characters long, but the SOUNDEX() function returns an arbitrarily long string. You can use SUBSTRING() on the result to get a standard soundex string. All nonalphabetic characters in str are ignored. All international alphabetic characters outside the A-Z range are treated as vowels.
expr1 SOUNDS LIKE expr2

This is the same as SOUNDEX(expr1) = SOUNDEX(expr2).

SPACE(N)
It returns a string consisting of N space characters.

mysql> SELECT CONCAT('Tech',SPACE(6),'Volcano');
+------------------------------------------+
| CONCAT('Tech',SPACE(6),'Volcano') |
+------------------------------------------+
| Tech      Volcano                                   |
+------------------------------------------+

SUBSTR(str,pos) / SUBSTR(str FROM pos) / SUBSTR(str,pos,len) / SUBSTR(str FROM pos FOR len)
SUBSTR() is a synonym for SUBSTRING().

mysql> SELECT SUBSTRING('Volcano',4);
+------------------------------+
| SUBSTRING('Volcano',4) |
+------------------------------+
| cano                                    |
+------------------------------+

mysql> SELECT SUBSTRING('Volcano' FROM 4);
+--------------------------------------+
| SUBSTRING('Volcano' FROM 4) |
+--------------------------------------+
| cano                                                |
+--------------------------------------+

mysql> SELECT SUBSTRING('Volcano',2,3);
+----------------------------------+
| SUBSTRING('Volcano',2,3)     |
+----------------------------------+
| olc                                              |
+----------------------------------+

mysql> SELECT SUBSTRING('Volcano', -3);
+--------------------------------+
| SUBSTRING('Volcano', -3)   |
+--------------------------------+
| ano                                          |
+--------------------------------+

mysql> SELECT SUBSTRING('Volcano', -5, 3);
+----------------------------------+
| SUBSTRING('Volcano', -5, 3) |
+----------------------------------+
| lca                                              |
+----------------------------------+

mysql> SELECT SUBSTRING('Volcano' FROM -4 FOR 2);
+-----------------------------------------------+
| SUBSTRING('Volcano' FROM -4 FOR 2) |
+-----------------------------------------------+
| ca                                                                  |
+-----------------------------------------------+

If len is less than 1, the result is the empty string.

SUBSTRING_INDEX(str,delim,count)
It returns the substring from string str before count occurrences of the delimiter delim. If count is positive, everything to the left of the final delimiter (counting from the left) is returned. If count is negative, everything to the right of the final delimiter (counting from the right) is returned. SUBSTRING_INDEX() performs a case-sensitive match when searching for delim.

mysql> SELECT SUBSTRING_INDEX('www.techvolcano.co.in', '.', 1);
+------------------------------------------------------------+
| SUBSTRING_INDEX('www.techvolcano.co.in', '.', 1) |
+------------------------------------------------------------+
| www                                                                                 |
+------------------------------------------------------------+

mysql> SELECT SUBSTRING_INDEX('www.techvolcano.co.in', '.', 2);
+------------------------------------------------------------+
| SUBSTRING_INDEX('www.techvolcano.co.in', '.', 2) |
+------------------------------------------------------------+
| www.techvolcano                                                            |
+------------------------------------------------------------+

mysql> SELECT SUBSTRING_INDEX('www.techvolcano.co.in', '.', -2);
+-------------------------------------------------------------+
| SUBSTRING_INDEX('www.techvolcano.co.in', '.', -2) |
+-------------------------------------------------------------+
| co.in                                                                                    |
+-------------------------------------------------------------+

TRIM([{BOTH | LEADING | TRAILING} [remstr] FROM] str), TRIM([remstr FROM] str)
It returns the string str with all remstr prefixes or suffixes removed. If none of the specifiers BOTH, LEADING, or TRAILING is given, BOTH is assumed. remstr is optional and, if not specified, spaces are removed.

mysql> SELECT TRIM('  volcano   ');
+----------------------+
| TRIM('  volcano   ') |
+----------------------+
| volcano                    |
+----------------------+

mysql> SELECT TRIM(LEADING 'x' FROM 'xxxvolcanoxxx');
+---------------------------------------------------+
| TRIM(LEADING 'x' FROM 'xxxvolcanoxxx') |
+---------------------------------------------------+
| volcanoxxx                                                         |
+---------------------------------------------------+

mysql> SELECT TRIM(BOTH 'x' FROM 'xxxvolcanoxxx');
+----------------------------------------------+
| TRIM(BOTH 'x' FROM 'xxxvolcanoxxx') |
+----------------------------------------------+
| volcano                                                        |
+----------------------------------------------+

mysql> SELECT TRIM(TRAILING 'ano' FROM 'volcano');
+----------------------------------------------+
| TRIM(TRAILING 'ano' FROM 'volcano') |
+----------------------------------------------+
| volc                                                               |
+----------------------------------------------+

UCASE(str)
UCASE() is a synonym for UPPER().

mysql> SELECT UPPER('Tech Volcano');
+---------------------------+
| UPPER('Tech Volcano') |
+---------------------------+
| TECH VOLCANO          |
+---------------------------+

Thanks for reading this article !

Read More