IBM Analytics Ideas

Welcome to the idea forum for IBM Analytics Clients! 

 

IBM Employees:

The correct URL for entering your ideas is https://hybridcloudunit-internal.ideas.aha.io

 

Clients:

Our team welcomes any feedback  and suggestions you have for improving our offerings / products!  This forum allows us to connect your offering / product improvement ideas with IBM product and engineering teams.

 

If you have not registered on this portal please click on the following link and register.  To complete registration you will need to open the email you will receive from Aha to confirm your identity.  https://ibm.biz/AnalyticsIdeasPortalRegister

 

Implement data loading process which utilizes multiple threads

As reported in the case TS001233758, in AS 3.1.1 it takes about 50 minutes to load 21 mln rows from Oracle to Hadoop; in our test environment it took less time to load data from Hadoop to Oracle, but the process is still slow.
While working on this case, IBM informed that this behaviour is explained by the fact that AS is loading data in one thread, not in multiple threads, as could be expected with HDFS.
We suggest to implement data loading process which utilizes multiple threads (as opposed to single thread), so that the import and export operations from/to relational databases to/from Hadoop are performed faster.

  • Eduard Reshetilov
  • Oct 2 2018
  • Needs review
Role Summary performance
  • Attach files

NOTICE TO EU RESIDENTS: per EU Data Protection Policy, if you wish to remove your personal information from the IBM ideas portal, please login to the ideas portal using your previously registered information then change your email to "anonymous@euprivacy.out" and first name to "anonymous" and last name to "anonymous". This will ensure that IBM will not send any emails to you about all idea submissions